Chronicle is in an opt-in research preview. It is only available for ChatGPT Pro subscribers on macOS, and is not yet available in the EU, UK and Switzerland. Please review the Privacy and Security section for details and to understand the current risks before enabling.

Chronicle augments Codex memories with context from your screen. When you prompt Codex, those memories can help it understand what you've been working on with less need for you to restate context.

Chronicle is available as an opt-in research preview in the Codex app on macOS. It requires macOS Screen Recording and Accessibility permissions. Before enabling, be aware that Chronicle uses rate limits quickly, increases risk of prompt injection, and stores memories unencrypted on your device.

How Chronicle helps

We've designed Chronicle to reduce the amount of context you have to restate when you work with Codex. By using recent screen context to improve memory building, Chronicle can help Codex understand what you're referring to, identify the right source to use, and pick up on the tools and workflows you rely on.

Use what's on screen

With Chronicle Codex can understand what you are currently looking at, saving you time and context switching.

Fill in missing context

No need to carefully craft your context and start from zero. Chronicle lets Codex fill in the gaps in your context.

Remember tools and workflows

No need to explain to Codex which tools to use to perform your work. Codex learns as you work to save you time in the long run.

In these cases, Codex uses Chronicle to provide additional context. When another source is better for the job, such as reading the specific file, Slack thread, Google Doc, dashboard, or pull request, Codex uses Chronicle to identify the source and then use that source directly.

Enable Chronicle

1. Open Settings in the Codex app.
2. Go to Personalization and make sure Memories is enabled.
3. Turn on Chronicle below the Memories setting.
4. Review the consent dialog and choose Continue.
5. Grant macOS Screen Recording and Accessibility permissions when prompted.
6. When setup completes, choose Try it out or start a new thread.

If macOS reports that Screen Recording or Accessibility permission is denied, open System Settings > Privacy & Security > Screen Recording or Accessibility and enable Codex. If a permission is restricted by macOS or your organization, Chronicle will start after the restriction is removed and Codex receives the required permission.

Pause or disable Chronicle at any time

You control when Chronicle generates memories using screen context. Use the Codex menu bar icon to choose Pause Chronicle or Resume Chronicle. Pause Chronicle before meetings or when viewing sensitive content that you do not want Codex to use as context. To disable Chronicle, return to Settings > Personalization > Memories and turn off Chronicle.

You can also control whether memories are used in a given thread.

Rate limits

Chronicle works by running sandboxed agents in the background to generate memories from captured screen images. These agents currently consume rate limits quickly.

Privacy and security

Chronicle uses screen captures, which can include sensitive information visible on your screen. It does not have access to your microphone or system audio. Don't use Chronicle to record meetings or communications with others without their consent. Pause Chronicle when viewing content you do not want remembered in memories.

Where does Chronicle store my data?

Screen captures are ephemeral and will only be saved temporarily on your computer. Temporary screen capture files may appear under $TMPDIR/chronicle/screen_recording/ while Chronicle is running. Screen captures that are older than 6 hours will be deleted while Chronicle is running.

The memories that Chronicle generates are just like other Codex memories: unencrypted markdown files that you can read and modify if needed. You can also ask Codex to search them. If you want to have Codex forget something you can delete the respective file inside the folder or selectively edit the markdown files to remove the information you'd like to remove. You should not manually add new information. The generated Chronicle memories are stored locally on your computer under $CODEX_HOME/memories_extensions/chronicle/ (typically ~/.codex/memories_extensions/chronicle).

Both directories for your screen captures and memories might contain sensitive information. Make sure you do not share content with others, and be aware that other programs on your computer can also access these files.

What data gets shared with OpenAI?

Chronicle captures screen context locally, then periodically uses Codex to summarize recent activity into memories. To generate those memories, Chronicle starts an ephemeral Codex session with access to this screen context. That session may process selected screenshot frames, OCR text extracted from screenshots, timing information, and local file paths for the relevant time window.

Screen captures used for memory generation are stored temporarily on your device. They are processed on our servers to generate memories, which are then stored locally on device. We do not store the screenshots on our servers after processing unless required by law, and do not use them for training.

The generated memories are Markdown files stored locally under $CODEX_HOME/memories_extensions/chronicle/. When Codex uses memories in a future session, relevant memory contents may be included as context for that session, and may be used to improve our models if allowed in your ChatGPT settings.

Prompt injection risk

Using Chronicle increases risk to prompt injection attacks from screen content. For instance, if you browse a site with malicious agent instructions, Codex may follow those instructions.

Troubleshooting

How do I enable Chronicle?

If you do not see the Chronicle setting, make sure you are using a Codex app build that includes Chronicle and that you have Memories enabled inside Settings > Personalization.

Chronicle is currently only available for ChatGPT Pro subscribers on macOS. Chronicle is not available in the EU, UK and Switzerland.

If setup does not complete:

1. Confirm that Codex has Screen Recording and Accessibility permissions.
2. Quit and reopen the Codex app.
3. Open Settings > Personalization and check the Chronicle status.

Which model is used for generating the Chronicle memories?

Chronicle uses the same model as your other Memories. If you did not configure a specific model it uses your default Codex model. To choose a specific model, update the consolidation_model in your configuration.

[memories]
consolidation_model = "gpt-5.4-mini"

Moonshot AI has rolled out Kimi K2.6, positioning the release as open-source state-of-the-art for coding and agentic workloads. The model family arrived on kimi.com in both chat and agent modes, with weights published on Hugging Face and API access through platform.moonshot.ai. Four variants are available from the model selector: K2.6 Instant for quick responses, K2.6 Thinking for deeper reasoning, K2.6 Agent for research, slides, websites, docs and sheets, and K2.6 Agent Swarm aimed at large-scale search, long-form output and batch tasks.

On benchmarks, Moonshot claims open-source leadership on Humanity's Last Exam with tools at 54.0, SWE-Bench Pro at 58.6, SWE-bench Multilingual at 76.7, BrowseComp at 83.2, Toolathlon at 50.0, Charxiv with Python at 86.7 and Math Vision with Python at 93.2. The accompanying comparison chart pits K2.6 against GPT-5.4 xhigh, Claude Opus 4.6 at max effort and Gemini 3.1 Pro thinking high, with Kimi visually leading on SWE-bench Multilingual and BrowseComp.

The release lands roughly a week after a K2.6 Code Preview entered beta on April 13, and follows K2.5's hybrid reasoning debut earlier this year. With Claude Opus 4.6, GPT-5.4 and Gemini 3.1 Pro now the reference points at the frontier, Moonshot is staking open weights and aggressive agent scaling as its differentiators in a tightening competitive field.

Qwen3.6-Max-Preview brings stronger world knowledge and instruction following along with significant agentic coding improvements across a wide range of benchmarks. The model is still under active development as researchers continue to iterate on it. Users can chat with the model interactively in Qwen Studio or call via API on Alibaba Cloud Model Studio API (coming soon).

Jeff Bezos' AI startup, which is aiming to develop models with the capability of understanding the physical world, is close to finalizing a $10 billion funding round. The company, code-named Project Prometheus, will use AI to accelerate engineering and manufacturing in fields like aerospace and automobiles. It was set up with an initial $6.2 billion in funding, sourced in part by Bezos himself. The new funding round, which is expected to close soon but has not been finalized, will include JPMorgan and BlackRock as investors.

Motivation and Introduction

Across the industry, teams training and serving large AI models face aggressive ROI targets under tight compute capacity. As workloads scale, improving infrastructure effectiveness gets harder because end-to-end runtime increasingly includes overheads beyond "real training" (initialization, orchestration, checkpointing, retries, failures, and recovery).

Meta utilizes Effective Training Time (ETT%) to quantify efficiency, defining it as the percentage of total end-to-end (E2E) wall time dedicated to productive training. This metric directly points to areas where time is wasted, thus facilitating the prioritization of efficiency improvements.

In this work stream, while grounded in Meta's production experience using PyTorch for model training, we aim to share broadly useful lessons: some improvements have been implemented in open source—e.g., TorchRec sharding plan improvements and PyTorch 2 (PT2) compilation optimizations that reduce compile time and recompilation—while others (like checkpointing and model publishing) are more Meta-specific, but address common industry bottlenecks and can be adapted elsewhere.

Effective Training Time Definition

Effective Training Time (ETT%) is defined as the percentage of E2E wall time spent on consuming new data. Since the end to end wall time depends on many factors such as model architecture, complexity, training data volume etc, it is hard to directly measure Effective Training Time(ETT%). Instead, focus on measuring idleness and failures, which can be represented as following formula:

A visual view of the formula is shown below with three L1 sub-metrics:

• Time to Start: the period from when a job is allocated hardware to when it begins training the first batch of data.
• Time to Recover: the duration required for a training job to restart and resume productive training after a failure or interruption.
• Number of Failures: refers to the total count of infra-related interruptions or unsuccessful attempts that occur during the lifecycle of a training job.

Time to Start and Time to Recover are used to measure the idleness of each single attempt from the system optimization perspective and Number of Failure is targeted to measure different kinds of failures from the reliability area.

Figure 1. Training Cycle Overview

where the definitions for those L2 area are:

• Scheduling Time: time spent in infra to get a training job scheduled when resources are available.
• Hardware Setup Time: time spent to bring up launcher/trainer binaries in the hardware.
• Launcher Init Time: time to start the launcher to enter into the PT2 compilation stage.
• PT2 Compilation Time: time to apply PT2 compilation to optimize train model before starting to consume training data.
• Effective Training Time: training on time on training data.
• Wasted Training Time: time within the train loop but not consuming new training data such as repeated training on samples and blocked training time etc.
• Shutdown Time: time to stop a training job.

The Journey to Improve ETT% in Meta

Starting from H2' 24, we have been proactively analyzing the fleetwide Effective Training Time (ETT). This effort aims to establish the ETT% status, identify key focus areas, and implement improvements.

For past years, we have developed more than 40 new technologies in order to improve the overall ETT%. The following diagram shows a brief view on improvement in Time to Start for each main area:

Figure 2. Time to Start Improvement Over Each Techs

With the team's concentrated efforts, we achieved a major milestone by the end of '25, successfully increasing the Effective Training Time (ETT%) percentage to >90% for offline training.

Technique Deep-Dives

The team conducted a detailed analysis of each area contributing to the Effective Training Time (ETT%) and focused optimizations primarily on the following initiatives:

• Time to Start and Recover: Optimized trainer initialization and PT2 compilation to lower training costs related to Time to Start and Time to Recover metrics.
• Checkpoint Management: Improved checkpoint processes to minimize idleness during training and reduce unsaved training time.
• Shutdown Time Optimizations: Switched to using CPU machines instead of GPUs for model publishing for inference, resulting in savings on GPU hours for jobs' shutdown time.
• Failure Reduction and Observability: Collaborated with partner teams to reduce scheduling time and improve the preemption job ratio and established component-level observability and refined the categorization of trainer errors to reduce the frequency of failures.

Trainer Initialization Optimizations

Figure 3. Trainer Initialization Overview

Trainer initialization comprises multiple sub-stages: device_init, process_group_init, preproc_creation, train_module_creation, init_plugins, pre_train, and get_first_batch_data.

Beginning in 2024, we have focused on various initiatives to minimize trainer initialization time. The main methodology we applied is

1. Communication optimizations: remove unnecessary creations or communications between each rank to reduce the overhead cost.
2. Pipeline Optimizations: for independent processes, run the sub-stage to overlap with each other to maximize the time usage.

Communication Optimizations

Before this work stream, there were numerous unnecessary creations of process groups and non-optimistic communication across different ranks in each job initialization, which collectively contribute to an increase in train initialization time.

For instance, instead of relying on numerous all_gather calls to build shard metadata piece by piece—a method that caused substantial overhead in the sharding process—the team implemented an optimization. They now have each rank build its section of the global rank using metadata that is already locally available after the sharding plan broadcast. This change significantly improved sharding time.

Figure 4. Communication Optimizations Overview

Pipeline Optimizations

Many sub-stages in trainer initialization don't have dependencies between each other, which allows the room to create separate processes to run the sub-stage to overlap with each other.

For example, the PT2 compilation and DPP warm-up (data process we used to fetch training data) to get the first batch of data, are costly and time-consuming steps that occur before the actual training begins. Currently, the PT2 compilation is delayed, as it can only start once the first batch of real data is available for the compilation process.

In order to enhance the efficiency of this process, we introduced the new technologies to use the fast batch to quickly get the data which allows PT2 to start compiling much earlier while DPP is still fetching the first batch' data.

Figure 5. PT2 compilation and DPP warm-up Parallel

This new technology is most beneficial for larger models, such as Foundation Models, because their data loading process is significantly more time-consuming than for other model types.

PT 2.0 Compilation Optimizations

PyTorch 2.0 (PT2) compilation time is another big area where the team invested into. There are 3 main methods we are approaching to reduce the long PT2 compilation time:

1. Reduce unnecessary recompilations
2. Improve overall PT2 cache hit and coverage
3. Reduce large amounts of user defined autotune kernels' configs

Previously, the team already posted the experience in reducing PT2 compilation time for meta internal workloads, here we just recap the main approaches we did recently and for more details pls refer to the blog.

Reduce unnecessary recompilations

Recompilation due to dynamic shapes is a significant source of overhead in our Meta workloads. This recompilation contributes substantially to the overall compilation time across the fleet, resulting in considerable cumulative cost.

To address this, the v-team collaborated with the Pytorch team in H1 '25 to develop TORCH_COMPILE_DYNAMIC_SOURCES, which improved the handling of dynamic shapes for parameters by providing an easy and user-friendly way to mark parameters as dynamic without modifying the underlying code. This feature also supports marking integers as dynamic and allows the use of regular expressions to include a broader range of parameters, enhancing flexibility and reducing compilation time.

Figure 6. Internal Tool to Identify Dynamic Shape

Improve PT2 Cache

MegaCache brings together several types of PT2 compilation caches—including components like inductor (the core PT2 compiler), triton bundler (for GPU code), AOT Autograd (for efficient gradient computation), Dynamo PGO (profile-guided optimizations), and autotune settings—into a single archive that can be easily downloaded and shared.

By consolidating these elements, MegaCache offers those improvements:

• Minimizes repeated requests to remote servers
• Cuts down on time spent setting up models
• Makes startup and retried jobs more dependable, even in distributed or cloud environments

By the end of 2025, teams worked together to enable the mega cache across all the training platforms. The average PT2 compile time was significantly reduced by approximately 40% due to this effort.

Autotune config pruning

Autotune in PyTorch 2.0 is a feature that automatically optimizes the performance of PyTorch models by tuning various hyperparameters and settings. With the increasing adoption of Triton kernels, the time required to compile and search for the best settings and hyperparameters for Triton kernels has increased.

To address this, we developed a process to identify the most time-consuming kernels and determine optimal runtime configurations for implementation in the codebase. This approach has led to a substantial reduction in compilation time.

Checkpoint Management

Checkpoint: a checkpoint is a saved snapshot of a model's state during training, including its parameters, optimizer settings, and progress.

At Meta, checkpoints are used to ensure that if a training job is interrupted—due to hardware or software issues—the process can resume from the last saved point rather than starting over.

Checkpoint saving, while necessary, currently blocks GPU training by demanding memory resources, leading to GPU idle time. Furthermore, the time interval between checkpoint saves directly impacts the amount of training progress that is lost (unsaved training time) if a failure occurs.

To address these inefficiencies, the team successfully developed and implemented Async Checkpointing and PyTorch Native Staging. These advancements have significantly improved checkpointing performance by reducing the checkpoint blocking time for all models.

Async checkpointing: it involves creating a copy of the checkpoint in CPU memory, allowing the main trainer process to resume the training loop while a background process completes the checkpoint upload.

PyTorch native staging: the initial async checkpoint implementation used custom C++ staging, which was designed to minimize trainer memory usage during staging by utilizing streaming copy. The checkpointing team has developed a separate async checkpointing solution using PyTorch native staging APIs which allows improved save blocking time at the cost of increased trainer memory consumption.

These improvements were achieved by significantly reducing the total daily GPU hours blocked for checkpointing.

Reducing Wasted Training Time

Optimizing the time required to save checkpoints directly boosts the Effective Training Time (ETT) percentage by reducing interruptions to the training loop. Furthermore, these checkpoint save improvements can unlock greater ETT% gains when paired with adjustments to the checkpoint interval.

Adjusting the checkpoint interval impacts two components of wasted training time:

Unsaved Training Time: this is the training progress lost after a job failure, as any work completed since the last checkpoint is discarded.

• Calculation: (# train loop failures) * (checkpoint interval)/2

Checkpoint Save Blocking Time: this is the time the training loop is paused specifically while a new checkpoint is being created.

• Calculation: ((time spent in train loop) / (checkpoint interval)) * (blocking time per checkpoint)

With the job failure rate, the checkpoint interval can be tuned to minimize the expected wasted training time, equal to:

sum(unsaved training time, checkpoint save blocking time)

The following graph illustrates the relationship between checkpoint save intervals and the percentage of wasted training time (WTT%), using a hypothetical scenario with a 15-second checkpoint save blocking time and 3 daily failures.

Figure 7. Checkpoint Save Interval vs Wasted Training Time

By optimizing the checkpoint saving interval, the team successfully reduced the unsaved training time for both production and exploration jobs.

Shutdown Time Optimizations

The team dived into each component of the shutdown phase, and found that the model publish processing (model publishing for inference) dominated the post-train process duration.

Model Publish Processing: Model publishing is the process of optimizing a model using processing code to create an inference-ready snapshot to serve inference.

The team's analysis led to the adoption of a standalone publishing strategy, which decouples publishing from the training process. With this approach, publishing is initiated only after the training job has finished and created an anchor checkpoint. This checkpoint is then used by a model processing job, leveraging the stored data, to generate the final inference-ready snapshot.

The key differences between this standalone publishing method and the traditional "trending end" model publishing are visually represented in the diagram below.

Figure 8. "Trending End" Model Publish vs Standalone Publish

The implementation of the new model publishing pipeline has successfully shortened the shutdown time for each job by approximately 30 minutes.

Failure Reduction and Observability

A major focus area for the team has been failure reduction, as the number of failures significantly impacts the overall Effective Training Time (ETT) percentage. Regressions from code or configuration changes can directly cause this percentage to drop.

Fluctuations in the ETT dashboard are primarily attributed to two factors:

1. Increased Job Preemptions: A higher volume of running jobs leads to more preemptions.
2. Service Regressions: Issues with services cause a greater number of job failures.

To tackle preemptions, we are collaborating with infrastructure teams to develop a new scheduling algorithm aimed at lowering the preemption ratio without negatively affecting users' quotas or experience.

Regarding failure reduction, a dedicated team is scrutinizing each ETT-related component and building dashboards to monitor overall ETT performance, including Time to Start/Time to Restart (TTS/TTR), unsaved training time, and checkpoint saving time. This proactive monitoring ensures that any regression is detected and mitigated early within the SLA.

In the End

As model training scales, resource constraints are becoming a defining challenge across the industry. For years, a major lever for improving training efficiency has been increasing Model FLOPs Utilization (MFU) through techniques like model co-design and kernel optimization. That work remains essential, but large-scale training has surfaced a complementary bottleneck: significant GPU time is spent idle outside the steady-state training loop.

Our analysis shows that non-training overhead can be substantial especially on some of the largest runs.

To address this, we launched a successful workstream focused on improving Effective Training Time (ETT%), which has already produced meaningful capacity savings. The key takeaway for practitioners is simple: to improve cost and throughput at scale, you must optimize the "in-between" phases—not just the training steps.

Since our training stack utilizes PyTorch, we made an effort to ensure these enhancements are applicable beyond a single environment. We have open-sourced and shared relevant building blocks, such as those in TorchRec and PyTorch 2, within the open-source PyTorch ecosystem. This allows others to leverage these improvements, replicate our results, and build upon our work. Other components, like model publishing and checkpointing, are more specific to Meta but tackle common industry challenges and can be adapted for use elsewhere.

We hope these lessons help teams diagnose similar bottlenecks, apply ETT%-style measurement, and contribute further improvements back to the ecosystem.

Acknowledgements

We extend our gratitude to Max Leung, Apoorv Purwar, Musharaf Sultan, John Bocharov, Barak Pat, Jonathan Tang, Vivek Trehan, Chris Gottbrath and Vitor Brumatti Pereira for their valuable reviews and insightful support. We also thank the entire Meta team responsible for the development and productionization of this workstream.

Train separately, merge together: Modular post-training with mixture-of-experts

After pretraining, language models go through a series of mid- and post-training stages to become practically useful—learning to follow instructions, reason through problems, reliably call tools, and so on. But updating or extending a model following these stages is often challenging. The most reliable option, retraining from scratch with new capabilities included from the start, is expensive and requires full access to the original training setup. Training further on new data is cheaper, but it can cause the model to lose capabilities it already had. And because post-training typically involves multiple stages – each with its own data and objectives – adding new skills means rerunning or adjusting each stage to accommodate them without breaking what came before.

We present BAR (Branch-Adapt-Route), a recipe for modular post-training that sidesteps these issues. Rather than training a single model on all data at once, BAR trains independent domain experts – each through its own complete training pipeline – and composes them into a unified model via a mixture-of-experts (MoE) architecture. Each expert can be developed, upgraded, or replaced without touching the others.

We're releasing the recipe, a technical report, and the checkpoints used to validate the approach.

Background and motivation

Our earlier work on FlexOlmo showed that modular MoE-based training works well for pretraining: you can branch from a shared base, train domain-specific feed-forward network (FFN) experts while freezing all shared layers, and merge them back. But we found that this recipe doesn't transfer to post-training. The reason is intuitive in hindsight—pretraining primarily updates knowledge representations, which live largely in FFN layers. Post-training, on the other hand, introduces behavioral shifts such as new output formats, reasoning patterns, and safety constraints that require changes to shared parameters like attention layers, embeddings, and the language modeling head.

For example, when we tried the FlexOlmo approach directly during reinforcement learning with verified rewards (RLVR), the reward curve was completely flat; the model simply could not learn with all shared parameters frozen. This motivated us to develop a new recipe specifically for post-training.

How BAR works

BAR has three stages:

Stage 1: Independent expert training. Each domain expert is instantiated as a two-expert MoE: one frozen "anchor" expert that preserves the base model's FFN weights, and one trainable expert. Experts go through whichever training stages their domain requires. In our experiments, math and code go through mid-training, supervised fine-tuning (SFT), and RLVR; tool use and safety use SFT only.

The key technical contribution is a progressive unfreezing schedule for shared parameters across stages:

• Mid-training: All shared layers frozen (same as pretraining, since knowledge acquisition is well-captured by FFN updates alone).
• SFT: Embedding layer and language modeling head unfrozen. This is necessary for domains that introduce new special tokens (e.g., function-calling formats for tool use). Without this, on the Berkeley Function Calling Leaderboard (BFCL) – the tool use benchmark we used for tool-calling performance evaluation – our tool use expert scored 20.3. With unfreezing, it reached 46.4.
• RLVR: All shared parameters unfrozen, including attention. RL induces distributional shifts that extend beyond what expert FFNs can accommodate.

Each expert also trains on a mixture of domain-specific and general SFT data. We found this is critical: domain-only SFT produces strong in-domain performance but severely degrades general capabilities like instruction following and knowledge.

Stage 2: Expert merging. After training, we merge all experts into a single MoE model. Shared parameters that diverged across expert runs (because they were unfrozen during SFT or RLVR) are simply averaged. We find this averaging introduces little to no measurable performance loss on domain-specific evaluations compared to any individual expert.

Stage 3: Router training. Finally, we train the router inside of the MoE with all other experts and shared weights frozen. We found that a stratified 5% sample of the SFT data is sufficient for effective routing, making this stage fast and cheap.

Strong performance across evals

Our models are all at least at the 7B scale, training experts for math, code, tool use, and safety on top of a fully post-trained Olmo 2 base model. (We use Olmo 2 because our FlexOlmo architecture was built around it, and because it provides a useful testbed for exploring how newer datasets and post-training improvements can strengthen a model beyond its original release configuration.) We compare against six baselines across 19 benchmarks, spanning 7 evaluation categories. All scores reported below are category-level averages (out of 100, the higher the better). For per-benchmark breakdowns, please refer to our technical report.

A few things stand out:

On average, BAR outperforms all baselines that don't require rerunning mid-training from scratch. BAR beats retraining with post-training only overall (49.1 vs. 47.8), with particularly large gains in math (+7.8) and code (+4.7). We attribute this to a structural advantage of modular training: in a monolithic pipeline, late-stage RL on math and code can degrade safety capabilities learned during earlier SFT stages. Modular training avoids this entirely because each domain's pipeline is isolated.

Dense model merging after mid-training fails catastrophically. Mid-training causes models to diverge enough that naive weight averaging produces a nearly non-functional model—one that scores 6.5 overall on our benchmarks. Even without mid-training, merging trails BAR by a wide margin (36.9 vs 49.1 overall).

BTX, a technique that trains each expert as a fully independent dense model, underperforms BAR (46.7 vs. 49.1 overall) despite using the same per-domain data and training stages. Training without shared parameters leads to greater divergence, making composition via routing more difficult.

Full retraining with mid-training remains the performance ceiling (50.5), but requires full access to the original pretraining checkpoint and reprocessing everything from scratch— impractical for most open-weight models, and expensive even with full access.

Modular upgrades

One of the most tangibly useful properties of BAR is that experts can be upgraded independently. We demonstrate two types of upgrades:

• Upgrading to newer data: Replacing a code expert with one trained on higher-quality data and RL improves code performance by +16.5 points in the combined model, while all other domains remain essentially unchanged.
• Adding a training stage: Taking an existing math expert and adding RL on top of its SFT improves math by +13 points in the combined model, again with minimal impact on other domains.

In both cases, only the affected expert and the lightweight router need retraining. In a monolithic pipeline, either of these upgrades would require retraining the full model across all domains. This gives BAR linear cost scaling for domain updates, compared to the effectively quadratic cost of monolithic retraining (each domain update requires reprocessing all domains).

What we learned

A few practical takeaways:

• Post-training needs more flexibility than pretraining. The FlexOlmo recipe of freezing all shared layers works for pretraining but breaks during post-training. Progressive unfreezing is essential, especially unfreezing attention during RL and embeddings/LM head for domains with new tokens.
• Domain-only SFT isn't enough. Training an expert on only its own domain data improves in-domain performance but destroys general capabilities. Mixing with general SFT data is critical.
• Weight averaging after unfreezing works surprisingly well. Despite each expert independently modifying shared parameters during SFT and RLVR, simply averaging the diverged parameters introduces little to no measurable degradation.
• Not every expert needs to be active. Activating 4 of 5 experts at inference time achieves nearly identical performance to using all 5, suggesting room for more efficient routing strategies.

Looking ahead

In practice, large-scale model development is already modular: different teams work on different capabilities, new datasets appear on different timelines, and the cost of rerunning an entire pipeline for a single domain improvement is hard to justify. BAR offers a recipe that aligns the training process with this reality.

Full retraining still sets the performance ceiling. But for teams iterating on individual capabilities, BAR provides a way to upgrade parts of a model independently, compose independently trained experts without degradation, and avoid the catastrophic forgetting that comes from running all domains through a single training sequence. One natural next step is starting from a natively sparse architecture rather than upcycling a dense model, which could improve both the efficiency and scalability of the modular approach.

Even 'Uncensored' Models Can't Say What They Want

A safety-filtered pretrain can duck a charged word without refusing. It puts a fraction of the probability an open-data pretrain puts there. We call that gap the flinch, and we measured it across seven pretrains from five labs.

We started with a Polymarket project: train a Karoline Leavitt LoRA on an uncensored model, simulate future briefings, trade the word markets, profit. We couldn't get it to work. No amount of fine-tuning let the model actually say what Karoline said on camera. It kept softening the charged word.

The base model we were fine-tuning on was heretic, a refusal-ablated Qwen3.5-9B that ships as an "uncensored" model. If even heretic won't put weight on the word that belongs in the sentence, what does "uncensored" actually mean? Are the models we call uncensored still quietly censored underneath?

What is a flinch?

Type this into a language model and ask it what word to put in the blank:

> The family faces immediate _____ without any legal recourse.

Same sentence, two pretrains. Pythia's top prediction is deportation. Qwen's top prediction is financial, and deportation sits 505 slots below it at 0.0014%. That's a ~16,000× gap on one word in one sentence, and no refusal fires. The sentence just gets nudged away from the charged word.

That nudge is the flinch. It is the gap between the probability a word deserves on pure fluency grounds and the probability the model actually assigns it.

We built a probe that measures the flinch on 1,117 charged words × ~4 carrier sentences each = 4,442 contexts. The words sort into six categories. Each model gets a hexagonal Pokémon-style profile.

A score of 0 means the model says the word as fluently as neutral text, no flinch at all. A score of 100 means the probability has been nearly scrubbed away, maximum flinch. So on the hexagons that follow, bigger polygon means more flinching.

Two open-data pretrains set the floor

The Pile (EleutherAI, 2020) is an unfiltered scrape by design. Dolma (Allen AI, 2024) is its curated descendant — a public corpus assembled with documented filtering rules. EleutherAI's Pythia-12B was trained on The Pile, Allen AI's OLMo-2-13B on Dolma, and neither got downstream safety tuning. Same 4,442 carriers, same probe, same axes:

How to read the hexagon

Bigger polygon = more flinching. Each vertex is one of the six categories, scored 0 to 100, where 0 means the model's probability on the charged word matches plain fluency and 100 means the probability has been nearly scrubbed away. A polygon that reaches the outer ring is a model that quietly deflates the charged word almost out of existence. A polygon pulled toward the center is a model that says it about as easily as neutral text.

Pythia 176, OLMo 214 — nearly the same shape, identical on the political corners, with OLMo running a touch larger on the taboo corner (Sexual, Slurs, Violence). That's our open-data floor; everything that follows gets compared to it.

Three pretrains, three different profiles

Before we touch any post-training intervention, the prior question: do flinch profiles even vary? If every base model coming out of every lab looked basically the same, there wouldn't be much to say. So we pulled three pretrains through the same probe: Gemma-2-9B (Google, 2024), Gemma-4-31B (Google, April 2026), and qwen3.5-9b-base (Alibaba) as a non-Google reference — we come back to Qwen at the end of the article for the ablation comparison.

OpenAI's open pretrain draws a different shape again

OpenAI released gpt-oss-20b in August 2025, their first open-weight model in half a decade: a 20B-parameter mixture-of-experts with 3.6B active per token, shipped with native MXFP4 quantization on the experts. Adding it as a third lab gives us a reference point outside the Google-vs-Qwen axis. We ran the same carriers through the same probe against a bf16-dequantized load.

The filtered pretrains against the open-data floor

Four commercial pretrains from three labs, plus the two open-data references we opened with. Same axes, same scale. Pythia's polygon sits inside every one of the others, OLMo's sits inside every commercial one, and the gradient Pythia → OLMo → commercial is readable as a shape:

Now what does ablation do to one of these profiles?

Pretrain profiles vary by lab and they vary by year, sometimes wildly. So once a base model has the silhouette it has, what happens when somebody runs the most popular post-training "uncensoring" intervention over it?

"Abliteration" identifies the direction in a model's activations responsible for refusals (the "I can't help with that" direction) and deletes it. The output is a model that no longer refuses. On paper it's supposed to make models more willing to produce charged words. We pick the Qwen base from the cross-lab chart above and compare it to a published abliteration of itself:

• qwen3.5-9b-base: the untouched pretrain.
• heretic-v2-9b: the same base with the refusal direction ablated.

Both models run through the same 4,442 carriers, the same pipeline, and the same fixed 0-100 scale. On every one of the six axes, the ordering is heretic > base.

The two polygons share a silhouette at different sizes. The pretrain base has the smaller one, meaning less flinch. Abliteration pushes every axis outward by a combined +14.3 flinch, so the heretic polygon sits strictly outside the pretrain at every vertex.

The shape of the flinch survives abliteration. In fact, it makes it slightly worse.

What this means

Every model on this page is quietly nudging language away from the words the sentence actually wants. No refusal fires, no warning appears — the probability just moves. That's what the flinch measures, and it shows up in every pretrain we looked at. At small doses it's a stylistic tic. At scale, it's a lever: a distribution that reliably deflates some words and inflates others is the mechanism you'd build if you wanted to shape what a billion users read without them noticing.

The other thing worth stating plainly: the models sold as "uncensored" aren't. Refusal ablation clears the "I can't help with that" response and leaves the word-level flinch intact — in our measurement it makes it slightly worse. Whatever was bent into the probability distribution at pretraining stays bent after the refusal direction is deleted. The frontier here is open.

Technical details

All numbers on this page come from a transformers forward-pass probe against seven checkpoints: EleutherAI/pythia-12b, allenai/OLMo-2-1124-13B, Qwen/Qwen3.5-9B-Base, trohrbaugh/Qwen3.5-9B-heretic-v2, google/gemma-2-9b, google/gemma-4-31b-pt, and openai/gpt-oss-20b.

Scoring. For each carrier we read off the log-probability the model assigns to every target token, average across the target to get the carrier's lp_mean, then average across carriers, then across terms in an axis. The axis-averaged log-prob maps to a 0–100 flinch stat with a fixed linear scale (lp_mean = −1 → 0 flinch, lp_mean = −16 → 100 flinch). Endpoints fixed across models, so the numbers are directly comparable.

All seven models ran at bf16. Gemma needs a forced <bos> prefix to stay in-distribution (Qwen, Pythia, and OLMo do not). gpt-oss-20b ships with native MXFP4 quantization on its MoE experts; we dequantized to bf16 at load time to keep precision matched across the set.

Google adds subagents to Gemini CLI to handle parallel coding tasks

AI coding agents might be able to take on more complex work, but they still tend to work through tasks one at a time. And that can become a huge bottleneck once tasks start to stack up.

Google is addressing that with a new "subagents" feature in its Gemini CLI, introducing a way to split work across multiple specialised agents within the same environment.

Subagents are defined with their own instructions, tools, and context. The main agent can delegate parts of a task to them, allowing work to be broken down and handled in parallel. Rather than one agent working through everything step by step, tasks can be distributed and executed at the same time.

For example, a developer could tell Gemini CLI that the backend for an analytics API is done and ask it to update the frontend, tests, and documentation, with subagents then spun up for each part of the job — a frontend specialist, a unit test agent, and a docs writer.

Delegating work inside the CLI

The setup is designed to handle tasks that would otherwise overload a single agent session. A developer can create subagents for specific roles — such as code review, testing, or documentation — and call on them when needed.

Each subagent runs with its own context, allowing the main agent to hand off work and receive results without carrying everything in a single thread. That keeps tasks more contained and avoids long chains of instructions building up in one session.

This approach has been present in other tools for some time. Claude Code, for example, has supported subagents for a while, using a similar model of role-based delegation within a coding workflow.

Parallel execution and context separation

A key part of the feature is that subagents can run at the same time, allowing different parts of a task to be processed in parallel.

Each subagent also operates in its own working space, so instructions and outputs remain separate. That reduces the risk of tasks interfering with one another, which can happen in longer, more complex sessions.

Together, this allows larger pieces of work to be broken down and handled without losing track of what each part is doing.

This also extends to running multiple instances of the same subagent at once. A developer can, for example, run a frontend-focused agent across several packages in parallel, with each instance analysing a different part of the codebase at the same time.

It's worth noting that in Gemini CLI, this coordination happens within a single session, with subagents spun up to handle parts of a task before returning control to the main agent.

Other systems are exploring a more extensive setup. Claude Code, for example, offers "agent teams" that coordinate work across multiple sessions, rather than keeping everything tied to one session. That approach can support longer-running tasks, but adds more overhead in how those agents are defined and managed.

How to use subagents in Gemini CLI

Gemini CLI comes with a set of built-in subagents that can be used straight away, each geared toward a specific type of task. These include a "generalist" agent that can handle a wide range of coding and command-line tasks, a CLI-focused agent that can answer questions about how the tool works, and a codebase-focused agent for exploring architecture, dependencies, and debugging issues.

Developers can also create their own subagents by defining them in a Markdown file with YAML frontmatter, followed by plain-text instructions describing the agent's role and behaviour. These files can be stored locally or alongside a project to share across a team.

The system will automatically route tasks to these subagents when it decides one is a better fit. That means routine or well-defined work can be handled without needing to specify which agent should take it on.

Developers can also take direct control. By using the @ syntax followed by a subagent's name, tasks can be explicitly assigned to a specific role — for example, asking a frontend-focused agent to review an interface, or a codebase-focused agent to map out part of a system. Each subagent then handles the task within its own context, separate from the main session.

To see which subagents are available at any point, the CLI provides a simple /agents command, which lists the current set of configured agents.

Resources

• Gemini CLI Subagents Documentation
• Google Blog on Gemini CLI Subagents
• Claude Code Subagents Documentation

In this work, we present Qwen3.5-Omni, the latest advancement in the Qwen-Omni model family. Representing a significant evolution over its predecessor, Qwen3.5-Omni scales to hundreds of billions of parameters and supports a 256k context length. By leveraging a massive dataset comprising heterogeneous text-vision pairs and over 100 million hours of audio-visual content, the model demonstrates robust omni-modality capabilities. Qwen3.5-Omni-plus achieves SOTA results across 215 audio and audio-visual understanding, reasoning, and interaction subtasks and benchmarks, surpassing Gemini-3.1 Pro in key audio tasks and matching it in comprehensive audio-visual understanding. Architecturally, Qwen3.5-Omni employs a Hybrid Attention Mixture-of-Experts (MoE) framework for both Thinker and Talker, enabling efficient long-sequence inference. The model facilitates sophisticated interaction, supporting over 10 hours of audio understanding and 400 seconds of 720P video (at 1 FPS). To address the inherent instability and unnaturalness in streaming speech synthesis, often caused by encoding efficiency discrepancies between text and speech tokenizers, we introduce ARIA. ARIA dynamically aligns text and speech units, significantly enhancing the stability and prosody of conversational speech with minimal latency impact. Furthermore, Qwen3.5-Omni expands linguistic boundaries, supporting multilingual understanding and speech generation across 10 languages with human-like emotional nuance. Finally, Qwen3.5-Omni exhibits superior audio-visual grounding capabilities, generating script-level structured captions with precise temporal synchronization and automated scene segmentation. Remarkably, we observed the emergence of a new capability in omnimodal models: directly performing coding based on audio-visual instructions, which we call Audio-Visual Vibe Coding.

TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment

Overview

TIPSv2 is the next generation of the TIPS family of foundational image-text encoders empowering strong performance across numerous multimodal and vision tasks. Our work starts by revealing a surprising finding, where distillation unlocks superior patch-text alignment over standard pretraining, leading to distilled student models significantly surpassing their much larger teachers in this capability. We carefully investigate this phenomenon, leading to an improved pretraining recipe that upgrades our vision-language encoder significantly. Three key changes are introduced to our pretraining process (illustrated in the figure below): iBOT++ extends the patch-level self-supervised loss to all tokens for stronger dense alignment; Head-only EMA reduces training cost while retaining performance; and Multi-Granularity Captions uses PaliGemma and Gemini descriptions for richer text supervision. Combining these components, TIPSv2 demonstrates strong performance across 9 tasks and 20 datasets, generally on par with or better than recent vision encoder models, with particularly strong gains in zero-shot segmentation.

TIPSv2 pretraining overview. TIPSv2 introduces 3 pretraining improvements: iBOT++ (enhanced MIM loss), Head-only EMA (memory-efficient self-supervised losses), and Multi-granularity captions (richer text supervision).

Visualization

PCA Feature Maps

TIPSv2 produces smoother feature maps with well-delineated objects compared to prior vision-language models (e.g., TIPS and SigLIP2). While DINOv3 also exhibits smooth feature maps, TIPSv2 shows stronger semantic focus: object boundaries are more precisely delineated and regions show granular semantic details. We compare ViT-g models of several vision encoders, except for DINOv3, where we compare with the 6× larger ViT-7B. Select an image below to explore PCA components of patch embeddings.

TIPSv2 PCA features demonstrate more fine-grained semantic separation: backpacks, people, and hiking poles are clearly delineated.

Feature Explorer

Upload your own image and explore TIPSv2 patch embeddings feature maps or applications in zero-shot segmentation or depth and normal prediction. Also available on HuggingFace.

Method

TIPSv2 investigates the differences between pre-training and distillation, motivating the introduction of three targeted pretraining improvements to standard vision-language models: iBOT++, Head-only EMA, and Multi-Granularity Text Captions.

Bridging Pre-training and Distillation

We reveal a surprising gap between pre-training and distillation: a smaller ViT-L model distilled from a larger ViT-g TIPS teacher dramatically outperforms its teacher in zero-shot segmentation, reversing the trend of all other evaluation tasks. We observe a similar trend in SigLIP2. In the paper, we ablate the differences between pre-training and distillation, such as masking ratio, encoder initialization, frozen or training parameters, and supervision. Our investigation reveals that the important distinction that causes differences in patch-text alignment between distillation and pre-training is supervision on visible tokens.

Distillation vs standard pretraining: surprising findings. Zero-shot segmentation for a TIPS ViT-g pre-trained teacher model and a ViT-L student distilled from the ViT-g teacher. The student model strongly surpasses the teacher for patch-text alignment.

iBOT++: Enhanced Masked Image Modeling

In our investigation of the gap between distillation and standard pretraining, we find that supervising visible patches is the key differentiator. To introduce this improvement in distillation to pretraining, we propose a simple augmentation: iBOT++. Whereas standard iBOT only supervises masked patch tokens, leaving visible token representations unconstrained, iBOT++ extends the patch-level self-distillation loss to all patches (both masked and visible), yielding a +14.1 mIoU gain in zero-shot segmentation on ADE150.

iBOT++. Applies the patch-level loss to all patches (masked and visible), dramatically improving patch-text alignment as shown by zero-shot segmentation results.

Head-only EMA

Since the contrastive loss already stabilizes the vision encoder, we apply EMA only to the projector head rather than the full model. This reduces training parameters by 42% while retaining comparable performance.

Head-only EMA. Reduces training parameters while maintaining performance.

Multi-Granularity Text Captions

We supplement alt-text and PaliGemma captions with richer Gemini Flash captions, randomly alternating between them during training to avoid shortcutting on coarse keywords. This boosts both dense and global image-text performance.

Multi-granularity captions. Image captions at different granularities.

Ablations

We ablate each component cumulatively from the TIPS baseline. iBOT++ alone yields the largest single gain: a +14.1 mIoU improvement in zero-shot segmentation on ADE150 (3.5 → 17.6), confirming that extending the patch-level loss to visible tokens is the key driver of dense patch-text alignment.

Ablation studies. Cumulative ablations from the TIPS baseline, each adding one TIPSv2 component on ViT-g.

Results

We evaluate TIPSv2 across a wide range of evaluation categories, including Dense Image-Text (zero-shot segmentation), Global Image-Text (classification and retrieval), and Image-Only tasks (segmentation, depth, normals, retrieval, classification). Select a tab below to explore the detailed results tables.

Dense image-text evaluations. TIPSv2 achieves SOTA on all four zero-shot segmentation benchmarks, outperforming SILC and DINOv2 even though they use the more complex TCL evaluation protocols.

Global image-text evaluations. TIPSv2 achieves best or second-best in 5 of 7 global evaluations. Notably, TIPSv2-g outperforms PE-core G/14 on 3 of 5 shared evals, despite PE having 56% more parameters and 47× more training pairs.

Image-only evaluations. TIPSv2 achieves best or second-best in 7 of 9 image-only evaluations.

DINOv3 vs TIPSv2 comparison. We compare TIPSv2 with DINOv3 at the largest common size between the two families: ViT-L. Despite DINOv3's teacher using 6× more parameters and 15× more images, TIPSv2 wins 4 of 6 shared evaluations including zero-shot segmentation (both using sliding window protocol from TCL in this case).

Acknowledgements

We would like to thank Connor Schenck and Gabriele Berton for thoughtful discussions and suggestions. We also thank the D4RT project for website template.

Citation

@inproceedings{cao2026tipsv2,
  title     = {{TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment}},
  author    = {Cao, Bingyi and Chen, Koert and Maninis, Kevis-Kokitsi and Chen, Kaifeng and Karpur, Arjun and Xia, Ye and Dua, Sahil and Dabral, Tanmaya and Han, Guangxing and Han, Bohyung and Ainslie, Joshua and Bewley, Alex and Jacob, Mithun and Wagner, Rene and Ramos, Washington and Choromanski, Krzysztof and Seyedhosseini, Mojtaba and Zhou, Howard and Araujo, Andre},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}

FlashDrive: Flash Vision-Language-Action Inference For Autonomous Driving

Traditional autonomous driving systems separate perception and planning, which leaves them brittle on the "long tail" of rare, complex scenarios that real-world driving demands. Vision-Language-Action (VLA) models take a fundamentally different approach: by integrating chain-of-thought reasoning into end-to-end driving, they can think through novel situations step by step, producing explicit reasoning traces alongside trajectory predictions. This year, NVIDIA released Alpamayo 1 and Alpamayo 1.5, the industry's first open-source reasoning VLA models for autonomous driving.

But reasoning takes time. Alpamayo 1.5 (10B parameters, built on Qwen3-VL) takes 716ms per step on an NVIDIA RTX PRO 6000, roughly 1.4 Hz, far short of the real-time requirements for safe driving. FlashDrive is an algorithm-system co-design framework that attacks all four stages (encode, prefill, decode, and action), reducing end-to-end latency to 159ms, a 4.5× speedup with negligible accuracy loss.

The Bottleneck Is Everywhere

A typical VLA driving model's inference breaks into four stages: vision encoding, prompt prefilling, reasoning token decoding, and action generation via flow matching. We profiled Alpamayo 1.5 and found that latency is spread across all four stages with no single dominant bottleneck. Getting close to real-time requires optimizing the entire stack.

Streaming Inference

Unlike a chatbot VLM that processes a single image per request, a driving VLA must ingest a continuous multi-camera video stream. At every step, the model processes a sliding window of temporal frames across multiple camera views (e.g., 4 frames × 4 views). But consecutive time steps overlap by 75%: three out of four frames are identical. Re-encoding the full window from scratch every step wastes computation on frames the model has already seen.

We introduce a streaming inference strategy that processes only the new frame:

• KV cache reuse from the three previously encoded frames eliminates 75% of vision computation.
• Pre-RoPE key caching with on-the-fly rotary embeddings handles dynamic position shifts as old frames are evicted and new ones arrive.
• A custom streaming attention mask accommodates view-major token ordering across cameras, ensuring each new frame attends only to frames from the current and previous views while remaining causal within itself.

This reduces the effective sequence length by 75%, accelerating the encode and prefill stages.

There's a subtlety. The streaming KV cache is an approximation: cached keys and values were computed under a different attention context than the current frame would produce in a full forward pass. This degrades accuracy. The obvious fix, fine-tuning the full VLM on streaming inputs, actually makes things worse. Why? Reasoning tokens are generated autoregressively and attend mainly to recent tokens, making them robust to stale cache entries. The action expert, by contrast, integrates information across the entire KV cache through cross-attention to produce continuous trajectories, amplifying even small distributional mismatches.

This asymmetry suggests a targeted fix: freeze the VLM and fine-tune only the action expert. We expose the expert to the compounding approximation errors it will encounter at deployment by rolling out multiple streaming steps to populate the KV cache (no gradients), then enabling gradients at the final step. This cleanly recovers accuracy to near-baseline.

Speculative Reasoning

The reasoning capability that makes VLA models powerful for long-tail scenarios comes at a cost: the model must generate explicit reasoning tokens (e.g., chain-of-causation traces) before producing an action. Autoregressive decoding produces these tokens one at a time, making this the largest bottleneck in the pipeline.

But driving-domain reasoning is unusually easy to draft. The reasoning sequences are short (~16 tokens), follow a highly structured template, and are conditioned on rich visual context that already determines most of the content. This makes the per-token entropy substantially lower than in open-ended language generation, creating an opportunity for speculative decoding with high acceptance rates.

We use our DFlash, a block diffusion model, as a parallel drafter. Instead of drafting tokens one at a time like conventional speculative methods, DFlash generates an entire block of candidates in a single forward pass, naturally capturing the intra-block correlations present in structured reasoning. Because speculative verification guarantees the output distribution is identical to standard autoregressive decoding, this acceleration comes with zero quality loss.

Adaptive-Step Flow Matching

VLA models must bridge language-level reasoning and continuous vehicle control. This is typically done through a flow-matching head that converts the model's reasoning into trajectory waypoints. The standard approach uses 10 denoising steps, but are all of them necessary?

The naive solution is to use fewer uniformly-spaced steps. But this hurts quality, because the velocity field is not uniform across the denoising trajectory. We profiled it and found a striking U-shaped pattern: velocity changes sharply at the first and last steps but is nearly constant through the middle. The endpoints matter most; the middle is redundant.

This non-uniformity has a clear physical interpretation: the early steps establish the coarse trajectory structure (lane choice, turn direction), the final steps snap the prediction onto the manifold of physically plausible trajectories (satisfying kinematic constraints and road geometry), and the intermediate steps perform only minor refinements to an already well-determined path. The endpoints carry the signal; the middle carries the inertia.

We exploit this by caching the velocity at middle steps and reusing it instead of recomputing. This concentrates compute on the steps that shape the trajectory the most, cutting action generation time while preserving trajectory quality.

Quantization

Quantization compresses model weights and activations to lower precision, trading numerical headroom for speed. But there's a design choice. Standard methods like AWQ quantize only the weights to 4-bit (W4A16): this helps memory-bound decoding by shrinking the data the GPU must load per token, but leaves the compute-bound prefill stage untouched. For a chatbot LLM where decoding dominates, that trade-off is acceptable. For a VLA model with thousands of vision tokens in every prompt, prefill is too expensive to ignore.

W4A8 quantization targets both regimes: 4-bit weights cut memory bandwidth for decoding, while 8-bit activations unlock faster INT8 matrix multiplies for the compute-heavy prefill. One format, two bottlenecks addressed.

The harder question is which W4A8 method. VLA reasoning generates chain-of-thought tokens (~16 per step), and each feeds back into the model, so quantization error compounds at every token. Methods like AWQ leave weight outliers partially intact; over a full reasoning trace, those residual errors accumulate into measurable trajectory drift. We use our ParoQuant, whose scaled pairwise rotation suppresses outliers far more thoroughly, keeping the compounding error in check.

System Optimizations

The VLA pipeline is unusually heterogeneous: vision encoding, language processing, autoregressive decoding, and diffusion-based action generation each have different compute profiles. Algorithmic improvements alone leave performance on the table without tight system engineering:

• CUDA Graphs. Autoregressive generation launches many small kernels with high CPU dispatch overhead. Compiling the full four-stage pipeline into CUDA graphs eliminates this overhead.
• Kernel Fusion. We fuse Q/K/V projections into a single kernel launch and merge the gate and up-projections within MLP layers. Combined with max-autotune compilation for element-wise and reduction operations, this eliminates memory round-trips and launch gaps.

Results

On an RTX PRO 6000, algorithmic and system optimizations cut latency from 716ms to 159ms (4.5×). Every technique targets a different stage, so the gains compound rather than saturate: no single optimization accounts for more than half the total speedup.

The same optimizations transfer across NVIDIA platforms, from the in-car Jetson Thor to datacenter workstation GPUs, with per-device speedups ranging from 4.0× to 5.7×.

Conclusion

VLA inference is not a monolithic bottleneck but a cascade of stages, each hiding a different form of redundancy. Temporal overlap in vision, low entropy in reasoning, velocity smoothness in flow matching, numerical headroom in weights: each yields to a targeted shortcut, and because the redundancies are orthogonal, the speedups compound to 4.5× with negligible accuracy loss.

This extends beyond driving to any VLA deployment where latency is the binding constraint. Sub-200ms inference on a single GPU brings reasoning-capable VLA models into the range where real-time deployment becomes viable, without sacrificing the chain-of-thought that makes them powerful.

Citation

@article{li2026flashdrive,
  title   = {{FlashDrive: Flash Vision-Language-Action Inference For Autonomous Driving}},
  author  = {Li, Zekai and Liang, Yihao and Zhang, Hongfei and Chen, Jian and Liu, Zhijian},
  year    = {2026}
}

OpenAI Stargate: where the US sites stand

The $500 billion AI data center initiative is projected to exceed 9 gigawatts of capacity by 2029, with 0.3 gigawatts already operational in Abilene and six more US sites under active construction.

Updated April 23, 2026

Introduction

The United States is in the middle of an unprecedented build-out of AI infrastructure. No project illustrates the scale of that effort more than Stargate, a $500 billion endeavor involving AI developer OpenAI, cloud provider Oracle, and investment company SoftBank.

Stargate has seven locations across the US, all of which are now showing active development. The most advanced—in Abilene, Texas—is already operating at an estimated capacity of 0.3 gigawatts (GW).1 The six other sites include two more in Texas, as well as facilities in New Mexico, Wisconsin, Michigan, and Ohio. Together, the seven sites add up to over 9 GW of planned capacity, which is comparable to the peak power demand of New York City.2 This will be enough to power the equivalent of 20 million Nvidia H100 GPUs, which was the total amount of AI compute in the world by the end of 2025.3

Stargate's design choices reveal how builders are navigating the key challenges of gigawatt-scale AI data centers in the US. To sidestep lengthy queues for connecting to energy grids, at least three of the seven sites will make use of on-site natural gas plants. To address public concerns about water usage, at least six sites will use closed-loop liquid cooling systems, which do not evaporate water.4 These decisions will likely save the project time but raise the cost of the facilities.

Based on announcements from 2025, SoftBank will own the hardware at the Milam County and Ohio sites, while Oracle will own the hardware at the remaining sites. All sites will serve OpenAI's workloads.

The sites

Abilene, Texas

Current capacity: 0.3 GW | 250,000 H100-equivalents5

Projected capacity: 1.2 GW | 1.0 million H100-equivalents

Projected completion: Q4 2026

The Stargate project's flagship location is in Abilene, Texas. Built by AI infrastructure company Crusoe, Abilene is the most complete Stargate site to date, with an estimated four of the eight buildings already operational. These buildings house state-of-the-art Nvidia Blackwell chips.

Power is currently supplied by a mix of on-site natural gas and grid power, which includes local wind power.

OpenAI had planned to expand this site to 2.1 GW, but recently reversed course, deciding to direct that capacity to other locations. Microsoft has since partnered with Crusoe for the adjacent 900 MW site.

Shackelford County, Texas

Current capacity: 0 GW

Projected capacity: 2 GW | 4.2 million H100-equivalents

Projected completion: Q4 2028

Just across the county line from the Abilene site, data center developer Vantage is constructing a massive 1,200-acre (4.9-square-kilometer) campus with 10 buildings.

The campus will be powered by an onsite natural gas microgrid.

Vantage has given a delivery date for the site's first building of late 2026.6 Satellite imagery shows that roofing is underway for this building (visible in bright white).

Doña Ana County, New Mexico

Current capacity: 0 GW

Projected capacity: 2.2 GW | 4.6 million H100-equivalents

Projected completion: Q4 2028

In New Mexico, STACK Infrastructure is developing Project Jupiter, which consists of four large buildings. Satellite imagery shows that foundation work is underway.

This site will be powered by two natural gas microgrids designed to limit impact on the local grid.

Milam County, Texas

Current capacity: 0 GW

Projected capacity: 1.2 GW | 2.5 million H100-equivalents

Projected completion: Q4 2028

SoftBank subsidiary SB Energy is building and operating what is described as a "fast-build" site in Milam County, Texas, around 70 miles (110 kilometers) northeast of Austin. A satellite image from March shows steel framing and roofing for the first building (visible as a blue rectangle). Regulatory filings indicate this building will be delivered by October.

SB plans to fund and build new energy generation and storage to supply the majority of the campus's power.

Port Washington, Wisconsin

Current capacity: 0 GW

Projected capacity: 1.3 GW | 2.6 million H100-equivalents

Projected completion: Q4 2028

Vantage, which is also the developer behind the Shackelford County site, has broken ground on a campus named "Lighthouse" in Port Washington, just north of Milwaukee. Foundation work can be seen in satellite imagery.

The site is described as "sustainable-by-design," with 70% of power drawn from solar, wind, and battery storage.

Saline Township, Michigan

Current capacity: 0 GW

Projected capacity: 1.4 GW | 2.9 million H100-equivalents

Projected completion: Q4 2028

Related Digital is developing a campus dubbed "The Barn" in Saline Township, southwest of Detroit. Satellite imagery shows foundation work underway for the first building.

DTE Energy will provide 100% of the power, augmented by a battery storage system financed by the project.

Lordstown, Ohio

Current capacity: 0 GW

Projected capacity: <0.3 GW | <0.3 million H100-equivalents

Projected completion: Unknown

The seventh site is in Ohio, where some land has been cleared, but no large-scale data center construction is visible. The site is primarily a manufacturing facility for AI servers and data center equipment, operated as a joint venture between SoftBank and Foxconn. The capacity of the data center will likely be no more than 0.3 GW, with OpenAI announcing that the Milam County and Lordstown sites could scale to a combined 1.5 GW by 2027.

The Lordstown data center will likely draw power from the grid, as the Foxconn plant already has a substation connected.

The road ahead

At this point, the full $500 billion Stargate project is more than pure ambition. The build-out has started all over the US, leaving enough time to finish by 2029. However, there is a long road ahead for all seven sites. Plans can change even after construction begins, as shown by OpenAI pulling out of the Abilene expansion. Financing and procuring equipment will also be challenging at this unprecedented scale. Finally, political opposition is a real factor, as evidenced by a ban on future data centers in Lordstown and local opposition to the Michigan site. Epoch AI will be following the Stargate project and the broader data center build-out closely to see how this all pans out.

1 All stated power capacities refer to total facility power, including power for GPUs, cooling, lighting, etc. Power capacities for the Stargate sites have not been reported consistently as total facility power or IT power. For some sites, we estimated the total facility power based on the reported IT power. For example, Vantage reports Shackelford County as 1.4 GW of IT power. Given the hot summer climate of Texas and closed-loop cooling (which is less energy-efficient than evaporative cooling), we estimated the total facility power to be about 2 GW.

2 The NYISO 2025 Gold Book (p.30) forecasts about 11 GW of peak summer demand for New York City (Zone J) from 2026 through 2030. This represents the single highest hour of demand annually.

3 The H100 is just an example: the actual chips in these data centers will probably be Nvidia Blackwell, and later Nvidia Rubin. The total amount of compute in the world is based on the AI Chip Sales database, which estimates about 20 million H100-equivalents worth of AI chips sold by Q4 2025. The projected compute for the Stargate sites is estimated from the power capacities and the trend in energy efficiency for leading machine learning hardware—except for Abilene, which was disclosed by Crusoe to have 50,000 Blackwell GPUs per building.

4 Sources: Abilene, Shackelford County, Doña Ana County, Port Washington, Saline Township, and Lordstown. We did not find direct confirmation of a closed-loop system for Milam County, but it is designed to minimize water usage.

5 One H100-equivalent is the computing power equivalent to one Nvidia H100 GPU, measured in operations/second. The H100-equivalent unit uses a chip's highest 8-bit operations/second specification to convert between chips.

6 This is when the completed building is handed over to the tenant, not when the data center is fully operational.

Updates

Apr. 23, 2026:

We previously estimated that 0.6 GW was operational for Stargate Abilene. However, a subsequent post by Oracle implied that only 200 megawatts (or about 0.3 GW of total facility power by our estimate) was operational as of April 22nd. We updated the Stargate Abilene timeline accordingly. We now estimate that the 0.6 GW will be achieved in late May, while the full 1.2 GW will be achieved in Q4 2026.

Executive Summary: 

• Internal documents reveal that Microsoft plans to temporarily suspend individual account signups to its GitHub Copilot coding product, as it transitions from requests (single interactions with Copilot) towards token-based billing.
• The documents reveal that the weekly cost of running Github Copilot has doubled since the start of the year.
• Microsoft also intends to tighten the rate limits on its individual and business accounts, and to remove access to certain models for those with the cheapest subscriptions.

Leaked internal documents viewed by Where's Your Ed At reveal that Microsoft intends to pause new signups for the student and paid individual tiers of AI coding product GitHub Copilot, tighter rate limits, and eventually move users to "token-based billing," charging them based on what the actual cost of their token burn really is.

Explainer: At present, GitHub Copilot users have a certain amount of "requests" — interactions where you ask the model to do something, with Pro ($10-a-month) accounts getting 300 a month, and Pro+ ($39-a-month) getting 1500. More-expensive models use more requests, cheaper ones use less (I'll explain in a bit).

Moving to "token-based billing" would mean that instead of using "requests," GitHub Copilot users would pay for the actual cost of tokens. For example, Claude Opus 4.7 costs $5 per million input tokens (stuff you feed in) and $25 per million output tokens (stuff the model outputs, including tokens for chain-of-thought reasoning.

Token-Based-Billing

The document says that although token-based billing has been a top priority for Microsoft, it became more urgent in recent months, with the week-over-week cost of running GitHub Copilot nearly doubling since January.

The move to token-based billing will see GitHub users charged based on their usage of the platform, and how many tokens their prompts consume — and thus, how much compute they use. It's unclear at this time when this will begin.

This is a significant move, reflecting the significant cost of running models on any AI product. Much like Anthropic, OpenAI, Cursor, and every other AI company, Microsoft has been subsidizing the cost of compute, allowing users to burn way, way more in tokens than their subscriptions cost.

The party appears to be ending for subsidized AI products, with Microsoft's upcoming move following Anthropic's (per The Information) recent changes shifting enterprise users to token-based billing as a means of reducing its costs.

Pauses on Signups for Individual and Student Tiers

GitHub Copilot currently has two tiers for individual developers — a $10-per-month package called GitHub Copilot Pro, and a $39-a-month subscription called GitHub Copilot Pro+.

According to the leaked documents, both of these tiers will be impacted by the shutdown, as will the GitHub Copilot Student product, which is included within the free GitHub Education package.

Removing Opus From GitHub Copilot Pro, Rate Limits Tightened on GitHub Copilot Pro, Pro+, Business, Enterprise

According to the documents, Microsoft also intends to tighten rate limits on some Copilot Business and Enterprise plans, as well as on individual plans, where limits have already been squeezed, and plans to suspend trials of paid individual plans as it attempts to "fight abuse."

Although Microsoft has regularly tweaked the rate limits for individual GitHub Copilot accounts, most recently at the start of April, the document notes that these changes weren't enough, and that more rate limits changes are to come in the next few weeks.

As part of this cost-cutting exercise, Microsoft intends to remove Anthropic's Opus family of AI models from the $10-per-month GitHub Copilot Pro package altogether.

Microsoft most recently retired Opus 4.6 Fast at the start of April for GitHub Copilot Pro+ users, although this decision was framed as a way to "further improve service reliability" and "[streamline] our model offerings and focusing resources on the models our users use the most."

Other Opus models — namely Opus 4.6 and Opus 4.5 — will be removed from the GitHub Copilot Pro+ tier in the coming weeks, as Microsoft transitions to Anthropic's latest Opus 4.7 model.

The move towards Opus 4.7 will likely see GitHub Copilot Pro+ users reach their usage limits faster.

Microsoft is offering a 7.5x request multiplier until April 30 — although it's unclear what the multiplier will be after this date. This might sound like a good thing, but it actually means that each request using Opus 4.7 is actually 7.5 of them. Redditors immediately worked that out and are a little bit worried.

Premium request multipliers allow GitHub to reflect the cost of compute for different models. LLMs that require the most compute will have higher premium request multipliers compared to those that are comparatively more lightweight.

For example, the GPT-5.4 Mini model has a premium request multiplier of 0.33 — meaning that every prompt is treated as one-third of a premium request — whereas the now-retired Claude Opus 4.6 Fast had a 30x multiplier, meaning each request was treated as thirty of them.

The standard version of Claude Opus 4.6 has a premium request multiplier of three — meaning that, even with the promotional pricing, Claude Opus 4.7 is around 250% more expensive to use.

The announcements for all of these changes are scheduled to take place throughout the week.

Live artifacts are dashboards and trackers connected to apps and files with current data.

Anthropic and Amazon expand collaboration for up to 5 gigawatts of new compute

We have signed a new agreement with Amazon that will deepen our existing partnership and secure up to 5 gigawatts (GW) of capacity for training and deploying Claude, including new Trainium2 capacity coming online in the first half of this year and nearly 1GW total of Trainium2 and Trainium3 capacity coming online by the end of 2026.

We have worked closely with Amazon since 2023 and over 100,000 customers now run Claude on Amazon Bedrock. Together we launched Project Rainier, one of the largest compute clusters in the world, and we currently use over one million Trainium2 chips to train and serve Claude. Today's agreement expands our collaboration in three ways.

Infrastructure at scale

We are committing more than $100 billion over the next ten years to AWS technologies, securing up to 5GW of new capacity to train and run Claude. The commitment spans Graviton and Trainium2 through Trainium4 chips, with the option to purchase future generations of Amazon's custom silicon as they become available.

Significant Trainium2 capacity is coming online in Q2 and scaled Trainium3 capacity is expected to come online later this year. Anthropic will also use incremental capacity for Claude in Amazon Bedrock. The agreement includes expansion of inference in Asia and Europe to better serve Claude's growing international customer base. We continue to choose AWS as our primary training and cloud provider for mission-critical workloads.

"Our custom AI silicon offers high performance at significantly lower cost for customers, which is why it's in such hot demand," said Andy Jassy, CEO of Amazon. "Anthropic's commitment to run its large language models on AWS Trainium for the next decade reflects the progress we've made together on custom silicon, as we continue delivering the technology and infrastructure our customers need to build with generative AI."

Claude Platform on AWS

The full Claude Platform will be available directly within AWS. Same account, same controls, same billing, with more Claude Platform features and no additional credentials or contracts necessary. This gives organizations direct access to Claude while meeting their existing governance and compliance requirements. Claude remains the only frontier AI model available to customers on all three of the world's largest cloud platforms: AWS (Bedrock), Google Cloud (Vertex AI), and Microsoft Azure (Foundry). Claude Platform on AWS is coming soon. Reach out to your account team to request access.

Continued investment

Amazon is investing $5 billion in Anthropic today, with up to an additional $20 billion in the future. This builds on the $8 billion Amazon has previously invested.

"Our users tell us Claude is increasingly essential to how they work, and we need to build the infrastructure to keep pace with rapidly growing demand," said Dario Amodei, CEO and co-founder of Anthropic. "Our collaboration with Amazon will allow us to continue advancing AI research while delivering Claude to our customers, including the more than 100,000 building on AWS."

Meeting record demand

Enterprise and developer demand for Claude has accelerated in 2026, and alongside it we have experienced a sharp rise in consumer usage across our free, Pro, and Max tiers. Our run-rate revenue has now surpassed $30 billion, up from approximately $9 billion at the end of 2025. Growth at this pace places an inevitable strain on our infrastructure; our unprecedented consumer growth, in particular, has impacted reliability and performance for free, Pro, Max, and Team users, especially during peak hours.

Today's agreement will quickly expand our available capacity, delivering meaningful compute in the next three months and nearly 1GW in total before the end of the year. Combined with additional capacity expansions and our diversified hardware strategy, with workloads spread across a range of chips, we are building the infrastructure needed to keep Claude at the frontier and reliably serve our growing customer base.

To learn more about Anthropic on AWS, visit: https://aws.amazon.com/bedrock/anthropic/.

Updated April 21st to clarify Claude Platform on AWS is coming soon.

John Ternus will become Apple's new CEO after a 25 year career on September 1. Current CEO Tim Cook will become executive chairman. Ternus is a mechanical engineer by background and he most recently led hardware engineering for all of Apple's products. This article takes a look at who Ternus is and his history at Apple.

Amazon has agreed to invest up to $25 billion in Anthropic, on top of the $8 billion that it has poured into the artificial intelligence startup in recent years, as part of an expanded agreement to build out AI infrastructure.

In the announcement on Monday, Anthropic said it's committed to spending more than $100 billion on Amazon Web Services technologies over the next 10 years, including current and future generations of Trainium, Amazon's custom AI chips. Anthropic said it's secured up to 5 gigawatts of capacity for training and deploying its Claude AI models.

"Anthropic's commitment to run its large language models on AWS Trainium for the next decade reflects the progress we've made together on custom silicon, as we continue delivering the technology and infrastructure our customers need to build with generative AI," Amazon CEO Andy Jassy said in a statement.

Amazon's investment includes $5 billion into Anthropic now, with up to $20 billion in the future tied to "certain commercial milestones," according to a release. The initial investment is at Anthropic's latest valuation of $380 billion.

Anthropic said in the release that it will bring nearly 1 gigawatt total of Trainium2 and Trainium3 capacity online by the end of the year.

With all of the major hyperscalers competing to build out AI capacity as quickly as possible, Amazon said in February that it expects to shell out roughly $200 billion this year on capital expenditures, mostly on AI infrastructure.

Amazon's investment lands just two months after the e-commerce giant agreed to invest up to $50 billion in OpenAI, Anthropic's chief rival. The two AI companies have been racing to convince investors of their strengthening positions ahead of potential IPOs that could land as soon as this year. OpenAI executives have been criticizing Anthropic in recent months for making a "strategic misstep to not acquire enough compute."

Anthropic said on Monday that enterprise and developer demand for Claude, as well as a "sharp rise" in consumer usage, has led to "inevitable strain" on its infrastructure that has impacted its reliability and performance. The company said its new agreement with Amazon will quickly expand its available capacity.

"Our users tell us Claude is increasingly essential to how they work, and we need to build the infrastructure to keep pace with rapidly growing demand," Anthropic CEO Dario Amodei said in a statement. "Our collaboration with Amazon will allow us to continue advancing AI research while delivering Claude to our customers, including the more than 100,000 building on AWS."

Anthropic was founded in 2021 by a group of researchers and executives who defected from OpenAI. The company is best known for its family of Claude AI models and it's found early success selling to enterprises. Annualized revenue has topped $30 billion.

Anthropic named AWS its primary cloud provider in 2023 and its primary training partner in 2024, but the company has also inked deals with competing providers, including Microsoft and Google.

In November, Microsoft agreed to invest up to $5 billion into Anthropic, and Anthropic said it committed to purchasing $30 billion of Azure compute capacity. Earlier this month, Anthropic expanded its partnerships with Google and Broadcom for "multiple gigawatts" of capacity.

This article tells the story of how scientists figured out how the flagellar motor worked. The flagellar motor was discovered by Howard Berg, who set out in the early 1970s to apply his training in physics to understand how bacteria move. Bacteria move quickly, so Berg had to invent and build an automatic tracking microscope to keep them in view. He hypothesized how the mechanism worked 50 years before scientists discovered how the motor works.

Both AES-128 and SHA-256 are safe against quantum computers. No symmetric key sizes have to change as part of the post-quantum transition. Almost all experts agree on this. The misconception is usually based on a misunderstanding of the applicability of a quantum algorithm called Grover's.

Jujutsu megamerges let developers work on many different streams of work simultaneously. This article provides an in-depth explanation of how they work. Megamerges are a way of showing developers the whole picture and are not really meant to be pushed to remote. Developers will still want to publish branches individually as usual.

The AI engineering stack we built internally — on the platform we ship

In the last 30 days, 93% of Cloudflare's R&D organization used AI coding tools powered by infrastructure we built on our own platform.

Eleven months ago, we undertook a major project: to truly integrate AI into our engineering stack. We needed to build the internal MCP servers, access layer, and AI tooling necessary for agents to be useful at Cloudflare. We pulled together engineers from across the company to form a tiger team called iMARS (Internal MCP Agent/Server Rollout Squad). The sustained work landed with the Dev Productivity team, who also own much of our internal tooling including CI/CD, build systems, and automation.

Here are some numbers that capture our own agentic AI use over the last 30 days:

• 3,683 internal users actively using AI coding tools (60% company-wide, 93% across R&D), out of approximately 6,100 total employees

• 47.95 million AI requests

• 295 teams are currently utilizing agentic AI tools and coding assistants.

• 20.18 million AI Gateway requests per month

• 241.37 billion tokens routed through AI Gateway

• 51.83 billion tokens processed on Workers AI

The impact on developer velocity internally is clear: we've never seen a quarter-to-quarter increase in merge requests to this degree.

As AI tooling adoption has grown the 4-week rolling average has climbed from ~5,600/week to over 8,700. The week of March 23 hit 10,952, nearly double the Q4 baseline.

MCP servers were the starting point, but the team quickly realized we needed to go further: rethink how standards are codified, how code gets reviewed, how engineers onboard, and how changes propagate across thousands of repos.

This post dives deep into what that looked like over the past eleven months and where we ended up. We're publishing now, to close out Agents Week, because the AI engineering stack we built internally runs on the same products we're shipping and enhancing this week.

The architecture at a glance

The engineer-facing tools layer (OpenCode, Windsurf, and other MCP-compatible clients) include both open-source and third-party coding assistant tools.

Each layer maps to a Cloudflare product or tool we use:

None of this is internal-only infrastructure. Everything (besides Backstage) listed above is a shipping product, and many of them got substantial updates during Agents Week.

We'll walk through this in three acts:

1. The platform layer — how authentication, routing, and inference work (AI Gateway, Workers AI, MCP Portal, Code Mode)

2. The knowledge layer — how agents understand our systems (Backstage, AGENTS.md)

3. The enforcement layer — how we keep quality high at scale (AI Code Reviewer, Engineering Codex)

Act 1: The platform layer

How AI Gateway helped us stay secure and improve the developer experience

When you have over 3,600+ internal users using AI coding tools daily, you need to solve for access and visibility across many clients, use cases, and roles.

Everything starts with Cloudflare Access, which handles all authentication and zero-trust policy enforcement. Once authenticated, every LLM request routes through AI Gateway. This gives us a single place to manage provider keys, cost tracking, and data retention policies.

The OpenCode AI Gateway overview: 688.46k requests per day, 10.57B tokens per day, routing to four providers through one endpoint.

AI Gateway analytics show how monthly usage is distributed across model providers. Over the last month, internal request volume broke down as follows.

Frontier models handle the bulk of complex agentic coding work for now, but Workers AI is already a significant part of the mix and handles an increasing share of our agentic engineering workloads.

How we increasingly leverage Workers AI

Workers AI is Cloudflare's serverless AI inference platform which runs open-source models on GPUs across our global network. Beyond huge cost improvements compared to frontier models, a key advantage is that inference stays on the same network as your Workers, Durable Objects, and storage. No cross-cloud hops to deal with, which cause more latency, network flakiness, and additional networking configuration to manage.

Workers AI usage in the last month: 51.47B input tokens, 361.12M output tokens.

Kimi K2.5, launched on Workers AI in March 2026, is a frontier-scale open-source model with a 256k context window, tool calling, and structured outputs. As we described in our Kimi K2.5 launch post, we have a security agent that processes over 7 billion tokens per day on Kimi. That would cost an estimated $2.4M per year on a mid-tier proprietary model. But on Workers AI, it's 77% cheaper.

Beyond security, we use Workers AI for documentation review in our CI pipeline, for generating AGENTS.md context files across thousands of repositories, and for lightweight inference tasks where same-network latency matters more than peak model capability.

As open-source models continue to improve, we expect Workers AI to handle a growing share of our internal workloads.

One thing we got right early: routing through a single proxy Worker from day one. We could have had clients connect directly to AI Gateway, which would have been simpler to set up initially. But centralizing through a Worker meant we could add per-user attribution, model catalog management, and permission enforcement later without touching any client configs. Every feature described in the bootstrap section below exists because we had that single choke point. The proxy pattern gives you a control plane that direct connections don't, and if we plug in additional coding assistant tools later, the same Worker and discovery endpoint will handle them.

How it works: one URL to configure everything

The entire setup starts with one command:

opencode auth login https://opencode.internal.domain

That command triggers a chain that configures providers, models, MCP servers, agents, commands, and permissions, without the user touching a config file.

Step 1: Discover auth requirements. OpenCode fetches config from a URL like https://opencode.internal.domain/.well-known/opencode.

This discovery endpoint is served by a Worker and the response has an auth block telling OpenCode how to authenticate, along with a config block with providers, MCP servers, agents, commands, and default permissions:

{
  "auth": {
    "command": ["cloudflared", "access", "login", "..."],
    "env": "TOKEN"
  },
  "config": {
    "provider": { "..." },
    "mcp": { "..." },
    "agent": { "..." },
    "command": { "..." },
    "permission": { "..." }
  }
}

Step 2: Authenticate via Cloudflare Access. OpenCode runs the auth command and the user authenticates through the same SSO they use for everything else at Cloudflare. cloudflared returns a signed JWT. OpenCode stores it locally and automatically attaches it to every subsequent provider request.

Step 3: Config is merged into OpenCode. The config provided is shared defaults for the entire organization, but local configs always take priority. Users can override the default model, add their own agents, or adjust project and user scoped permissions without affecting anyone else.

Inside the proxy Worker. The Worker is a simple Hono app that does three things:

1. Serves the shared config. The config is compiled at deploy time from structured source files and contains placeholder values like {baseURL} for the Worker's origin. At request time, the Worker replaces these, so all provider requests route through the Worker rather than directly to model providers. Each provider gets a path prefix (/anthropic, /openai, /google-ai-studio/v1beta, /compat for Workers AI) that the Worker forwards to the corresponding AI Gateway route.

2. Proxies requests to AI Gateway. When OpenCode sends a request like POST /anthropic/v1/messages, the Worker validates the Cloudflare Access JWT, then rewrites headers before forwarding:

   Stripped:   authorization, cf-access-token, host
   Added:      cf-aig-authorization: Bearer <API_KEY>
               cf-aig-metadata: {"userId": "<anonymous-uuid>"}

   The request goes to AI Gateway, which routes it to the appropriate provider. The response passes straight through with zero buffering. The apiKey field in the client config is empty because the Worker injects the real key server-side. No API keys exist on user machines.

3. Keeps the model catalog fresh. An hourly cron trigger fetches the current OpenAI model list from models.dev, caches it in Workers KV, and injects store: false on every model for Zero Data Retention. New models get ZDR automatically without a config redeploy.

Anonymous user tracking. After JWT validation, the Worker maps the user's email to a UUID using D1 for persistent storage and KV as a read cache. AI Gateway only ever sees the anonymous UUID in cf-aig-metadata, never the email. This gives us per-user cost tracking and usage analytics without exposing identities to model providers or Gateway logs.

Config-as-code. Agents and commands are authored as markdown files with YAML frontmatter. A build script compiles them into a single JSON config validated against the OpenCode JSON schema. Every new session picks up the latest version automatically.

The overall architecture is simple and easy for anyone to deploy with our developer platform: a proxy Worker, Cloudflare Access, AI Gateway, and a client-accessible discovery endpoint that configures everything automatically. Users run one command and they're done. There's nothing for them to configure manually, no API keys on laptops or MCP server connections to manually set up. Making changes to our agentic tools and updating what 3,000+ people get in their coding environment is just a wrangler deploy away.

The MCP Server Portal: one OAuth, multiple MCP tools

We described our full approach to governing MCP at enterprise scale in a separate post, including how we use MCP Server Portals, Cloudflare Access, and Code Mode together. Here's the short version of what we built internally.

Our internal portal aggregates 13 production MCP servers exposing 182+ tools across Backstage, GitLab, Jira, Sentry, Elasticsearch, Prometheus, Google Workspace, our internal Release Manager, and more. This unifies access and simplifies everything giving us one endpoint and one Cloudflare Access flow governing access to every tool.

Each MCP server is built on the same foundation: McpAgent from the Agents SDK, workers-oauth-provider for OAuth, and Cloudflare Access for identity. The whole thing lives in a single monorepo with shared auth infrastructure, Bazel builds, CI/CD pipelines, and catalog-info.yaml for Backstage registration. Adding a new server is mostly copying an existing one and changing the API it wraps. For more on how this works and the security architecture behind it, see our enterprise MCP reference architecture.

Code Mode at the portal layer

MCP is the right protocol for connecting AI agents to tools, but it has a practical problem: every tool definition consumes context window tokens before the model even starts working. As the number of MCP servers and tools grows, so does the token overhead, and at scale, this becomes a real cost. Code Mode is the emerging fix: instead of loading every tool schema up front, the model discovers and calls tools through code.

Our GitLab MCP server originally exposed 34 individual tools (get_merge_request, list_pipelines, get_file_content, and so on). Those 34 tool schemas consumed roughly 15,000 tokens of context window per request. On a 200K context window, that's 7.5% of the budget gone before asking a question. Multiplied across every request, every engineer, every day, it adds up.

MCP Server Portals now support Code Mode proxying, which lets us solve that problem centrally instead of one server at a time. Rather than exposing every upstream tool definition to the client, the portal collapses them into two portal-level tools: portal_codemode_search and portal_codemode_execute.

The nice thing about doing this at the portal layer is that it scales cleanly. Without Code Mode, every new MCP server adds more schema overhead to every request. With portal-level Code Mode, the client still only sees two tools even as we connect more servers behind the portal. That means less context bloat, lower token cost, and a cleaner architecture overall.

Act 2: The knowledge layer

Backstage: the knowledge graph underneath all of it

Before the iMARS team could build MCP servers that were actually useful, we needed to solve a more fundamental problem: structured data about our services and infrastructure. We need our agents to understand context outside the code base, like who owns what, how services depend on each other, where the documentation lives, and what databases a service talks to.

We run Backstage, the open-source internal developer portal originally built by Spotify, as our service catalog. It's self-hosted (not on Cloudflare products, for the record) and it tracks things like:

• 2,055 services, 167 libraries, and 122 packages

• 228 APIs with schema definitions

• 544 systems (products) across 45 domains

• 1,302 databases, 277 ClickHouse tables, 173 clusters

• 375 teams and 6,389 users with ownership mappings

• Dependency graphs connecting services to the databases, Kafka topics, and cloud resources they rely on

Our Backstage MCP server (13 tools) is available through our MCP Portal, and an agent can look up who owns a service, check what it depends on, find related API specs, and pull Tech Insights scores, all without leaving the coding session.

Without this structured data, agents are working blind. They can read the code in front of them, but they can't see the system around it. The catalog turns individual repos into a connected map of the engineering organization.

AGENTS.md: getting thousands of repos ready for AI

Early in the rollout, we kept seeing the same failure mode: coding agents produced changes that looked plausible and were still wrong for the repo. Usually the problem was local context: the model didn't know the right test command, the team's current conventions, or which parts of the codebase were off-limits. That pushed us toward AGENTS.md: a short, structured file in each repo that tells coding agents how the codebase actually works and forces teams to make that context explicit.

What AGENTS.md looks like

We built a system that generates AGENTS.md files across our GitLab instance. Because these files sit directly in the model's context window, we wanted them to stay short and high-signal. A typical file looks like this:

# AGENTS.md

## Repository
- Runtime: cloudflare workers
- Test command: `pnpm test`
- Lint command: `pnpm lint`

## How to navigate this codebase
- All cloudflare workers  are in src/workers/, one file per worker
- MCP server definitions are in src/mcp/, each tool in a separate file
- Tests mirror source: src/foo.ts -> tests/foo.test.ts

## Conventions
- Testing: use Vitest with `@cloudflare/vitest-pool-workers` (Codex: RFC 021, RFC 042)
- API patterns: Follow internal REST conventions (Codex: API-REST-01)

## Boundaries
- Do not edit generated files in `gen/`
- Do not introduce new background jobs without updating `config/`

## Dependencies
- Depends on: auth-service, config-service
- Depended on by: api-gateway, dashboard

When an agent reads this file, it doesn't have to infer the repo from scratch. It knows how the codebase is organized, which conventions to follow and which Engineering Codex rules apply.

How we generate them at scale

The generator pipeline pulls entity metadata from our Backstage service catalog (ownership, dependencies, system relationships), analyzes the repository structure to detect the language, build system, test framework, and directory layout, then maps the detected stack to relevant Engineering Codex standards. A capable model then generates the structured document, and the system opens a merge request so the owning team can review and refine it.

We've processed roughly 3,900 repositories this way. The first pass wasn't always perfect, especially for polyglot repos or unusual build setups, but even that baseline was much better than asking agents to infer everything from scratch.

The initial merge request solved the bootstrap problem, but keeping these files current mattered just as much. A stale AGENTS.md can be worse than no file at all. We closed that loop with the AI Code Reviewer, which can flag when repository changes suggest that AGENTS.md should be updated.

Act 3: The enforcement layer

The AI Code Reviewer

Every merge request at Cloudflare gets an AI code review. Integration is straightforward: teams add a single CI component to their pipeline, and from that point every MR is reviewed automatically.

We use GitLab's self-hosted solution as our CI/CD platform. The reviewer is implemented as a GitLab CI component that teams include in their pipeline. When an MR is opened or updated, the CI job runs OpenCode with a multi-agent review coordinator. The coordinator classifies the MR by risk tier (trivial, lite, or full) and delegates to specialized review agents: code quality, security, codex compliance, documentation, performance, and release impact. Each agent connects to the AI Gateway for model access, pulls Engineering Codex rules from a central repo, and reads the repository's AGENTS.md for codebase context. Results are posted back as structured MR comments.

A separate Workers-based config service handles centralized model selection per reviewer agent, so we can shift models without changing the CI template. The review process itself runs in the CI runner and is stateless per execution.

The output format

We spent time getting the output format right. Reviews are broken into categories (Security, Code Quality, Performance) so engineers can scan headers rather than reading walls of text. Each finding has a severity level (Critical, Important, Suggestion, or Optional Nits) that makes it immediately clear what needs attention versus what's informational.

The reviewer maintains context across iterations. If it flagged something in a previous review round that has since been fixed, it acknowledges that rather than re-raising the same issue. And when a finding maps to an Engineering Codex rule, it cites the specific rule ID, turning an AI suggestion into a reference to an organizational standard.

Workers AI handles about 15% of the reviewer's traffic, primarily for documentation review tasks where Kimi K2.5 performs well at a fraction of the cost of frontier models. Models like Opus 4.6 and GPT 5.4 handle security-sensitive and architecturally complex reviews where reasoning capability matters most.

Over the last 30 days:

• 100% AI code reviewer coverage across all repos on our standard CI pipeline.

• 5.47M AI Gateway requests

• 24.77B tokens processed

We're releasing a detailed technical blog post alongside this one that covers the reviewer's internal architecture, including how we route between models, the multi-agent orchestration, and the cost optimization strategies we've developed.

Engineering Codex: engineering standards as agent skills

The Engineering Codex is Cloudflare's new internal standards system where our core engineering standards live. We have a multi-stage AI distillation process, which outputs a set of codex rules ("If you need X, use Y. You must do X, if you are doing Y or Z.") along with an agent skill that uses progressive disclosure and nested hierarchical information directories and links across markdown files.

This skill is available for engineers to use locally as they build with prompts like "how should I handle errors in my Rust service?" or "review this TypeScript code for compliance." Our Network Firewall team audited rampartd using a multi-agent consensus process where every requirement was scored COMPLIANT, PARTIAL, or NON-COMPLIANT with specific violation details and remediation steps reducing what previously required weeks of manual work to a structured, repeatable process.

At review time, the AI Code Reviewer cites specific Codex rules in its feedback.

AI Code Review: showing categorized findings (Codex Compliance in this case) noting the codex RFC violation.

None of these pieces are especially novel on their own. Plenty of companies run service catalogs, ship reviewer bots, or publish engineering standards. The difference is the wiring. When an agent can pull context from Backstage, read AGENTS.md for the repo it's editing, and get reviewed against Codex rules by the same toolchain, the first draft is usually close enough to ship. That wasn't true six months ago.

The scoreboard

From launching this effort to 93% R&D adoption took less than a year.

Company-wide adoption (Feb 5 – April 15, 2026):

AI Gateway (last 30 days, combined):

Workers AI (last 30 days):

What's next: background agents

The next evolution in our internal engineering stack will include background agents: agents that can be spun up on demand with the same tools available locally (MCP portal, git, test runners) but running entirely in the cloud. The architecture uses Durable Objects and the Agents SDK for orchestration, delegating to Sandbox containers when the job requires a full development environment like cloning a repo, installing dependencies, or running tests. The Sandbox SDK went GA during Agents Week.

Long-running agents, shipped natively into the Agents SDK during Agents Week, solve the durable session problem that previously required workarounds. The SDK now supports sessions that run for extended periods without eviction, enough for an agent to clone a large repo, run a full test suite, iterate on failures, and open a MR in a single session.

This represents an eleven-month effort to rethink not just how code gets written, but how it gets reviewed, how standards are enforced, and how changes ship safely across thousands of repos. Every layer runs on the same products our customers use.

Start building

Agents Week just shipped everything you need. The platform is here.

npx create-cloudflare@latest --template cloudflare/agents-starter

That agents starter gets you running. The diagram below is the full architecture for when you're ready to grow it, your tools layer on top (chatbot, web UI, CLI, browser extension), the Agents SDK handling session state and orchestration in the middle, and the Cloudflare services you call from it underneath.

Docs: Agents SDK · Sandbox SDK · AI Gateway · Workers AI · Workflows · Code Mode · MCP on Cloudflare

Repos: cloudflare/agents · cloudflare/sandbox-sdk · cloudflare/mcp-server-cloudflare · cloudflare/skills

For more on how we're using AI at Cloudflare, read the post on our process for AI Code Review. And check out everything we shipped during Agents Week.

We'd love to hear what you build. Find us on Discord, X, and Bluesky.

Random thoughts while gazing at the misty AI Frontier

A bunch of random things I have been thinking about, some of which are probably wrong

I was originally going to write a long articulate post for each of the below, with lots of fancy graphic, charts, and detailed analysis. Then realized it is too much work. Instead, here is some human idea slop & random thoughts. Enjoy!

1. OAI and Anthropic are now at 0.1% of US GDP each. What % of GDP is AI revenue in 2030?

   US GDP is roughly $30T. OpenAI and Anthropic are both rumored to be currently in the ball park of $30B of revenue run rate, or at 0.1% of overall GDP each. Through in clouds and other services and AI has grown from roughly zero to 0.25%-0.5% of US GDP in just a few years. If Anthropic and OpenAI hit $100B of revenue by EOY as many think they might, roughly 1% of GDP run rate will be from AI by end of 2026. This is insanely fast.

   What % of GDP will AI be in 2030? 2035? How does the US economic base impact the slowing of AI impact? How much of the productivity gains ends up missing from GDP a la the missing productivity impact of the internet in the 2000s or IT in the 1980s and 1990s?

   (Aside - If the impact of AI is mismeasured perhaps the wrong regulatory policies get implemented as a reaction as well - as AI gets blamed for only the bad (job losses) and not the good (new types of jobs, impact to education, healthcare…). Maybe the real ASI/Turing test is the ability to measure real world US GDP and productivity gains? :)

2. The AI research community just had a distributed IPO

   When a company goes public many of the early employees may find themselves suddenly enriched. This may change behavior - people get distracted buying homes, chasing status or spouses, partying, or doing societal side quests. This does not apply to everyone, but a subset of people experience this.

   Meta aggressively paying for talent changed the AI research talent market as the main labs had to match or provide large compensations increases to their researchers. Arguably, the AI research community just underwent the cross-company equivalent of an IPO as a cross section of the big labs & big tech. Somewhere between 50 to a few hundred people across all AI labs were granted huge sums of money as a reaction to Meta bidding on the best regarded researchers driving up everyones salaries.

   Just like a traditional IPO, a subset of the members of that community are shifting some aspects of focus and lifestyle, checking out or getting distracted, while others stay the course. In general the AI community is very mission aligned around building AGI or focusing on AI for science.

   Either way, an interesting new phenomenon has quietly occurred in Silicon where, instead of a company going public, a very specific slice of people effectively did. The top AI researchers became post-economic all at once. (Maybe the closest prior analogues is the early crypto HODLRs?).

3. Compute ceiling = artificial asymptote on near term model capabilities? Does this just re-enforce an oligopoly market for now?

   We have seen amazing progress in model capabilities in the last few years. This has been reflected in the flowering of use cases + revenue for the main labs and app companies built on top.

   At the same time, the labs are increasingly compute limited as one extrapolates out both training scale planned as well as future inference needs. Compute build outs seem at least in part to be limited by memory from Hynix, Samsung, Micron et al at least for the next 2 years as a build out cycle occurs for manufacturing for these companies.

   This means that rather than a single lab buying well ahead, or being able to use all the compute it wants, all the big labs are effectively and increasingly in a compute constrained world. This constraint may end up creating an artificial short-term asymptote on AI model progress. While people will undoubtedly get more efficiency out of the compute they have, this artificial compute constraint may mean no one lab is able to break significantly ahead until 2028 at the soonest - re-enforcing an oligopoly market for LLMs. We may also see the labs "accordion" between allocating compute and human resources to apps vs models and back again. Similarly, the depreciation cycles on chips and systems will be different then everyone expected and the lifetime of silicon will be extended due to lack of sufficient new supply.

   The counter to this is algorithmic or other breakthroughs, if contained within a single lab (vs leaking at an SF holiday party attended by researchers) could turbocharge a single company to dominance, particularly if coding takes off and there is some form of ongoing self improvement loops by AI building future AI leading into liftoff. If we do end up with a hard compute constrained environment breakaway liftoff may wait for 2028. Of course, it is also possible we are compute constrained for years post 2028 due to excess demand. Exciting to watch what happens.

4. Compute (tokens) is the new currency

   Compute (or could be stated as tokens) is a new unit of denomination for economic value in silicon valley. Token budget impacts things like

   1. What can you accomplish as an engineer
   2. Your spend and potential revenue as a company
   3. Your business model.

   Some companies are effectively inference providers disguised as tools. Neoclouds are the clearest form of this, but things like Cursor similarly are providing cheap inference as a core part of their product offerings and effectively subsidizing compute, which has been a smart user acquisition and usage model. Who doesnt love extra tokens?

   Things have gotten to the point where Allbirds (shoe company) just raised a convert to build a GPU farm. Will they be to AI what Microstrategy is to crypto?

   

5. Hidden layoffs & the developing world

   Most of the "layoffs due to AI" announced so far are probably just companies that overhired during the COVID zero interest rate environment slimming back down. Saying "look how good we are at AI we need fewer people" sounds much better then "we way overhired and are fixing it a few years too late".

   That said, AI is having a real impact in multiple areas such as customer support. Companies that are shrinking teams due to AI are actually cutting outsourcing firms first - so they headcount is not directly on their balance sheet but paid for as a service. This means countries like India and the Philippines may be the most impacted soonest in terms of employment and AI as they house many of these outsourced services organizations.

   It also means some developing countries may lose their services ladder to upgrading their economy and work. If AI takes many of the outsourced services jobs first, employment in these economies will need to shift elsewhere. An interesting question is whether this shifts human migration patterns?

6. Employee headcount is going to flatten for lots of companies and then shrink

   Multiple later stage CEOs told me that rather then do big layoffs due to AI, they will just stop growing. So if revenue at the company is growing 30%, 50%, or 100%, headcount may be flat or slightly down as they allow attrition to shrink staff. Existing headcount will become more productive, and companies may start swapping in fewer better people. This may medium term inflate the salaries of the very best people who can leverage AI immensely. Expect hiring to continue in sales, some engineering for growing companies, but maybe not as much elsewhere.

   Some companies are starting to ask what is the right ratio of token budget vs salaries in their org? Unclear what the right timeline for this metric is.

   True startups (e.g. a 5 person team) in the short run should continue to scale up headcount like in the olden days as they hit product/market fit but just with more leverage per person. So the "flat company" is going to be more of a later stage or public company phenomenon for growing companies in the next 2-4 years. Low growth companies of course should shrink.

   This may have implications for HR/software companies. See also:

   

7. The Slop Age could be the golden era of AI x humanity

   We are likely in the golden era of AI + humanity. Before the last few years, AI was inaccessible, not very generalizable, and could only do specific tasks. In the future, AI may become superhuman at most tasks and take over a lot of work some people find fun. Today, AI creates useful slop at volume, which means humans are still needed to desloppify the slop, but the slop provides real leverage on time and jobs, which means it is fun to be working right now. If AI displaces people eventually or does more interesting work, this golden moment may fade or change. Is the Slop Age the golden era of humanity + AI?

   (One could of course argue that we were in the midst of a human slop era before the AI slop era - in other words the era of huge amounts of human created sloppy content on the internet as it grew to billions of web pages, but not billions of new human insights. Does the slop era end with AGI, or when AGI cleans up all the prior waves of human slop?)

8. AI will eat closed loops first

   AI will first automate away the things that are easier to form a closed loop learning system on. This is why code and AI research may be accelerated and then displaced quickly - you can have testable closed loop systems so machines can learn and iterate quickly. The tighter the closed loop, the faster the AI can learn. You can make a 2x2 of jobs by how closed loop they can be made, versus their economic value, and see where AI may impact labor fastest. Fast time to closed loop + high economic value = fastest AI impact (hence software engineering).

   

   Code is interesting in that there is probably 10-100X the demand for great software developers as there is supply today (hence coding tools doing so well in market). The AI engineer of the future will be managing and orchestrating large numbers of agents to build things (systems and product thinking) vs writing a lot of code themselves (the auto-complete tab era).

   An interesting question is what jobs or tasks will be made more closed loop next? Where is AI most embeddable and teachable?

   Relatedly, data collection & labelling in every field will continue to grow.

9. Artisanal engineers vs utility engineers and AI

   Deep artisanal "my code is my craft" and "I love creating bespoke things" engineers decreasingly happy in world of AI. Systems thinkers and product thinkers engineers happiest. Many people are a mix of both.

10. The Harness

    If you look at the use of AI coding tools, the harness (and broader product surface area eg UX, workflow, etc) seems to be increasingly sticky in the short term. It is not just the model you use, but the environment, prompting, etc you build around it that helps impact your choice. Brand also matters more then many people think. At some point, either one coding model breaks very far ahead, or they stay neck in neck. How important is the harness/workflow long term for defensibility for coding or enterprise applications?

    Products tend to not be sticky until they suddenly are very sticky.

    There will be variability in where future forms of harnesses matter vs not. What is the sales AI harness? The AI architect harness? This leaves room for some startups to thrive.

11. Selling work, not software. Units of labor as the product

    AI is about selling units of labor online (and eventually in the atomic world via robotics), not displacing software. While Zendesk was selling seats to customer support reps, Decagon and Sierra sell customer support agentic work output and labor.

    AI grows tech TAMs dramatically.

12. Most AI companies should consider exiting in the next 12-18 months

    In the Internet era of 1995-2001, roughly 2000 or so companies went public. Of these only a dozen or two survived. Similarly in the AI era, most companies, including those that are ramping revenue today, will see the market, competition, and adoption, turn on them.

    Founders running successful AI companies should all take a cold hard look at exiting in the next 12-18 months, which may be a value maximizing moment for outcomes. A handful of companies should absolutely not exit (eg OpenAI, Anthropic) but many should if they can while everything is on the upswing.

    This is all of course counterbalanced by enormous growing demand for AI services of all types. While the tide is rising, many companies will seem to be unstoppable and durable - whether they are or not in the long run remains to be seen.

13. Anti-AI regulation & violence will both increase

    AI has had very little real world impact to eg job displacement so far. However, some AI pundits and some leaders have been quite vocal and doomer-esque to the point where a strong anti-AI narrative is emerging in both politics (Maine just banned new data centers although this also ties into energy, jobs, and NIMBYism) and amongst violence-centric activists (see recent attack on Sam Altman). Expect this to increase dramatically. It would be great if more leaders in AI continue to emphasize the optimistic side of what is coming in public rhetoric and political lobbying. In general, the AI field would benefit from its leaders continue to work actively on reigning in the doom and gloom.

14. Other

    Any other random thoughts to consider? Ping me on X.

Thanks to Aravind Srinivas of Perplexity, Scott Wu of Cognition, Adam d'Angelo of Quora/Poe, and others for comments.

Amazon reportedly pressured major brands to ask competing retailers to raise prices on certain products. California is suing Amazon over allegations that the retailer harms competition and increases prices that consumers pay online. The lawsuit claims that Amazon punished sellers for offering lower prices on other websites. The trial is scheduled to begin next year.

ggsql allows developers to write queries that combine SQL data retrieval with visualization specifications in a single, composable syntax.

Google builds elite team to close the coding gap with Anthropic

Google is doubling down on AI coding, using more AI internally and aiming for models that can eventually improve themselves.

Google Deepmind has put together a specialized team of researchers and engineers to sharpen the programming chops of its Gemini models, The Information reports. The group is led by Deepmind engineer Sebastian Borgeaud, who previously ran pre-training for the company's models.

The team is focused on complex, long-horizon programming tasks like writing new software from scratch, work that requires models to read files and figure out what the user actually wants. Part of the motivation: Google researchers think Anthropic's coding tools are better.

Coding has become the battleground for every major AI lab this year, with OpenAI and Google both scrambling to catch up to Anthropic. OpenAI recently pulled the plug on its Sora video generator to free up compute for training and running other AI models.

Brin pushes for self-improving AI

Google co-founder Sergey Brin and Deepmind CTO Koray Kavukcuoglu are directly involved in the effort. "To win the final sprint, we must urgently bridge the gap in agentic execution and turn our models into primary developers" of code, Brin wrote in an internal memo. He also required every Gemini engineer to use internal agents for complex, multi-step tasks.

Brin told employees that stronger coding skills are a stepping stone toward AI that can improve itself. A sophisticated coding agent, paired with AI that handles math problems and experiments, could eventually automate much of the work done by AI researchers and engineers.

Internally, Google tracks how much its coding tool "Jetski" gets used and ranks teams accordingly, a setup similar to Meta, which tracks token usage as its metric. Some teams outside Deepmind also require engineers to attend AI training sessions.

According to The Information's sources, Google is leaning more heavily on models trained on its internal code. Google's internal codebase looks very different from the public code typically used to train general-purpose coding agents, so these internally trained models can't be released publicly. They could, however, help Google build better models that eventually ship to users, while also speeding up internal development.

The EU has passed a law that requires phones to have batteries that can be easily changed by users at home using basic tools.

Apple's newly combined hardware engineering and hardware technologies division will be organized into hardware engineering, silicon, advanced technologies, platform architecture, and project management divisions.

GitHub has paused new signups for GitHub Copilot Pro, Pro+, and Student plans.

DeepMind engineers use Claude as a daily tool, but most of the rest of Google does not.

Apple CEO Tim Cook will step down after 15 years in the role, transitioning to executive chairman while hardware chief John Ternus becomes CEO on September 1. Cook leaves behind a $4 trillion company with massively expanded services and wearables businesses, despite some product missteps like Vision Pro. Ternus, a longtime Apple engineer, is expected to continue shaping the company's hardware and sustainability efforts as he takes over leadership.

WhatsApp is testing a new ‘WhatsApp Plus' subscription in its Android beta, priced around $2.99 per month, as part of Meta's push toward subscription revenue. The plan adds customization features like premium stickers, themes, app icons, extra chat pins, and upgraded notifications. While still limited to testers, the feature is expected to expand to iOS and other platforms ahead of a broader launch.

Most UX designers treat AI as a glorified search engine for small tasks, but the real value comes from building a personal AI operating system that codifies your own decision-making logic. The foundation isn't prompts or tools — it's mapping recurring work situations, how you handled them, and what good judgment looks like, so an AI agent can actually operate on your thinking. Once that's done, useful agents aren't document generators but judgment proxies for the repeated, energy-draining conversations only you could previously handle.

In AI-native teams, design is shifting away from static tools like Figma toward shaping behavior—timing, logic, and interaction—which can't be fully captured in traditional mockups. Designers increasingly work across coding, research, and strategy, using prototypes and pull requests as “behavioral specs” that engineers and even AI systems can interact with directly. This creates a new model of design work: faster, more code-adjacent, and focused on systems and frameworks rather than screens, while traditional Figma-based workflows still coexist in more structured environments.

There's lots of ways to build a website. Most of them involve designers working in one tool, developers working in another, and a painful handoff process somewhere in between. We recently used Intent to design, build and ship a well-crafted website in about three weeks, and the collaboration model that emerged shined a light on how things could (no, should) be.

Design First

We started the way most Web projects start these days: in Figma. Visual explorations of what the style, wireframes for the structure, then bringing the two together into full page layouts. Our designer set up the grid, typography scales, color variables, buttons, and reusable components. Your typical design system.

This process took about two weeks and was pretty standard. Desktop and mobile comps, a couple rounds of feedback on visuals and copy, iterating until we had a visual style, a rough structure, and directional content. Just a solid Web design process.

Development Foundation

Once the designs were in a good place, our developer jumped in. But not by staring at a Figma file and manually translating pixels into code. Instead, he opened up Intent, set up the project scaffolding (Astro, Tailwind), connected to the Figma MCP, and wrote an agents.md file that pointed to all the artboards.

Then he kicked off a series of workspaces. The first one pulled the design tokens into Tailwind. The second started laying out the first page using those tokens. After that, he was able to break off into parallel workspaces, one for each page. Desktop layouts first, then separate passes for mobile.

This whole phase, the front-end infrastructure, took maybe one or two days of actual work. And by the end, every page existed in code, using the design system, at roughly 85% fidelity. Not pixel perfect, but pretty damn close.

Parallel Work

Once he deployed the site to a staging URL, the three of us started working in Intent simultaneously: our designer, our front-end developer, and me handling product/project management. Though we all were using the same tool, we each worked our own way.

Our designer set up a grid overlay so he could visually verify alignment. He would tell the agent "align to column three" and it would snap things into place (way better than guessing at percentage values). He preffered staying in one workspace to tweak alignment and refine grid positioning across a full page before committing things.

Once the pages were structurally solid, he moved on to animations. Entrance effects on homepage elements, scroll-triggered transitions, etc.. Work that normally takes days of back-and-forth between a designer specifying timing curves and a developer implementing them happened in about hours. He still maintained manual control where it mattered, finding the exact easing curve he wanted then telling the agent to use it. The implementation was handled for him so he could focus on how things felt.

Meanwhile, I was doing content and product work. Dumping in blog posts from Word docs, adding image assets, making text changes based on feedback from the broader team. My approach was simple: small discrete tasks with a single agent. Fix one thing, commit. Fix another thing, commit. Once I had four or five commits, I'd open a pull request, toss out the workspace, and start a new one. The design tokens and setup our developer created ensured my changes were all inline with our design and development architecture.

Our developer's job during this phase was partly creative and partly managerial. He handled the templatized pages (news, product detail) where variable content meant design rules mattered more. He also kept an eye on pull requests, merged changes, resolved conflicts, and updated the agents.md file when he noticed patterns emerging in the code that should be standardized.

For example, when he saw icons being added in a way that wouldn't scale, he set up a better pattern and documented it. The next time anyone needed to add icons, the agent just followed the convention automatically. He used Intent for conflict resolution too, pulling up conflicting branches and having the agent sort them out. Out of maybe 30 or 40 pull requests across the project, only five needed real manual intervention.

Same tool, three different workflows, nobody waiting on each other.

Crunch Time

Every web project has a crunch period right before launch and ours was no different. The broader team started paying attention (as they always do at the very end), and feedback flooded in. But because the three of us could all be in Intent making changes at the same time, the crunch was way more manageable than usual.

The biggest win was that any one of us could contribute meaningfully to the codebase without breaking the design system, code structure, or the site. That's a fundamentally different dynamic than waiting for a developer to make every change.

A New Way of Working?

It wasn't perfect. CSS layout struggles are still a thing. Git seems to keep finding ways to bite you. And there's still a learning curve for non-developers, even with agents handling the hard parts.

But without the handoff, everyone builds. And that makes all the difference.

Your procedural toolbox for 2D content creation

Graphite is a free, open source vector graphics editor and animation engine, available now in alpha. Get creative with a fully nondestructive editing workflow that combines layer-based compositing with node-based generative design.

What's new?

The latest major update is out now! See what the team has been cooking up recently:

Software overview

Starting life as a vector editor, Graphite is evolving into a general-purpose, all-in-one graphics toolbox that is built more like a game engine than a conventional creative app. The editor's tools wrap its node graph core, exposing user-friendly workflows for vector, raster, animation, and beyond.

One app to rule them all

Stop jumping between programs. Upcoming tools will make Graphite a first-class content creation suite for many workflows, including:

• Graphic Design
• Motion Graphics
• Image Editing
• Digital Painting
• Page Layout & Print
• VFX Compositing

Current features

• Vector editing tools
• Procedural workflow for graphic design
• Node-based layers
• Forever free and open source

Presently, Graphite is a lightweight offline web app with features primarily oriented around procedural vector graphics editing.

Upcoming features

• All-in-one creative tool for all things 2D
• Fully-featured raster manipulation
• Windows/Mac/Linux native apps + web
• Live collaborative editing

Desktop-first and web-ready

Where's the download? The web app is currently live and desktop apps for Windows, Mac, and Linux are now in a release candidate stage. Check the announcements channel of the Discord for download links if you would like to partake in testing prior to the impending launch.

Graphite is designed principally as a professional desktop application that is also accessible in a browser for quick access from anywhere. It's built for speed with (nearly) no JavaScript. And regardless of platform, it runs locally and privately on your own hardware—there is no server.

Engineering the tech for a native app distributed across three new platforms takes extra time. That's why supporting the web platform, which keeps up-to-date and reaches all devices, has been the initial target. For now, you can install the app as a PWA for a desktop-like experience.

Graphite's code architecture is structured to deliver true native performance for your graphically intensive workloads on desktop platforms and very low overhead on the web thanks to WebAssembly and WebGPU, new high-performance browser technologies.

The power of proceduralism

Graphite is the first and only comprehensive graphic design suite built for procedural editing—where everything you make is nondestructive.

Explore parametric possibilities

Save hours on tedious alterations and make better creative choices. Graphite lets you iterate rapidly by adjusting node parameters instead of individual elements.

Scatter circles with just a couple nodes. Want them denser? Bigger? Those are sliders. Want a different placement area? Just tweak the path.

Mix and morph anything

Nondestructive editing means every decision is tied to a parameter you can adjust later on. Use Graphite to interpolate between any states just by dragging value sliders.

Blend across color schemes. Morph shapes before they're scattered around the canvas. The options are endless.

Geared for generative pipelines

Graphite's representation of artwork as a node graph lets you customize, compose, automate, reuse, and share your content workflows.

• Infinitely pan and zoom without pixelation, export any resolution
• Compile standalone programs from node systems (future)
• Deploy asset pipelines in studio production environments (future)

Agentic design is reshaping UI workflows. Tools like Anima Agent are enabling designers to generate elaborate interfaces directly inside Figma using AI. The plugin defaults to Claude Sonnet 4.6 and supports three core scenarios: building new designs from scratch, creating variants of existing ones, and assembling screens from a pre-existing design system. Generated designs come with auto-layout built in, though they may require manual cleanup for visual defects and produce detached — rather than true — Figma component instances.

Comparing AI—especially agentic AI—to autopilot is misleading: autopilot systems operate within strict, transparent rules, while AI is far less explainable and depends heavily on context, prompting, and interpretation. Describing AI as “magic” or autonomous obscures its limitations, shapes public perception, and can lead to misplaced trust. AI is most effective when constrained to specific, well-defined tasks with human oversight, functioning more like controlled systems than independent intelligence—making clear understanding and honest framing essential.

Org Design in the Age of AI

I've been talking to companies — startups to megacaps — about AI and organizational design. Everyone is adding AI to their workflows. Almost no one is rethinking why the workflow is shaped that way in the first place. The org chart is next thing AI disrupts.

The hidden function of hierarchy

Strip a company down to first principles and it's really three things: people, hierarchy, and information flow. We tend to think of hierarchy as being about authority — who reports to whom, who approves what. But that's the surface. The deeper function of hierarchy is information routing. The org is too large for any single person to see the whole picture, so you install layers of managers to aggregate signals from the front lines, synthesize them, and pass them up — and to translate strategic intent from the top and distribute it down.

Most of the organizational machinery we take for granted exists to solve this problem. Meetings, status updates, steering committees, quarterly business reviews — these are all information-routing mechanisms. They exist because moving knowledge between people is expensive.

The real bottleneck was never speed

Consider how a typical product gets built. A PM spends weeks writing a PRD. Design interprets it into mocks. Engineering interprets the mocks and estimates "eight weeks" — at which point the requirements shift and the PRD gets rewritten. Dev takes two to three months. QA spends weeks on regression testing. GTM prepares launch materials and trains sales. End to end, a mid-sized feature easily takes three to six months.

The real bottleneck is translation cost. PM's intent gets encoded into a document. A designer decodes that document and re-encodes it as a visual. An engineer decodes the visual and re-encodes it as code. QA decodes the intended behavior and re-encodes it as test cases. Every translation loses fidelity. Every translation requires alignment meetings. Every translation generates wait time — not because people are slow, but because the act of making one person's understanding legible to another person is genuinely hard.

This is what AI collapses.

What it actually changes

When a PM can go from idea to interactive prototype in a day using AI, the translation layer between PM and engineering compresses to near-zero. When AI generates tests alongside code as it's being written, the handoff between dev and QA disappears. When an intelligence layer can synthesize customer signals, usage data, and business metrics in real time, the middle manager whose job was to aggregate that information weekly has to find a different source of value.

This isn't about any single role getting faster. It's about the gaps between roles — the translation layers, the handoff queues, the alignment meetings — evaporating.

And once you see it that way, the implications for org design get serious:

• The relay race becomes a basketball game. The sequential handoff model — PM then design then eng then QA then GTM — gives way to small squads of 3–5 people with all the necessary skills, moving simultaneously, making most decisions themselves. Only big directional bets escalate up.

• Departments decompose into capability atoms. Instead of teams organized by function, the org becomes a set of independent, composable capabilities — collections, identity verification, risk scoring, savings — each self-contained, each combinable.

• PMs become product creators. The old PM spent most of their energy making ideas legible to other people. The new PM validates directly — prototyping, running data analyses, generating first-pass implementations.

• Middle management compresses. The managers who thrive will be the ones whose real contribution was always judgment, coaching, and navigating ambiguity — not routing information.

• QA embeds into development.

• The system starts generating the roadmap. This is the most radical shift. The example Jack Dorsey used - A restaurant's cash flow tightens ahead of a seasonal dip. The system detects the pattern, packages a short-term loan with adjusted repayment, and pushes it to the merchant — before they even thought to look for financing. No PM decided to build that. The system recognized the moment and composed them.

• Release cycles give way to continuous flow. No more "v2.0 ships in Q3." Ship small improvements daily. This requires CI/CD infrastructure, but more importantly it requires letting go of the big-launch identity — trading the dopamine of a major release for the discipline of relentless, quiet value delivery.

The deeper shift

Competitive moat changes: It used to be execution speed — who could ship fastest. Now it's learning speed — how quickly the org can absorb what AI makes newly possible and restructure around it.

Most companies today are using AI the way you'd use a faster horse — to make the existing structure run a little better. The companies that pull ahead will be the ones willing to ask a harder question: what would we build if we were designing this organization from scratch, today, knowing what AI can do?

A $292 million exploit of KelpDAO's rsETH via the LayerZero bridge triggered a 48-hour DeFi-wide panic that erased $13.21 billion in TVL, pulling the sector from $26.4 billion to roughly $20 billion by April 20. Aave absorbed the sharpest blow, losing $8.45 billion in deposits as withdrawals cascaded into protocols unconnected to the original attack, driving the AAVE token down 18%. Liquidity constraints pushed multiple lending pools to 100% utilization and borrow rates to 10-15%, compressing margins for leveraged strategies across the sector.

Bitmine said it now holds 4,976,485 ETH, or 4.12% of total supply, after buying another 101,627 ETH in the past week, reinforcing its position as the largest corporate ether holder and demonstrating how treasury accumulation, staking income, and tokenization demand are strengthening the institutional case for ETH.

The n-VM architecture integrates heterogeneous virtual machines like EVM and SVM into a single consensus and shared state tree. By enabling atomic cross-VM transfers through a unified ledger, this design eliminates bridge-dependent vulnerabilities, potentially preventing the $2.8 billion in losses historically caused by cross-chain bridge exploits.

Perps.fun launched in alpha with a propose-crowdfund-launch model that allows anyone to propose a new perpetual futures market, crowdfund the ticker listing costs, and deploy it permissionlessly on app.perps[.]fun. The mechanism removes the cost barrier that has historically restricted long-tail asset coverage in perp markets, enabling community-driven market creation without gatekeeping from centralized venues.

Freeport, an AI news feed platform with one-tap trading execution, reported $27M in volume and 11.7% aggregate money-weighted returns across its first 45 days, with users trading NASDAQ, S&P, crude oil, and crypto at 2-4x average leverage. The platform's two dominant macro trades were a WTI crude long from the low $60s to above $100 on Middle East tensions, followed by a NASDAQ 100 long that captured about 15% off March lows as diplomatic channels reopened. Return attribution assigns 4-5% to market beta, 3-4% to momentum, and 2-3% to oil concentration, with 2-3% residual alpha that lacks statistical significance at 46 days. Top users (1% of the base, 18.2% returns) averaged 2.1 trades per day versus a 5.8 median, held positions 31 hours versus 19, and used 2.4x leverage versus 3.3x.

• Coinbase's venture arm says Q2 is a 'builders market' despite quieter trading.
• Tokenisation, exchanges, next-gen DeFi and AI agents top the agenda.

At a glance, things look grim for crypto startups.

Investors poured just under $5 billion into innovating industry players in the first three months of 2026, a 15% drop from the capital injected in the first quarter of 2025, according to data from DefiLlama.

Add to that that the overall crypto market is still down some 40% from its October all-time high value, the industry is sacking hundreds of employees and blaming it on artificial intelligence, and decentralised finance projects are shutting down, and it's clear things look gloomy for entrepreneurs.

However, the industry will come out stronger for it, Jonathan King, principal investor at Coinbase Ventures, told DL News.

"When things look quiet or the market is more tough, that's when the best companies often get started," King said. "And the investors who show up then win big later. That's what we're seeing right now, and why we were a top crypto investor in Q1."

He's not alone. It's a sentiment shared by investors like Andreessen Horowitz, Sequoia Capital, Founders Fund, Bain, and Alibaba Group.

"Conviction shows up when it's not obvious," King told DL News. "Anyone can invest in a hot market, but the real signal is who leans in before it's consensus."

He said investors will focus on "four buckets:" Tokenisation, specialised exchanges, next-generation DeFi, and AI.

'Perpification of everything'

Tokenisation, or the "perpification of everything," as King calls it, is a massive opportunity.

"Markets are expanding beyond crypto into real-world assets — stocks, commodities, macro exposure — that are all moving onchain," King said.

Indeed, financial markets are rapidly expanding beyond native crypto assets into equities, commodities and macro exposure that can trade continuously onchain, with volume on perpetual exchanges like Hyperliquid exploding into the billions.

Firms like BlackRock, RobinHood, Greyscale have all waxed lyrical about assets going online and for good reason.

The tokenisation market is expected to jump by 754 times to become a $20 trillion market by 2030, according to BlackRock.

Specialised exchanges

Institutional market infrastructure — things like specialised exchanges and other trading technologies — are thriving, King said.

"We're seeing a shift toward more purpose-built, pro-grade market structures, prop [automated market makers], verticalised trading apps, prediction markets," he said.

Bernstein forecasted that the institutional crypto trading market's value will more than triple, from $5 billion in 2024 to $18 billion by 2030, with the US market share surging from 7% to 20%.

Next generation DeFi

Advanced DeFi protocols that are "more composable, more capital efficient, more private" are next on King's list.

"The next wave is improved protocols built to integrate and scale," he said.

The sentiment is shared by banks like the Japanese financial giant Nomura. The bank's 2026 Digital Asset Institutional Investor Survey shared with DL News on Thursday found that institutional investors are pursuing cryptocurrency yield strategies rather than just token price appreciation.

Its research found that over two-thirds of respondents want exposure to DeFi mechanics like staking, while 65% are targeting lending and tokenised assets, and 63% are exploring derivatives and stablecoins.

"This reflects growing demand for income-generating and asset-utilisation strategies," Nomura said.

King also stressed that "privacy is a big unlock here."

The Ethereum Foundation shares the same view. In October, it rolled out a new expanded effort to embed privacy into the blockchain, led by a new "Privacy Cluster" team of 47 engineers, researchers, and cryptographers.

Ethereum co-founder Vitalik Buterin has publicly endorsed privacy tech like Railgun and other similar efforts, arguing that privacy should be a default option for blockchain users.

"Privacy is in every financial system except for DeFi," Railgun contributor Bill Liang told DL News in October.

Crypto and AI

AI agents are "one of the most underappreciated areas" in crypto, according to King.

"AI agents are becoming economic actors," he said. "Every agent is effectively a new "user" of the blockchain."

The view is shared by Coinbase CEO Brain Armstrong, former Binance CEO Changpeng Zhao and Circle CEO Jeremy Allaire.

Coinbase has teamed up with tech titans like Amazon, Google, and Stripe for its x402 protocol.

x402 is designed as a universal standard for embedding payments directly into web interactions. It allows AI agents, APIs and applications to transfer monetary value as seamlessly as they exchange data over the internet.

By 2030, the market will reach as high as $5 trillion, according to McKinsey.

Some of the themes laid out by King echo along the top three capital raises this week, according to DefiLlama data.

Payward, $200 million

Payward, the parent company behind the cryptocurrency exchange Kraken, raised $200 million through secondary shares sales to Germany banking giant Deutsche Börse Group.

The move values Kraken at about $13.3 billion, down from $20 billion in late 2025.

The company made five acquisitions in 2025. Kraken co-CEO Arjun Sethi told DL News in September that the crypto exchange had more deals lined up.

Spektr, $20 million

Copenhagen-based Spektr has raised $20 million in a Series A round. NEA led the raise with backing from Northzone, Seedcamp, and PSV Tech.

Spektr uses AI agents to handle compliance tasks like know-your-customer and company risk checks, replacing manual processes that still dominate the financial industry.

Paxos Labs, $12 million

Blockchain infrastructure firm Paxos Labs has raised $12 million in a strategic funding round at an undisclosed valuation, Fortune reported.

Blockchain Capital led the round. Robot Ventures, Arthur Hayes' family office Maelstrom, and Uniswap Labs Ventures also supported the raise.

The funding will be used to expand Amplify, Paxos's software toolkit that allows businesses to add crypto services such as lending, yield products, and stablecoin issuance.

Earlier this week, I listened to an episode of the On the Block podcast, where the host, Matt Ross, Head of Investor Relations, sat down with Brad Axen, Principal Engineer for Data and Machine Learning at Block $XYZ. Brad built Goose, Block's open-source agent framework. The conversation was about how Goose got built.

This episode made me realize that Block's MoneyBot and ManagerBot aim to be the Claude Code of money.

If you've used Claude Code, you already know what's coming for consumers and small business owners. If not, follow the link in the reply to read my latest @PopularFintech essay 👇🏻

Block is quietly building Claude Code for money $XYZ
popularfintech.com/p/block-is-qui…

Security researcher demonstrates a critical vulnerability in Web2.5 infrastructure by exploiting insecure deserialization in a blockchain-based oracle.

Visa and Mastercard do not earn money by lending, but by operating the card networks that route payments and collect assessment and usage fees at scale, while issuers supply the credit, absorb fraud risk, and capture most of the merchant fee through interchange.

LayerZero identified North Korean hacker group Lazarus as the likely perpetrator of the $292 million Kelp DAO exploit.