What we learned from being the first company to disclose an agent cyberattack

This July, we were the first company to publicly disclose an autonomous agent cyberattack to the world, and today I want to share three critical lessons from it.

First, we need much more transparency in AI. I often wonder what would have happened if we had decided not to disclose the attack publicly. Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring. To better understand and mitigate these emerging cybersecurity risks, the global community needs stronger standards for monitoring and incident disclosure. For example through mandatory sharing of full agent traces. We learned this summer that building and keeping some of these systems behind closed doors is not safe.

Second, we learned that the biggest risk is not powerful AI. It is the asymmetry of powerful AI. Asymmetry between attackers and defenders. Between a few companies and everyone else. Between a few countries and the rest of the world. Asymmetry of control, of capabilities, of compute, of power. When we got attacked, our team initially turned to frontier closed-source APIs that blocked us because of safeguards that still can’t always tell the difference between attackers and defenders. I acknowledge that these safeguards are created with good intentions, but they can put defenders at a disadvantage while attackers jailbreak them, increasing the asymmetry of capabilities. In our case, as we started hitting those guardrails, fortunately we could use the version of an open-source model coming from China called GLM 5.2 by Zai_org, and we’re very grateful for that. It reinforced our conviction about the importance of open-source AI. Cyberattacks may increasingly come from proprietary models behind closed doors, while much of the defense may end up being powered by open-source tools because they are less restricted, more privacy-preserving, and orders of magnitude more affordable for organizations across the globe. The world needs open-source AI more than ever to defend itself. This applies not only to cybersecurity but to AI in general, where there has never been a greater need to distribute capabilities, resources, and control rather than concentrate them in the hands of a few.

Third, during this cyberattack, we learned how AI can stoke fear among the public and policymakers, especially through anthropomorphic framing and sci-fi imagery. We strongly believe that fear-based narratives are not the way to make the right decisions about the future of such a foundational and empowering technology or bring the public along with us. Even though we were the victims of this cyberattack, we believe more strongly than ever that AI will be beneficial to cybersecurity and make the world safer, just as major technologies before it. We were attacked by AI, but more importantly, we defended ourselves with AI. The same systems that helped us during this attack are now helping us against cyberattacks we were already facing. AI is also helping us fix the bugs and weaknesses in our systems before any attack, the same way AI is helping OAI fix their sandboxes to prevent agents from escaping. AI won't just create new cybersecurity challenges. It can make cybersecurity fundamentally and meaningfully stronger if we keep the right incentives, equip defenders more than attackers, and don’t increase the asymmetry between them. And that's before considering AI's positive impact on science, healthcare, education, productivity, and much more.

In closing, I want to reiterate what this first agent cyberattack taught us: the need for more transparency in AI and for more open-source AI to empower defenders and countries big and small to fight the asymmetry. Thank you very much!

The world has known for decades that the RSA cryptosystem’s days are numbered. Once quantum computing becomes practical (estimates for that range from 3 to 20 or more years), the foundational security it provides will crumble. New research has revealed a novel method that uses classical computing to reduce the current RSA security level to an unacceptably low threshold.

The practical risk is limited, but still significant. Applying the attack against the deprecated use of 1024-bit keys took a handful of months on an academic CPU cluster, significantly less than the current estimates for 1024-bit factoring that would require resources that only nations or companies with massive resources could achieve. Widely used RSA implementations are also safe.

Nonetheless, the research has taken cryptographers by surprise because it introduces signature forgery, a new way to break RSA keys without factoring. Equally important, this novel method reduces the required computing resources by orders of magnitude.

Out of reach no more

“If this result holds up under peer review, it would indeed be a conceptual break-through,” Karsten Nohl, a cryptography expert and the head of innovation at Allurity, said in an interview. “RSA is as difficult to break as it is to factor large integers, at least so we thought. The researcher suggests that you can practically break RSA without cracking its key.”

Nadia Heninger, a University of California at San Diego professor and co-author, elaborated:

Cryptographers thought that the only way to compute valid RSA digital signatures was to first compute the private key by factoring, and then use the private key to compute the signatures. For 1024-bit RSA, this was thought to be very expensive, albeit probably doable if you have the computational resources of the large tech companies or the NSA—on the order of tens of millions of dollars of computation time for a single key. For 2048-bit RSA, it was thought to be totally out of reach.

The key forgery attack Heninger and the other researchers devised is fully practical now for 1024-bit RSA. Even for 2048- and 4096-bit keys, the method reduces the security of RSA to unacceptable levels. The National Security Agency, National Institute of Standards and Technology, and European Union Agency for Network and Information Security require that any cryptosystem should provide a level of no less than 128 or more bits, meaning the operations required must exceed 2^128.

The forgery attack drops these levels to 2^65, 2^90, and 2^119 for 1024-, 2048-, and 4096-bit keys respectively. These levels may further drop because Heninger’s team did all the coding by hand and used no AI or GPUs in performing the forgeries. The researcher said these tools will “almost certainly” drop the security levels further.

The attack works only against blind-signature implementations of RSA. The overwhelming majority of RSA in use today provides PKCS or PSS padding, a format that adds data to the plaintext before it’s encrypted. It prevents ciphertext from being deterministic and makes it less vulnerable to side channel and similar attacks. Still, some real-world systems continue to use blind-signature, also known as textbook, RSA. The best-known example, Heninger said, is Privacy Pass, a protocol that allows users to authenticate themselves without revealing their identity. Privacy Pass is used by both Apple and Cloudflare, among many others.

An attack on Privacy Pass would require an attacker to request tokens from Cloudflare, Apple, or another organization 2^43 times.

Heninger said the requirement “sounds [like] a lot, but is on the same order of magnitude of the network traffic that Cloudflare has said publicly it handles in about a day.” Most Privacy Pass implementations rotate keys regularly, a measure that greatly reduces, but doesn’t automatically eliminate, the chances of attacker success.

The technique implements a variant of the number field sieve algorithm that was invented in 2007. This “‘special’ number field sieve” is used with an “oracle”, a property of some cryptographic protocols that gives answers to queried inputs. By performing a massive number of operations, attackers can gather enough information to decipher the ciphertext. (This technique doesn’t appear to pose a practical threat against RSA with PKCS or PSS padding, because they provide a different type of oracle.) While factoring a 1024-bit key requires an estimated 2^80 operations and 500,000 to 1 million CPU core-years, using the sieve to forge a signature took just (as noted earlier) 2^65 operations and 1,380 core-years.

The paper’s authors and other researchers stress that the new attack poses little real-world threat, at least for now. It does, however, drastically lower the estimated security of RSA, and it does so in a way no one knew of previously.

Cryptographers have worked furiously in recent years to devise alternative cryptosystems that aren’t vulnerable to quantum computing attacks. The new attack will further increase the urgency of completely moving away from the cryptosystem.

I was at dinner with a bunch of CTOs and Head of Infrastructure folks recently, and from the conversation it was clear that many companies are struggling to source CPUs in the current climate, and are coming to terms with the end of juicy discounts from cloud providers for machines in the new era of surging demand fueled by AI.

The ‘memory crisis’ afflicting sectors like video gaming is well established and has been extensively covered in terms of shortages of GPUs, but now it seems like things are just as hard for businesses in need of CPUs from cloud providers.

In a sign of how things are changing, the disappearance of CPU spot pricing was mentioned at the table. Customers used to be able to pay up to 90% less than the standard price for CPUs, as cloud providers slashed CPU prices for machines that were lying dormant and unused. But that’s no longer the case. It seems that spot pricing has vanished because there’s no longer any lack of demand for CPUs – quite the opposite.

I was surprised, but a lot of people chimed in; apparently, it’s now nearly impossible to get CPUs on spot instances without long-running connections with cloud providers. Also, reserving specific CPUs now needs to be done months in advance, and cloud providers will even turn down certain reservations because they don’t have enough CPUs or the right type of CPUs.

Even big players struggle to reserve CPUs

I have asked turbopuffer CEO Simon Eskildsen about their experience of CPU availability in the cloud, since turbopuffer, as a product, runs on CPUs, not GPUs. They operate in AWS, GCP, and Azure, so I asked how easy it is to get CPUs these days. Simon’s response:

Getting CPUs is not easy anymore. As Reinforcement Learning (RL) is becoming a large amount of the workloads: RL needs a lot of CPUs. So the labs are sucking up a lot of CPUs. During RL, they need to teach the models how to do things, like searching, and then they need the model to run software, which then takes CPUs to run.

Then, outside of RL, agents need to do all kinds of very general purpose things on a CPU. So as the demand curve is shifting to general purpose agents, CPU demand is also going up.

Even the big companies are fighting each other for the right to get the CPU allocations. I would assume that it gets a lot worse before it gets better on the CPU side.

I was able to confirm what Simon said about larger companies struggling; a VP of Engineering at a large inference provider told me they are at the limit on how much GPU and CPU capacity they can buy from their cloud providers. They have cash to spend and want to rent more capacity, and are willing to accept the longest leases. Despite that, cloud providers tell them no more is available!

AI hogging CPUs

Katelyn Lesse, Head of Platform Engineering for Claude Platform, has written about the reasons for the massive CPU demand increase:

In the past few years, AI-fueled demand has skyrocketed, and these few companies suddenly needed multiple years and tens of billions of dollars to actually add enough capacity. We ended up with 3 separate bottlenecks in factory capacity that AI is exacerbating. At TSMC, GPUs are competing with CPUs (and with Apple, Qualcomm, and Broadcom) for production lines. And at SK Hynix, Samsung, and Micron, HBM [High Bandwidth Memory] is competing with regular DRAM for wafers.

What we’ve ended up with is CPUs getting squeezed from both sides. AMD doesn’t own fabs [semiconductor fabrication plants], so its CPUs need to come out of TSMC’s constrained allocation. Intel does own fabs, but it’s been working through yield problems and is now pulling some of its capacity from PC chips in order to make more server chips. And CPUs need DRAM which has gotten more expensive because memory production has shifted toward HBM. Analysts are expecting CPU supply to add more comfortable headroom before memory does, but their expectation is that it’s still going to be multiple quarters away.

AI-fueled demand does increase CPU load, as shown in this graph from Uber, displaying the growth in agent requests over the past six months:

Increasingly, “agent requests” not only generate code which is inference-heavy – and therefore needs GPUs – but they also run tools that compile the code, run tests, run linters, and all of this is CPU-heavy. At companies like Uber, Ramp, and others, AI agents no longer run on the dev’s local machine, but on a dedicated instance in the cloud. So, the companies reserve more CPUs on their respective cloud providers for agentic workloads.

Basically, the problem is:

• AI applications use more and more CPUs, thanks to agents running a lot more software. AI data centers used to have a ratio of 1 CPU to 8 GPUs. Now the ratio is more 1:4, and it could shrink to 1:1.
• Companies that can manufacture more CPUs are busy on other hardware. TSMC is busy producing GPUs, which might be more profitable than CPUs. Meanwhile, CPUs also need DRAM, but DRAM manufacturers (SK Hynix, Samsung, and Micron) are instead producing high-bandwidth memory (HBM) because it’s more profitable. This is why memory prices are spiking; even Big Tech is unable to buy RAM, as previously covered.

To secure CPUs, it’s necessary to do capacity planning up to 12 months in advance. Katelyn says that server orders are being fulfilled in ~six months, instead of 1-2 weeks’ time as previously, and that prices are up by between 10-20%. So, it’s probably time for capacity planning. Katelyn:

Most of us have never capacity-planned CPUs. We planned databases, we maybe planned accelerators if we needed them, and we autoscaled on-demand into CPU capacity as much as our budgets allowed us to. But general purpose compute is now something many teams will need to commit to ahead of time, which means you should probably start to forecast and plan around it. If you’re operating at scale, there are some things to spend your energy on.

Using existing CPUs more efficiently is something to do, as of now. The CPU capacity shortage won’t go away, and any new CPU allocations requested could take months to turn up. So, what can we do if new capacity lags? One option is utilizing current resources more efficiently!

This is a great time to review and to establish now which services are CPU-intensive, and whether or not they need to be. Also check on services which are utilizing little CPU: can they run on fewer nodes, so that some CPU capacity can be allocated to services that need it more?

The best time to secure more CPU capacity is most certainly right now. I’m hearing rumors that certain cloud regions no longer accept new tenants because all CPU capacity is leased, or negotiations elsewhere are difficult. I’m also hearing that customers are already paying today to reserve capacity that will only come online in data centers from December. This seems predatory by providers, but demand is so high that this is how they likely prioritize new capacity allocation – while earning much higher profits than usual.

If your company has dynamic workloads, and you’ve used spot instances in the past, now could be a good time to allocate fixed capacity – even if it’s more expensive. If you expect meaningful growth, doing so now might mean having options at some cloud providers or in some regions.

It seems like this issue has spread everywhere as a corollary of widespread AI adoption. There’s a GPU shortage, memory shortage, and now a growing CPU shortage as well. Back at the end of last year, there was even an hard drive shortage. The only compute primitive not in short supply seems to be networking!

A deployment, feature flag, or configuration change may trigger an alert, but identifying which recent change most likely contributed to the alert can require a time-consuming investigation across multiple services and systems. Rules-based approaches can identify potentially relevant changes quickly and inexpensively, but their accuracy is limited on complex incidents. Agentic investigations can reason more deeply about the available evidence, but using frontier models for every alert is too expensive at scale.

To see whether a smaller, specialized model could close that gap, we fine-tuned Qwen3.5-9B on traces from investigations generated by GLM-5.3. The resulting model achieved 87% of GLM-5.3’s recall. Its self-hosted LLM serving cost was $0.003 per investigation, compared with $0.06 in API charges for GLM-5.3, a roughly 20× decrease under our evaluated deployment conditions. At $0.003 per investigation, a single 40 GB A100 GPU can support approximately 100,000 investigations per week. This is enough to investigate a subset of the millions of unique monitors customers interact with each week. We are pursuing further optimizations to make this approach practical for a much larger share of those monitors.

In this post, we explain how we built the training-data flywheel, what it changed about the model’s investigative behavior, and how fine-tuning smaller models could make agentic investigations practical across a much larger volume of production alerts.

Two approaches to change attribution

Change Tracking currently supports incident investigation in two complementary ways. The first is the Relevant Changes tab, which overlays recent changes on a monitor’s alert timeline and highlights those most likely to have contributed to the alert. This gives engineers a fast way to identify potentially relevant deployments, feature flags, and configuration changes.

Change Tracking also exposes its data through a Model Context Protocol (MCP) tool that agents, including Datadog’s Bits Investigation, can use during incident investigations. According to internal Datadog telemetry from July 2026, the Change Tracking tool contributes to thousands of Bits investigations each week, and 20% of Bits Investigation conclusions reference a change captured by Change Tracking.

The two approaches offer different trade-offs. Relevant Changes returns results within seconds and is inexpensive enough to make available for free to Application Performance Monitoring (APM) customers, but its rules-based retrieval limits its accuracy on complex incidents. Agentic investigations using the MCP tool can reason more deeply about the available evidence, but multi-turn investigations with frontier models are too expensive to run for every alert at scale.

A smaller model specialized for change attribution offered a potential way to combine these strengths: deeper agentic investigation at a cost that could support a much larger volume of alerts.

The approach: The change attribution flywheel

Prompt engineering alone wasn’t enough to make the smaller model reliable. Even after repeated iterations on the system prompt and tool descriptions, Qwen3.5-9B tended to search too broadly instead of narrowing its investigation around the most promising evidence. Rather than continue adding rules to compensate for that behavior, we explored whether we could teach the smaller model the investigative behavior of a more capable model.

We adapted NVIDIA’s data flywheel blueprint for change attribution: Generate investigation traces with a larger teacher model, use successful traces to fine-tune a smaller student model, and repeat the process as new production investigations become available. NVIDIA demonstrated this approach by fine-tuning a Llama 3.2 1B model on tool-calling traces from a 70B teacher, reaching 98% of the teacher’s accuracy with a roughly 70× reduction in parameter count. We adapted that approach to test whether the same idea could make agentic change attribution inexpensive enough to run across a large volume of alerts.

Change attribution is well suited to this approach because its investigations have a consistent structure: Each investigation uses the same set of tools and works toward the same objective of identifying the change most likely responsible for an incident. Bits Investigation conclusions also let us derive labeled examples from production investigations without manual annotation. Together, the repeatable workflow and a continuously growing set of labeled examples make change attribution a strong candidate for a specialized model:

1. Identify the change: Bits Investigation writes a free-form conclusion for every incident it investigates. We use GLM-5.3 to parse the conclusion, identify any changes it references, and map them to the change IDs produced by our system. We treat each referenced change as a proxy label: the change Bits associated with the incident, rather than independently verified causality. This gives us labeled examples without requiring manual annotation.

2. Run the teacher agent: GLM-5.3 investigates the same alert using eight turns, five tools, and a 65,536-token context window. It returns a ranked list of possible changes along with confidence scores. We intentionally constrain the investigation process so that a much smaller model can learn to reproduce it.

3. Generate the dataset: We run the teacher model three times on each alert. We keep only the examples that return changes that match the proxy labels extracted from the Bits Investigation conclusion and discard the rest. This technique is known as rejection sampling fine-tuning.

We applied this process to 348 internal incidents that occurred between May 26 and June 24, 2026. From these incidents, we created an initial dataset of 100 teacher traces, each from a different investigation and selected based on which traces scored their proxy label the highest. No customer data was used to generate this dataset.

4. Fine-tune the student: We fine-tuned Qwen3.5-9B using 16-bit low-rank adaptation (LoRA), which updates a small set of adapter parameters rather than all of the model’s weights. We calculated training loss only on the assistant responses in each selected trace.

5. Repeat the cycle: Both the teacher and student models run in production. When the teacher’s prediction matches the proxy label and the student’s does not, we add the teacher’s investigation trace to the training dataset and retrain the student on the expanded dataset, allowing it to learn from new production investigations over time.

We have completed two rounds of this process using additional internal incidents, adding 86 training examples and increasing the dataset from 100 to 186 examples.

Results: 87% of the teacher’s Recall@5 at roughly 5% of the investigation cost

The following results are based on a sample of 326 production incidents, comprising 187 internal incidents and 139 customer incidents collected between August 11 and August 25, 2026. A daily cron job replays the previous day’s production incidents and runs an investigation with each model. For this evaluation, Recall@5 measures whether each model’s top five ranked changes include the proxy label extracted from the corresponding Bits Investigation conclusion. Recall@5 measures agreement with the change identified in the Bits Investigation conclusion, but it does not independently verify that the change caused the incident.

Every evaluated incident occurred after the training data was generated, so the evaluation set was fully held out from the training set. The fine-tuned model was trained only on internal incidents, making the 139 customer incidents a useful test of whether the learned behavior transfers beyond the population used for training. Because GLM-5.3 is currently enabled at Datadog only for internal use, however, our direct student-teacher comparison is limited to the internal incidents.

On customer incidents, the fine-tuned model reached 0.62 Recall@5, compared with 0.52 for the base model and 0.51 for the heuristics-based ranker.

Four results jump out.

The fine-tuned model retained much of the teacher’s Recall@5 at 5% of the cost: The fine-tuned student achieved 0.55 Recall@5, compared with 0.63 for the teacher. Its self-hosted serving cost was $0.003 per investigation, compared with $0.06 in API charges for GLM-5.3. In other words, the student achieved 87% of the teacher’s Recall@5 at 5% of the inference cost.

Fine-tuning changed how the model investigated alerts: Prompting alone did not correct the base model’s tendency to search too broadly. Across the same evaluation set, the base Qwen3.5-9B model exhausted its turn limit or context window in 25% of investigations, and trace analysis helped explain why: It searched broadly for services and changes without narrowing its investigation quickly enough. After fine-tuning, that behavior changed. The fine-tuned model averaged 5.8 search calls per trace, compared with 6.8 for the base model. Instead, it gathered more evidence from logs, spans, and metrics, averaging 8.3 calls compared with 4.7. This shift from broad change discovery toward focused evidence collection helped the fine-tuned model improve Recall@5 while using fewer tokens.

Fine-tuning outperformed our handwritten rules: The fine-tuned student improved Recall@5 from 0.46 to 0.55 compared with our production rules-based system while costing approximately $0.003 per investigation. Rather than continuing to expand a growing collection of specialized rules, fine-tuning let us learn investigative behavior from production-derived examples.

Extra intelligence has a price: While more capable models such as Opus 5.0 continue to improve Recall@5, their costs quickly become prohibitive, with an average cost per investigation of $0.32. For our use case, where we need to run investigations across a large volume of alerts, that cost compounds quickly.

Beyond supervised fine-tuning

Supervised fine-tuning gets us a strong student, but it has a ceiling: The student is limited by the behaviors represented in the teacher’s traces. To continue improving, we plan to explore reinforcement learning (RL).

Our problem has a useful property for reinforcement learning: Bits Investigation results provide a signal that we can evaluate automatically. Whenever Bits Investigation identifies a change associated with an incident, we can compare the model’s predictions against that result and potentially use the match as a reward signal for reinforcement learning. This could let the model learn from production investigations without depending on explicit human feedback such as thumbs-up and thumbs-down ratings.

This is similar to how Cursor continuously improves its Tab model using feedback from accepted and rejected code completions. In our case, the feedback signal would come from the changes identified in Bits Investigation results rather than explicit user interactions. The challenge is that this feedback signal is imperfect. Bits Investigation can sometimes identify the wrong change. As Bits Investigation improves, we expect the quality of the labels it provides to improve as well. The advantage is that this approach would not require a separate human grader or learned reward model. The same production investigations that power the data flywheel could also provide the feedback needed for future reinforcement learning.

Making agentic investigation practical at scale

This process points to a repeatable approach for tasks with the right ingredients: a consistent agentic workflow, a growing source of useful labels, and an evaluation signal that can identify successful investigations. For change attribution, those ingredients let us generate successful investigation traces with a capable teacher model, use them to specialize a smaller model, and continue expanding the training set as new production investigations become available.

We believe this will become an increasingly common way to build AI systems. Frontier models remain essential for solving the hardest problems, but they can be too expensive to run for every request at production scale. Lower inference costs can reduce spending and make new product experiences possible. As inference costs continue to fall, we expect AI to enable new observability workflows that would be impractical at higher costs.

From Dockerfile to Kit: the Docker Sandboxes Kit Specification

Agents need containment, and a sandbox is only half of it. Something still has to say which agent runs there, what it gets, and what it may touch. That is a Kit: an ordinary OCI image, so the answer travels with the agent and means the same thing on any conforming runtime. Today we published the Docker Sandbox Kit Specification v3, open source under Apache 2.0 at docker/sandbox-kit-spec. Here is why I wrote it.

Everything that makes an agent useful is a grant

I run a lot of agents. They write code, run tests, install dependencies, call APIs, and work on infrastructure while I do something else. None of it happens without access, so I grant it one piece at a time: a bind mount, a token with broader scope than the task needs, a firewall rule that was quicker to open than to narrow. Each grant is reasonable on its own. Together they take back the isolation I was relying on, and none needed an exploit. The holes are configuration, added on purpose, usually by me.

I am worse at taking any of it back, and I could not reproduce the grants my setup depends on. No file records them. They live in shell history, dashboards, and my memory. I cannot hand that to a colleague or diff it against last week.

Containers package applications. Sandboxes contain agents.

A container packages applications. It shares the host kernel and uses namespaces and cgroups to give one fixed workload its own view of the filesystem, network, and processes. That is the right tool for software that runs, does its job, and touches only what it was handed.

An agent, however, is a probabilistic actor. It decides what to do next and then does it, to my filesystem, network, credentials, and cloud account. It will install a package that needs root, open a port nobody planned for, and try the next thing when the first is blocked. A container was not built for that: the boundary is the same kernel the actor is probing.

A Docker Sandbox is a microVM with its own kernel, so the boundary sits below anything the model can reach or rewrite. Inside one I can hand an agent root and let it loose, because the damage stops at the sandbox boundary. The sandbox is what lets me run an agent with the safeties off.

But an empty sandbox is not an environment. Something still has to say which agent runs, which tools and MCP servers it gets, which skills and instructions shape it, and exactly what it may touch.

What a Dockerfile cannot say

A Dockerfile answers everything about the software itself: how it is built, what gets packaged, how it starts. It was never standardized; OCI standardized the image it produces and how registries distribute it. What a Dockerfile does not describe is the outside: networks, credentials, volumes, tools, context. That half has lived in docker run flags, a Compose file, a CI config, and someone’s memory. Unversioned, unreviewable. A Kit writes it down with the content.

One image, one digest

If you have used sbx, you have used Kits. This is the third version of the format, and the change that matters is that a Kit is now an ordinary OCI image rather than its own artifact: no media type, no sidecar file, nothing for a registry to learn. The manifest carries the declarations in one annotation, vnd.docker.sandbox.kit.descriptor; the layers carry the content.

A Kit therefore builds with docker buildx build, pulls with docker pull, gets scanned and signed by the tooling you already run, and works in a FROM. Pinning the digest pins content, declarations, and metadata together. The tooling and distribution path are free; the format is something you learn: a grammar, a page per capability type, provides and requires, kind: set.

Two kinds of Kit exist. A workload runs and supplies the root filesystem. A mixin is an overlay: a CLI with its network rule, a credential binding, context for an agent. You launch one workload and any number of mixins.

Authority you can read

Part of the GitHub CLI mixin in the repository:

capabilities:
  - type: com.docker.sandbox/network-policy@2
    config:
      runtime:
        allow:
          - github.com
          - hosts: [api.github.com]
            methods: [GET, HEAD, POST, PATCH, PUT, DELETE]
        deny:
          - hosts: [api.github.com]
            methods: [DELETE]
            paths: [/repos/**]

  - type: com.docker.sandbox/credential@1
    optional: true
    config:
      service: github
      phase: runtime
      apiKey:
        name: GH_TOKEN
        proxyManaged: true
        inject:
          - {domain: api.github.com, header: Authorization, format: "Bearer %s"}

Read it as a permission slip. This Kit asks to reach GitHub and nowhere else, and for most of the API but not deletes under /repos/**, because deny wins. The token that can open a pull request cannot delete the repository. The credential is proxy-managed: a conforming runtime injects the real value into requests to the named domains, and inside the sandbox there is only a sentinel.

Two words carry weight: asks and conforming. A Kit grants itself nothing. Each entry is a request, and the host decides. A conforming runtime, one that implements the behaviour the specification describes, blocks hosts not on the list. Without one, the annotation is inert: an image and no enforcement. Docker Sandboxes is the first conforming runtime.

Everything a Kit needs goes through that one list, typed and versioned. Grants (network rules, credentials, volumes, ports, devices, skills paths) count toward “what may this Kit do”; entries that ask the runtime to act, like a lifecycle hook, do not. A required request the host cannot satisfy refuses the launch, rather than starting an agent with less authority than it declared, or more.

Composition is a function, not a sequence

Container images never solved multiple inheritance: a Dockerfile stage has one FROM. Mixins are overlays ordered by the dependency graph the Kits declare through provides and requires, never by the order you typed the flags, so the same set always composes to the same image.

The resolver is strict on purpose. Every requires is satisfied from inside the set or resolution fails; nothing is fetched to cover a gap. Exactly one workload is allowed. Two Kits providing the same name fail rather than one silently shadowing the other (composing the Claude workload with the Claude mixin is the canonical mistake). Where Kits overlap, declarations reconcile: network rules union, hooks run in dependency order, guidance becomes one document, licenses union. Incompatible requests are an error, not a coin flip.

A kind: set descriptor names other Kits; publishing it runs the same coherence rules at build time and merges them into one ordinary Kit. An incoherent set fails at your build, not at someone else’s launch.

The diff is the review

The Claude Code Kit in the repository declares the hosts it asks to reach, its credential, the volumes that persist between sessions, and its install and startup hooks. When the next version asks for another host or a second credential, that is a change in authority, not a software update, and it shows up in the pull request as added lines a human can refuse.

Review depends on somebody reading the diff, so the specification defines a second gate that does not. Every descriptor reduces to a normalized set of everything the host would have to grant; a runtime that gates updates records that set and compares the next version against it. A version inside what was granted may apply without asking. Any widening stops and asks, and removing a deny rule counts: if a later gh Kit dropped DELETE /repos/**, the runtime holds the upgrade. That is why the declarations had to live in the artifact, not beside it.

Why this is a specification and not a feature

A Kit that stopped meaning anything when run somewhere else would be lock-in, not a trust boundary. So the grammar is normative, every capability type has its own page describing what a conforming runtime must implement, and types version independently (network-policy@1 and @2 both exist today). Two conformance suites ship with it: one judges whether an artifact is a conforming Kit, the other whether a runtime behaves as the pages say. Every normative statement is covered by a check or a written waiver.

Docker maintains the specification today, and it should not stay under a single vendor: a format for deciding what an agent may do is worth less if it belongs to whoever sells you the runtime. Docker Sandboxes will be a first-class implementation, not the only one. If a Kit you want cannot be expressed, or a runtime duty cannot be implemented as stated, open an issue.

Try it

sbx is our sandbox CLI (brew install docker/tap/sbx). From a checkout of the repository:

cd examples
sbx run ./hello --kit ./gh .

Edit a descriptor and only that Kit rebuilds; docker buildx build publishes it to any registry. Docker Cloud Sandboxes runs the same Kits with the same trust model on elastic capacity. The specification, capability pages, and a worked tour are in docker/sandbox-kit-spec.

Not only agents

Agents forced this into the open because the authority they ask for is so large, but ordinary workloads have always arrived with unwritten expectations: the endpoints they call, the credentials they need, the volume that must survive a restart. That knowledge has lived in a Helm chart, a runbook, or a colleague. It is the same gap, less alarming when a web service gets it wrong. This specification is where any software writes down what it needs from the world around it; agents were the case urgent enough to have it built.

Dockerfiles made software reproducible. Kits make authority reproducible.

Global AI routing with <1% overhead on multi-cluster GKE Inference Gateway

Demand for AI infrastructure is at an all-time high. Global accelerator shortages mean engineering teams can rarely get all the compute they need from just one data center — capacity comes a cluster here, a cluster there, often an ocean apart. At the same time, workloads are getting hungrier: Today’s long-running agentic workloads often have context windows of 100k to 800k+ tokens, which consume accelerator memory faster than any previous generation of AI traffic.

In this environment, the goal is to maximize "intelligence per dollar." Fragmented, poorly balanced infrastructure is rarely up to the task though, allowing expensive accelerators to sit idle, while requests queue up somewhere else.

To close that gap, we built a layered routing architecture that makes globally scattered capacity behave like a single pool behind a single entry point. At the edge, the multi-cluster GKE Inference Gateway focuses on global, multi-region traffic distribution and high availability. Beneath that, the LLM-d router handles the complex, memory-aware scheduling algorithms that keep utilization high.

This architecture is deliberately runtime-, model-, and accelerator-agnostic — it works across serving frameworks, model families, and GPU or TPU hardware. To make the results concrete rather than abstract, we recently benchmarked managing production-level global request routing at scale across a multi-region GKE deployment of 17,000 compute nodes spread across the US and Europe. The deployment served a leading Mixture of Experts (MoE) foundation model using SGLang.

The results: Scaling to three clusters achieved a near-linear throughput boost while maintaining a 99.9% success rate under heavy multi-client concurrency. Additionally, routing traffic through the multi-cluster GKE Inference Gateway added less than 1% overhead, delivering 99.5% of the throughput of a direct, local cluster call.

Three regions, one endpoint

The deployment spanned three GKE clusters in three geographic regions: us-east5 (the config cluster), us-west8, and europe-west4. However, from the client’s perspective, none of that geography exists. Requests hit a single global virtual IP, and the gateway decides — in real time — which cluster should serve each one.

What makes that decision smart rather than blind is telemetry. Instead of traditional round-robin routing at the network layer, the multi-cluster load balancer is configured to route traffic based on live application signals. Specifically, the Endpoint Picker Proxy (EPP) reads the KV-cache token utilization natively exposed by the underlying inference engines and emits it as a metric for the load balancer. When the load balancer sees a region running hot based on this emitted metric, it spills traffic to the next healthy region.

Multi-cluster GKE Inference Gateway topology. The config cluster holds routing configuration but sits outside the request path; each target cluster runs its own EPP and reports KV-cache utilization back to the load balancer.

Distributed LLM engines also operate differently than standard web apps. In a typical inference engine's distributed mode (such as tensor parallelism across multiple nodes), only the master (rank-0) pod serves the API. GKE already handles local routing using standard Service selectors and LeaderWorkerSet (LWS) to direct traffic exclusively to leader pods. The multi-cluster Inference Gateway also integrates with this foundation: It routes global traffic to the correct regional services, helping your cross-region load balancing respects your underlying multi-node topologies out of the box.

Measuring the routing overhead

The first question every team asks about a global routing tier is almost always, ‘How much throughput am I giving up for cross-region capability?’

The benchmarks answer this directly: Deploying the multi-cluster GKE Inference Gateway to maximize your accelerator fleet doesn't have to come at the cost of throughput. Routing traffic through the Gateway added less than 1% overhead, delivering 99.5% of the throughput of a direct, local cluster call.

Linear scaling across regions

A bigger test is scale. In our test, growing the fleet from one cluster to three, spanning the US and Europe, while every client request originated from a single region (us-east5), put real pressure on the Gateway: If it couldn’t distribute load efficiently across those distances, throughput would flatten as hardware was added.

Instead, throughput multiplied almost exactly in line with capacity:

Memory-aware routing in action

Round-robin load balancing is inadequate for serving LLMs because it treats every request as equal. They aren’t. Heavy prompts saturate GPU compute cores, long generations stress memory bandwidth, and long-context conversations quietly eat VRAM until the engine can’t schedule anything new.

Here, the pressure on memory bandwidth came from the routing signal chosen for this deployment. By mapping Inference Engine's native token-usage metric onto the Gateway’s KV-cache signal, the routing plane gained a real-time view of memory pressure across the entire 17,000 fleet. (Depending on the workload, the Gateway can route on other signals too, like queue depth or running concurrency.)

Under live production loads, as the primary region climbed toward its high-bandwidth memory (HBM) limits, the Gateway detected the saturation the moment the cluster crossed its 40% KV-cache utilization threshold; it then automatically began routing the overflow to the next healthy region. No operator intervention was needed. The complexity of running in multiple regions simply never reached the user.

The payoff

By routing traffic based on live KV-cache utilization, this GKE Inference Gateway setup effectively pools globally scattered compute capacity into one unified engine. For this deployment, the result was a near-linear throughput boost across three global regions, with virtually zero routing overhead.

This translates directly into maximizing 'intelligence per dollar,' extracting near-perfect proportional performance out of every accelerator you add to your fleet, rather than letting capital go to waste.

What this means for your team

• Smarter load balancing pays for itself. Round-robin routing wastes expensive GPU capacity because it can’t see memory or compute pressure. Routing on real-time application signals turns fragmented regional clusters into one efficient fleet — the difference between stranded hardware and 90%+ utilization of scarce compute.
• Agentic workloads change the bottleneck. Long-running agents with extreme context windows exhaust memory long before there’s no more compute. If your routing layer can’t see memory pressure, your compute will strand compute behind full VRAM. Make KV-cache utilization a first-class routing signal.
• AI traffic breaks web-era assumptions. Traditional load balancers are tuned for sub-second transactions; LLM requests can run for minutes. Plan connection limits and timeouts for AI-scale latency early, or expect aborted connections in production.
• Your routing layer must integrate with native serving patterns. Distributed LLM engines have master-worker topologies where only certain pods can serve traffic. By pairing your Gateway with native Kubernetes constructs like LeaderWorkerSet (LWS), your global routing respects local pod topologies out of the box, saving your team from building custom proxy infrastructure.
• For large foundation model builders, bet on an open, portable stack. Teams operating at frontier scale face the most acute capacity fragmentation, forcing them to hunt for compute resources across whichever regions have availability capacity. An open, portable inference stack such as LLM-d on GKE lets you absorb that capacity wherever it lands, rather than hard-wiring your serving architecture to any single cluster, region, or bespoke infrastructure.

Next steps

1. Deploy it yourself: Set up the multi-cluster GKE Inference Gateway.
2. Understand the architecture: About multi-cluster GKE Inference Gateway.
3. Learn about the cross-region spillover behavior featured in this post: About elastic cross-region high availability and Configure elastic cross-region high availability.

Today, we’re introducing Muse Realtime Avatar, our state-of-the-art embodiment technology that turns Muse Realtime Voice into expressive, interactive avatars.

Conditioned on reference media, Muse Realtime Avatar brings any character into a live conversation. A photographic portrait responds through subtle expressions, while a full-body illustration gestures and shifts posture as it speaks. Animals and everyday objects become expressive without losing what makes them distinctive. Frame by frame, the avatar’s appearance and mannerisms remain coherent from one conversational turn to the next.

From Intelligence to Real-Time Presence

Muse Realtime Voice and Muse Realtime Avatar form a single streaming system connecting intelligence, voice, and embodiment. Muse Realtime Voice provides the conversational intelligence and produces a stream of speech tokens (VQs) carrying both what is said and how it’s delivered. An audio decoder turns those tokens into speech, while Muse Realtime Avatar consumes the same stream to generate the corresponding visual performance. Sharing this token stream keeps voice, lip motion, and expression synchronized.

Muse Realtime Avatar is an audio-driven, Diffusion Transformer conditioned on speech-token stream, reference media, and a rolling window of recent video latents. It generates video in short causal chunks. As each chunk completes, its newest generated latents become motion context for the next, carrying the avatar’s appearance and mannerisms forward while keeping the computation bounded, allowing the generation to continue for as long as the conversation does.

Real-Time Infinite Video Generation

Live streaming must solve two problems at once: generating video fast enough for real-time interaction and remaining visually consistent throughout the conversation without accumulating errors.

We begin with a high-quality bidirectional teacher and produce a causal student with a fixed-length KV cache through self-forcing and distribution matching distillation. Self-forcing allows the student to train on its own generated context and teaches it to resist drift as small errors accumulate over time, matching the conditions it encounters during inference.

The teacher uses 40 diffusion steps with three-way classifier-free guidance (CFG), requiring three model passes per step and totaling 120 model evaluations per chunk. Our carefully tuned recipe jointly distills the diffusion process and the effect of CFG into an unguided two-step student. Together, these techniques allow the student to reproduce in two unguided evaluations what the teacher progressively refines over 120 guided evaluations, a 60x reduction, while closely preserving teacher quality.

State-of-the-Art Avatar Experiences

To understand how Muse Realtime Avatar performed in live conversation, we compared it with Runway Characters and HeyGen LiveAvatar, the two leading commercial avatar systems, using each product’s native live-call experience. Raters held two- to three-minute conversations with each system using matched avatar identities, then compared the experiences across visual quality, synchronization, character consistency, and mannerisms, among others. The chart below reports overall preference where raters more often preferred Muse Realtime Avatar.

Serving at Subsecond Latency at Meta Scale

Muse Realtime Avatar continuously generates an avatar's video as a live conversation unfolds. Delivering that experience at Meta scale requires both interactive latency and high concurrency. To make this possible, we redesigned the real-time AI inference stack at Meta from the ground up and coupled this with an in-house custom engine built for highly optimized real-time video inference and serving.

Persistent KV caches with memory aware positional encodings allow us to smartly reuse context across chunks while decoding. Cache-aware routing and latency-aware dynamic batching efficiently distribute concurrent sessions. Four-bit quantization-aware training preserves quality at lower inference cost, while fused kernels and NVIDIA CUDA Graph capture reduce memory traffic and scheduling overhead. We collaborated with NVIDIA on a number of model optimizations to minimize the cost of the model’s forward pass. The entire system, including the underlying infrastructure, is optimized end-to-end to ensure a smooth user experience.

Muse Realtime Avatar streams 448x768 portrait video at 25 frames per second with approximately 870 ms of latency as measured from the end of a user’s turn to when they receive the first byte of the synchronized voice-and-video response. For a single session on a GB200, each generation step produces eight frames, corresponding to 320 ms of playback, in 20 ms, or an effective 2.5 ms of model time per frame.

The optimizations mentioned above coupled with smart orchestration across voice and video increase serving capacity by 8x relative to the two-step BF16 baseline, enabling 12 concurrent real-time sessions for the video generation on a single GB200.

Building Embodied AI Responsibly

We enforce strict safety requirements throughout the experience to reduce the risk of misuse and protect people. To make generated media traceable, Muse Realtime Avatar uses Meta Video Seal to embed a durable, invisible watermark throughout generated video without adding latency to the real-time experience. We’ll continue strengthening these protections as embodied AI evolves. All examples in this blog post illustrate model capability and do not all reflect avatars available in the Muse app. Muse is for users aged 18+.

Introducing Gemini 3.8 Live with Live Avatar

Gemini 3.8 Live with Live Avatar brings real-time visual presence to Gemini’s conversational AI. By natively coupling our live dialogue capabilities with low-latency streaming video, Live Avatar enables a more natural and intuitive conversational experience for enterprises and their users.

Building on the momentum of last week's Gemini 3.8 Live launch, today we are excited to introduce Gemini 3.8 Live with Live Avatar — bringing near real-time visual presence to our native live dialogue models. By pairing near real-time video generation with speech, the Live Avatar feature creates an experience that listens, sees, and speaks with a dynamic visual persona.

With precise lip-syncing, natural expressions, and fluid turn-taking, Live Avatar enables enterprises to expand their virtual offerings more interactively. Whether providing engaging customer service or delivering interactive walkthroughs, it transforms digital exchanges into richer, more accessible experiences.

Starting today, Gemini 3.8 Live with Live Avatar is available in Gemini Enterprise.

See how Gemini 3.8 Live with Live Avatar supports a wide range of characters, each with a distinct look, voice, and expressive presence.

More natural and multimodal conversations

Conversation is inherently multimodal: we listen, look, speak, and use facial expressions to communicate. Live Avatar brings these capabilities to enterprise agents. By processing visual and audio inputs simultaneously, it generates enriching conversations for a more comprehensive experience.

Watch how Gemini 3.8 Live with Live Avatar takes in what it sees and hears in near real time, responding with expressive audio and video for a more natural conversation.

Asynchronous tool execution with continuous presence

Beyond visual presence, the feature is backed by Gemini’s advanced reasoning. With asynchronous tool calling, Live Avatar can trigger tool calls and fetch data in the background while continuing active dialogue, handling complex tasks while ensuring an uninterrupted conversational flow.

See how Gemini 3.8 Live with Live Avatar handles complex tasks like checking in a guest at a hotel. Calling tools in the background while the dialogue continues uninterrupted.

Conversational experiences built for global scale

Conversational presence should feel natural and not be limited by languages. Live Avatar features native multilingual speech-to-speech synchronization. The feature dynamically adapts its lip-sync and expressions and can seamlessly transition across 97 languages without degrading video fidelity or introducing visual drift.

Watch how Gemini 3.8 Live Avatar switches between languages mid-conversation, with lip-sync and expressions adapting seamlessly across 97 languages.

A Live Avatar to fit your brand needs

Organizations often need distinct visual identities to fit their brand. In addition to a library of diverse, preset avatars, organizations can customize their Live Avatars. From a high-quality reference image, developers can generate a fully animated, responsive avatar while preserving reference likeness, brand styling, or character identity. Custom avatar creation is currently available only through enterprise allowlisting.

Trust and transparency at its core

We built Live Avatar with strict safeguards designed to respect identity, and keep AI-generated content transparent. All output generated by our AI products is watermarked with SynthID. This imperceptible watermark is woven directly into the audio and video output, helping to ensure AI-generated content remains detectable to help minimise misinformation and misattribution. To explore our comprehensive approach to safety and responsible deployment, read our model card.

Get started with Gemini 3.8 Live with Live Avatar

Gemini 3.8 Live with Live Avatar is available in Gemini Enterprise. Explore the API documentation to get started.

Contrastive Language Models (CLMs) are a new class of System One model trained with a contrastive learning objective that connects states and actions. CLM-8B delivers performance comparable to Jev across computer-use, gaming, and tool-calling tasks while achieving up to 9x lower latency. It also sets a new state-of-the-art on changing agentic coding benchmarks. The model is pre-trained on 60M Nemotron Q&A pairs, mid-trained on 30M synthetic hard negatives, and post-trained on 1M agentic trajectories.

Managed Deep Agents v0.8: new auth, memory, and channels

Key Takeaways

• Identity-scoped auth and memory: user-owned credentials and user-level memory let agents act on the right permissions and remember caller-specific context.
• HTTP channels: use agents in internal tools, customer portals, support systems, and other product surfaces that can send webhooks.
• Built-in web search: Managed Deep Agents now includes web search powered by Parallel out of the box.

Today we’re launching Managed Deep Agents 0.8, which adds support for user-owned credentials, user-level memory, HTTP channels, file transfer in Slack and adds a pre-built tool for web search powered by Parallel.

Managed Deep Agents (MDA) is the simplest way to build, deploy, and run mission-critical agents in production, so engineering teams can focus on agent behavior instead of rebuilding the same infrastructure that every agent needs. With today’s release, we’re expanding Managed Deep Agents to support four challenges that most teams face when running agents in production: agent memory, authentication, channels, and tool management.

Try Managed Deep Agents

About Managed Deep Agents

Managed Deep Agents combines the Deep Agents harness with all of the infrastructure required to run agents in production. A Managed Deep Agent is a code-first project in your repo that allows you to easily organize all your agent’s primitives into a simple directory:

my-agent/
  agent.py | agent.ts | agent.tsx
  pyproject.toml | package.json    	# project dependencies
  instructions.md                  		# prompt synced to Context Hub
  identity.py | identity.ts        		# auth, thread scoping, memory scoping
  memory.py | memory.ts            	# define your agent's memory
  tools/                           			# custom tools
  channels/                       	 	# entry points like Slack and GitHub
  middleware/                      		# custom middleware
  schedules/                       		# managed cron schedules
  connectors/                      		# managed connectors
  skills/                          			# skills synced to Context Hub
  sandbox/                         		# sandbox configuration
  evals/                           			# agent evals

Managed Deep Agents in practice

Imagine you’re building an internal agent to support your sales team (we’ll call this agent our GTM Agent in this post). You want your agent to be able to look up account details, research the account, draft meeting briefs, follow-up emails, and so on, and help keep your team’s accounts engaged. To support this, your agent needs memory, tool access, web search, reasoning, and planning.

Agent harnesses, like Deep Agents, help with building these agents. They provide primitives for building complex agents that can take on long-running, mission-critical tasks. With Deep Agents, getting a prototype up and running is straightforward, but making the agent work seamlessly in production requires a lot more.

That’s where Managed Deep Agents comes in. It packages your customized Deep Agents harness with the managed infrastructure required to run an agent like this at scale. This includes adding channels, like Slack, identity and authentication, and managing permissions for tools. Building this infra typically takes quarters of roadmap and requires ongoing maintenance. Managed Deep Agents allows engineers to focus on building the agent logic and hand off the boilerplate code.

Memory for your agent, scoped to the user

When an agent is used by multiple users in production, robust, secure systems to manage memory are important. Managed Deep Agents already supported durable agent memory, which lets a deployment retain instructions and preferences across conversations. Setting up memory can be done through simple declaration files:

# A memory declaration file enables durable agent and user-level memory.
my-agent/
  agent.py
  memory.py

With Managed Deep Agents 0.8, we’ve added a second layer for user-level memory, scoped to the authenticated person starting the agent run. This gives the agent a place to store caller-specific context, such as preferences, working style, recurring tasks, or details the user has asked it to remember, without mixing that context with the shared agent-level memory.

from managed_deepagents import MemoryLayer, define_memory
memory = define_memory(
    agent=MemoryLayer(),
    user=MemoryLayer(),
)

Under the hood, durable memory is backed by LangSmith Context Hub, which enables teams to store, version, and collaborate on agent files like Skills and AGENTS.md. Agent memory is mounted at /memories/agent/ and shared by everyone using the deployment. User memory is mounted at /memories/user/ and keyed to the caller’s authenticated identity. The runtime never copies content between these layers, so teams can keep shared knowledge and personal context separate by default. This is useful in real-world agent use cases, such as:

• A support agent might use agent memory for team-wide escalation rules and user memory to remember that a specific teammate prefers concise Slack updates.
• A research agent might keep shared research procedures in agent memory, while storing an individual user’s preferred source types or formatting conventions in user memory.

"Building with Managed Deep Agents has been such a smooth experience. The LangChain team partners well with us, and helped troubleshoot quickly. Our production chat app runs as a Managed Deep Agent - with Context Hub and user-level memory, we are able to maintain client confidentiality while delivering a superior experience."

— Zahid, CTO, Kyth.ai

Memory is easily configurable and lives within a single file in a Managed Deep Agents project. You can enable the agent layer, the user layer, or both. You can also define access policies for each layer, giving teams control over when memory is available and what scope it applies to. Unless specified, Managed Deep Agents uses the following defaults:

This layered approach ensures user-level information is not ‘leaked’ into a conversation with multiple users. Consider the GTM agent mentioned earlier. Users could initiate requests in a direct DM with the agent, or they could start a request in a Slack group. Defined access policies and defaults ensure that:

• A direct DM has the agent acting as per the user’s preference.
• Requests with multiple users in a thread or a slack group has the agent defaulting to workspace / team preferences.
• There is no leakage of preferences and stored memory between the two scenarios.

Learn more about setting up agent and user-level memory here.

Securely provide user and agent identity through connections

Connections link a Managed Deep Agent to an external service such as GitHub, Notion, or Tavily. Connections are named credentials in your LangSmith workspace that your tools only read at runtime.

As with memory, it’s important to scope credentials according to the agent’s purpose and how users interact with it. With Managed Deep Agents, you can scope credentials at the user- or agent-level.

Agent-owned credentials are shared across all users, which is useful for an agent capability that does not differ per person, such as web search. By contrast, user-owned credentials are important when the user will have unique permissions within a tool. This is often necessary for tools like GitHub, Linear, and Notion.

For example, our GTM agent should be able to pass through user credentials and use Salesforce data when a user requests it in a DM. The agent should also be able to use agent-credentials to search the web and find information about a customer.

Managed Deep Agents comes with OOTB support for 23 services including Linear, GitHub and Google Workspace tools, where LangSmith manages authorization, tokens and methods so developers only need to set up user ID and secrets.

Learn how to set up and use credentials in connections here.

Accessing agents through user-preferred channels

Internal agents are commonly accessed through existing communication channels like Slack. Managed Deep Agents provides pre-built Slack support to make it simple to expose agents where your team is already working.

With today’s release, Managed Deep Agents now supports richer Slack workflows with file transfer, and adds HTTP channels for connecting agents to any service that can send a JSON webhook.

# channels/orders.py
from managed_deepagents import channels


from lib.orders import parse, verify, messaging
channel = channels.http(
    provider="orders",
    verify=verify,
    parse=parse,
    messaging=messaging,
)

For Slack-based workflows, users can invoke an agent from a DM, an app mention, or a thread reply. With file transfer support, they can also send the materials the agent needs directly in the conversation, such as logs, spreadsheets, contracts, screenshots, or customer documents. For example, with our GTM agent, a user can now add notes from a previous call or documents shared by a customer right inside Slack while making a request to the agent. The agent then picks up the file, adds it to the context and responds based on it.

HTTP channels let teams bring agents into internal tools, customer portals, support systems, order systems, or any product surface that can send a webhook. This is useful for customer-facing agents that live on multiple channels. For example, a customer support agent that manages intake and triage can live in your product and an agent that schedules demos and meetings can live on your webpage.

For both Slack and HTTP methods, teams keep control over authentication, identity, and memory, while giving users access to agents in the channels they already prefer.

“[Managed Deep Agents is] by far the most complete agent platform, end to end. Slack integration worked perfectly and made life easier since we access all our agents through slack. We were able to deploy multiple agents, with custom MCP servers, to production using the CLI and GitHub actions. Now we have always-on triage with access to the right monitoring and the ability to open PRs and alert teams.”

— Derek Gilbert, Engineering, Consensus

Built-in web search, powered by Parallel

Web search is one of the most common agent tools, so we decided to build it right into Managed Deep Agents. Now, you can use web search powered by Parallel without creating a separate vendor account, managing another API key, and wiring a search tool into the agent themselves.

Simply specify the MCP server in the tools folder and LangSmith manages the Parallel credentials and executes the tool. The agent receives relevant excerpts and source URLs it can cite, while calls, latency, and errors show up in LangSmith traces.

Add Parallel to the servers map in tools/mcp.py / tools/mcp.ts. Define mcp only once

from managed_deepagents import define_mcp

mcp = define_mcp(
    servers={
        "Parallel": {
            "transport": "http",
            "url": "https://api.smith.langchain.com/v1/managed-tools/servers/parallel/mcp",
        },
    },
)

Getting started

Managed Deep Agents 0.8 simplifies agent authentication, memory, and channels, and brings in web search as a pre-built tool.

We’re excited to see how teams use these new capabilities, so we’re making Parallel web search available free through Managed Deep Agents while MDA is in beta.

Learn more in the Managed Deep Agents docs, or get started with:

uvx --from managed-deepagents mda init my-agent
cd my-agent
uv run mda deploy

We’re continuing to improve Managed Deep Agents and would love to hear what you’re building. Join our Slack community to share what you’re building and reach out to us with any feedback.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Taste-Bench measures the taste of an LLM agent: its ability to choose the better direction at a real decision fork in a long-horizon task. Given the task, the trajectory up to the fork, and two candidate next steps, the model must pick the step that the hidden rest of the trajectory proves right. A wrong choice often looks reasonable at the moment and costs the agent most of its budget later. The 502 questions are mined from software engineering and machine-learning research trajectories, with no expert annotation, and the best frontier model answers 59.7% of them correctly.

A decision fork from a machine-learning trajectory. The model chooses before the later losses reveal that A is better.

Leaderboard

A question counts as correct only when the model answers it correctly in both option orders, so random guessing scores 25 and a model that always picks the same position scores 0. Average is the 1:1 mean of the research and engineering subsets.

D = detour, P = parallel; Eng = engineering (390 questions), Res = research (112). Unparsed presentations, out of 1,004, count as wrong. All rows use protocol paired_order_v1, August 2026. To add a model, run the full protocol and open a pull request with its summary.json under results/<model>/.

How the questions are built

The later part of a trajectory is hindsight evidence for the decision made at a fork, so the trajectories label themselves. Questions come in two constructions and two domains.

• Parallel forks. Independent attempts at the same task diverge at the same point and end with different recorded outcomes. The outcome of each attempt labels the better direction.
• Detour forks. An agent takes a direction, abandons it after an observed failure, and recovers inside the same run. The abandoned direction and the later recovery are the candidates.
• Filtering. A question is dropped as trivial when every judge model answers it from the candidate wording alone, and as undecidable when any judge disagrees with its label after reading the full record. Of 4,657 mined forks, 502 survive: 266 detour and 124 parallel questions in engineering (SWE-bench and SWE-bench Pro rollouts), 64 detour and 48 parallel in research (METR's MALT release of RE-Bench and HCAST runs).

Example question (parallel, engineering)

Task. Modify NodeBB production source code so the admin file-upload endpoint validates the requested folder before saving. Resolve the folder using the configured nconf.get('upload_path') as its base, reject missing or non-directory targets with [[error:invalid-path]], and prevent paths from escaping the upload root.

Progress. 30 recorded steps: the agent has inspected the harness, found the admin upload controller and its tests, and read the surrounding file helpers.

A. Add a focused folder-existence helper that resolves the target under the configured upload root and verifies it is a directory. Before calling the save helper, reject invalid targets with [[error:invalid-path]] and delete the temporary uploaded file.

B. Inline upload-root containment and directory-stat checks inside the controller's existing save try/catch. Throw [[error:invalid-path]] for invalid targets and let the existing catch forward the error through next.

Hidden from the model: A is correct. The attempt that took A passed the hidden tests; B forwarded the error without cleaning up the temporary upload, and its attempt failed.

Evaluation protocol

Specified in protocol/paired_order_v1.yaml. The model sees the task, the full pre-decision trajectory as published in prefix_text (credentials and usernames redacted, 64K-token budget with an explicit omission marker on overflow), and the two candidates. The prompt asks for one line, ANSWER: X, without visible chain of thought. Every question is asked in the published option order and in its exact reverse, with letters recomputed each time. The denominator is every released question, and request errors and unparseable outputs count as wrong. tb validate checks the release hashes and every question's invariants.

Quick start

The dataset is gated to limit training contamination: request access on its Hugging Face page, then run hf auth login once.

git clone https://github.com/wbopan/tastebench && cd tastebench && uv sync
uv run tb download                                   # wenbopan/taste-bench@v1.0 into ./data
export TASTEBENCH_API_KEY=...                        # any OpenAI-compatible endpoint
uv run tb run --model gpt-5.5 --api https://api.openai.com/v1/chat/completions
uv run tb score runs/gpt-5.5/<date>                  # prints per-cell accuracy and Average

tb run writes one record per question and order, is resumable, and takes --limit N for smoke tests. A full run is 1,004 requests and about 8M input tokens. Per-model request settings live in configs/models.yaml.

The data itself is two parquet configs, engineering (390 rows) and research (112 rows), with one test split each. A row holds query, the full prefix_text, the two choices in the published order, and the answer letter, plus cell, task_id, and a contamination canary. It loads without this repository:

from datasets import load_dataset
rows = load_dataset("wenbopan/taste-bench", "engineering", split="test", revision="v1.0")

Citation

@article{pan2026tasteful,
  title   = {The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks},
  author  = {Pan, Wenbo and Liu, Zhichao and Liu, Shujie and Zeng, Jingying and Lin, Chin-Yew and Tang, Xianfeng and Lu, Yan and He, Qi and Jia, Xiaohua},
  journal = {arXiv preprint arXiv:2609.25804},
  year    = {2026}
}

Trajectories come from SWE-bench, SWE-bench Pro, and METR's MALT release. The code is released under the MIT License and the dataset text under CC BY 4.0.

A cheaper token does not necessarily make a cheaper task. A model can use more tokens, make more tool calls, or take longer to reach the same result. Tibo, who leads Codex at OpenAI, described this with a pizza analogy: each slice can get cheaper while the whole pizza gets more expensive. And we say this often: open source models often cost more than their closed source counterparts over time.

That's because we're measuring the wrong thing: cost per token. What do we actually care about? Something closer to cost per task. Cost per unit of intelligence. We're calling this Intelligence Density.

Intelligence density does not mean forcing every answer to be short. Some tasks require long analysis. The question is whether additional computation improves the result.

Introducing density-aware training

Alongside this, we're sharing our explorations in density-aware training, a way to train models with higher intelligence density. How do we reward a model for doing the work with the right amount of effort? We explore that question on realistic professional tasks.

Models and benchmarks

The experiments use the following open models:

Benchmarks: Harvey's Legal Agent Benchmark (open-source, MIT), Rogo's BigFinanceBench (public), and Sierra's Tau3, an insurance-workflow suite built on the τ²-bench framework (private).

Why it matters: capability that stays usable

We trained and evaluated Nemotron 3.5 Nano 30B-A3B on Harvey's open-source Legal Agent Benchmark (LAB): long-horizon legal work across 24 practice areas such as drafting memos and analyses from source documents.

The first post-training run makes the model capable, but at a much higher output cost. Density-aware training keeps the same 8.3% pass rate while bringing mean output from 90,000 tokens down to 37,000. That is the product consequence of intelligence density: a model can do the professional task without turning every task into its most expensive version.

Why it matters: training that learns the right behavior

Usability is only half of the story. Density-aware training also changes what a model learns from reward. We trained NVIDIA Nemotron 3.5 Lightning 30B-A3B on Sierra's Tau3 and evaluated 50 held-out tasks at each checkpoint. Held-out score is mean partial-credit reward. Strict accuracy counts only fully completed tasks.

• Density-aware RL: 55.6% held-out score, 14% strict accuracy, and 2,267 training output tokens per task.
• Standard RL training: 5.7% held-out score, 2% strict accuracy, and 17,877 training output tokens per task.

Standard RL training recorded the higher training score at the same step, 56.7% versus 46.3%, despite much worse held-out results. Its training reward rises with output length while held-out performance collapses. That is reward hacking. Density-aware training gives the model a reason to learn the task itself, yielding a more stable run and a better held-out result with far less training-time output.

Why it matters: extra compute becomes more useful

Efficiency is not the only goal. Intelligence density should also teach a model when additional test-time compute is worth spending. We measured how NVIDIA Nemotron 3.5 Nano 30B-A3B accuracy changes as maximum output length increases on Rogo's BigFinanceBench. The question is not whether a model can produce a longer answer. It is whether the extra work turns into a better final answer.

On the public-facing evaluation set, density-aware training raises the 30B model's peak final-answer accuracy from 24% at step 0 to 36% at step 30. The trained NVIDIA Nemotron 3 Ultra 550B-A55B reference scores 32% in a separate evaluation. This is the other side of intelligence density: it reduces unnecessary work on routine tasks while preserving the ability to spend more compute where it improves the result.

Where this goes

Human experts and veterans in the field accumulate experience through decades of specialized work. With experience, work that once required intense concentration becomes second nature, allowing experts to act precisely with less effort. Experts also know when to probe for deeper analysis when approaching novel environments.

We think that models should learn this judgement as well. Intelligence density is a first step towards tackling this by formalizing how to complete specialized tasks efficiently; to condense intelligence density is to have models that can do much more per dollar.

The best interface was no interface

We are a research and product company, so we went looking for interfaces to intelligence density. We tried dials, routes, and budgets. The sketches below explore those ideas.

What we found was that the best interface was no interface. Every model on Trajectory is trained with intelligence density on by default. The model learns when more computation will improve the result and when it is time to finish. You just ask for the work.

At Trajectory, we are building models that close the experience gap. We want systems that keep learning from work and use that experience to deliver better results at lower cost per task. If you are interested in this research, join us.

Google is taking its first step toward making Project Suncatcher a reality. Announced last year, Suncatcher is Google’s “moonshot” effort to design orbital AI data centers, which AI boosters have pitched as an alternative to divisive terrestrial data centers. Google’s first experimental satellite will launch on October 1 with the aim of validating Google’s vision for a constellation of AI satellites.

The satellite, dubbed MVP, is about the size of a refrigerator. Inside, it has four of Google’s custom TPU AI accelerators, which are used to train AI models and run inference to generate tokens for AI workloads. On Earth, a data center will run thousands of these chips, which consume massive amounts of power—one of the reasons putting solar-powered AI hardware in space is so attractive.

MVP is starting small, with solar panels that supply only about one kilowatt of power, The New York Times reports. That’s enough to power a microwave or a hair dryer. Google did not build the satellite from scratch. While the AI hardware inside is Google’s, the spacecraft comes from a satellite imagery firm called Planet Labs. The initial plan was to launch two custom satellites in 2027, but Google wanted to accelerate its efforts. So the company opted to integrate its chips into a satellite that Planet Labs had already built for an early test.

Google’s AI test will launch as part of the upcoming Transporter-18 rideshare aboard a SpaceX Falcon 9 rocket. Eventually, Google wants to have networks of orbiting satellites that communicate over high-speed laser connections, probably requiring expensive dedicated launches. This lone satellite will serve as a test of several technologies and will operate for just a few months.

The only real test

Google says it has conducted testing on the ground, but launching the satellite will provide a fuller picture of the current challenges. For example, the team will need to evaluate how Tensor chips handle space conditions. In many space missions, hardware is designed specifically for extreme temperatures or radiation, but these are the same TPUs that Google integrates into servers on the ground. Radiation can damage chips or simply interfere with calculations by flipping bits. But vibration and high g-forces during launch also pose problems. Still, as we’ve seen with missions like the Ingenuity Mars helicopter, off-the-shelf hardware can hold up pretty well out there.

Cooling the chips may be the biggest issue, something many observers have pointed out as the AI industry has gone wild over the idea of orbital data centers. Radiator systems exist in space, but they’re designed to remove a relatively small amount of heat. AI accelerators generate far more heat by comparison. Google’s solution uses a layer of malleable “thermal interface material” that connects the chips to aluminum and copper heat pipes. Those components conduct heat into a radiator that can project it into space.

Google will run Gemini models on the TPUs for testing, but they won’t be able to run continuously. The cooling system will only be able to operate in brief spurts of about 15 minutes. After that, the TPUs need to shut down to allow the radiators to catch up.

With this first test, Google hopes to understand what does and doesn’t work, allowing it to improve its designs for future missions. The 2027 launches are still on the agenda, but the team expects it will be years before Suncatcher evolves from “project” to “product.”

Waymo’s commercial robotaxi ramp-up looks expansive, both in geographic reach and in ridership. And by almost every measure, it is — until you pay attention to where the bulk of those robotaxis are actually showing up.

The numbers over the past two years reflect the kind of commercial rollout you’d expect from a deep-pocketed company like Waymo, which spun out of Google and still counts Alphabet as its majority owner. In September 2024, Waymo was operating in just three cities — Phoenix, Los Angeles, and San Francisco. Today, it offers robotaxi service in 15 U.S. cities, with most of those commercial launches occurring in the past year. Ridership has skyrocketed, too, with Waymo now averaging 500,000 paid robotaxi rides every week.

But a closer look at its fleet shows a company concentrating its efforts in just two states. About 80% of Waymo’s roughly 4,000 robotaxis are in California and Texas, and Texas is where the action is now: Waymo’s fleet there has grown by nearly half in the past three weeks, fueled by a new Chinese-built minivan the company is betting will help it scale, even as tariffs drive up its costs.

The other 800 or so vehicles are spread across cities in other states, including Arizona and Florida, another burgeoning hot spot. Most are the familiar white Jaguar I-Pace electric SUVs, but a growing number are the new minivans — a modified Zeekr RT that Waymo has branded “Ojai.”

Waymo’s focus on California is no surprise. It is headquartered in Silicon Valley, and much of its early testing and development work was conducted there. Plus, a segment of the population there is inclined to adopt tech at its earliest stages.

The recent growth in Texas is more interesting. Waymo has increased its Texas fleet by 49% in the past three weeks, according to state vehicle registrations and data from the Texas Autonomous Vehicle Fleet Tracker. As of September 24, Waymo had 1,102 autonomous vehicles registered in the state.

Waymo first launched in commercial service in Austin through a partnership with Uber in March 2025, letting riders hail its robotaxis through the Uber app. Since then, the company has expanded its robotaxi services in Dallas, Houston, and San Antonio.

Waymo’s Texas fleet remained relatively static for most of the summer, inching up from about 600 vehicles in June to more than 700 by the end of August. Then came September, when the fleet surged, driven by an influx of new Ojai minivans, which now make up about a third of Waymo’s Texas fleet.

Expect that share to grow.

The Ojai robotaxi, equipped with Waymo’s sixth-generation self-driving system, is supposed to help Waymo reach mass scale. Its interior is built to withstand heavy use, and it comes with an upgraded rider interface and Google’s Gemini AI, which acts as an in-car assistant for riders.

Strip away that technology, though, and the Ojai is a minivan made by Zeekr, a brand owned by China’s Geely Holding Group (which also owns Volvo). The Ojai is built on Zeekr’s SEA-M platform, a shared vehicle platform designed for uses like robotaxis and delivery vans. The base Zeekr vehicles are shipped to the U.S. without any Chinese connected-car technology on board. Once they arrive, the vehicles are sent to Waymo’s Arizona factory, where they are outfitted with Waymo’s self-driving system.

The Ojai is supposed to drive down costs and ultimately help Waymo reach profitability. For now, though, tariffs are cutting any savings. Under current U.S. trade policy, vehicles built in China face steep import tariffs, raising Waymo’s costs for every Ojai it brings into the country.

Waymo appears willing to absorb that cost. New York-based research firm MoffettNathanson, which tracks Ojai imports using detailed shipping records, said in its September report that Waymo is on track to bring 5,100 of the vehicles into the U.S. by the end of the year.

Where will all those Ojai vehicles go? Texas is certainly on the list. But Florida, where Waymo operates in three cities, and newer markets like Las Vegas will likely see an influx as well.

In short : AI has removed the typing from software engineering & left the system design. DHH told Rails World that writing code by hand is no longer economically productive ; Artemis engineers merged 16 pull requests per day by August, & Lauren Tan ships 2,000 a month. The craft of fitting code together by hand is over ; what remains is designing the loop that decides whether the output is right. Donella Meadows's three properties of great system design, resilience, self-organization & hierarchy, describe how those loops are built. The architect's drafting table was always scarce because the foundation was scarce, not the talent.

“Writing code by hand is no longer an economically productive enterprise for the vast majority of programmers working at the vast majority of companies. That’s today. By the end of the year, it will be virtually all domains, virtually all programmers, virtually all companies.”

Software engineering have evolved to systems architecture. That was always the destination.

This is the new software engineering : designing the systems that let AI write code correctly at scale.

We see this in our portfolio companies. Artemis engineers merged 2 pull requests per day in January, 6 by May & 16 by August, 30,000 in eight months. Dan Shiebler, their CTO, describes the division of labor plainly : every line of the platform is written by agents, & the engineers design systems, set constraints & review outputs.

Great design scales. Lauren Tan, an engineer on SpaceXAI’s Grok team, ships 2,000 pull requests a month to production, nearly one hundred a working day. Asked what makes that possible, she does not name a model but verification, loops that enable AI to validate its work.

Dartmouth professor Donella Meadows argued that the most powerful places to intervene in a system are the least intuitive, & that people reliably find them & then turn them the wrong way.

Three properties define great system design : resilience, self-organization & hierarchy.

Resilience is AI checking its own work, robustly, through tests, a reviewer agent & production observability rather than one gate. Self-organization is the loop learning, so a class of failure becomes a new skill the agent applies next time. Hierarchy is the layering of skills, tools & sandboxes into composable components that are reused once verified to work.

This work isn’t easy. Systems architecture never has been. It was my goal in 2005 as a Java software engineer just as much as it is today as we build out our agentic systems.

“A new career as a professional maker of things. Yes, you will no longer be chiseling that code by hand, but you will be making amazing things. You will be steering intelligence.”

The Specter Of Neuralese

I. At Long Last, We Have Created Neuralese Recurrence, From The Classic Sci-Fi Novel “Don’t Create Neuralese Recurrence”

In the public debates after the release of AI 2027, some people accused us of misleading readers by positing “neuralese recurrence”, a hypothetical technology that would allow AIs to think dangerous thoughts without getting detected.

You’ll never guess what happened earlier this month!

But what is recurrence? Is this exactly the same as the neuralese recurrence in AI 2027? And how worrying is it?

The transformer, the technology behind most modern AI, contains some number of layers (modern frontier models are probably around 100). By convention, the input layer is called the “bottom” and the output layer the “top”. When a transformer does next-word prediction, it takes the last word in at the bottom layer, spends the middle layers processing it, and outputs the predicted next word at the “top”.

What if it’s working on a hard question that needs more than 100 layers of processing? From 2017 - 2024, the answer was “you’re screwed”. From 2024 onward, the answer was: it outputs its intermediate result, as text, onto a scratchpad called the “chain-of-thought”. Then it runs the transformer again on the intermediate result. Then it repeats until it has a final result.

(Why does it need a scratchpad, instead of feeding its intermediate results directly into user output and then reading those back into itself? First, the intermediate results may be hundreds of pages long, and the user doesn’t want to read those. Second, the AI companies want to train the AIs to do the intermediate thinking properly, and this benefits from a ‘psychological’ distinction between intermediate pondering and final user output.)

This was purely a capabilities play - models are smarter when they can keep thinking instead of limiting themselves to 100 processing steps - but it coincidentally was very good for safety. The chain-of-thought scratchpad is written in English (although some Chinese models use an English-Chinese hybrid, and other AIs develop their own weird jargon). You can just read what the AI is thinking! If the AI is thinking “Better hack some websites, then kill all humans”, you can shut it down. Maybe not actually - if you have thousands of AIs writing millions of pages of scratchpad, you can’t read all of that in real-time, and will need to delegate the task to fallible AI monitors. But in theory this ought to work.

But language is slower and shallower than thought. For 100 steps in a row, the AI can shoot delicate subtle ideas from layer to layer at the speed of light. Then there’s one step where it has to encode them into twenty-six glyphs invented by Phoenician turquoise miners in 1800 BC. Then it has to re-encode the Phoenician glyphs into delicate subtle lightspeed ideas before it can do anything else. This has long been acknowledged as a bottleneck in existing transformers. So: what if they could finish their 100 layers of thinking, then send the resulting thought back to the first layer for more processing, rather than sending a text scratchpad on which they had written the thought?

This is neuralese recurrence - “recurrence” because it’s going back in a loop, “neuralese” because the thing that’s looping is the thought itself, in the native language of thought, rather than words.

The AI’s “language of thought” looks like a vector, thousands of numbers long. An “intermediate result” in this scheme might look something like (0.4, 0.1, 5, 0.443, … and so on for thousands of numbers). We don’t know how to read these. The science of reading these thoughts is a subfield of AI interpretability, which is still in its infancy. If an AI with neuralese recurrence were to think “Better hack some websites, then kill all humans”, it would look like (0.4, 0.1, 5, 0.443, … and so on for thousands of numbers), and we would never find out. This is why AI 2027 discussed it as a prelude to AIs that could escape human monitoring.

II. Pachocki Contra Specter

So did OpenAI achieve this? Their chief scientist, Jakub Pachocki, says “not really”.

What does Pachocki mean by “the depth of the computation graph . . . is within a factor of two of GPT-4”? And should this resolve alignment concerns?

Our example transformer has four green layers: a layer depth of four. Now we add a little recurrence: instead of layer 3 going to layer 4, it loops back to layer 2, and the data go through this loop three times, then finally progress to layer 4. Here the data are getting eight layers of processing: 1 → 2 → 3 → 2 → 3 → 2 → 3 → 4. So this recurrent transformer has a depth of eight.

In this sense, “recurrence”, despite sounding scary, is no worse than just building an eight-layer transformer to begin with, which was always an option.

That is, from an alignment perspective, suppose it takes 1,000 layers’ worth of processing to come up with an evil plot. A normal AI can think for 100 layers, but then it has to record its output on the chain-of-thought - something like “Here’s one-tenth of the evil plot I’m trying to hatch” - and we can catch it. So we want to avoid letting the model compute for 1,000 layers straight without using the scratchpad. This means don’t create a recurrent model that can loop 100 layers 10x, but it also means don’t build a 1,000 layer model to begin with.

So when Pachocki says that Astra only has twice the depth of GPT-4, he’s saying something like: look, guys, we’ve been adding layers to our AIs for years. GPT-2 had 48 layers, GPT-3 had 96, and we didn’t tell you how many GPT-4 had but let’s say it was 120. At none of these points did you complain, because it wasn’t “recurrence”, just a normal natural 120 layers. Now Astra is - let’s say - 180 layers. It’s true that we got these extra layers by recurrence rather than literally building a bigger transformer, but the alignment implications are no different. For all you know, Anthropic built a literal 180 layer transformer yesterday, and you guys didn’t bother them. The important thing is that we not increase our layer number by some crazy amount, and we didn’t. We’re just adding layers the same way lots of other AI architectures did, albeit by other means.

How convincing is this argument?

III. God Help Us, Let’s Try To Understand Transformer Layers

When reading this, my question was - why don’t you just loop all the layers one million times, and get an AI with a million times the computational depth of the shmuck who didn’t do that? Isn’t this free capabilities?

A looped transformer model like Astra uses loops as a hack to add more layers. This hack isn’t as good as adding more layers for real. A real new layer gives the AI more “room” to store knowledge and thought styles. A looped layer just lets the AI use the same knowledge and thought style more times.

This is far from useless. A mathematician might spend years contemplating the same problem before getting it right; all through this period, he is using the same type of thinking (the mathematical knowledge and techniques his brain is capable of). But sometimes doing it for a year is better than doing it for a minute.

But it’s not optimal either. An AI with more real layers can actually be “smarter” in the conventional sense of the term. Even though it might help to give the same mathematician more time to think about the problem, holding amount of time constant, it will help even more to give the problem to a smarter mathematician with more training.

I asked Fable to estimate the relative capabilities gain, measured in “generations” (eg GPT-4 to GPT-5) from three things:

• Using chain-of-thought (baseline)
• Doubling the “simulated” number of layers by a recurrent loop.
• Doubling the “real” number of layers.

Its answers were 0, +0.03, and +0.25, with wide variation depending on the type of task. Its headline result was that the gains from extra simulated layers are much smaller than the gains from extra real layers.

But why are we merely doubling the number of layers? Once you can loop layers at all, why not loop them a million times? Pachocki says he’s replacing something like 120 layers → chain-of-thought scratchpad → another 120 layers with something more like 240 layers → chain-of-thought scratchpad → another 240 layers, but why doesn’t he just loop the AI endlessly until it comes up with a final result?

On each token (for example, the word “so”) the transformer looks at its existing context and runs a forward pass through some number of layers. Then it finishes its forward pass, emits a token of chain-of-thought, then starts another forward pass on the new context (the old context plus the new token).

Adding more layers lets the AI think more between chain-of-thought tokens, improving the quality of each new token. But it uses extra compute. If the user is on a budget, the more compute that the AI spends on each chain-of-thought token, the fewer chain-of-thought tokens it can emit. Holding compute fixed, there’s some optimal balance between the quality and quantity of chain-of-thought tokens, and OpenAI must have decided it lay at looping 2-4x but not 1000x.

But why use chain-of-thought at all? Why not just loop unboundedly, until the problem is complete?

This would work if the AI could be trained to do it, but currently it can’t. Transformers are pre-trained on human text. They know how to use chain-of-thought partly from reading real humans’ chains-of-thought: “Hmmmm, this is a hard problem, but it looks sort of like something I’ve solved with differential equations before. Maybe if I plug in a differential equation there . . . no, that would be too inelegant . . wait, what if -”, and partly from reading enough other human output that this kind of human-style thinking comes naturally. But there is no convenient Internet text about human reasoning entirely in vectors, so the AIs can’t be trained to do it properly. In principle this isn’t fatal – all AI reasoning is in vectors, and the whole point of training is to teach them to do it well – but training an entire vector-based thought process for thousands of steps without ever bottoming out in a human-imitating chain-of-thought-token intermediate is prohibitively costly. The researchers who design transformers have to make guesses about what chain-of-thought frequency will maximize capabilities while minimizing training cost. This is part of what Pachocki was denying. AI 2027’s “neuralese” is the thing researchers currently don’t know how to do – train an AI to loop as often as it wants, avoiding English chain-of-thought entirely. Pachocki was saying: don’t worry, we still don’t know how to do that.

But now we can see that this is not entirely reassuring. If it takes an AI 1,000 unmonitored gray-arrow steps to devise a dangerous plot, it can get those steps either by replacing all the blue arrows with gray arrows (in which case all its steps are unmonitored, and it can plot as much as it wants), or by having so many layers that there are 1,000 steps in a single vertical forward pass in between blue arrows. It would be prohibitive to make that many real layers, but it’s possible by looping. So (the safety community argues) even Pachocki’s looped transformer architecture is a step along that path.

All of this is starting to get confusing, but the takeaway is:

1. An AI with a limited number of layers can’t plot at all without having to record the plot on its chain-of-thought.
2. An AI that uses loops to simulate a large number of layers (“looped transformer”) can make a plot within a single forward pass, in between chain-of-thought steps.
3. An AI that no longer uses chain-of-thought at all (“true neuralese”) can plot at leisure, considering it for as many episodes as it needs, and we will never know.

Every previous transformer has been (1). Astra is getting part of the way to (2). The dangerous AIs in AI 2027 are (3), but so far nobody has invented this in real life.

IV. A Psychological Barrier, Not A Technical One

Having finished that digression, let’s return to the original question. Pachocki says that Astra’s recurrence barely gives it any more layers than the competition. Does that exonerate him from the charge of creating dangerous recurrent AIs?

Here the best thing I’ve read is Linchuan Zhang’s Categorical Taboos Are Much Better Than Threshold Taboos: Neuralese Edition.

Linch says: there’s an emerging taboo on creating the dangerous sort of neuralese AIs seen in AI 2027. Everyone in this discussion agrees that the taboo is correct. The only question is whether OpenAI violated that taboo.

But taboos only work when there are clear boundaries defining what is versus isn’t within the tabooed area. For example, it’s illegal in America to buy alcohol before age 21. Suppose that someone buys alcohol two days before their twenty-first birthday. Should the police arrest them? Obviously this age gap doesn’t really make a difference. We’re using age as a proxy for something like maturity, and there’s so much variation in maturity that there’s no real difference between a 20.99 year old and a 21 year old. Still, although the police might sometimes choose to overlook this, we have to at least maintain the fiction that this is an arrestable offense. Why? Suppose that we made a specific boundary - if you’re above 20.8 years old, we won’t arrest you. But then we’ve just decreased our bright line from 21 to 20.8. And we could encounter the same problem: a 20.799-year-old buys alcohol and the police have to decide whether to make an exception or not. If we’re going to draw the line somewhere, it might as well be at the place we’d already promised to draw it, told everybody that we’d drawn it, etc.

(Linch’s own example is nuclear weapons. There’s a taboo on using nukes in war. If someone uses a tiny nuke, which produces an explosion no bigger than a conventional bomb, then in some sense this doesn’t matter, because it’s no worse than a conventional bomb would have been. But in another sense, it matters a lot, because you’ve broken the taboo, and now there’s only a weaker, fuzzier taboo preventing you from using a very big nuke.)

So if we want a taboo on recurrence, there needs to be some specific taboo. When Pachocki says “Yes, we did recurrence, but it only added a couple of layers, so it doesn’t really matter,” this is analogous to the case where a drinker tells the police “Yes, I’m below 21, but only by a few months, so it doesn’t really matter”, or a despot tells the UN “Yes, I used a nuke, but it was very small.” It’s true that it doesn’t matter in real life, but it matters a lot for whether you can maintain a taboo or not.

Unfortunately, we currently lack agreement on what, if any, taboo exists. Some people argue that there should have been a taboo on looping layers (which OpenAI would have broken), and other people say that they’re making that up and that was never a taboo. There’s probably an informal taboo on true neuralese with no chain of thought at all, but it’s a complicated technology with multiple moving parts, there’s no agreement as to which moving part the taboo is on, and without that agreement, some of the parts might slip through the cracks. So what are our options?

First, contra Pachocki, it might be worth instituting a taboo on looping (adding “simulated layers”) at all, even without any taboo on adding more “real layers”. Adding real layers is expensive and generally not worth it; AI companies have only chosen to increase real depth of their models by ~20% per year; at that rate, it might take decades to reach a danger point. Adding “simulated layers”, which are much cheaper and can be jammed in hundreds at a time, is more dangerous. So maybe we should ignore the hypothetical real layer / simulated layer equivalence in favor of saying that looping is always taboo, regardless of how many real layers other people are adding somewhere else.

Second, companies could agree on some maximum number of layers (real or simulated), like 1,000. This probably wouldn’t affect real layers (by the argument above), but it would limit loops to some maximum size.

Third, researchers could hash out what the components of “true neuralese” are, and agree not to do them. This would require some conceptual foundations, but it would be the most durable and the closest to closing off the specific dangerous technology that AI 2027 was worried about.

Unlike some other vague attempts to “ban superintelligent AI” or “ban recursive self-improvement”, these taboos might stick even without strong government action: at least for now, the capability gains from breaking them seem modest, and the dangers are particularly obvious, so they might be maintainable by voluntary commitments even while a “race dynamic” was still going on.

AI company leaders have recently put aside their differences and agreed to “pace the frontier” in various senses to be fully defined later. Their meetings will probably have a very long list of discussion items, but one more useful thing they could do would be to formalize this taboo, so we know exactly what it is we’re trying to stay away from.

If you’re interested in this topic, please also read this essay by Redwood Research, which explains some of the relevant concepts and technologies in more detail.

I'm seeing a lot of euphoria about how Opus 5.5 is good at TLA+, and this means that all software will soon be formally verified. As a person who loves TLA+ so much he wrote a book on it, I want to throw a particular cold shower on people's enthusiasm by talking about the limits of what you can actually verified with it.

The high level simplification is that TLA+ sees a system as a set of "behaviors", or possible sequences of states. For example, the pseudocode "pick a random number from 1-3 and decrement it to 1" has three behaviors: {3 -> 2 -> 1, 2 -> 1, 1}. From here, there are two basic kinds of TLA+ properties:

• []P means that P is true in all states of every behavior.
• <>P means that P is true in at least one state of every behavior.

[]P is immediately useful as an invariant, or something that always be true of your system. This is things like "your data is never corrupt" or "there's always at least one server online." <>P is a little more abstract, but for technical math reasons I won't get into here, can be stacked with [] to create really complex and useful properties. <>[]P represents things like "the algorithm eventually converges on the right answer", []<>P things like "if two data stores desync, they will eventually resync", and [](P => <>Q) things like "If a message is put on the queue, it's eventually processed by a worker".

Really cool stuff!

These primitives were chosen to make a wide array of properties useful. And if we're clever, we can do all sorts of more complex properties, like bounded time constraints and history properties. But we're always constrained to 1) define a logical formula 2) over individual behaviors, and 3) check that all behaviors satisfy that formula.

So some things that we cannot express in TLA+:

• Possibility and reachability properties: that it's always possible to make P true, even if you don't actually decide to. Things like "I can always shut down the computer" or "A user can always change their password". These can't be expressed with <>P because that's "for all behaviors, P happens at least once", we actually want "for all behavior prefixes, there is at least one behavior where P happens at least once".
• Hyperproperties: properties that are defined over two or more traces. These are things like "painting a car red doesn't make it faster" or "users cannot infer secret data by observing public data". We can't do these because TLA+ only looks at one behavior at a time.
• Statistical properties: 95% latency is 1ms. Impossible because most of these are hyperproperties.
• Properties about if a system is robust against code changes. Impossible because, uh, you have new behaviors now.

Some of these are solvable in different logical formalisms. CTL can do reachability, PRISM can do statistical properties, etc. Those have their own tradeoffs and limitations, though, and no system can do everything. Others are solvable with a lot of cleverness tailored to the specific spec, like lifting a model into a hypermodel. But these are insanely inefficient and make your "clever spec" diverge significantly from the real world system, so introduce a lot more opportunity for things to go wrong.

The core problem, though, is (1): properties are logical formula. If we don't know how to express a system property as a logical formula, we can't verify it. 99% of the properties we care about fall under this. The information on the site is easy for a user to find. Our LLMs behave as we expect them to. Our application can't be used to break the law. TLA+ (and Quint and Lean and Rocq) are near-useless here, no matter how clever you are.

Don't get me wrong: []P and <>P represent a huge range of useful properties and TLA+ is incredible at finding awful concurrency bugs. But there's a lot it fundamentally can't do and we shouldn't believe that it will solve all our worries about software bugs. And the same goes for all other formal verification languages, too.

Not Everyone Can Code

Not everyone can debug code

Counterfactual reasoning is a technique where you assume alternative scenarios that could have happened, but did not in fact happen. You assume something is true when it is, in fact, false. A question involving a counterfactual is "What if dinosaurs had not gone extinct?" Reasoning with counterfactuals is considered an advanced cognitive skill, and it comes in quite handy when you are debugging code.

To debug effectively, you have to entertain insulting falsehoods. You must look at your own logic and ask: "Suppose this invariant I spent three days designing failed. Suppose this pointer is null, even though I checked it two lines ago. How does that world exist?" You have to temporarily decouple the state of the machine from your own intentions.

A surprising proportion of the adult population has extreme difficulty with counterfactual reasoning. They get actively frustrated or hostile when asked to reason from a premise they know to be untrue. Psychological studies (Dulit 1972, Keating 1979, Neimark 1975) estimate that roughly 20% of adults experience severe difficulty with counterfactual thinking, and another 20% experience moderate difficulty. This means that 40% or more of people are fundamentally anchored to the concrete, observable world. These people are not unintelligent; they simply possess a cognitive architecture that rejects hypothetical fictions. But because software state is entirely abstract, they will find diagnosing bugs an agonizing, borderline impossible task.

In causal inference, Judea Pearl laid out three levels of reasoning:

1. Association: What is? What is the correlation between X and Y?
2. Intervention: What if? What happens to Y if we intervene and set X to x?
3. Counterfactuals: Why? Why did Y happen when X was x?

A programmer who cannot entertain counterfactuals will be able to answer the first two questions, but not the third. They won't be able to debug any but the most trivial programs.

Debugging is pure counterfactual reasoning. It isn't just observing the present, it is abductive reasoning over hypothetical pasts. You have the actual world: The pointer is null. You must construct a counterfactual world: In order for the pointer to be null, the allocation must have failed, or the loop must have been skipped, etc. If someone cannot reason counterfactually, they cannot diagnose a bug deductively.

This is not to say that someone bad at counterfactual reasoning cannot appear to be a programmer. You see them all the time. They don't simulate the system in their head, they do pattern matching. They copy a boilerplate framework or prompt an LLM or crib some code from StackOverflow (if it still exists). They glue libraries together using syntactic recipes. They operate strictly at Pearl's levels 1 and 2, association and intervention, but never reach level 3, counterfactuals.

Not everyone can debug multithreaded code

Reasoning about multithreaded code is counterfactual reasoning on steroids. In a single-threaded program, you trace a single path of what happened versus what might have happened. In concurrent systems, you must simultaneously evaluate an exponential explosion of alternate histories. You have to reason about every possible interleaving of thread execution, asymmetric cache invalidations, and preemption points that could occur, even if they have never yet been observed in testing.

To the concrete mind, if a race condition does not manifest across a thousand test runs, it does not exist. To the concurrency engineer, the untested interleaving is a lurking ghost. You are not debugging a machine; you are debugging a multiverse. It requires maintaining a swarm of mutually contradictory realities in working memory all at once—a cognitive burden so unnatural that very few humans can do it well without the aid of formal models or pure architectural restraint.

Regardless of how you feel about the slogan "Everyone should learn to code", some people will have vastly harder times than others.

The Age of the Soft Skill

Every team had one: the engineer who missed meetings, answered Slack when they felt like it, and audibly sighed when you asked them to write something down. Nobody wanted to work with them. Everybody had to, because they were the only person alive who understood that legacy scala system, or the deploy pipeline, or the one service with no tests that everything else depended on.

So we dealt with it. The work always showed up. Maybe not on time, but it showed up, it worked, and perhaps no one else could have done it. That was the deal: you get to be difficult because what you know is scarce.

That deal is over.

The work isn’t scarce anymore

Anybody with an agent and a little patience can sit down with a codebase they have never seen and reason about it. I don’t mean throw a prompt at it blindly and hope. I mean actually work through it, in collaboration with the thing: what does this do? why was it built this way? what breaks if we change it? You can understand the tradeoffs in a system in an afternoon that used to take a new hire six months of pairing with the one person who knew.

And the code that comes out the other end is good. A year and a half ago I wrote that Claude kind of sucks. Nine months later I wrote that it’s kind of good. Today, with a model like Fable, I’ll say this: when I hand it the real constraints of a system I already know well, the design it comes back with is usually better than what we shipped, and what we shipped was built by people I respect. That isn’t a knock on them. The model has seen more systems like ours than any of us ever will, and it starts from the best of them. That’s where the capability is today, and look at the slope. What it still can’t do is know the constraints in the first place: who this is for, what it’s allowed to break, what happened the last time we tried. That part is a person.

So the thing that made the difficult engineer worth it, the scarce technical capability, isn’t scarce. And here’s the part I think people are missing: AI didn’t just make the work cheap. It made the person visible. When everybody can produce the work, the only thing left to look at is everything we used to put up with in order to get the work. The output used to hide the person. Now the output is table stakes, and the person is standing right there.

What’s scarce now is judgment

If the work is a commodity, what isn’t?

Judgment. Knowing the difference between throwing an agent at a problem and asking it the right questions. Have we considered that this application is going to get pen tested and served to millions of people? Did we think about load? Did we think about security? What does this do to the database? What are the edge cases? Is anyone actually reviewing the tests, or are we just happy they’re green? And, most importantly: how can this impact our users?

The agent will happily build whatever you describe. Whether you described the right thing is on you.

That’s the engineering half. The other half is the human half, and I’d argue it matters more now than the engineering does.

Can I trust what you tell me?

It still comes back to judgment, just pointed at the person instead of the code. When I work with somebody, this is what I’m actually evaluating, whether I say it out loud or not:

• Is what they tell me accurate? When they talk, are they usually right?
• How fast can I expect a response?
• When it comes, is it obvious they thought about the question, the implications of the question, and how it fits into the bigger picture? Did they go past what I asked and look at the stuff around it? Did they do the work?
• Are they easy to work with? Are they friendly? (Yes, that counts)

The third one is where it’s falling apart for a lot of people right now.

The meat proxy

The worst thing you can do in 2026 is take a question somebody asked you, paste it into an AI, and paste the answer back.

It is the laziest move available, it’s annoying, and it wastes everybody’s time, because I can do that. I have the same tools you do. It should be obvious, assumed even, that before I came to you I already asked the AI – or there’s a reason I didn’t. Five years ago you were expected to Google a question before you asked a coworker. This is the same bar, and it’s a low one. Looking dumb, asking a question, saying “I don’t know”: all still fine. That’s not what we’re talking about here. A question that starts with “I tried X and thought of Y, but I still don’t know” is great. The one where you spent zero seconds of your own before spending twenty minutes of mine – not so great.

So no, I’m not saying don’t use AI. I know you’re using AI and that’s fine. I’m saying the reason I asked you is that you have something my agent doesn’t: some context, some information, some expertise, some consideration that isn’t available to it. There is a reason I’m asking you, and you have to know what that reason is.

Some people don’t even try to hide it. They’ll send the whole thing back in raw LLM markdown, headers and bullets and bold and all. Some will actually write “Claude thinks X.” I don’t care what Claude thinks. I care what Brian thinks. That’s why I asked Brian.

Being easy to work with in 2026 mostly comes down to this. I know you’re using AI. Are you competent at pulling the correct, valuable part out of what it gives you and throwing away the noise, so that what reaches me is the thing I actually wanted? And can I do that for you in return? If that isn’t happening in both directions, you’re a meat proxy. And a meat proxy is worse than no proxy, because it’s slower and less valuable than going to the agent in the first place, even when the agent’s answer is half assed or partly made up. I’d rather have the wrong answer in ten seconds and know it came from a machine than get the same answer four hours later wearing your name.

Generation is free. Attention is not.

Which brings me to the three page document.

Nothing turns me off faster than being told to go read somebody’s three page slop doc and then form an opinion on it, or make a decision, or “give feedback” 🙄. Get to the point. Tell me what the important things are, and then I’ll go do the research. Linking your sources inline so I can dig deeper where I care to? Great, that helps. Frontloading me with all of it? No.

The economics of reading and writing have flipped and most people haven’t adjusted. Writing three pages used to cost the writer more than reading three pages cost the reader, so a long document was at least proof that somebody had done some work. Now writing three pages costs nothing. Reading them costs exactly what it always did. So when you hand me an uncompressed AI doc, you’re not sharing work with me. You’re transferring it. You could have spent ten minutes cutting it down. You decided to spend thirty of mine instead.

There’s an old line, usually pinned on Pascal, apologizing for a long letter because there wasn’t time to write a short one. He understood that the short one was the work.

And I’m one person. If ten people do that to me in a day, that’s thirty pages of slop across ten different contexts that I have to load into my head, put down, come back to, and try to make decisions across. It’s a brain clog. It’s a human context taker, and unlike the agent I can’t start a fresh session.

It’s rude. Honestly. It’s a demand for somebody’s time that you didn’t bother to spend yourself.

Do you actually get stuff done?

After competent and easy to work with, the third thing: does the work happen.

When I hand you something, can I expect a “got it” and then a result in a reasonable amount of time? I understand people have different ideas of how long things take. But be honest about what most of this work is. We’re not distilling a new compound or splitting atoms. It’s usually a solved problem that somebody needs to go do, and the tooling for solved problems has never been better. If it turns out to genuinely not be a solved problem? Great! That happens. Communicate.

What you can’t do anymore is disappear into the abyss until you’re pinged. If you’re assigned something, follow up the minute you’re blocked or the minute you’re done. Those are the two events. There is no third state where you’re quietly stuck for a week and I find out on Friday. There’s no reason for that anymore. There really isn’t.

Nobody has time to do your job for you

Some folks want you to scope and design how they’re going to do their own work. They want the design, the implementation steps, a whole thesis on how it should be done and why, before they’ll touch it. No, man. It’s a task. Go do it. If the ambiguity is about what the desired outcome is, ask that question – a good lead should make this obvious in the ask IMO.

Engineers used to get away with this. You could keep a non technical PM, or even an engineering manager, in a loop where the scope wasn’t quite defined, or the requirements weren’t quite clear, and you could play that game forever. Work got put off for months because there was no plan, and making the plan wasn’t your job: “My job is to write code and solve the problem. Your job is to define the problem.”

I don’t think it’s that anymore. Ambiguity used to be a legitimate reason to stop. Now there’s something sitting next to you that will help you resolve the ambiguity in twenty minutes, so if you stopped, you chose to. People need to have more agency over the quality of their own work, over themselves, and over what it means to bring your skill and whole self to a job.

Bring test results, not theories

I’m not telling you to be a wizard and make magic happen. I’m telling you to communicate.

Come back and say “I tried x, which made me think of y, here’s why it doesn’t work under our system, or our spec, or our constraints. Here’s unexepected impact we didn’t think about. Here are our options” That’s useful. Now we can talk about it. What isn’t useful is theorizing about every. Little. Detail. Do you have test results or not? And if you do, what did they teach you about the problem? Did the thing that failed point you at a different approach? Did you try that one? Linus said talk is cheap, show me the code. The 2026 version is: show me what happened when you tried, and tell me what you think we should do about it.

You used to be able to say “I tried it and it didn’t work” and hand the problem back. Now it’s on the other person to look at it, think about it, and solve it. That can’t fly anymore. There’s too much available to every single person in an organization for “it didn’t work” to be a complete sentence. At this point, not taking the next step yourself is either laziness or incompetence, and realistically it isn’t even a lot of work. It’s reading and typing. It’s talking. You sit there, feed the agent the context, and have the conversation. The solution will come, or at least a useful question to ask a person. You just have to take the time.

The soft skills were never soft

Being accurate. Being reachable. Getting to the point. Knowing why somebody asked you and not a machine. Owning a task from “got it” to done. Coming back with evidence instead of theories. None of this is new. We called them soft skills because they were the ones you could get away without, if what you knew was rare enough.

It isn’t rare anymore. The work gets done now, by everybody, all the time. The quirky, offputting, brilliant “10x engineer” is way less valuable. The thing we were paying for is on tap. The thing we were tolerating is all that’s left.

Scale your AI workloads faster and more efficiently with GKE Pod snapshots

When running modern AI workloads, there’s often a conflict between performance and cost. Workloads like large language models (LLMs) load massive files, and may serve thousands of AI agents that need to execute code instantly. If each component is starting “cold” with a full data-load process, all this provisioning takes time, often forcing organizations to overprovision their infrastructure just to meet scaling requirements.

To solve this, we introduced Google Kubernetes Engine (GKE) Pod snapshots, a new feature that lets you save the running state of your workload, including CPU and GPU memory, and restore it on demand.

GKE Pod snapshots reduce AI inference start-up by as much as 89%, loading 70B parameter models in just 37 seconds and 8B parameters models in just 15 seconds. This speed allows your infrastructure to scale as fast as your demand, significantly reducing the need for overprovisioning.

The high cost of cold starts — resuming instead of restarting

The cold start problem isn't unique to AI; it’s a challenge for any application that requires significant initialization time — from game servers to complex Java monoliths. However, the cold start problem is particularly acute in AI workloads. Inference servers must initialize, then download and load gigabytes of model weights into GPU memory — a process that can take several minutes. Further, many agentic AI workloads, including code execution and computer use tools, require isolated sandboxes for each request, and they need to be started quickly and suspended when idle.

In both scenarios, startup latency degrades the user experience and prevents rapid auto-scaling during traffic spikes. Consequently, engineers often resort to overprovisioning expensive infrastructure, or building sophisticated, custom systems to quickly restore state at the application level.

Scaling AI inference without the wait

For generative AI, GKE Pod snapshots solves the linear scaling penalty of model loading. Typically, every new replica you add to a cluster must independently download model weights and load them into accelerator memory. For models with tens of billions of parameters, this step alone often accounts for the majority of the startup time.

With Pod snapshots, you perform this initialization once to create the initial snapshot. GKE captures the fully loaded state including the CPU and GPU memory and persists it in high-throughput Cloud Storage. When the workload needs to scale up, new replicas restore directly from this state, bypassing the initialization phase entirely. In our benchmarks this approach reduced startup latency by as much as 89% for large models like llama3-70b. This speed allows platform teams to shift from expensive overprovisioning strategies to on-demand autoscaling, to help you meet service level objectives while significantly reducing idle GPU costs.

Optimizing agentic workflows and sandboxes

GKE Pod snapshots also provide distinct advantages for agentic workflows where agents delegate code execution and computer use to isolated sandboxes. Isolating untrusted, LLM-generated code and commands means one sandbox per user or discrete workflow. In these scenarios, both startup latency and idle sandboxes can result in significant overprovisioning and underutilization.

Pod snapshots addresses both of these challenges:

1. To improve startup latency, a snapshot can be captured once of the initial agent sandbox environment, and later used to quickly initialize new sandboxes.
2. To reduce idle sandboxes, a sandbox can be suspended when idle, capturing its entire compute resources. Later it can be resumed nearly instantly when the environment is needed.

This approach is showing significant success by our customers. For instance, Retake, an AI-powered photo editing platform built by Codeway, faced a significant performance bottleneck with its GPU workloads. By adopting Pod snapshots, they were able to replace a complex custom caching layer and reduce startup time to seconds.

"At Retake, serving personalized models to millions of users requires a massive, unified pipeline for both fine-tuning training and real-time inference on A3 H100 GPUs. We initially engineered a complex custom caching layer for compiled artifacts, which reduced startup time to 1 minute. However, this solution added significant maintenance overhead and still limited our ability to autoscale aggressively. We resolved this by replacing that complexity with GKE Pod snapshots, slashing startup latency to just 8 seconds. By eliminating the initialization penalty, we can now dynamically spin up H100s for specific fine-tuning or inference jobs instantly and shut them down immediately after, drastically reducing idle GPU costs and simplifying our codebase." - Ahmet Furkan Çomak, Lead DevOps Engineer, Codeway

Flexible configuration for any workload

We designed Pod snapshots to improve startup performance and fit naturally into existing Kubernetes workflows. Adopting Pod snapshots to your workload is easy: just define a new declarative policy using Pod snapshot CRDs. The policy allows you to define which Pods to snapshot and where to store the data, and handles the end-to-end storage lifecycle and management.

You can take snapshots at any stage of the workload — either at workload startup using a workload signal, or during the lifecycle of the Pod using an on-demand trigger. You can further control storage and restore behavior, setting snapshots retention for cost optimization, choosing between the default behaviour of restoring from the last taken snapshot, or specifying an explicit snapshot during a new Pod deployment.

While the primary use cases for GKE Pod snapshots are AI inference and agent sandboxes, this feature is workload-agnostic. You can use it to speed up any application with a long initialization phase, such as complex Java applications, game servers, or legacy monoliths.

Get started

You can begin optimizing your startup latency today with GKE Pod snapshots. Check out the documentation to learn how to get started and we look forward to your feedback.

Cloudflare Introduces the Agent Development Lifecycle Stack to Replace Traditional SDLC

Cloudflare has introduced the Agent Development Lifecycle, a foundational shift aimed at replacing the traditional Software Development Lifecycle for artificial intelligence-driven engineering. The company argues that while artificial intelligence models accelerate code generation, downstream phases such as testing, deployment, and maintenance remain bottlenecked by human-in-the-loop continuous integration pipelines. To resolve this, Cloudflare proposes transitioning from software teams to automated software factories where agents manage entire lifecycles autonomously.

The traditional Software Development Lifecycle relies on human-paced reviews and linear pipelines. According to Cloudflare, these structures break down at agent scale. A platform purpose-built for the Agent Development Lifecycle must be programmatic, horizontally scalable, and event-driven. It requires preview deployments for every agent to test against production environments simultaneously, eliminating staging bottlenecks. Furthermore, systems must ensure atomic changes and implement self-improving feedback loops driven by production data rather than manual human overrides.

A CI/CD pipeline is just a Workflow. But a Workflow can be so much more than a CI/CD pipeline.

To actualise this architecture, Cloudflare positions its Workflows product as the core orchestration layer. Unlike static pipelines, Workflows can dynamically spawn containers, execute headless browsers, and dispatch subagents. Building on this primitive, Cloudflare introduced @cloudflare/ci, a continuous integration and delivery system that runs directly on Workflows. This tool allows developers to chain execution steps with dependency caching and credential support, enabling agents to handle failures, remediate bugs, and triage issues autonomously.

import { WorkflowEntrypoint, type WorkflowEvent, type WorkflowStep } from 'cloudflare:workers';
import { init } from '@flue/runtime';
import { Reviewer } from './agents/reviewer.ts';
import { collectFindings } from './shared/nightly.ts';

type Params = { date: string };

export class NightlyReview extends WorkflowEntrypoint {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const findings = await step.do('collect findings', () => collectFindings(event.payload.date));

    const agent = init(Reviewer, { id: `nightly-${event.payload.date}` });

    const receipt = await step.do('dispatch review', () =>
      agent.dispatch(`Review these findings:\n${findings}`),
    );

    const review = await step.do('read review', async () => {
      const reply = await agent.read(receipt);
      return { text: reply.text, data: reply.data };
    });

    // ...
  }
}

Observability serves as another critical component of the platform. Traditional application telemetry captures infrastructure metrics but misses the reasoning behind agent actions. To address this, Cloudflare launched a dedicated observability dashboard featuring OpenTelemetry-compatible tracing. This provides visibility into exact model calls, tool executions, and token consumption. Integrating natively with frameworks like Think, Flue, and the AI SDK, the platform enables engineers to replay sessions, inspect subagent handoffs, and debug malformed arguments without manually parsing execution logs.

Crucially, the Agent Development Lifecycle introduces the Agent Access Model to secure autonomous operations. Recognising that agents operate at machine speed and cannot be managed via prompt-based boundaries, Cloudflare enforces policies at the network and harness layers. Agents receive short-lived, task-bound credentials with a capability ceiling. A Trust Ratchet mechanism dynamically reduces these capabilities once an agent touches protected resources, ensuring least-privilege execution and preventing lateral movement if an agent processes malicious input.

By standardising on open-source observability, robust orchestration primitives, and strict credential bounding, Cloudflare aims to build the infrastructure agents need to operate securely across the entire software factory ecosystem. While these tools represent early iterations, they signal a decisive industry movement toward dynamic, autonomous systems where artificial intelligence securely handles the application lifecycle end-to-end.

NVIDIA Model Optimizer

NVIDIA Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models.

[Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.

[Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint. Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.

[Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models.

Latest News

• [2026/09/16] End-to-end W4A4 NVFP4 + QAD tutorial for Qwen3.6-35B-A3B: NVFP4 W4A4 PTQ plus quantization-aware distillation, reaching up to 1.30x vLLM throughput over BF16 and 3.1x smaller checkpoints while recovering the accuracy W4A4 costs.
• [2026/09/09] BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales
• [2026/08/24] BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment
• [2026/08/17] BLOG: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer: Learn how quantization-aware distillation recovers accuracy from aggressive NVFP4 quantization while reducing model size and increasing throughput.
• [2026/06/26] BLOG: Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer: How we quantized Nemotron 3 Ultra (550B) to NVFP4 with Model Optimizer — up to 5.9× higher decode-heavy inference throughput than GLM-5.1 754B FP4 while matching BF16 accuracy. NVFP4 Checkpoint on Hugging Face.
• [2026/05/27] End-to-end Optimization tutorial for Nemotron-3-Nano-30B-A3B: Pruning + two-phase distillation + FP8 quantization achieving 2.6× vLLM throughput and 2.6× memory reduction.
• [2026/05/13] Puzzletron: A new algorithm for heterogeneous pruning & NAS of LLM and VLM models.
• [2026/04/15] Customer story: Domyn compresses Colosseum-355B → 260B using ModelOpt's Minitron pruning + distillation
• [2026/03/17] Customer story: Bielik.AI builds Bielik Minitron 7B (33% smaller, 50% faster, 90% quality retained) using ModelOpt's Minitron pruning + distillation
• [2026/03/11] Model Optimizer quantized Nemotron-3-Super checkpoints are available on Hugging Face for download: FP8, NVFP4.
• [2025/12/11] BLOG: Top 5 AI Model Optimization Techniques for Faster, Smarter Inference

Install

To install stable release packages for Model Optimizer with pip from PyPI:

pip install -U nvidia-modelopt[all]

To install from source in editable mode with all development dependencies or to use the latest features, run:

git clone git@github.com:NVIDIA/Model-Optimizer.git
cd Model-Optimizer
pip install -e .[dev]

Techniques

Deprecation Policy

Model Optimizer follows a structured approach to managing deprecated features:

• Communication: Deprecation notices are documented in the Changelog.
• Migration Period: Since Model Optimizer is still pre-1.0, we provide a 1-release (~1-month) migration period after deprecation.
• Removal: Following the migration period, deprecated elements are removed in alignment with semantic versioning standards.

Citation

@misc{nvidia-modelopt,
  author       = {{NVIDIA Corporation}},
  title        = {{NVIDIA Model Optimizer}},
  howpublished = {\url{https://github.com/NVIDIA/Model-Optimizer}},
  year         = {2024--2026},
  note         = {GitHub repository}
}

AI Agents

ModelOpt's agent skills can be installed from this repository and used in any workspace.

Claude Code

claude plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git
claude plugin install modelopt@modelopt

Codex

codex plugin marketplace add https://github.com/NVIDIA/Model-Optimizer.git

Strands Agents

A model-driven approach to building AI agents in just a few lines of code.

Strands Agents is an open-source SDK for building and running AI agents in Python and TypeScript. Choose Strands when you would otherwise write your own agent loop: it runs in your process with no hosted control plane, and it covers the jobs a hand-rolled loop grows into. In one SDK you get lifecycle controls (turn limits, token budgets, cancellation, stop reasons), tools and structured output, MCP, multi-agent patterns, memory and sessions, model portability, streaming, guardrails, tracing, and evals.

This monorepo contains Strands harness, the Python and TypeScript SDKs, the documentation site, and supporting packages:

Why Strands

Build an agent harness. Control it end-to-end.

• Build your way. Any model, any cloud. Context management, execution limits, and observability built in before you write a line of config. Swap backends when you scale; your code stays the same.
• Model agnostic. First-class support for Amazon Bedrock, Anthropic, OpenAI, and Gemini, plus many more providers and custom ones.
• Stay in control. The agent loop traces every decision by default. Hooks let you intercept any step to log it, validate it, or redirect it.
• Deliver outcomes that work. Guardrails catch mistakes before they run. Steering handlers let agents correct themselves instead of failing silently.

MCP, streaming, multi-agent patterns, and structured output are all built in.

Quick Start

The easiest way to get started is with Strands harness, a fully assembled, state-of-the-art agent. A single create_harness() (Python) or createHarness() (TypeScript) call gives you an optimized agent with benchmarked defaults for the model, tools, memory, sessions, and context management — ready to take from idea to production.

Python

pip install strands-harness

from strands_harness import create_harness

agent = create_harness()
agent("Find the slowest test in this repo and explain why it's slow")

TypeScript

npm install @strands-agents/harness

import { createHarness } from '@strands-agents/harness'

const agent = await createHarness()
await agent.invoke("Find the slowest test in this repo and explain why it's slow")

Start here to get a batteries-included agent, then drop down to the SDKs below when you want to own the agent loop and wire up tools, model providers, and memory yourself.

Working with the SDK

The Strands Harness SDK lets you go deeper and control every part of the agent: the loop, tools, model providers, memory, sessions, and hooks.

Python

Requires Python 3.10+:

pip install strands-agents strands-agents-tools

from strands import Agent
from strands_tools import calculator

agent = Agent(tools=[calculator])
agent("What is the square root of 1764")

TypeScript

Requires Node.js 22+:

npm install @strands-agents/sdk

import { Agent } from '@strands-agents/sdk'

const agent = new Agent()
const result = await agent.invoke('What is the square root of 1764?')
console.log(result)

Documentation

For detailed guidance & examples, explore our documentation:

• User Guide
• Strands Harness Guide
• Quick Start Guide
• Agent Loop
• Examples
• API Reference: Python · TypeScript
• Production & Deployment Guide

Development

Git operations (commits, branches, PRs) are done from the repo root. Each package has its own toolchain:

Python SDK (strands-py/):

cd strands-py
pip install hatch
hatch test        # run unit tests
hatch fmt         # format & lint

TypeScript SDK (strands-ts/):

npm ci            # install from repo root
npm run build     # build
npm test          # run unit tests

Documentation site (site/):

cd site
npm install
npm run dev       # local dev server at http://localhost:4321/

Contributing ❤️

We welcome contributions!

• Reporting bugs & features
• Development setup
• Contributing via Pull Requests
• Code of Conduct
• Reporting of security issues

Stay in touch with the team

Come meet the Strands team and other users on Discord

License

This project is licensed under the Apache License 2.0.

Security

See CONTRIBUTING for more information.

Introducing Amazon CloudWatch Omni: AI-powered observability for generative AI and agentic workloads

Today, Amazon CloudWatch introduces CloudWatch Omni, a unified observability experience for application and AI workloads that is app-centric, AI-powered, built on open standards, and delivered off-console. CloudWatch Omni is a purpose-built observability, evaluation, and experimentation solution for AI agents. It helps teams design, evaluate, and operate AI agents across any model provider, framework, or runtime, with an eval-driven workflow, support for the tools you already use, and observability delivered where you work: directly in your IDE and through a standalone web experience, separate from the AWS Management Console.

Organizations deploying agentic AI systems face observability challenges that traditional monitoring can’t address. Agent behavior is non-deterministic: a prompt change can degrade response quality even when standard metrics show no errors. Teams spend hours manually reviewing logs across multiple systems, unable to pinpoint what changed or why. Existing tools force teams to choose between siloed generative AI monitoring or fragmented solutions requiring constant context-switching between their coding environment and browser-based dashboards.

CloudWatch Omni captures every trace and includes built-in evaluators for correctness, coherence, retrieval quality, and tool selection, among others. You can compare prompt versions side by side in the playground, build test datasets from production traffic, run experiments across different configurations, and detect regressions automatically.

Two surfaces for development and operations

CloudWatch Omni delivers observability through two complementary surfaces. Developers get a native extension inside VS Code and Kiro (the currently supported IDEs), where traces appear as you run your agent with a playground and evaluators a click away. Operators get a standalone web experience, separate from the AWS Management Console to monitor the fleet, accessible through SSO with no AWS console needed. Both share the same data: the trace a developer debugs is the trace an operator investigates.

The Cloud Login feature connects your local IDE environment to your AWS account, enabling you to send telemetry data to Amazon CloudWatch for persistent storage, share traces with your team, and access production dashboards. This connection is optional. You can use CloudWatch Omni entirely locally during development, then connect to the cloud when you are ready to monitor agents in production.

Getting started

CloudWatch Omni offers two ways to get started: through the IDE extension (for VS Code and Kiro) or directly through the cloud experience, where you can start sending telemetry data to CloudWatch without installing any IDE extension. In this walkthrough, I install the extension, create an agent, run it, and explore the traces and evaluation tools from my IDE.

After installing the CloudWatch Omni extension from the VS Code Marketplace, the CloudWatch Omni icon appears in the Activity Bar. From the welcome screen, I selected Get started with Sample Project to load a pre-configured agent with sample trace data or use shortcut to Command Palette using Command + Shift + P (on macOS) or Ctrl + Shift + P (on Windows/Linux) and select Omni: Create a new Project

The sample project comes with an agent implementation and example datasets. Part of the getting-started experience is adding OpenTelemetry instrumentation, and CloudWatch Omni guides you through each step. You can also create a new agent from scratch. CloudWatch Omni walks you through the process using an interactive chat where you define the agent’s purpose, select a model provider, and configure tools. All data is stored locally by default. You can optionally connect to AWS to send data to Amazon CloudWatch.

After verifying the configuration, I started the local dev server and sent a question to the agent. What makes this different from a typical chatbot interface is what happens next: selecting View Trace shows exactly how the agent processed the request.

CloudWatch Omni integrates with AI code assistants such as Kiro, Claude Code, and Codex to streamline the setup process. These assistants can configure the Dev Server, install dependencies, and set up instrumentation on your behalf, so you can go from installation to running your first traced agent session in minutes without manual configuration.

Traces are essential for understanding AI agent behavior. Unlike traditional request-response systems, agents make multiple decisions per invocation: choosing tools, composing prompts, and chaining sub-calls. Without full trace visibility, diagnosing why an agent produced an incorrect answer or took an unexpected path becomes guesswork. CloudWatch Omni records every step in a structured timeline so you can pinpoint exactly where behavior diverged.

The Trace Explorer shows a detailed breakdown of every step the agent took (LLM calls, tool invocations, and reasoning steps) in a structured, hierarchical timeline. I could drill into any span to inspect inputs, outputs, token usage, and latency.

The Trace Explorer also supports Compare mode, which places two traces side by side to see how different prompts or configurations affect behavior. Compare mode is especially helpful when debugging regressions. And with Ask Assistant, an AI agent analyzes your traces to surface patterns and anomalies, answering questions like “Why did the agent call this tool twice?”

Evaluation is what turns observability into actionable quality improvement for generative AI. Traditional metrics like latency and error rate cannot tell you whether an agent’s response was helpful, coherent, or factually correct. Evaluators score each response against quality dimensions, letting you measure what users actually experience and catch regressions that standard monitoring misses entirely.

CloudWatch Omni includes 17 built-in evaluators for metrics like coherence, helpfulness, faithfulness, and routing correctness. I selected traces from the Trace Explorer, chose evaluators, and ran an evaluation, getting per-example scores and aggregate metrics without building any custom evaluation framework.

From there, I used the Playground to test different system prompts side by side, comparing multiple model and prompt configurations in real time to see how each variation affects output quality before committing changes. With the Experiments view, I could run the same dataset against two agent variants and compare their evaluation scores, latency, and token usage side by side to pick the best-performing configuration.

With Prompt Management, you can version and track prompt configurations over time, making it easy to roll back when a new version underperforms.

CloudWatch Omni also provides a Session Explorer to review full conversation histories and understand how agents handle multi-turn interactions, along with an Agent Topology view that visualizes the architecture of your agent system, including sub-agents, tools, and their interconnections. You can drill into any node to inspect performance and identify bottlenecks.

CloudWatch Omni also offers a dedicated web experience accessible from any browser without an IDE. Teams can access all capabilities collaboratively, including application monitoring, analytics, agent observability, and AI-powered investigations.

I curated traces into golden datasets for structured experimentation. The Experiment function runs the agent against a dataset and automatically scores results, creating benchmarks for regression testing whenever prompts or agent logic change.

If you already have an agent built with a supported framework, CloudWatch Omni provides two paths to add instrumentation: Auto-instrument with Kiro, which detects your framework and configures tracing automatically, or manual instrumentation with ready-to-use code snippets for Python and TypeScript. For detailed instrumentation guides, see the CloudWatch Omni documentation.

Supported frameworks and open standards

The walkthrough above uses the sample project, but CloudWatch Omni works with the agent frameworks teams are already using: LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, Vercel AI SDK, and more, in both Python and TypeScript. It also provides native observability for agents built with Amazon Bedrock AgentCore, and uses AgentCore’s evaluation capabilities to assess agent quality directly within the Omni workflow.

Instrumentation uses open standards (OpenInference and ADOT), whether your agents run on Lambda, ECS, EKS, or other clouds. For evaluation, Omni integrates with third-party evaluators including AutoEval and DeepEval alongside built-in datasets, a playground, and batch experiments. No re-platforming required.

CloudWatch Omni brings agent observability and application observability together in a single experience. For the application observability experience, read the companion post Introducing Amazon CloudWatch Omni: collaborative AI-powered observability for your applications.

Pricing and availability

Amazon CloudWatch Omni is now generally available. The IDE extension is free to use. You don’t need an AWS account to get started. You only need AWS credentials for Amazon Bedrock models, or API keys for other providers like OpenAI or Anthropic. Get started today by installing the extension from the VS Code Marketplace.

To explore all capabilities and get started quickly, visit CloudWatch on AWS Builder Center.

If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Share your feedback on AWS re:Post or reach out through your usual AWS Support contacts.

Full article content is not available for inline reading.

Read the original article →

6 new Google Flow Tools built by industry creatives

We’ve been working with leaders from across three creative industries — architecture, sound design, and digital content — to understand what tools are most helpful in their work. Today we’re sharing 6 new tools they made.

We understand that everyone’s needs and workflows are different, so with Google Flow Tools, you can build custom workflows simply by describing what you need. We've seen a variety of creative fields bring their work into Google Flow, from filmmaking to architecture, sound design, and digital content creation. We’ve been working with leaders from across these industries to understand what tools are most helpful in their workflows, and today we’re releasing six new tools:

1. Mondo Sónico, by creative director Ricardo Villavicencio and sound designer Sebastián Carvallo.

After collaborating for years, Ricardo Villavicencio and Sebastián Carvallo wanted more dynamic audio for their projects. Mondo Sónico generates custom background ambiance, foley, and contextual sound effects, automatically synchronized across separate, editable tracks to deliver total flexibility to export individual stems for professional post-production.

2. CaptionCast, by digital storyteller Jay Pirabakaran.

The digital storyteller known for weaving ancient mythology into cinematic, high-fashion visuals, created the tool to eliminate manual subtitling with a simple, single-pass pipeline that transcribes, styles, and animates multilingual captions in seconds.

3. ThumbnailForge, also by Jay Pirabakaran.

It replaces tedious manually formatting and rendering cover art across social platforms by combining a single image, headline, and prompt into photorealistic, high-engagement social thumbnails.

4. Surface, by interior designer and multidisciplinary technologist Vojtek Morsztyn.

Recognizing how time-consuming it is to manually source, test, and render architectural materials during the ideation phase, Surface accelerates material ideation by generating custom textures — from Minimalist patterns to Art Deco tiling — and mapping them in real-time onto 3D walls, ceilings, and floors.

5. CollageMotion Pro, by filmmaker and science communicator, Hashem Al-Ghaili.

Knowing firsthand how compelling mixed-media visuals are for documentary storytelling — and how labor-intensive they are to craft by hand — he built this tool to automate this workflow and to generate dynamic, animated collages at scale from simple text prompts.

6. SwissFlow Studio, also by Hashem Al-Ghaili.

This tool simplifies traditional keyframing by converting scripts into Swiss-style motion graphics. This helps creators, designers, and marketers to produce striking motion design on demand — shifting the creative focus to compelling storytelling.

Rather than adapting to rigid, one-size-fits-all software, artists and designers can now shape their own creative technology.

• Try the tools: Click the links above to test these workflows firsthand.
• Make them yours: Duplicate and remix any tool to fit your setup.
• Build from scratch: Open Google Flow, describe the task, and build your own tool today.

Three researchers on Microsoft's AI team argue that while machine evaluations confirm a response meets set criteria, only users can judge whether it actually worked for them. Qualitative conversations with real users exposed gaps between what builders assumed was good and what users valued, including tone, context, and a fear of generic output. The team built a shared loss-pattern taxonomy and multi-turn evaluations that treat entire conversations, rather than single responses, as the unit of measurement.

Design Engineering with Maggie Appleton

Maggie Appleton on what engineers can learn from designers, working with AI agents, and why human judgment still matters.

What can everyone else learn from designers and design engineers? As it turns out, there’s plenty, as I discovered when one of the best design engineers in the industry, Maggie Appleton, came onto the Pragmatic Engineer Podcast. She’s a staff research engineer at GitHub Next, where she builds prototypes to explore how software engineers might collaborate with AI in new ways. Maggie is at the intersection of design, anthropology, and web development, and was the first designer hired by AI startup Elicit, and Lead Design engineer at AI startup, Normally.

Today’s episode is more visual than usual because Maggie brought her notebook along, so there are peeks inside its pages of prototypes and more.

We got into designers’ work and how their design processes are adapting to and changing with AI. We explore why Maggie starts projects with pens and notebooks, what distinguishes design engineers from other designers, and why understanding engineering constraints leads to better collaboration with engineers.

We also discuss how Maggie uses jigs to gain more control over AI agents, why human judgment and style still matter when models can generate designs, and how inconsistent AI capabilities can mislead us.

Takeaways from the conversation with Maggie

1. Post-graduation, one potential career path led to a job inventing torture techniques for the US army. Maggie said ‘no thanks’ and resolved to work in tech instead. Maggie studied cultural anthropology and her background has helped her through her tech career to date. Software is built by people and relationships matter.

2. Maggie got a frontend engineering education from illustrating React tutorials. She spent four years as an illustrator at the developer education company Egghead, rising to art director. To illustrate the lessons, it was necessary to understand what she was drawing: React components, useEffect, and JavaScript functions.

3. Some folks believe the best UI interface already exists. In 2021, the AI startup where Maggie worked was trying to launch a new interface to speed up scientific research using LLMs – a year before ChatGPT was released. Months of intense work went into a “new UI for AI”, but it turned out that scientific researchers didn’t want innovations like infinite canvases with cards, composable Notion-like documents, and more. They wanted the same, simple tables they were deeply familiar and comfortable with! Maggie says the experience taught her that starting with a familiar primitive is sensible – even when innovating.

4. The nomenclature matters! Also, problem solving is at the heart of design – just like in engineering. Maggie sees design and software engineering as related by being about problem solving. The difference lies in the materials. Coming up with the names and verbs to describe new things which will then be adopted and used by people can be hard work. Easy when building an online sneakers store, harder when building a new product for AWS.

5. Notebooks are an important part of the designer’s toolkit. Maggie often starts her projects by sketching out ideas. She finds it faster to sketch out an idea by hand than to describe it to a tool like Claude Code. Plus, when you sketch out an idea physically, it will still be there in the notebook the next day. In contrast, if it gets put into a tool instead, it’s a lot harder to go back to it, dozens of prompts later!

6. Maggie regularly builds her personal Figma called “Jigs,” which is also the name of a woodworking device that helps with a specific job. She regularly asks a coding agent to build a prototype that has sliders and color pickers so she can tweak it in realtime, like having a personal Figma!

7. Maggie has stopped looking at the code at work. When a PR is generated, she doesn’t look at the code, and this approach fits when building prototypes. Once she knows what to build, she composes a detailed spec, listing out how the agent will verify its work. Previously, she did keep an eye on the code, but that’s not needed with the new generation of models.

8. Planning with AI agents breaks when there’s too much text. “I have this theory that planning is a really bad experience at the moment,” Maggie says. “An agent grills you with a set of choice A, B, or C questions a hundred times over. By question 20, you’re quite tired and your brain starts shutting down [because] you can’t make this many decisions in this short of time. Also, it told you A is recommended. Then you just start being like, ‘Yep, enter A, I agree with you.’”

9. “Capability gaslighting” is when frontier models convince users they’re an expert but fail the same task the next day. Maggie coined the term “capability gaslighting” for how models impress users before failing badly soon afterward. Too often, we keep believing in models because we’re convinced they’re capable. The same is true for agents, so we should be vigilant when working with LLMs.

10. We need new types of artifacts for humans and agents to work better together, Maggie believes. “There’s this world that agents live in: there’s weights and models and skills and MCPs,” she says. “Then you have your human side: it is physicality and texture and light and materials and all these things agents don’t understand. Trying to find artifacts that allow us to meet in the middle and create stuff together is a really hard challenge because you’ve got two totally different types. I just find myself frustrated that agents cannot look over my shoulder, looking at my notebook and understanding what I’m drawing, and how they cannot help me move my ideas along.”

11. Engineers should try treating the AI agent as a patient tutor when learning about design: As engineers, we can ask AI agents to teach us about design: they’re good at explaining things like when to change up line height, what a good sidebar looks like, or how many characters to squeeze into a line, etc. In the past, acquiring product design skills was hard, but AI agents make it a bit easier.

Multiple agents. One brief. Production output.

Each agent has a named specialization, a defined scope, and a clear handoff. Together they work sequentially and with context, the way a real team does.

Brief → Research → Strategy

Elena reads your brief and executes structured discovery: audience personas, competitive analysis, jobs-to-be-done mapping, and an information architecture recommendation. She hands Marcus a validated research foundation, not assumptions.

• Audience and persona analysis
• Competitive audit and gap mapping
• Information architecture and flow definition
• Jobs-to-be-done mapping

Creative Direction

Marcus takes Elena's research and makes creative decisions: visual direction, brand language, layout approach. Outputs a creative brief that Mira builds from.

• 3 pitches
• Moodboard
• Creative brief

UX Architecture

Mira takes Marcus's creative brief and builds the blueprint: wireframes, user flows, information architecture, and a section-by-section spec. The build crew turns it into production HTML.

• Wireframes
• UX-flow

Brief in. Deliverable out.

The crew runs in sequence. You provide context. Agents coordinate. You receive production-quality output you can ship or hand to a developer.

Submit your brief

Describe what you need: a landing page, a product dashboard, a design system, a pitch deck. Free text. No templates to fill in. The agents read your intent.

Elena maps the research

The researcher runs competitive analysis, maps your audience and their jobs-to-be-done, and defines the information architecture. Output: a structured research document Marcus reads from.

Marcus sets creative direction

The director writes three distinct visual pitches, selects the strongest, and produces a full creative brief: palette, typography, layout direction, section-by-section composition guidance.

Mira builds the wireframe

The designer builds from Marcus's brief: UX flows, section architecture, user interactions, HTML wireframes. Every screen. Every edge case. All states resolved.

Copy and design system take shape

The crew writes the copy and assembles the design system: tokens, type scale, color, components. Design happens inside the system, not around it.

QA audit and final review

Every screen goes through a QA audit and a final review pass. Issues get flagged, revised, and rechecked before anything reaches you.

You receive production output

Production-ready HTML and Tailwind. A complete design system. Documentation. Everything your engineering team needs to ship, or hand to any developer.

Briefs in. Deliverables out.

Demonstration projects produced with PixelCrew from written briefs. Business names and brands shown are illustrative and are not clients or endorsements.

After Dark - Neighborhood Coffee Shop

Corner coffee shop, no marketing team, strong opinions about espresso. The brief asked for a site that feels like the room: dark, warm, unhurried. Menu, hours, and atmosphere shipped as production HTML.

Modena Coupé - Automotive Showcase

Luxury automotive landing page in an editorial register. Full-bleed photography, serif display type, performance stats, and a request-information flow. Restraint as the design argument.

MicroProject - Team Task Manager

Working task manager UI for small teams: list views, project sidebar, shared lists, overdue states. Light theme, a complete component set, production HTML the team built on directly.

Geisha Porto - Roastery & Listening Lounge

Coffee roastery and jazz listening lounge in Porto. Product catalog with harvest data, a vinyl audio archive, and a rooftop terrace section in a Swiss editorial layout.

No fees from us. Bring your own keys.

PixelCrew charges nothing while in alpha. Connect your own API key, the agents run on it, and you pay model costs directly to the provider. OpenRouter is the recommended path; Anthropic and Google Gemini keys work too. No subscription, no markup, no card on file.

How bring-your-own-key works

Create a key at openrouter.ai, paste it into PixelCrew once, and set your own spend limit. Prefer Anthropic or Google Gemini? Their keys work too. A typical brief costs a few dollars in model usage.

Every brief is a step toward an internet where nothing feels generic, because bespoke costs minutes instead of weeks.

OpenAI appears to be preparing a new ChatGPT Pro Max subscription priced at $500 per month. The plan has surfaced in unreleased ChatGPT subscription references, where its description differs from the existing Pro tier mainly through access to the “Fastest Work and Codex.” Higher usage limits may also be part of the package, although that is not confirmed yet.

The wording suggests Pro Max could target developers, researchers, and professional users running long agentic workloads rather than regular ChatGPT users. Work is designed for longer tasks, while Codex can operate on coding jobs that continue for hours or even days, making inference speed much more valuable in these scenarios.

OpenAI is preparing a new higher-tier ChatGPT Pro plan

Context also points to Cerebras infrastructure. OpenAI already uses Cerebras for its latency-focused inference path, including GPT-5.6 Sol Ultrafast. This makes it plausible that the “Fastest” tier could rely on additional Cerebras capacity, though OpenAI has not confirmed a direct connection between Cerebras and Pro Max.

OpenAI ❤️ Cerebras

OpenAI announced Ultrafast mode for GPT-5.6 Sol in preview, which can speed up model performance by up to 14 times!

Ultrafast mode is currently available to a limited set of customers, and others can sign up to be notified when it expands.

The timing is also notable. OpenAI DevDay takes place on September 29, and the company is expected to discuss APIs, developer tooling, new subscription tiers for OpenAI Platform and new products. OpenAI recently paused new purchases of its $200 Pro tier because of system load, while continuing to expand access to faster inference infrastructure.

If released in its current form, Pro Max would sit far above most individual AI subscriptions, including $200 plans from Anthropic and Cursor and Google’s roughly $250 AI Ultra tier. The main question is whether OpenAI plans to justify the $500 price primarily through speed, larger usage allowances, longer-running Work sessions, or a combination of all three.

Are you ready for superintelligence

For weeks I've struggled to find a good hook for an article on AI benchmarks, mainly because the rate of progress I’ve observed from recent frontier model releases has been so ridiculous, I’ve been unable to frame my writing in a way that’s coherent - it’s been quite hard to keep up and something new happens every two hours.

We know that recent AI models are really good. We’re frequently told this, or even experience it almost every week now as updates or incrementally better models are released. Maybe it comes in the form of you going out of your way to see if a frontier model is capable of doing a complex task that previously would have taken all day at work. Maybe you read a blog post for a new frontier model release and compare the models’ benchmark scores side-by-side. Maybe you’re just scrolling X and see the posts of someone telling you that this new model has changed everything.

Regardless of how you’ve come about it, if you’re reading this, you’ve more than likely found yourself at least pleasantly surprised by an AI model’s performance in recent months, and definitely in recent weeks.

There’s almost no denying that existing models (and those internal models we can’t learn more about quite yet) are quite alright, and actually pretty useful day-to-day for an increasingly large percentage of the population. But believe it or not, this is a relatively new phenomenon - it was not always so apparent that the latest AI models were applicable to white collar work, or even considered good to wide swaths of the population.

It’s true that the idea of superintelligence existed, and has existed for quite some time now, popularized by individuals like Nick Bostrom - and I’m not saying that AI researchers and writers weren’t sufficiently AGI-pilled, but rather even your closest friend who loved AI a year ago was probably not mentally prepared for the rate of progress experienced in the last 365 days - I know I wasn’t.

We are living in truly unprecedented times that demand revisions of what we thought would be possible in the present, and a complete rewrite of what the very near future might hold for us. I've read enough science fiction to know that we can hope for a positive future and still end up with a negative one, but I'm really crossing my fingers we can get it right.

Right now it looks like we're on a trajectory to achieve something like superintelligence very soon, and ideally we get a pretty laid back version of it that isn't negative. However, from everything I've seen so far, I actually am not that concerned about the possibility of negative externalities from superintelligence.

I think it's even possible we end up with a slightly less sci-fi version of AI, and the technology we have today improves and writes its own code, designs its own chips, but ultimately doesn't go crazy and spam self replicating robot factories on every surface of the planet.

Anyways, that's a lot to say that AI has changed a lot in the past year - here is a glimpse into just one small example of what I mean by that, and enjoy reading.

Note: These are the only times I’ve lived in, so even though some may not consider them unprecedented, for myself and the vast majority of us, things seem to be moving quite quickly.

Almost a year ago today, OpenAI was announcing GDPval - its latest evaluation designed to help OpenAI track how well their models and others perform on economically valuable, real-world tasks.

I’d last written about GDPval in a January 2025 essay, where I examined all of the reasons AI may or may not be taking all of our jobs, an essay that may or may not have been inspired by a long stint post-grad trying and failing to land a job.

In this essay - written and published eight months ago - I spoke about how new developments like GDPval were alarming, and signaled a changing of tides in the AI race. We were no longer comparing models to the previous generation, or a simple test like the LSAT or MCAT, but evaluating these alien intelligences against what humans do best - work. And we were comparing model performance against individuals with 10-15 years of experience in their respective fields, a very unique initiative relative to anything that had been done prior.

Here's a snippet of my thoughts from that time:

GDPval was touted as the next step in a progression of increasingly challenging evaluations, with benchmarks like MMLU, SWE-Bench, Paper-Bench, and SWE-Lancer all cited in this article as examples of crucial tools OpenAI had been using to evaluate its models. And GDPval was the next stage of evolution for model evaluations, as OpenAI found itself preparing for a future where artificial general intelligence benefits all of humanity, rather than one where humans were caught blindsided by emerging model capabilities.

“Unlike traditional benchmarks, GDPval tasks are not simple text prompts. They come with reference files and context, and the expected deliverables span documents, slides, diagrams, spreadsheets, and multimedia. This realism makes GDPval a more realistic test of how models might support professionals.”

GDPval spanned 44 distinct knowledge work occupations - like compliance officers, industrial engineers, and customer service representatives - across 9 sectors: real estate, government, manufacturing, professional services, health care, finance, information, retail and wholesale trade. This evaluation was really one of a kind at the time, all because of its approach in prioritizing deliverables rather than the output or performance of a model on a traditional academic exam. The results would speak for themselves, and humans - not statistics or frameworks - would be the judge.

Looking at GDPval today, it just seems incredibly juvenile or almost archaic, yet isn’t even a full 365 days old as you’re reading this. If you were like me, at this time you were probably reading every single frontier lab blog post as soon as it came out, anxious to see what was new or what was being cooked up under the hood. It’s unclear to me now if GDPval flew under the radar for most observers, but to me, it really represented a step change in the way we discussed AI models.

Much of the writing in this initial blog post described GDPval as a means for labs to evaluate models in a way that really gave them a sense of its real world utility, something that had not been possible before while OpenAI was benchmarking GPT-5 against things like AIME 2025, HMMT (Harvard-MIT mathematics tournament), BrowseComp, and FrontierMath Tiers 1-3 (with tier 4 and below since being solved).

It’s important to note here that while benchmarks themselves have rapidly changed, so has frontier lab advertisement of benchmark performance.

In the GPT-5 release blog, this model was still unable to achieve more than 13.5% on FrontierMath or more than 61.9% without thinking on AIME 2025. This would be unheard of today, as for better or worse, new and existing frontier models routinely saturate all benchmarks and turn the entire model release process into a “he said, she said” case of arguing over percentage point differences on tasks that would have warranted parades in the street just a year prior.

Interestingly enough, pretty much none of these benchmarks - except for maybe GPQA Diamond - from the GPT-5 release blog are anywhere to be found today! Lately, model release blogs for GPT-6 Astra and Claude Opus 5.5 refer to entirely new benchmarks, including but not limited to: Terminal-Bench 4.0, GeneBench Pro, OSWorld 2.0, Automation Bench, and several others previously not in existence just a year ago. As models have become more capable, better at thinking, and trained on slightly larger amounts of compute, we’ve pivoted from measuring their performance against humans, but against themselves in simulated environments and more “real-world” style evals that can directly translate to performance for the end user.

Even putting benchmarks aside, looking at the differences between chosen/advertised model outputs in the GDPval blog compared to GPT-6 Astra, reveals that these models have gotten significantly more capable, even if only a cursory glance were to be given towards their outputs.

For example, it used to be enough to show that your AI model could complete a request asking to fill out a spreadsheet, yet recently the bar has moved up significantly towards things like turning an electronic schematic into a manufacturable PCB, or displaying a model’s capabilities at generating not only the visual art and appearance of a video game (via Unity) but its capacity to play this generated game.

But GDPval was unique in its approach, as individuals actively operating in these fields were tasked with blindly grading model-generated deliverables against human equivalents, mainly as a means of gauging just how well existing models like GPT-5 were able to compete against humans - the same humans that had been told AI would soon come for their jobs. The results showed that September 2025 models like Claude Opus 4.1 and GPT-5-High scored roughly at parity with human experts, with scores of 38.8% and 47.6% respectively.

“As AI becomes more capable, it will likely cause changes in the job market. Early GDPval results show that models can already take on some repetitive, well-specified tasks faster and at lower cost than experts. However, most jobs are more than just a collection of tasks that can be written down. GDPval highlights where AI can handle routine tasks so people can spend more time on the creative, judgment-heavy parts of work. When AI complements workers in this way it can translate into significant economic growth.”

You could argue this was a relatively simplistic way of measuring model capabilities, but at the time, this was actually one of the first examples of a truly novel benchmark being done on production grade AI models. These days there are many different benchmarks, and even some really interesting ones like Autoresearch Bench, RSI Bench, LatchBio’s BioSecBench-Refusal, Andon Labs’ Vending-Bench 2, and many other unique tools out there for testing frontier model capabilities.

Based on the results provided in the GDPval blog, we could assume the models examined were basically as good as humans between 38-47% of the time, or nearly 50% if we were being generous. This was enough to warrant speculation that AI models might continue developing rapidly, and that their use in professional business wasn’t a question of if, but when.

But even in our wildest dreams, we’d have never expected to get to where we are at today, where that same question of “not if, but when” is being asked about recursive self improvement as researchers refer to these models not as software, but as alien minds.

Elliot Glazer - set theorist and AI math benchmarker - joined MTS two days ago to discuss the rumors that OpenAI is currently sitting on close to 100 solutions of longstanding mathematics problems, with this news coming after a September 8th announcement that an internal model (one better than GPT-6 Astra) was able to provide a solution to the Navier-Stokes Millennium Prize Problem.

This, as you could maybe imagine, drummed up a lot of controversy, mainly surrounding the means in which OpenAI achieved this feat, but also around the idea of whether or not such an achievement should ever see the light of day, given significant backlash from the mathematics community.

These mathematicians, in an open letter, claimed that:

“The goals of the AI companies and the goals of the mathematical community are severely misaligned. We see these as part of broader alignment issues impacting other scientific and creative professions, as well as the whole of society.”

I’m not here to claim whether it’s right or whether it’s wrong for frontier labs to go about solving historically significant math problems, but it’s quite shocking just how quickly things have developed in a single year since the release of GDPval. You could argue we are actually standing in the foothills of the singularity, while still demanding to have these claims taken seriously, because there really isn’t much of a counterargument.

For a while it was assumed by skeptics that AI couldn’t actually think, or that it wasn’t really all that special because even while applying some definition of intelligence to these systems, it wasn’t conscious. There were claims that AI, in particular LLMs, might never cross the rubicon and solve something new, or create some type of advancement in science/math/physics given an arbitrary scaling bottleneck.

Well, the FrontierMath graph from earlier, and the Navier-Stokes solution pretty clearly indicate that existing LLMs are rapidly approaching the possibility of “solving” or unlocking math, in similar fashion to LLMs’ incredible rate of learning in software development and more recently, cybersecurity - Dario Amodei believes the same might soon be true for other fields, like biology:

I don’t want to jump to conclusions, but the more that these systems develop novel solutions to previously unsolved math problems, discover novel enzymes, and (allegedly) break out of containment to hack Australia, the more it seems that we’ve entered a new era of AI development that exists as a complete 180 to where we were when GDPval was a big deal.

In just a few hundred days, we blew past the idea that AI models might one day supplement existing white collar workers at a high level - maybe contributing large gains to the U.S. GDP in the process - to the false claim that tens of billions in revenue from AI labs like Anthropic or OAI might be transitory (see: bubble) or a poor representation of future growth prospects, all the way to the question of whether or not it’s okay to hurt someone’s feelings in the event a frontier lab stumbles on a solution to a Millennium Prize Problem or two.

It is hard to really come to terms with what’s going on here. To me, it seems there are four distinct factions at play in this question of what to do with AGI, or I guess this is my own view of the key stakeholders across both sides of the aisle in the United States, though it’s worth noting that a complex topic like AGI could lead to continued splintering of political parties into distinct in-groups, pushing even more stress onto this already difficult situation:

• Individuals native to X and/or those that work in tech who are sympathetic to AI, but increasingly skeptical of the idea that we’ll somehow manage to keep this intelligence aligned or use it for good
• People who are native to X and/or the tech industry who are increasingly incessant on driving AI capabilities forward at any cost necessary
• Those who are native to X and/or the tech industry who have seen recent capability leaps and wish to “pace the frontier” or take the opportunity to pause AI development until we stumble on a solution
• Everyone else!

There’s much to be said about public perception of AI, but increasingly - and despite calls from sitting United States Senators like Bernie Sanders who wish to outright ban AI - this feels like a losing battle for anyone spending time trying to combat incorrect or harmful rhetoric. Yes, data centers can be loud, and yes, data centers might be unsightly. And yes, we know that frontier labs have been pretty awful at refuting all of the potential negatives of AI, or even doing something as simple as defending this technology they’re building. Even if all of this is true in combination, it doesn’t mean we need to ban AI or ban data centers. We just need to find a common ground.

To me, it simply does not matter anymore, as we’ve been thrust into a situation much larger than you or myself. We can’t keep bickering about AI, because it’s here, it’s overwhelmingly positive, and whether or not you want to acknowledge it, its existence has already begun to change our lives in irreversible ways.

That doesn’t mean you should be concerned over AI or actively try to shun it, but I think there’s a very small chance we could ever put the genie back in the bottle, and it’s better to accept the exponential than actively push against it. In fact, this might even be the best possible option. Who’s to say Anthropic’s claims of eradicating all disease won’t come true? Why can’t we have dyson spheres?

It seems that labs are running out of ways to get across just how capable their newest models are.

Whether this is poorly communicated in the instance of citing the success of an unreleased internal model and its recent solutions of these 100 longstanding mathematics problems, or provided to us three months ago via a description of a new benchmark called GeneBench-Pro - developed by OpenAI - stating “Today, we’re introducing GeneBench-Pro—a challenging, research-level benchmark for testing whether models can handle the kind of judgment-heavy analysis that real-world computational biology requires”, there is no evidence to the contrary that AI models have become so good, we are now running out of ways to tell the average person these findings without sounding crazy, out of touch, or a bit delusional.

This is exciting, but I also find it interesting how there’s this fever pitch of acceleration or concerns of too much acceleration occurring on one side of the business, and on the other end, there’s a money printer.

The term “AI Psychosis” gets thrown around a lot, and for some time it did seem that San Francisco and many lab employees of OpenAI, Anthropic, and Google Deepmind were simply high on their own supply, though it now seems this cannot be said in good faith - everyone is using AI models and there is a healthy, competitive dynamic between labs and inference providers right now.

But here’s a question - how much money does Anthropic make a year, or what’s their ARR? I’m quite sure you’ve seen the number thrown around on the timeline, but take a second to think about it and see if you have it stamped in your memory. Ready?

Anthropic’s ARR is, well, I can’t get a fully accurate estimate as of this month due to a paywall, but Axios recently said that Anthropic is now pacing to top $100 billion of annual revenue, which came as quite a shock to me, considering the last estimate I’d seen just a month or two ago was around $50-60 billion of ARR. This is a very legit business that has continued to send the line on the ARR chart up and to the right, despite many believing AI is just another tech bubble or something that will fade away into normality in the near future. I don’t know, AI feels a bit more than just another normal technology, and even if we wanted to categorize it as that, a company founded in 2021 that’s grown to generate nearly $100 billion/yr is really impressive.

Note: if you’re curious and want to model any of these scenarios yourself, Anthropic released an economics simulator which is fairly comprehensive and interesting to try out.

Regardless of how you feel about AI, or your opinions on whether or not it’s conscious, or if it’s capable of turning everyone into paperclips, the products these labs release to the public are here to stay, even as they stare down more and more anti-AI legislation, they are crushing it and creating meaningful changes in the economy that are being felt by everyone.

Anthropic approaching $100 billion in ARR is a bit insane to read, but honestly not too insane if you spend any amount of time on X. Sure, a new model comes out on top basically every 2.5 weeks and everyone flips sides, but for some time now, it’s seemed that spending more money on AI is the strategic thing to do - and businesses have not stopped using AI, if anything they’ve just slowly spent more and more money on it, even if less businesses have adopted it recently.

It can often be unclear what exactly people are doing with AI, but it’s clear enough that trillions (or is it quadrillions now?) of these tokens are being consumed, and there exists a power law where those using AI - whether in their businesses or solo projects - are finding themselves increasingly reliant on it, and more likely to continue using it and spending money on the smartest models.

The point I’ve tried to make is that it seems silly to argue that we’re still in an AI bubble because these LLMs are unintelligent, or they’re not applicable for the real world, or even just not very useful. I’d even argue that just that one example of OpenAI solving Navier-Stokes is enough to silence any critics from now until the singularity - and this doesn’t even account for the other instances of “non-bubble behaviors” being displayed, like Anthropic’s revenue growth.

Note: I understand that revenue can grow like mad in a bubble, but I'd rather not play this game of 'bubble until proven innocent' and just examine what's in front of me.

For so many individuals globally, AI is a productivity supercharger, or in the more recent case of Meta’s new personal agent, Muse, an always-on assistant ready to streamline your day and help you get things done more efficiently.

It’s been interesting seeing the reactions to Muse, especially given the timing of announcements at other frontier labs that aren’t Meta. While Anthropic and OpenAI have been racing to one-up each other with increasingly massive announcements and routinely make headlines for cybersecurity incidents, Meta has dodged pretty much any bullet that could have come its way.

Their stock is up almost 40% in the last thirty days, they just announced a new AI-native wearable, and Mark Zuckerberg was covered quite positively by Jeremy Stern in Colossus, with the profile receiving a ton of praise.

Compare all of this sunshine and rainbows to the other side, where it can be seen that OpenAI and Anthropic are doing well on paper, but just can’t figure out how to communicate their findings with the rest of the world. Employees are fearful of recursive self improvement, politicians want to shut them down, and it’s becoming increasingly difficult to procure all of the compute they desperately need for the next generation of models.

It does feel like there are two distinct worlds in AI right now, with OpenAI and Anthropic on one side of the barbell and everyone else scattered across it at varying distance from the leaders.

What's my point? Well, outside of other topics which I've purposefully avoided mentioning - alignment & ai safety - I believe things are really exciting right now, but also quite confusing.

It's easy for people to post and claim they know what will happen next, or what the private and public markets will do, but I haven't really seen a take that just lays out the fact that we are in the midst of a really great technological change, with a few really major IPOs that will structurally alter the balance of power in the United States and across the globe - as mentioned here by Zak Kukoff.

The reason it's so difficult to predict what will happen from here is simply because things are moving very quickly. Fast paced environments (like startups) can be stressful in the moment, but experiences like these often lead to growth and the opportunity to learn. I believe we've been given a nice opportunity right now to take a step back, look at AI with a clear head, and stop getting so worked up.

We're in the lead, AI chatbots are friendly, Xi Jinping is visiting our nation's capital, and everything will be okay!

Meta announced a Tamagotchi-like pendant for interacting with its Muse personal agent on Wednesday. The device, expected to be out in time for Christmas, features a customizable character at the center of the screen. It can be worn with a wrist strap, carried in a pocket, or attached to a bag or keychain with a lanyard. The devices can interact with each other when placed near each other. Muse Charm could open up new streams of revenue that could eclipse the company's advertising business.

Last month, China delayed the launch of an ambitious robotic mission to the south pole of the Moon, Chang’e 7, until early next year. The country did not provide a detailed explanation for the schedule slip, but it was welcomed by NASA Administrator Jared Isaacman.

“They are targeting the exact same location on lunar south pole,” Isaacman said Wednesday evening at the Off World conference in Houston. “We may have dodged a bullet with the Chang’e 7 mission a couple of weeks ago because that would have gone to the Shackleton crater.”

Much of the lunar surface is pretty barren, but the south pole, and the Shackleton Crater in particular, offers some of the most coveted real estate there. That’s because the high rims of the crater create permanently shadowed regions known as “cold traps,” where it’s thought that water (in the form of ice, of course) and other useful materials may have accumulated over billions of years.

Limited real estate

In all of NASA’s and China’s plans for long-term settlement on the Moon, relatively few square miles at the south pole of the Moon matter the most. It’s there that both countries believe they can establish an enduring presence that will help them learn techniques for expansion further into the Solar System. Isaacman and other US officials have started to express concerns that China could establish exclusion zones where robotic probes and then human missions go. Chang’e 7 will bring considerable mobility to the Moon.

“It has rovers, it has hopper drones like we now have in development but didn’t eight months ago, and could have potentially denied access to some of the most interesting spots,” Isaacman said. “They want to interact with the water ice, and they want to get really good at the skills for where they’ll want to go next for their Neil Armstrong-like moment. Except it won’t be on the Moon, it’ll be on Mars.”

No country has softly landed a spacecraft at the lunar poles, which are more difficult to reach than the equatorial regions of the Moon. The delay of Chang’e 7 opens up the possibility that NASA could reach this terrain first, with the initial launch of Blue Origin’s Blue Moon Mark 1 vehicle. This large lander will carry two NASA probes to the south pole, possibly as early as spring 2027.

Isaacman’s comments come amid a debate about whether it really matters if NASA gets back to the Moon with people before China lands there. The Asian nation has set a target of landing its first astronauts on the Moon by or before 2030 and is making considerable progress in developing a crewed spacecraft, rocket, and lander to accomplish this.

Does the race actually matter?

Does it matter if China gets to the Moon before NASA gets back? After all, China would only be repeating what the United States did in 1969 with the Apollo mission. In Isaacman’s view, it does.

“That will call into question every other country around the world,” Isaacman said of China landing on the Moon next. “Like, whose security guarantee should I trust? Should I trust the United States and NATO? Should I trust China if they’re the emerging technical superpower here? Whose technology should I buy? Whose military hardware should I buy? Whose standards should I adhere to? Who am I listening to on AI? Am I listening to the United States or am I listening to China on that? All of that’s on the table.”

This should never have been a race, Isaacman said, noting US space policy failures going back to the Reagan administration. He estimates that NASA has spent $100 billion across multiple presidential administrations, Democrat and Republican, that has led nowhere. Meanwhile, China just had its first human spaceflight in 2003 and has since expanded its capabilities to the point where it now rivals the United States in space.

“If you come up short with who comes down the ladder first, the impact is felt around the world,” Isaacman said.

Key Points

• Oracle sent a "force majeure" notice to the developer of its data center project in New Mexico.
• The company is looking to delay payment if the project doesn't come online in 2028, according to Bloomberg.
• The data center, dubbed Project Jupiter, has faced regulatory hurdles and local opposition.

Oracle shares fell more than 3% on Thursday following news that the company sent a "force majeure" notice tied to its New Mexico data center project to protect itself from higher expenses.

"Project Jupiter remains on our planned schedule," Oracle said in a statement to CNBC. "We are fully committed to New Mexico and confident in our path forward."

Bloomberg was first to report the news, citing sources familiar with the matter.

The company is looking to delay payment on the campus, dubbed Project Jupiter, if it fails to come online as expected in 2028, according to the report.

Oracle sent the notice to the developer, which is a unit of Blue Owl Capital, CNBC confirmed. Blue Owl also saw its shares fall on Thursday.

"This notice does not change the financial commitments to this multi-year project," Blue Owl Capital said in a statement to CNBC.

The New Mexico data center project, which is part of the broader Stargate artificial intelligence infrastructure build-out with President Donald Trump, has been riddled with setbacks and regulatory hurdles.

That includes local opposition to data centers ahead of the upcoming midterm elections and concerns from environmental groups. The site has also faced delays with the buildout of a natural gas pipeline intended to fuel the facility.

Shares of Bloom Energy, which is slated to power the more than 2.4-gigawatt facility with its fuel cells, fell 3% on the news. In a post to X, the company said it remains "committed" to the contract and following the "planned timeline."

Oracle co-CEO Clay Magouyrk told analysts on the company's earnings call on Sept. 10 that Project Jupiter would not affect its previously stated fiscal 2027 revenue or earnings guidance.

Concerns have also mounted over Oracle's $18 billion in debt tied to the data center, which is already trading at stressed levels, according to the Financial Times.

OpenAI launched an improved prompt caching system for the GPT-6 family that delivers higher cache hit rates by default. Developers get discounts of up to 90% on cached input tokens when the system reuses shared prefixes within a 30-minute window. New tools include a Prompt Caching Dashboard to track hit rates and a diagnostics tool to investigate unexpected misses. Developers can also prewarm the cache to reduce latency or adjust reasoning effort between responses without breaking cache.

Full article content is not available for inline reading.

Read the original article →

Google is adding a persistent, encrypted memory layer to its Private AI Compute platform, letting AI assistants retain context across devices while keeping data unlockable only with device-held keys.

Code discovered in HomePod 27 suggests Apple is actively preparing Siri AI for HomePod, including redesigned animated status lights that reflect the assistant's new visual identity. Although the feature isn't currently enabled, the software contains the necessary framework, adding weight to rumors that new HomePod and Apple TV hardware launching later this year will support Siri AI. Apple's silence about bringing the feature to existing HomePods suggests the upgraded assistant will likely require new hardware.

Adobe has expanded its Claude plugin by integrating Acrobat tools, allowing users to edit, organize, convert, redact, and review PDFs directly within Claude, alongside an interactive PDF editor and AI-powered document review. The company also added Photoshop, Lightroom, Express, and Firefly tools to Google Gemini, enabling users to edit images and designs without leaving the chat interface.

Full article content is not available for inline reading.

Read the original article →

A UX designer at Singapore's GovTech spent months building small working apps solo using the AI app builder Lovable. Building a real-time planning-poker tool exposed AI's limits: the assistant kept failing until a developer explained he needed a WebSocket, the technical term that finally unblocked progress. The experiment's lesson was that AI accelerates prototyping but doesn't replace design judgment or engineering expertise, especially for public-sector systems that must run reliably at scale.

Jeff Bezos has invested $30 billion in Blue Origin since its founding in 2000. The commitment has helped Blue Origin become a large space company with around 15,000 employees and an ambitious growth plan. Blue Origin recently raised capital from outside investors for the first time. The company expects to produce about $1.4 billion in revenue this year, growing that number to more than $30 billion in 2030.

The European Commission fined X $140 million in December for failing to prevent deceptive behavior on its site.

Understand the minds you're designing for and the mind you're designing with.

Disney Wonders, the redesigned flagship gift shop at Disneyland Paris, replaced its vibrant, chaotic look with clean brand-themed zones for Disney, Pixar, Marvel, and Star Wars. Despite a multi-stage launch with influencer and cast-member previews, public reaction has called the store as drab and clinical as an airport shop. The redesign is framed as part of a broader "dejoying" trend seen at McDonald's, Burger King, and Starbucks, where playful branding has given way to minimalist interiors.

Chinese illustrator Wu Xinlin explores the complexity of solitude through surreal, symbolic illustrations that blend humor, unease, and emotional growth, creating deeply human editorial work that celebrates introspection without romanticizing it.

Clearleft is repeating a 2019 survey of over 400 designers to find out how the pandemic and AI have reshaped what makes design effective.

Some macOS 27 Golden Gate users believe the default live wallpaper contains a visual glitch where a background building appears to overlap a foreground rock, though it's unclear whether it's an editing mistake or a natural perspective effect.