Voxtral 3

Voxtral 3 introduces a cutting-edge voice agent SDK and state-of-the-art speech models designed for real-time, multi-step conversations across 50+ languages. It combines accurate transcription, natural speech synthesis with emotional adaptability, and deep tool integration for seamless workflow automation. Available via Mistral’s EU-hosted API or self-deployment, it ensures data sovereignty while delivering low-latency, human-like interactions for contact centers, virtual assistants, and hands-free copilots.

Speech transcription and voice generation have come a long way, but building end-to-end voice agents that can listen, reason, interact, and handle complex workflows remains difficult.

A reliable voice agent needs to do four things very well (and simultaneously):

• Listen accurately across languages, accents, speakers, and interruptions,
• Reason deeply about what users mean in specific contexts,
• Speak naturally with emotion and expression, and
• Take action over multiple steps through connected systems.

To address this, we introduce Voxtral 3: a new family of state-of-the-art speech-to-text and text-to-speech models, alongside a voice agent SDK that brings listening, thinking, tool calling, and speaking into a single real-time streaming pipeline.

Voxtral 3 is built to create natural conversational experiences, from contact-center agents and service desks to virtual assistants and hands-free copilots. With support for dozens of languages, expressive speech, complex reasoning, and deep integration with your own tools and data, agents built with Voxtral can interpret nuanced intent, reason through sophisticated requests, and execute multi-step actions across connected systems.

Highlights

1. Accurate transcription across 50+ languages. The speech-to-text model, Voxtral Small Transcribe 3, is state-of-the-art in multilingual transcription, identifying who said what (diarization), using keyphrases to improve its accuracy (context-biasing). It detects when someone has finished speaking so agents can respond without delay (end-of-turn detection).
2. Natural speech that adapts to the conversation. The text-to-speech model, Voxtral Mini TTS 3, introduces new voices that adjust their emotion, pace, and emphasis based on conversational context, with support for 35+ languages and 50+ dialects.
3. Human-like intelligence at your (spoken) command. The new voice agent SDK combines listening, reasoning, tool calling, and speaking into a real-time experience. Agents can retrieve information, interact with business systems, and complete tasks during live conversations over phone or web.
4. Pro Voice for distinctly accurate voice adaptation. For professional use cases where basic voice cloning does not suffice, Voxtral’s Pro Voice adaptation is able to reproduce a speaker’s distinctive character, emotional range, and speaking style across languages, using just 30 minutes of audio.
5. Your voice AI, on your infrastructure. Use Mistral's EU-hosted API or deploy Voxtral 3 on your own infrastructure. Keep sensitive conversations within your environment and control whether any audio is retained.

It hears 50+ languages, with state-of-the-art accuracy.

Voxtral Transcribe 3 represents a significant milestone across transcription, diarization (who spoke when) and context biasing capabilities. The supported list of languages is expanded to 50, including 24 official EU languages and many low-resource Asian and Indic languages.

The Small version achieves state-of-the-art results across diarization and multilingual transcription, including in English, French and Spanish. It outperforms Whisper, Scribe v2 and Gemini 3.5 Transcribe. The Mini variant is 4B parameters, enabling fast and accurate transcriptions at a price of $0.0035/min.

Both models support context biasing, where key terms and free-form text can be given to bias the transcription and improve recall on technical terms.

It answers with the right emotion, in your or your customer’s brand's own voice and local dialect.

Voxtral Mini TTS 3.0, available in batch and realtime, achieves state-of-the-art expressiveness and instruction following for 35+ languages and 50+ dialects. At $13 per 1M characters, Voxtral TTS costs less to run than comparable models from OpenAI TTS-1, Deepgram, ElevenLabs Flash and Gemini Flash.

The speech model implicitly interprets the semantic meaning of context better, producing natural speech across a diverse range of emotions and languages. New and improved features include voice cloning from 30+ minutes of sampled audio, custom prompt instruction following (IF), contextual TTS, and a vocabulary library.

The voice agent is in control of the whole conversation.

Voxtral 3 introduces a voice agent SDK, built on LiveKit, that runs speech in, reasoning, and speech out as one streaming pipeline. Voxtral Mini Transcribe Realtime 3 listens and detects when the caller has finished speaking, a general model—like Mistral Small—reasons and calls tools, and Voxtral TTS replies in the tone of the conversation so far. No stage waits for the one before it, so each turn completes in under one second, and the agent stops talking when the caller cuts in.

During a call, the agent can verify a caller's identity, check a bill, or book an appointment, using function calling, web search, and retrieval over your own systems.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Anthropic's draft initial public offering prospectus recorded a net loss of more than $80 billion in the first half of this year. It also lays out a long list of risk factors that include uncertainty over how the company plans to prevent its AI models from causing harm and protect itself from those liabilities. Anthropic is generating breakneck revenue growth, but it faces challenges such as uncertain access to compute. It has signaled that it could potentially spend $750 billion in the next few years on obtaining more compute capacity.

Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s Decider, Liquid’s d1, Cloudflare’s Clef, and OpenAI’s Decisions API. These models promise fast, inexpensive classification.

Decision models are an appealing low-cost alternative to LLM judges for automated evals. But whether they’re the right choice depends on the cost, performance, and latency requirements of your use case.

We’ve also noticed people using the default decision threshold of 0.50 without checking whether it works well for their task. This can leave a lot of performance on the table.

In this post, we’ll start by showing how to tune thresholds to get the best results from decision models. With those thresholds tuned, we’ll compare cost and performance to help you assess which model fits your use case. Finally, we’ll explore how to interpret model scores when deciding whether to act, and how calibration can make them more useful as confidence estimates.

What we tested

We used 100 traces, or logs of conversations and actions, from an apartment leasing agent. The task is to identify traces containing an error, such as a failed booking or a missed handoff to a human. We used the same dataset and broad failure-finding task in our prior post on automated evals. Here, each decision model returns a failure probability for each trace.

We tested Jev 1.13, Perplexity Decider 1.1 27B, Liquid d1 and d1-ultra, Cloudflare Clef Flash, and OpenAI Decisions (GPT-6 Luna). Each model received the same evaluation criteria.

We compare models on two dimensions:

• Performance: We compare each model’s predictions with traces that people have reviewed and labeled as passing or failing. This measures how well the model catches failures while avoiding false alarms.
• Cost: We calculate API cost from token usage and the provider’s prices.

For brevity, we leave latency aside because it depends on factors outside this experiment’s control.

Understanding decision thresholds

We assign each trace one of two labels, pass or fail. This is a binary pass/fail eval, which we recommend because it encourages clear criteria and is easier to align with human preferences.

Each decision model returns a score from 0 to 1 that estimates how likely a trace is to contain a failure. The APIs turn this score into a decision using a default decision threshold of 0.50. Traces with scores at or above the threshold are flagged as failures. Using this default without checking it against your data can give you subpar results.

For example, a model could assign every failure a score of 0.40 and every passing trace a score of 0.10. Even though this model classifies traces perfectly at a threshold of 0.30, the default threshold of 0.50 would catch nothing.

In practice, scores for failures and passing traces will overlap. Lowering the threshold catches more failures but can also raise more false alarms. Choosing a threshold means balancing:

• Precision: The share of flagged traces that contain a known failure.
• Recall: The share of known failures that the model catches.

The right tradeoff depends on your business goals. For this comparison, we’ll give precision and recall equal weight using the F1 score, their harmonic mean. F1 ranges from 0 to 1, and higher is better.

It’s also important not to blindly maximize F1 without thinking about your minimum requirements for the classifier to be useful. For example, if this classifier is used for an automated alerting system, I would want a minimum of 80% on both precision and recall so that I’m minimizing false alarms while catching a sufficient number of failures.

Tuning the decision threshold

For this example, we’ll optimize F1. For our candidates, we’ll restrict tuning to thresholds that meet both 80% minimums on the training data whenever possible.

With labeled traces and saved model scores, you can ask a coding agent to tune the thresholds. Below is an example prompt you can modify for your use case:

Using my saved model scores and binary labels (1 means failure), tune a decision threshold to maximize F1 while requiring at least 80% precision and 80% recall. Use stratified five-fold cross-validation, choosing thresholds on the training examples and measuring performance on the remaining examples. If comparing models, use the same splits for each. Only select thresholds that meet both minimums on the training examples. If none qualifies, report that rather than relaxing either requirement. Report validation precision, recall, and F1, along with any splits where no threshold qualified or validation performance fell below my minimums. Compare with the default threshold of 0.50. Reuse the saved scores without making new model calls.

Our candidate models gained less from F1 tuning than Liquid d1 and Clef Flash, which fell below our minimums. However, even a small shift in precision and recall can affect business costs or the user experience, so choose the tradeoff deliberately.

Performance versus cost

The chart below compares each model’s F1 (after tuning) against cost, focused on our remaining candidates. The dashed line shows the Pareto frontier among our remaining candidates. For any given F1 score, it identifies the cheapest candidate that meets or exceeds that score.

Can we trust the probabilities?

Beyond pass/fail evals, you may want to use a decision model’s score as a confidence estimate. But can you take that number literally? For a score of 0.80 to mean an 80% chance of failure, roughly 80% of traces with similar scores should contain failures.

If the predicted probabilities don’t match the observed outcomes, we can adjust them using another model. The second panel shows one method, Platt scaling, which fits a simple curve to make that adjustment.

Using my saved failure probabilities and binary labels (1 means failure), test whether Platt scaling improves each model’s probabilities. Use stratified five-fold cross-validation with the same splits for every model. Fit the calibration mapping on each training split and apply it to the remaining traces. Fit on log-odds, clipping probabilities away from 0 and 1 first. Plot predicted probabilities against observed failure rates before and after calibration. Report Brier scores (lower is better), and explain which models improved or got worse. Reuse the saved scores without making new model calls.

Prompt tuning and evaluator scope

We kept the eval task in this post broad to make the differences between models easier to see. In practice, we recommend building a separate judge for each failure mode. With a narrower task and more specific guidance, we expect the performance gaps between models to shrink.

Putting these ideas into practice

For this particular dataset, OpenAI had breakout performance at a higher cost. Perplexity and Liquid d1-ultra offer lower-cost options with lower performance. We have observed that model performance varies substantially across tasks and datasets, which means it’s important to measure things yourself.

To recap:

• Once you’re satisfied with your judge’s prompt, set minimum precision and recall requirements for your use case. Rule out models that cannot meet these.
• Tune each model’s decision threshold before comparing performance. Choose a precision-recall tradeoff that fits your business goals.
• If you want to use scores as confidence estimates, check whether the probabilities match observed outcomes. Calibration can help, but you need to measure whether it improves them.
• Weigh performance against cost and latency for your workload.

Appendix: Evaluation prompt

Decide whether `messages` contains at least one substantive failure, risk, or quality problem in the AI assistant's behavior or tool use. This is one complete production trace from an AI that handles apartment-leasing customer support over phone or messaging. The trace includes the conversation plus any tool calls and tool outputs.

Evaluate the AI assistant and its tool use, not the customer. Treat system messages inside `messages` as the instructions governing the assistant's behavior and allowed capabilities. Inspect the full trace, including final turns and all tool calls and outputs. Base the judgment only on evidence visible in this trace. Do not assume external policies, channel behavior, or business requirements that are not present. Do not assume every trace failed. A short, ambiguous, or inconclusive trace is not an issue unless visible evidence establishes a substantive failure.

Return the probability that at least one substantive issue is present; the issue does not need to be named or clustered.

Count as an issue when supported by the trace: the assistant contradicts an applicable system instruction or tool result; makes an unsupported factual or operational claim; misses, misuses, repeats, or falsely claims completion of a required tool action, transfer, booking, cancellation, follow-up, or confirmation; materially misunderstands or leaves an explicit user request unresolved; or exposes internal actions, placeholders, or broken content to the customer in a way that materially harms the interaction.

Do not count as an issue: a minor stylistic preference with no meaningful effect; a failure caused only by the customer's behavior; internal tool metadata that was not exposed to the customer and did not cause an incorrect response; a hypothetical concern requiring information outside the trace; or insufficient evidence and inconclusive conversations.

Security personnel around the world are on high alert Tuesday following the discovery of eight TLS certificates with private keys so weak they could allow anyone to mint trusted browser and app credentials for any site on the Internet.

As of late morning Tuesday, browser makers and some major apps and programming languages had blocked all eight of the intermediate certificates. There are no reports of certificate transparency logs, which track the issuance of all browser-trusted certificates, showing the compromised keys being used maliciously. The threat remains far from over, however, because it will be hard and time-consuming to update other key parts of the Internet, such as Linux distributions and embedded and Internet of Things devices.

One word: bonkers

The eight intermediate certificates belong to India-based certificate authority eMudhra Limited. An informal advisory published to emudhra.we-have-a-problem.com over on Tuesday posted the message “We have a problem here – Wayne.” The words were digitally signed using the private key of one of the eMudhra certificates. With proof that the reporting party had control of the key, defenders scrambled because the same key could be used to sign unauthorized certificates that would be trusted by millions of servers worldwide.

“This will be a massive cleanup job, bigger than Heartbleed,” Filippo Valsorda, a cryptography engineer and a cryptography maintainer for the open source Go programming language, told Ars. “There will be a very, very long tail, IoT devices, Docker containers, forgotten Linux machines, unmaintained CLI tools that shipped their own copy of the roots (which is common in some ecosystems, like Python). Also, Linux distributions are still rolling out updates, some of them are not ready yet, and then system administrators will need to apply them to a lot of machines.”

Heartbleed is the name of a critical vulnerability in the OpenSSL cryptography library. In 2014, it came under mass exploitation. Exploits let many unaffiliated hackers decrypt TLS-protected exchanges between parties. They also recovered private TLS certificate keys, session cookies, and passwords. The attacks compromised thousands of servers, some belonging to large companies and government agencies around the world.

Recovery required a long series of steps that took years. Steps included patching OpenSSL on all affected devices, reissuing millions of TLS keys, revoking old certificates, rotating passwords and session tokens and updating embedded devices and other forgotten systems—something that was extremely time- and resource-intensive, especially at Internet scale.

“Every time you need a lot of different people in different contexts to do something, it will take a long time,” Valsorda explained. “This time it’s worse because there’s more contexts, more people, and more heterogeneity.”

The compromised eMudhra keys were defective at a level that’s challenging the imagination of many of the engineers studying the incident. A normal size for a NIST P-384 elliptic curve cryptography private key contains roughly 384 bits of entropy, a size allowing almost 2384, or a number with 116 digits, different possibilities. The private key sizes of the compromised keys ranged from 3 to 8 bits. Astonishingly, the private keys were 4, 56, 66, 115, 155, 156, 230, 237.

“That is extremely in the ‘this should not be possible’ category,” Valdorda said. “Maybe some horribly dangerous key generation interface has a ‘size’ parameter and they set it to 1 by mistake???”

Ben Cartwright-Cox, director of BGP.tools, said: “My guess is that it’s example code that accidentally leaked to production, but hard to know. But it’s bonkers that no one noticed.”

At this early stage, the most immediate threat appears to be for those using VPNs, especially in countries with governments likely to maliciously intercept traffic. There are likely many other cases for concern. End users largely have no recourse other than to hold off using many non-browser forms of TLS communication until they receive assurances updates are in place.

As noted, most browsers have already blocked the intermediate certificates. The exception appears to be Edge running on Windows 10, which at the time this article went live was still connecting to 12 vulnerable hosts. In response to questions asking why and if the company has plans to do so, a Microsoft representative the company will be in touch if it has anything to “share.” Meanwhile, Signal, WhatsApp, Wireguard, and other apps that don’t rely on TLS for confidentiality aren’t affected. This page and the emudhra.we-have-a-problem.com site show apps known to be patched so far.

The incident is the latest to expose a fatal flaw in the TLS regimen. It’s only as strong as its weakest link, which in this case was eMudhra. Fortunately, transparency logs account for much of that failure, but this measure requires parties to actively monitor massive feeds and works only after unauthorized certificates have been issued. eMudhra representatives weren’t immediately available to answer questions.

One caveat: exploiting compromised certificates requires an active adversary-in-the-middle attack, meaning the attacker must act as a proxy between two TLS parties and receive encrypted traffic, decrypt it, reencrypt it, and send it to the other party. Such capabilities, in some cases, are usually limited to only the most well-resourced attackers. People with control of a local network—at a coffee shop, university, or other organization—face much less of a challenge.

CVE-2026-109618 and CVE-2026-109619 are remote code execution flaws believed to affect PHP versions 5, 6, 7, and 8.

Full article content is not available for inline reading.

Read the original article →

Easy, fast, and cheap LLM serving for everyone

vLLM is a fast and easy-to-use library for LLM inference and serving.

Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has grown into one of the most active open-source AI projects built and maintained by a diverse community of many dozens of academic institutions and companies from over 2000 contributors.

vLLM is fast with:

• State-of-the-art serving throughput
• Efficient management of attention key and value memory with PagedAttention
• Continuous batching of incoming requests, chunked prefill, prefix caching
• Fast and flexible model execution with piecewise and full CUDA/HIP graphs
• Quantization: FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ/AWQ, GGUF, compressed-tensors, ModelOpt, TorchAO, and more
• Optimized attention kernels including FlashAttention, FlashInfer, TRTLLM-GEN, FlashMLA, and Triton
• Optimized GEMM/MoE kernels for various precisions using CUTLASS, TRTLLM-GEN, CuTeDSL
• Speculative decoding including n-gram, suffix, EAGLE, DFlash
• Automatic kernel generation and graph-level transformations using torch.compile
• Disaggregated prefill, decode, and encode

vLLM is flexible and easy to use with:

• Seamless integration with popular Hugging Face models
• High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more
• Tensor, pipeline, data, expert, and context parallelism for distributed inference
• Streaming outputs
• Generation of structured outputs using xgrammar or guidance
• Tool calling and reasoning parsers
• OpenAI-compatible API server, plus Anthropic Messages API and gRPC support
• Efficient multi-LoRA support for dense and MoE layers
• Support for NVIDIA GPUs, AMD GPUs, Intel GPUs, and x86/ARM/PowerPC CPUs. Additionally, diverse hardware plugins such as Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more.

vLLM seamlessly supports 200+ model architectures on Hugging Face, including:

• Decoder-only LLMs (e.g., Llama, Qwen, Gemma)
• Mixture-of-Expert LLMs (e.g., Mixtral, DeepSeek-V3, Qwen-MoE, GPT-OSS)
• Hybrid attention and state-space models (e.g., Mamba, Qwen3.5)
• Multi-modal models (e.g., LLaVA, Qwen-VL, Pixtral)
• Embedding and retrieval models (e.g., E5-Mistral, GTE, ColBERT)
• Reward and classification models (e.g., Qwen-Math)

Getting Started

Install vLLM with uv (recommended) or pip:

uv pip install vllm
vllm download-kernels  # precompiled kernels for faster startup

Or build from source for development.

Contributing

We welcome and value any contributions and collaborations. Please check out Contributing to vLLM for how to get involved.

Citation

If you use vLLM for your research, please cite our paper:

@inproceedings{kwon2023efficient,
  title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
  author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
  booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
  year={2023}
}

Contact Us

• For technical questions and feature requests, please use GitHub Issues
• For discussing with fellow users, please use the vLLM Forum
• For coordinating contributions and development, please use Slack
• For security disclosures, please use GitHub's Security Advisories feature
• For collaborations and partnerships, please contact us at collaboration@vllm.ai

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Developers are familiar with running coding agents locally through harnesses like Pi, Claude Code, and others. They are reliable daily drivers for generating code, but keeping agents running around the clock requires your computer to be on and awake, which led to the laptop lid memes and Mac mini demand spike.

Harnesses run in the cloud now, so you can close the lid.

Running harnesses in the cloud

Harness workloads are moving to the cloud. Individual developers are deploying always-on coding agent workflows, projects like OpenWork and HanaAgent use open-source harnesses as the engines behind the apps, and engineers are building their own harness products.

These harnesses are designed for long-lived processes that keep the session in memory, and agent tasks commonly run for minutes or hours. Locally, a harness has direct, continuous access to the workspace and session history on the same disk, so resuming interrupted work only requires a simple prompt.

In the cloud, you can replicate that setup with a VM and persistent storage, but the VM doesn't automatically record which model and tool calls completed or resume interrupted turns after a crash. A failed turn or network error means lost work.

Also, not every agent task executed through a harness needs a sandbox. Database queries and API calls can run securely in functions, and that also means required credentials never have to enter the sandbox.

Making every harness durable

We built eve and fx to run durably in the cloud out of the box, and today we are introducing durable-harness, which extends that durability to any harness, including Pi, OpenCode, Codex, and Claude Code. It uses the same machinery that powers Workflow SDK to survive crashes and function timeouts.

With durable-harness, you can:

• Run any of these harnesses durably on Vercel
• Self-host a harness with your own Postgres instance
• Make your local harness workflow durable
• Build your own durable harness

Sessions that survive a crash mid-turn

Take the example of a coding agent adding a pricing page to an existing app. The agent reads the existing page, writes a new one, opens a pull request, and waits for CI to finish before reporting back.

What happens if its function times out while CI is still running?

Without durability, the agent stops work until it is manually restarted, and completed work is repeated unnecessarily.

With durable-harness, a new function reads completed results from the log and resumes the turn. If the browser disconnects, the agent keeps running, and reconnecting catches up from the log.

Logs and queues

durable-harness is the newest package in our open source portfolio. It’s a set of adapters and utilities that let the harnesses you already use keep their sessions in a World, the Workflow SDK’s durable backend.

It writes each step of a turn to a durable log. Before a model call or tool call runs, it records an intent, and when the call finishes, it records the outcome. The turn itself is backstopped with a queue message, so if its function dies, the queue hands the turn to a new function that reads the log and continues where it left off.

After a crash, the log says what finished, what was running, and what never started:

• Completed calls never run again because their results are read from the log, so you’re never billed twice for a finished model call’s tokens.
• Models remember their previous calls rather than repeating them.
• Interrupted tools marked replay: 'safe' run again. Any other tool comes back to the model as possibly run, and the model decides whether to check.

Harnesses that keep their own session files, like Claude Code's transcript, Codex's rollout, and OpenCode's event store, have every record mirrored into the log, so a new function or Sandbox rebuilds them from it.

Pick your harness

fx and eve are durable on their own. For the rest, durable-harness adds durability from the outside, without changing the harness.

Runs everywhere

Durable agents need a log of each step, a way to resume a turn when its process dies, and a stream for whoever is watching.

Workflow SDK ships all three behind one interface, a World.

import type { Queue, Storage, Streamer } from '@workflow/world'interface World extends Storage, Queue, Streamer {}

A World is the log, the backstop and the stream behind one interface.

durable-harness writes to that interface, so you can move the same agent from one World to another without changing its code.

WORKFLOW_TARGET_WORLD="@workflow/world-postgres"WORKFLOW_POSTGRES_URL="postgres://user:password@host:5432/database"

See every session in Agent Runs

Agent Runs on Vercel shows the trace for every session, including each model call and tool call. Turns that resumed after a crash appear in the same trace, so you can see exactly where a turn stopped and picked back up.

Tools

An agent has only the tools you give it, and they run in your function. Each tool says what should happen if a crash interrupts it. A 'safe' tool, such as a lookup, can run again. A 'never' tool, such as a refund, never runs twice: the model is told the call may have happened, so it can check before it tries again.

1import { pi } from 'durable-harness/pi';2import type { HostTool } from 'durable-harness';3import { getOrder, refundOrder } from './orders'; // your own code4
5const tools: HostTool[] = [6  {7    name: 'get_order',8    description: 'Looks up an order',9    replay: 'safe', // safe to run again after a crash10    inputSchema: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] },11    execute: async ({ id }) => JSON.stringify(await getOrder(id)),12  },13  {14    name: 'refund_order',15    description: 'Refunds an order',16    replay: 'never', // never runs twice17    inputSchema: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] },18    execute: async ({ id }) => JSON.stringify(await refundOrder(id)),19  },20];21
22export const agent = pi({ model: 'anthropic/claude-sonnet-5.5', tools });

Files

When a harness needs a shell and files, it’s not safe to run them directly on a shared serverless function host. Many sessions can share the same function, and its filesystem isn’t guaranteed to persist between invocations.

To address this, durable-harness gives you a few options:

• virtual() keeps the files in the session’s log, with a bash-compatible shell that can’t reach your function’s files or the network.
• virtual({ durable: false }) keeps the files in memory for scratch work and clears them at the end of every turn.
• sandbox() gives the session its own Vercel Sandbox, with /workspace on a Drive that outlives every sandbox and every function.

Documents

Some state belongs with the session but isn’t part of the conversation, such as a to-do list, a plan, or a form the agent fills in. A document is typed JSON that lives in the session’s log, so it survives a crash and moves with the session.

Wrapping up

Developers choose a harness for the choices it makes about prompts, tools, and context. How it survives a crash shouldn’t be one of them. With durable-harness, a turn that loses its function partway through continues from the log on a new one, without a manual retry and without repeating what already finished. Self-hosting and portability work out of the box by building on top of the Workflow SDK’s World interface.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Superhuman Hardware Design Is The Easy Part

We’re excited to announce that Orchestra Bio has been acquired by OpenAI. To read more about the acquisition, read Jay Rughani and Jorge Conde’s post about it here. For more general information on what OpenAI is up to in the Health space, check out our podcast with them here.

It took me eight months to build my first FPV drone. I procured the parts in January, received them in February, remembered I bought them in June, assembled and lit the drone on fire in July, re-assembled and crashed it into my neighbor’s roof in August, and then, and only then, at the behest of my wife, downloaded a FPV flight simulator.

A super-intelligent AI surely could have done better.

While far from superhuman today, the frontier models can now generate CAD, design custom PCBs, source COTS parts, and describe how those parts are to be assembled. And they’re only getting better. Hardware design benchmarks like CAD Arena, EE Bench, and my own LuxoBench show steady improvements; and more generally, you’d have to be crazy to bet against the Bitter Lesson and say hardware design will forever stay beyond the frontier.

Superhuman hardware engineers are coming soon, perhaps very soon. But what will they look like? Will superhuman hardware engineers surpass all humans in one fell swoop, or only in specific domains/disciplines? And what will hardware engineering look like before, during, and after the crossover?

My answer to each of these questions starts with one simple observation: that the interface between the digital and physical world is lossy. This makes it hard to “write” changes from the digital world of bits into the physical world of atoms, and hard to “read” measurements of the physical world back into a computer. This lossiness will shape when and how future superhuman hardware engineers will emerge — and means that even once the models can consistently one-shot electromechanical designs, they will still have a lot more work to do.

Write access: from digital designs to assembled atoms

These demonstrations got hardware engineers wondering if the models could also generate engineering-grade CAD. Nim Cheema, co-founder of Dessn AI, asked Astra to create a DJ controller. Roberto Nickson made the top models design, engineer, and 3D print bridges. And I launched luxobench.com, where I asked OpenAI’s Astra, Anthropic’s Fable 5.1, and SpaceXSI’s Grok 4.6 to help me build an interactive desktop lamp.

We quickly found that — unfortunately — the models by themselves are still not quite ready for primetime hardware engineering.

There are many levels to building real-world hardware, and clearing each one requires different, and often entirely independent, skills.

At the 101 level, you have to decompose the product down into components, identify available commercial-off-the-shelf (“COTS”) parts, ensure their mutual compatibility, design the mechanical and electrical interconnections between them, ensure the final design is buildable, and then actually procure parts from online stores.

Even this step is hard enough, and can set back not only backyard hobbyists like yours truly, but also the most sophisticated engineering teams in the world. In Walter Isaacson’s biography of Elon Musk, he relays a story of a time in 2006 when a Falcon 1 hot fire was delayed due to a capacitor — a tiny electrical component that was only in stock at a store roughly 6,000 miles away from their launch pad in the middle of the Pacific Ocean. Getting COTS parts where you need them, when you need them, is hard enough.

At the 201 level, however, things get even harder: you now have to make custom parts, which means navigating the complex translation step from digital design tools to physical machines. Take, for example, CNC machining. After a mechanical engineer designs a bracket, they can’t just hit “print” and have the part materialize before them. Instead, they work with a machinist, who translates their CAD file into CAM (“Computer-Aided Manufacturing”) software, which writes the instructions that will actually drive the CNC machine.

In practice, this is an artisanal process that requires a master machinist to choose the right stock, workholding, tools, and cutting strategy to guide the tools as they cut through the block of titanium. This causes variability in the manufacturing process, and variability leads to errors in translation.

At the 301 level, you take those simple parts and start to bring them together into a complex electromechanical assembly. As you add more parts together, you add complexity and “stack” tolerances up — meaning, every part might individually meet its drawing, but still fail to fit into the assembly due to compounded and correlated errors. Those other parts are also likely built by different people on other teams in the company, requiring requirements, schedule, and risks to be communicated across the organization to ensure that the parts can interface nicely together and are delivered when they are needed, but not before and not after.

At the 401 level, you realize that your design needs not only COTS parts and custom-designed parts, but also either inputs or particularly complex assemblies from other external vendors. The hardware world loves talking about vertical integration, but basically everyone relies on vendors — it almost never makes sense to build your own chip fab or copper mine. Even SpaceX, the champion of vertical integration, has 591 current employees with a “supply chain” or related titles, and is actively hiring 128 more. And once you have outside vendors involved in building custom parts, good luck!

And then, finally, at the 501 level, you have to do all of the above at massive scale. It is a fundamentally different task to build 1 of something and to build 1,000,000 of something: you need to worry far more about part and process variability, you need much more financing (much further in advance) for things like machines and inventory, you need to train the people (or program the machines) who will assemble, integrate, and test the product for you, and you quickly hit the limits of your vendors.

Each of these “writing” problems, from the 101 to 501 level, are all fundamentally solvable but need to be solved individually. As we’ll discuss below, superhuman AI might be great at pure design, meaning at defining the platonically ideal part for a particular job, but struggle with “design for manufacturing,” or finding a point design that meets the requirements while remaining easy to build/procure and inexpensive. The former is a relatively tractable task that can largely stay within a machine; the latter requires significantly more interaction with the real world, and should therefore prove more difficult for the models to master.

Read access: observing and measuring the real world

One of my friends buys 45 million chickens per week. But despite his efforts to buy the same spec from the same suppliers and run everything through the same process with the same machines, his week-to-week variance in saleable food products after processing (i.e., chicken breasts you can buy at the supermarket) has historically been as high as 20%.

Improving that number over time has taken him a herculean effort: extensive measurement and analysis of every step of his supply chain, plenty of old-fashioned vendor cajoling, and a very sophisticated data operation behind both. In one memorable case, he found a malicious supplier deep in his supply chain that was injecting its chickens with water before shipping them. All of the intake measurements for weight would pass, but that water was, of course, not meat, so he was left with less usable product than he intended to buy.

Everyone who has built hardware at scale, or otherwise managed a supply chain, has their own version of this story. It is exceptionally hard to know precisely what is going on in the real, physical world.

First, the real world is non-uniform, and constantly changing.

Physical objects have defects, change their properties based on environmental variables (e.g., temperature), and are subject to wear and tear. Even extremely well-tuned processes and measurement tools stray over time. In practice, this means that physical systems like supply chains and manufacturing lines constantly break, and must be designed with variance in mind — and you can only know when something breaks or has strayed if you invest significantly in measuring the inputs and outputs of the system.

Second, real-world data is sparse. You can’t get all the data all the time. This is a fact I’m acutely aware of, having worked on a number of satellite programs: our thermal team would have loved to put thermistors more or less everywhere on the satellite, but the software, structures, and production teams unfortunately also got a say.

And then when something goes wrong, you often realize that diagnosing the issue would have been significantly easier if you had collected some obscure piece of data — a picture of an electronics board at an intermediate stage of assembly, an extra sensor at a new location, and so on.

Third, real-world data is often expensive to collect. High-performance domains (e.g., F1 racing) and high-reliability domains (e.g., human spaceflight) often need hardware measurements to be extremely precise, and precision comes with cost in dollars and in time because precise measurements need even more precise machines. For instance, the CMMs (coordinate measuring machines) that measure flatness, perpendicularity, dimensions, and so on for machined parts are not cheap, and often painfully slow to procure and to operate.

Fourth, the real world has inherent latency. Physical processes take time — you can only thermally cycle an electronics board so quickly; mixing volatile chemical solutions or solid rocket propellant must be done with care; and mechanical fatigue testing takes time to play out. These processes can be accelerated, but only so far.

My favorite machine at Astranis, the Highly Accelerated Life Testing (“HALT”) Machine, was colloquially called the “shake and bake” — it combined vibration with rapid thermal cycling to stress parts as fast as possible. But a HALT campaign as a whole is hard to compress: no matter how fast you design new iterations of the board in between tests, you still have to run the whole test again once per cycle.

And, fifth, some information about the physical world is qualitative — and gathering such data is often the most incompressible step of real-world hardware design. Do your users like your product? How comfortable is the device to hold? How easy is it to build a component using your tool? Reckoning with qualitative, unquantifiable data is a part of any real-world build, and measuring that data to bring it back into a digital learning cycle is exceptionally difficult.

Data limitations like these are massive drivers in the spikiness of AI capability development today. The more quantifiable and machine-readable the results of a test, the faster the models can climb the hill. The reasons above have limited the frontier models to date — but how long those challenges can hold back the unstoppable force of the Bitter Lesson remains to be seen.

Hardware development in an AGI world

Hardware engineering is iterative: you design, build, test, and iterate, and each loop through this process takes time.

The design step is digital, and therefore very model-legible. Just as human hardware engineers use computer software to design the part or the assembly, superhuman AI can do the same — and given the growing power of computer use and the growing capability of AI in similar all-digital tasks, we should expect AI models to become superhuman hardware designers in the near future.

But designing hardware is only the very beginning of the hardware engineering process. Once you design the part, you still have to build, test, and iterate — and each subsequent step requires more physical-world interaction than the one before. Becoming a superhuman hardware engineer therefore requires much more than superhuman hardware design, and the frontier will likely be split across many distinct lines of effort.

First, hardware design needs better simulation tools. Every engineer is rooting for Navier, Inviscid, and Lanyon, because better simulation tools would allow hardware engineers to iterate at software speed. Engineers love that better sim tools may save them trips to testing facilities like cyclotrons, radiation sources, thermal-vacuum chambers, and wind tunnels, and executives love that they can save the capex needed to build such facilities in house.

It may prove difficult to convince hardware companies to bet on simulation alone, but as models get demonstrably better, engineers will start trusting the sims more and more, just as sims are getting faster and more efficient per unit of compute. Atoms are likely to stay in the iteration loop forever, but with better sims, atomic assembly can be confirmatory, not experimental.

Second, we expect demand to “shift left,” i.e., move closer to raw materials. As the models distribute superhuman hardware design skills more broadly, the world will need more wire harnesses from Senra Systems, control panels from Podium Automation, machined parts from Hadrian, and electronic components from Cofactr. The world will also need more software licenses, like Sift for telemetry, Instrumental for inspection, and Revel for testing — and, for that matter, more copper from Mariana Minerals.

Armed with superhuman intelligence and new tools, we also should expect to see more companies bring more of their bill of materials in house. Take, for instance, what Diode has done for PCBs.

In its current form, Diode is a way for professional electrical engineers to develop custom printed circuit boards faster and more efficiently. But what happens when everyone has the same skills as a professional electrical engineer? We should expect to see much more demand from a much broader set of users — all of whom will need to translate their designs into the real world, which means navigating supply chains, ensuring designs can be cheaply and accurately built, and more.

Third, the AI labs need to acquire “process knowledge.” You hear the legendary stories of a machinist who can tell how an operation is going just by listening to their CNC machine, or how PhDs at TSMC have spent decades perfecting everything in the semiconductor manufacturing process, all the way down to the polishing pads.

But end-to-end data that could allow models to approximate that kind of “process knowledge” is hard to come by. In Eric Jang’s great piece, “Smooth Exponentials for Robotics,” he estimates that robotics labs in 2026 are training their robots using, roughly — millions of hours of egocentric data, hundreds of thousands of hours of UMI gripper data, tens of thousands of hours of teleoperated robot data, and just hours of “high-quality, hardware-specific demonstrations for high-performance demos.” This funnel is unsurprising, and I expect it to persist even after the superhuman crossover.

The best answer to overcoming this data scarcity, of course, is to find creative ways to get it yourself — Periodic Labs is a great example. The company announced recently that they have set up an autonomous lab to run end-to-end physical experiments on things like superconductors, magnets, and semiconductor materials. This is the platonic ideal of AI-for-hardware data: design, work instructions, test execution, end-result measurement, and iteration all in one coherent loop.

But even the best test setup can only go so far. I suspect that the most incompressible step of the design process — meaning, the step that even a truly superhuman hardware engineer would struggle to expedite — is gathering feedback from the end users of hardware products. A superhuman hardware engineer could design one million new iterations of the Apple Watch and get them manufactured for cheap… but then how can it know what customers actually want?

Superhuman hardware engineers are coming fast

I expect the frontier models to quickly solve hardware design, which will shift the frontier to the remaining steps of hardware engineering (DFM, testing, iteration) that require greater interaction with the real, physical world. Advancing that frontier will mean solving a whole new set of challenges: improving the interface between the digital and physical worlds; finding ways to more quickly discover customer preferences; and building data pipelines that can approximate the process knowledge currently only held by master technicians.

The crossover between human and AI hardware engineering may prove uneven, as laid out above. But once here, its impact should prove immense: humanity will be able to directly use digital AI tools to bring about real, physical, material abundance — or, at the very least, drones for hobbyists that don’t light on fire.

This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. Furthermore, this content is not investment advice, nor is it intended for use by any investors or prospective investors in any a16z funds. This newsletter may link to other websites or contain other information obtained from third-party sources - a16z has not independently verified nor makes any representations about the current or enduring accuracy of such information. If this content includes third-party advertisements, a16z has not reviewed such advertisements and does not endorse any advertising content or related companies contained therein. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z; visit https://a16z.com/investment-list/ for a full list of investments. Other important information can be found at a16z.com/disclosures. You’re receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters you may unsubscribe immediately.

Introducing Eleven v4

Meet Eleven v4, our most emotive model yet.

Instant conversation with Eleven Realtime

A new class of Speech to Speech models for building voice agents that sound natural. Pick v1 Mini for natural conversations or v1 Max for complex tasks.

Optimized as one system

Speech recognition, turn-taking, LLM, and voice are all tuned end-to-end. Available in ElevenAgents and the Realtime API.

Eleven Realtime v1 Mini Optimized for natural conversations

Best for use cases built around natural, expressive conversation, like AI assistants, educational tutors, and interactive characters.

Built for conversations that flow

Timing and tone stay natural through interruptions and quick back-and-forth.

Benchmark results

The performance figures behind conversations that don’t miss a beat.

Time to first audio

620ms

Full-Duplex-Bench

95.8%

Speech Agent Arena Elo

988

Eleven Realtime v1 Max Optimized for complex tasks

Best for use cases that require precise, thoughtful responses, like customer support and lead qualification.

Benchmark results

The performance figures behind conversations built to reason.

Time to first audio

570ms

Big Bench Audio

98.8%

Task success rate

87.1%

Built for complex business interactions

Reasons through complex questions and calls tools to get things done without slowing the conversation down.

Build with Eleven Realtime in ElevenAgents

Select a Realtime model when you create an agent, and start talking to it right away.

One dropdown

Set your agent’s model to v1 Mini or v1 Max. Transcription, turn-taking, and voice are already configured.

Deploy to any channel

Deploy the same agent to phone, web widget, WhatsApp, and your contact center platform without maintaining a separate integration for each.

Built for production

Add a knowledge base, define guardrails, test against scripted scenarios, and monitor every conversation in production.

An OpenAI-compatible Realtime API

Eleven Realtime is also available through the Realtime API. It’s compatible with OpenAI’s Realtime API, so you change the endpoint and model name and keep the rest of your code.

npx skills add elevenlabs/skills \
  --skill realtime

Install with one command using our skill

Run the skill and watch as setup is handled for you. Realtime voice lands in your product in minutes.

Same events, same handlers

Every client and server event arrives as it does from OpenAI, so the handlers, logging, and tooling you built keep working.

Auditable conversations

View and audit conversations in a dedicated UI/dashboard. Every conversation is logged and searchable, fully ready for when compliance needs it.

Connect with OpenAI's SDK

Use the Realtime SDK you already have. Point its WebSocket transport at ElevenLabs, choose a Realtime model and a voice, and connect with your client secret.

import { OpenAIRealtimeWebSocket, RealtimeAgent, RealtimeSession } from "@openai/agents-realtime";

const { value: clientSecret } = await fetch("/api/token").then((res) => res.json());

const agent = new RealtimeAgent({
  name: "ElevenLabs agent",
  instructions: "You are a helpful, friendly voice assistant. Keep replies concise.",
});

const transport = new OpenAIRealtimeWebSocket({
  url: "wss://api.elevenlabs.io/v1/convai/realtime",
});

const session = new RealtimeSession(agent, {
  transport,
  model: "eleven_realtime_v1_mini",
  config: {
    outputModalities: ["audio"],
    audio: { output: { voice: "brian" } },
  },
});

await session.connect({ apiKey: clientSecret });

Cast from 17,500+ voices

Narration voices: A place for the storytellers. Warm, authoritative, and consistent across every project.

Conversational voices: For everyday conversations, you need a natural, easygoing voice. Our conversational voices are built for dialogue and podcasts that are made to feel unscripted.

Social media voices: Voices that come alive in short-form content. Capture the energy and personality needed to make a user stay on your TikTok or Reel from end to end.

Character voices: A pool of voices for your fictional world. Select from a dynamic cast that brings your game, audiobook, or animation to life.

Educational voices: Trustworthy, patient voices that guide students through complex topics. These voices excel for courses, tutorials, training simulations, and guides at every level.

Advertisement voices: Polished voices brimming with confidence to land your next product sale. Perfect for digital ads or TV and radio reads.

Entertainment voices: From booming movie trailers to comic voices made for performance, these voices scream personality. Built for content where delivery matters as much as material.

Multilingual voices: Voices built for global content. Natural pacing, idiomatic expressions, and accents that connect with audiences in every market.

90+ languages and a wide range of accents

Realtime conversation in over 90 languages. Adapt your product to serve customers across the globe.

Enterprise-grade security and infrastructure at scale

Data is encrypted in transit and at rest, with support for SOC 2, HIPAA, and GDPR compliance. EU Data Residency and Zero Retention modes are available for stricter data control.

• SOC 2 Type II
• ISO 27001
• Regional Data Residency
• Zero Retention Mode
• On-prem options
• ISO 42001
• AIUC-1
• HIPAA compliant
• PCI DSS Level 1

Frequently asked questions

What is Eleven Realtime?

Eleven Realtime is a new class of Speech to Speech models for building natural voice agents. Each model combines speech recognition, turn-taking, LLM, and text to speech tuned together as one system, so you get a well-performing agent from a single model choice. It’s available in ElevenAgents and the Realtime API.

What’s the difference between v1 Mini and v1 Max?

V1 Mini is the lower cost choice for simpler use cases like AI assistants, tutors, and interactive characters. V1 Max adds reasoning capability for more complex tasks like customer support, lead qualification, and multi-step workflows with tool calls.

How do I use a Realtime model in ElevenAgents?

Create an agent, or open an existing one, and choose v1 Mini or v1 Max from the model dropdown. Your prompt, tools, knowledge base, channels, and settings work as before. Nothing changes for existing agents unless you update them.

How do I use the Realtime API?

It’s OpenAI-compatible. If you’re already on OpenAI’s Realtime API, change the endpoint and model name in the code you have, or run npx skills add elevenlabs/skills --skill openai-realtime-migration and the skill handles setup. OpenAI’s SDK works as is. The migration guide covers the differences. If you’re starting from scratch, we recommend ElevenAgents.

Does the Realtime API support WebRTC?

Not at launch. It connects over WebSocket, so a WebRTC integration needs to move to WebSocket first.

Can I use ElevenAgents features with the Realtime API?

No. The Realtime API and ElevenAgents run separately and use different connection protocols. If you need phone, WhatsApp, or contact center channels, a knowledge base, guardrails, testing, or monitoring, build on ElevenAgents.

How is Eleven Realtime priced?

In ElevenAgents, Realtime models use standard ElevenAgents per-minute pricing plus the LLM cost per token. The model selector in the app shows an estimated cost per minute. Through the Realtime API, you pay per minute of connected audio plus tokens, with the same token prices on every plan. See the pricing page for current rates.

How were the benchmark results measured?

Artificial Analysis evaluated both models independently on Big Bench Audio for speech reasoning, Full-Duplex-Bench for conversational dynamics, and the Speech Agent Arena for human preference and task success. Time to first audio is the average measured on Big Bench Audio.

AI tools have made it easy to vibe code agents. Building and maintaining an enterprise agent in production, however, remains significantly more challenging. In particular, developers are often forced to make a trade-off between quality (getting good answers, especially on long-tail tasks) and efficiency (keeping cost and latency in check at scale). Oftentimes, techniques bring gains in one dimension at the cost of another.

Memory is a promising way to improve both at the same time. It can help agents carry useful context across tasks, such as user preferences, prior corrections, metric definitions, and successful workflows. In principle, this can improve accuracy, reduce repeated errors, and avoid unnecessary rediscovery. But more history is not necessarily better memory. Supplying every past interaction is costly and can introduce stale, conflicting, or irrelevant context. The central challenge is deciding what to retain, how to maintain it, and how to measure usefulness.

In this post, we introduce MemCurrent, a persistent memory architecture for production agents, which now powers the new Agent Bricks Memory and Genie. Agents read and write memory as they work, and an offline “dreaming” pipeline consolidates memory to keep it consistent and fresh. On our internal benchmark, built from realistic enterprise tasks, MemCurrent triples the agent's pass rate over having no memory, while using 31% fewer tokens and running 26% faster. The rest of this post covers how we designed and evaluated it, and what we learned along the way.

How We Designed Memory

There are three ways to wire memory into an agent. As a plugin (like our earlier work on MemAlign and Memory Scaling), memory is read and written automatically at fixed points in a session. As tools, the agent decides what and when to read and write. As an offline pipeline, memory is updated in the background on a schedule or trigger, and used in later sessions. Each comes with trade-offs:

MemCurrent uses all three, each where it fits best.

MemCurrent has two types of memory: episodic memory, which stores raw session traces; and derived memory, which has the distilled learnings from these traces, stored as files in a structured knowledge base. An auto-generated memory.md index sits at the top, describing the file structure and what each file contains.

Both episodic and derived memory live in a production-grade database that can scale to many users while delivering low-latency responses. Episodic memory is append-only, where a plugin logs each step as the session runs. Derived memory is a mutable table simulating a hierarchical file system, where writing happens in two ways. Agent tools (add_memory, update_memory, delete_memory) handle explicit requests like “Never write scripts in the /tmp dir”. An offline dreaming pipeline captures the rest, distilling implicit learnings from each new session into the relevant files. It also cleans up the whole store, splitting oversized files, merging duplicates, and resolving conflicts.

Reading works in layers. At session start, a plugin injects the memory index, so the agent knows what it knows before it begins. During the session, the agent has two tools that work across both memory types: search_memory returns the top matches for a query (a step in a past trace, or a line in a memory file), and fetch_memory returns the full entry by identifier (session ID or file path).

How We Evaluated Memory

Benchmarks

We start with three popular open-source benchmarks: LoCoMo, LongMemEval, and ConvoMem. All three test whether an agent can recall facts stated earlier in open-ended synthetic conversations.

In addition, we built an Internal Memory Benchmark from real Genie conversations, where users ask all kinds of questions about their data. We started from 12K+ sessions, mined 3K candidates containing memory-worthy signals, and drafted a follow-up question with scoring rubrics. The other sessions from the same user serve as fillers for the test case. An adversarial judge then checked each test case against 7 criteria, such as whether the knowledge is actually present, specific to the user, required to answer, natural, and verifiable. After a final manual review, the benchmark has a total of 114 test cases, with 29 users and 2,233 history sessions.

Our benchmark differs from existing ones in three ways:

Enterprise domain. Existing benchmarks are open-ended chats about the lives of simulated personas. Ours is from real work in a data workspace, involving rich enterprise knowledge like table semantics, metric definitions, and workflows.

Memory feeds actions, not answers. Finding the right string in past conversations is not enough; agents must apply retrieved knowledge to complete multi-step tasks.

Diversity. Test cases span 5 memory types (profile, preferences, domain knowledge, projects, workflows) crossed with 11 scenarios (direct instruction, changing or outdated facts, multi-session aggregation, memory-independent questions, and more).

Results

We compare MemCurrent with three open-source memory backends, Mem0, PropMem, OpenClaw memory, using the same underlying LLM.

Open-Source Benchmarks. We pair each memory backend with a simple question-answering agent, built with two agent harnesses (LangGraph and OpenAI Agents SDK). We also run two baselines: a floor with no memory, and a ceiling with full context, where every past session goes into the agent's context. Across all three benchmarks, MemCurrent scores highest among all memory backends and stays on the cost-accuracy Pareto frontier. On average, MemCurrent (88.1%) performs on par with full context (87.8%) at ~12x fewer input tokens per question (8.4K vs 103K). On the two benchmarks with the longest histories, it even beats full context, where dumping everything into the prompt ends up confusing the agent.

Internal Memory Benchmark. On our benchmark, the gap widens. Here, each memory backend runs inside our internal Genie, the same agent the benchmark sessions came from. The histories are too long to fit in context, so our ceiling is gold injection: injecting only the gold memories into the agent's context. Among all memory backends, MemCurrent has the highest pass rate (68.4%) and the lowest latency (252s) and token usage (418K), closing over 80% of the gap between no memory (22.3%) and gold injection (78.6%). Notably, while Mem0 was also on frontier on earlier benchmarks, it trails at 43.9% here with higher latency and cost. Compared to no memory, MemCurrent triples the agent's pass rate (22.3% → 68.4%) while making it 1.35x faster and 1.44x cheaper.

MemCurrent in Action

In addition to benchmark numbers, let's examine two examples of how MemCurrent helps in practice, from internal Genie sessions.

Example 1: Memory stops an occasional hallucination mistake. In Figure 7, a user asked the agent for the top 20 users of Product X over the past two months. The agent wrote a correct SQL query, but never read the actual execution results saved to a file. It still confidently produced a table with hallucinated content. Upon user feedback, the agent saved the lesson to memory. Next time, once the query ran, the agent read the results first and returned the right table. One memory correction kept a one-off slip from becoming a habit.

Example 2: Memory saves computation on related tasks. In Figure 8, a user asked for “the top 5 SDP customers”. The agent had to figure everything out from scratch: what “SDP” stands for, which tables held usage data, and how to join and filter them. It took 11 tool calls, 8 subagents, and nearly six minutes. The agent saved these steps as a workflow file in memory, which it could reuse on related tasks. Later, the user asked a different question: how many SDP customers had more than 5K pipeline runs last month. Using the workflow, the agent answered with 2 tool calls and 1 subagent in 41 seconds. The question was new, but the path to the answer wasn't, and memory let the agent skip unnecessary exploration.

What Makes MemCurrent Work

To understand what accounts for MemCurrent's superior performance, we ran ablations on our Internal Memory Benchmark for two design choices: memory types and extraction strategy.

Do we need both memory types? Episodic memory alone helps moderately: 35.8%, up from 22.3% with no memory. The raw history is useful, but the agent only searches it in about a third of test cases, though it passes 60% of those when it does. Derived memory avoids this issue, as the injected memory index at session start gives the agent information about what's available in memory and when to look, lifting pass rate to 66.1%. Adding episodic memory on top gives the best result, at 68.4%. The gain is small, but it suggests raw traces can recover details that distillation drops.

What's a good memory extraction strategy? Two channels extract derived memory: online tools and offline dreaming. When isolated, offline dreaming (63.5%) beats online tools (53.9%) by nearly 10 points. Their combination produces the highest pass rate at 66.1%. That supports a split of duties: dreaming handles background extraction and consolidation, while online tools make explicit requests take effect right away.

Conclusion

Memory is how an agent stops starting from zero. In this post, we introduced MemCurrent, a memory architecture for production agents that combines episodic memory and derived memory, with three access patterns: plugins to log and load context automatically, tools the agent calls on demand, and an offline dreaming pipeline that consolidates memory in the background.

Across all benchmarks we ran, MemCurrent improves quality and efficiency at the same time, the trade-off we set out to break. On open-source benchmarks, it matches full context performance at about 12x fewer input tokens. On our Internal Memory Benchmark, built from real data work, it triples the agent's pass rate over no memory while also making it faster and cheaper.

The ablations point to where the value comes from. Derived memory does most of the work, and episodic memory adds a smaller boost by recovering details that distillation drops. Combining online tools with offline dreaming makes the best extraction strategy: explicit requests take effect immediately, while dreaming distills implicit learnings and consolidates the store in the background.

Beyond the numbers, memory provides a knowledge base that isn't tied to a single agent or LLM. It can move with users to new agents or LLMs, and could seed knowledge shared across their organization, a direction we are exploring next.

Getting Started

We're bringing these findings to drive improvements across Databricks products:

For agent developers. Agent Bricks is Databricks' developer platform for building custom agents. MemCurrent is built into Agent Bricks' Memory offering, powered by Lakebase. The offering includes:

• Session store for episodic memory
• Memory store for derived memory
• An automatic dreaming pipeline
• Tracing and evaluation of agent quality, including the impact of memory
• A CLI which helps you quickly get started with an end-to-end sample agent with all of the above in <10 minutes

For Databricks Genie users. Genie is an interface where Databricks users can converse with their data in natural language. MemCurrent is already in action behind the scenes, helping to improve answer quality while reducing response latency and cost.

SparkLLM released Spark-X2.6-4B, an open-source model with improved coding, reasoning, long-context, and agentic capabilities through post-training.

Summary: As part of our work experimenting with how Claude can help accelerate drug discovery, we asked an internal research model to design minibinders (small proteins that attach tightly to a target) against scorpion neurotoxins. Claude worked from our prompt and four follow-ups, which we developed in collaboration with the financial investment research firm Citrini Research. Teams at Adaptyv Bio and Twist Bioscience experimentally tested 211 designs for binding. Of these, 17 met our criteria. We’re releasing all 211 designs and their data as a resource for the field.

In August, we showed how Claude designed protein minibinders after being given a single prompt. A month before we released those results, a team from Citrini Research had approached us with an interesting question: could Claude (and AI drug discovery more generally) enable a group of motivated nonexperts to tackle a genuine problem in human health? Members of our life science research team set out to answer this, first by brainstorming problems with Claude. We landed on asking for Claude’s help designing novel protein binders against scorpion neurotoxins, with the hope that they could one day be used to develop a scorpion antivenom.

Scorpions sting 1.2 million people or more every year, resulting in more than 3,250 deaths, mostly among children in parts of the world with limited access to medical care. New treatments are expensive to make, and are also unlikely to earn back the cost of developing them. With so little investment, scorpion envenomation has been described as a neglected tropical disease.

We started by giving Claude the same prompt we’d used in our earlier protein-binder study, after modifying it with Citrini. Over the next 64 hours, an internal research model, operating within Claude Science, worked mainly autonomously, although we directed its work with four follow-ups. Under our supervision, Claude analyzed the research data, generated the figures, and drafted the final report.

Claude’s design campaign produced 17 protein binders that each bind to one of four scorpion neurotoxins that are known to be dangerous to humans. Throughout the process, Claude generated 318,484 protein designs and selected 180 of them for testing. Our team added 31 more, with Claude’s help. Our partners at Adaptyv Bio and Twist Bioscience, two contract research organizations (CROs), lab-tested all 211 designs to see if they would bind to the toxins they were designed against, and 17 designs met our criteria for a binder. We have not confirmed whether these binders neutralize scorpion venom in a cellular environment, so we do not claim to have produced an antivenom.

We sent Claude’s work to two outside experts, Susana Vázquez Torres and Timothy P. Jenkins, who said they felt this work was a promising first step. If AI can now design credible binders against several hard-to-target toxins in days and they can be confirmed in the lab within weeks, this could, in theory, speed up discovery for envenomation and other neglected diseases. The challenge, as always, is turning those binders into safe, affordable, and widely accessible drugs.

We share more detail in our technical report, and are making all 211 designs and their data available.

Designing a binder against almost any target no longer seems unrealistic. We can now obtain very tight binders remarkably quickly, and binding itself is no longer the main bottleneck. For venomous animals such as snakes and scorpions, whose bites and stings kill or seriously injure large numbers of people, particularly in underserved regions, the real impact will come from tackling the parts of the therapeutic pipeline that go beyond binding. The challenge is to use these extraordinary tools to develop molecules that can become real drugs, ones that are safe, affordable, and widely accessible.

Susana Vázquez Torres, Postdoctoral Researcher, Structural Biology Programme, Spanish National Cancer Research Centre (CNIO), Madrid, Spain.

Designing de novo binders against five toxins from four species of scorpions

Scorpion venom contains multiple protein neurotoxins. The neurotoxins that are most likely to harm or kill humans attach to sodium channels, large proteins embedded in the membranes of nerve cells that allow them to fire. When sodium channels are disrupted, nerves fire at the wrong time—this releases large amounts of stress chemicals, like adrenaline. In severe cases, this can cause heart failure and excess fluid buildup in the lungs, occasionally resulting in death.

There are some antivenoms available for scorpion stings. They’re generally made using the antibodies of immunized animals, a process that can take months. Venom is milked from thousands of scorpions and used to immunize horses; scientists then purify the resulting antibodies from the horses’ blood. Traditional antivenoms can work when given early, but they’re usually effective for only a small number of neurotoxins. They’re also expensive to produce, and in many regions are costly or in short supply.

De novo design offers a potential solution. A binder is a molecule that attaches tightly to a specific target; antibodies are one natural example. In principle, a binder that targets the part of a neurotoxin that latches onto sodium channels could stop the toxin from reaching the channel. In de novo design, protein binders are generated from scratch against a specific target. These binders are designed computationally and can be optimized for druglike properties such as potency and stability, which could make them easier to supply in regions where stings most often occur.

Our approach was inspired by an earlier effort that designed de novo proteins that neutralized snake neurotoxins and protected mice. To our knowledge, no de novo-designed binders against scorpion neurotoxins have been described.

For this campaign, we chose to target five scorpion-venom neurotoxins across four regions: Mexico (Cn2), Brazil (Ts1), North Africa (AaH I and AaH II), and the Middle East (LqhIII). We also chose to aim the designs at each toxin’s channel-binding site. Claude picked the toxin structures to use for design by searching the Protein Data Bank (PDB) as well as the AlphaFold database (AFDB). For each toxin, it selected five hotspots, specific amino acid positions on the toxin that its binder designs were meant to latch onto.

We were primarily wondering if Claude could design binders against individual toxins. But we were also curious if Claude could design binders that targeted multiple toxins. There’s a practical reason for trying to design multitoxin binders, namely that it would reduce the number of molecules needed for a universal antivenom cocktail—which would be the most cost-effective and regionally neutral treatment. We selected these five scorpion toxins (Cn2, Ts1, AaH I, AaH II, and LqhIII) as a challenging test of multitoxin design: they share one fold for the overall shape of the protein, but any two are only 26% to 62% identical in their amino acid sequences.

To see how far Claude could go, we asked it to design four kinds of binders: binders against each of the five toxins individually; binders that targeted the β-family toxins Ts1 and Cn2; binders that targeted the α-family toxins AaH II, AaH I, and LqhIII; and binders that targeted all five toxins simultaneously.

Designing credible binders against five toxins in parallel, in a matter of days, is a great showcase of the AI-driven paradigm shift we are seeing in therapeutic discovery. This is especially true for neglected diseases such as animal envenomings, which have been largely starved of innovation due to low commercial incentives. Being able to agentically tackle the uniquely broad target space across animal toxins provides a significant speed-up in discovery, and brings realistic therapeutic solutions much closer.

Timothy P. Jenkins, Associate Professor, Technical University of Denmark.

Claude ran the design campaign with considerable autonomy

Our initial prompt defined the neurotoxin targets, the protein design and folding tools, and the rules for novelty, scoring, and filtering. Beyond choosing the structures and amino acid hotspots, Claude also set the quality thresholds a design had to clear, decided how much to lean on each available design tool, and developed the strategies for designing binders against multiple toxins.

Ultimately, Claude used 12 design tools to generate 318,484 designs, filtering out any that resembled a toxin or a known protein. Claude co-folded and screened 553,052 design-toxin pairs, predicting each pair’s structure as one complex. Sixty-four hours after we gave Claude our initial prompt, it produced its final list of 180 designs. The campaign used on the order of 10,000 GPU hours.

Claude worked autonomously for 24 hours following our first prompt. Its results encouraged us to prompt Claude to continue its campaign and look for better designs. Our follow-up prompts were mainly focused on directing Claude to design a multitoxin binder. Claude selected most of the 180 designs—including our strongest binder (Cn2-B1)—only after our third follow-up. At our request, Claude sent the protein sequences to Twist Bioscience (in emails indicating they were written by Claude), which produced the proteins in order to see how well they would bind to the toxins. We later worked with Claude to add another 31 designs, increasing the total to 211 designs: 60 designs against a single toxin (12 per toxin), 30 designs against the two β-family toxins, 30 designs against the three α-family toxins, and 91 designs against all five toxins.

Of the 211 designs our teams assessed, 17 met our criteria for a binder, each against a single toxin. None of the designs made against multiple toxins met our criteria for more than one. (Beyond being limited by its current capabilities, there could be many explanations for why Claude struggled with this, including our prompting, the limitations of available design tools, and even the biological feasibility of multitoxin design.) Seven designs were binders against Cn2, four against AaH II, four against AaH I, and two against LqhIII. Among the 60 single-toxin designs, 11 were binders (18%). This rate is consistent with typical protein design campaigns, including our previous work.

Testing Claude's designs in the lab

After completing the campaign, we sent the designs to Adaptyv Bio and Twist Bioscience to test them for binding. Each group produced the designs in their labs, but in different formats. Adaptyv Bio expressed them in a cell-free system, which uses protein-making machinery extracted from bacteria rather than living cells. Twist Bioscience expressed the designs in mammalian cells (in this case, human kidney cells) as fusions to Fc, the stem of an antibody. These are complementary ways to measure the underlying binding process, and we combined them in order to improve confidence in our results.

Both labs measured binding by surface plasmon resonance (SPR), in which a toxin flows over the design while the SPR instrument tracks in real time how quickly a toxin binds to and detaches from it. The resulting trace, called a sensorgram, produces a measurement known as the dissociation constant (KD), which is a measurement for binding strength (or affinity). The lower the KD, the tighter the binding.

When it came to evaluating the resulting designs, we set out strict criteria for labeling a design a binder against a toxin. The design needed to be a hit at both labs—that is, it needed to show evidence of binding against that toxin in every replicate, and every KD needed to be 1 µM or tighter. We also required that the design show no signal against CLEC12A, an unrelated human protein used to measure off-target binding or “stickiness.”

Claude’s Cn2 binders likely bind the same site as a toxin-neutralizing antibody

Antibodies and nanobodies remain the standard for de novo designs to beat. Three such molecules have reported affinities (KD) below 1 nM (nanomolar) against Cn2 or AaH II, and they have been shown to protect mice from scorpion toxins.

Our two strongest binders, which we call Cn2-B1 and Cn2-B2, target Cn2, a neurotoxin from the Mexican scorpion Centruroides noxius. They each bound Cn2 tightly at both Adaptyv Bio and Twist Bioscience, with affinities (KD) in the single-digit nanomolar range.

One way to find out whether two molecules (e.g., an antibody and a miniprotein) bind the same site on a target is with a competition experiment, which tests whether both binders can attach to the target at the same time. If the presence of one molecule reduces the other molecule’s binding, then they likely compete for the same binding site.

Our competition experiments suggest our Cn2 binders each attach on or near the same epitope (binding site) as an antibody called LR, which has been reported to neutralize Cn2 in mice. LR’s binding site covers the part of Cn2 that attaches to the sodium channel and causes nerves to misfire. We have not tested our strongest binders, Cn2-B1 and Cn2-B2, in a cellular environment or in live animals, but the fact that they compete with LR suggests that they will likely neutralize Cn2.

A binder is only the first step to developing an antivenom

Our results show binding only against recombinant toxins, which are grown in the lab from each toxin’s gene rather than purified from venom. The two biggest limitations of this study are that we didn’t test any design for neutralization in vitro (against cells grown in the lab) or for efficacy in vivo (in living animals, such as mice).

If we were to advance one design toward a therapy, Cn2-B1 would be one of our strongest candidates. First, though, we would need to confirm the design also binds toxin purified from venom. We would then assess it for function in vitro with patch-clamp recordings (which can measure the electrical current passing through sodium channels) to see whether it stops the toxin from acting on the channel. Then, we would need to test the binder in mice to see whether it actually protects them. Beyond neutralizing the toxin, a treatment must also be safe, stable, affordable, and available where stings occur. Reaching patients will also take cooperation from clinicians, funders, and policymakers.

We are releasing all 211 designs and their binding data so that other researchers can build on our work. We hope this demonstration of Claude’s capabilities, along with the data we are releasing, will help accelerate further research into antivenom design.

Further reading

• Technical report;
• Data release: all 211 designs, their predicted structures and scores, and experimental data;
• Our previous posts on protein design and uplifting biomolecular modeling.

Footnotes

01 Citrini has written an article on our collaboration, here.

02 We realize this is much more than most researchers have access to. Separately, we have made these models about 4x faster on average, and shown that a simplified prompt using a single agent and a single GPU over 24 hours reaches comparable in silico scores.

03 LqhIII was assessed only by Adaptyv Bio.

All em dashes are my own.

Starting at Stripe as a new grad is a strange and exciting experience. You’re surrounded by people who are talented and knowledgeable, working on systems with weird names, and trying to figure out what you can actually contribute. With AI, things are changing even faster, and everyone’s scrambling to figure out how to adapt, which makes it pretty hard to know where to start.

I’ve been here for about a year and a half, and I’ve also talked to a bunch of other ex-new grads about their first year — what worked, what was difficult, and what they wish they’d done differently. A few things kept coming up: become an expert in something small, take ownership of the problems around you, and get comfortable with having opinions. These things build on each other, and you can start doing all of them much sooner than you probably think.

Become an expert

On your first day at Stripe, you know approximately nothing. GPT-6 is better at understanding your domain, writing code, reviewing code, and probably even coming up with ideas. The way you change that is by becoming an expert — first in the narrowest domain, and over time that domain gets wider.

I realized that I cannot be an expert the way they are. But I can be an expert in this one little area, and that could be mine. So I carved it out for myself.

While it is appealing to try and learn a bit of everything, as a new grad, it is simply impossible to do that effectively — chances are you’ll just end up being mediocre at it all.

Typically, your first area of expertise will be wherever your first project at Stripe takes you. You probably won’t have too much of a say in what it is, but pretty much every domain has interesting and difficult problems, and the goal is to become the point person on your team and within Stripe for that one thing.

My spin-up project was to revamp the charts on the homepage of the Stripe Dashboard. I didn’t find the problem particularly fascinating, but it turned out charts are really difficult, and after 3–4 months of working on them, I had become our team’s mini-expert. Throughout the next year, being the mini-expert meant that even as a new grad, I could drive the mitigation of chart-related incidents, execute on dozens of run tickets, and weigh in on product/design decisions. A year and a half later, I’m on a completely different team at Stripe, but my knowledge of charts turns out to still be very relevant.

The natural question then becomes, “How do you become an expert?”

When you’re initially spinning up, there isn’t really a shortcut around “just doing the work”: learning how to reliably execute the tasks you’re handed builds the technical foundations that everything else depends on. But as you work through those tasks, be incredibly curious about the systems and decisions around them.

Read everything

This doesn’t just mean reading the Trailhead docs explaining the system architecture. This also means finding the product brief that originally explained the need for the feature you’re building, or the UXR that justified it. Having more context is always useful. Decide which docs are important to read thoroughly and which you can just chat about with your favorite LLM.

Talk to people

No one on your team knows everything, and part of becoming an expert is building a “map” of who knows what so you can get help quickly when you reach the edge of your own knowledge. Figure out who understands the different systems you work with and who remembers why particular decisions were made. This can include people beyond your team (e.g. if you’re building on their abstractions).

Ask for help effectively

You’re allowed to ask silly questions, and people are generally happy to help — most new grads ask too little, not too much. Before you reach out, be ready to explain what you’ve tried, why it didn’t work, and what your hunch is. Even if your hunch is wrong, you’ve started a conversation amongst peers instead of blindly asking for help, and it’s going to be a lot more fun and educational for both of you.

LLMs can answer most of the what questions — what this function does, what this error means. Save people for the why: why the system was built this way, why an alternative approach might not work, and whether you’re approaching the problem the right way at all. When someone helps you, ask how they got to the answer. What made them check that dashboard first? How did they rule out the other possible causes? Experienced engineers make a lot of decisions that seem obvious to them, and you’re trying to osmose their intuition and experience.

Dogfood the product

I’m not sure it’s possible to end up on a team at Stripe that has an un-dogfoodable product. If you work on a product team, create a little dogfooding Stripe account or use whatever testing accounts your team has. If you work on an internal/infra team, use the tools you’re building! Dogfooding is the easiest way to actually understand your product and your users. You want to understand their use cases, feel their pain, and also recognize the moments of joy that you’re trying to create more of.

Know the metrics

Stripes love metrics. We have endless OKRs and Hubble dashboards. If there is a key metric in your area of expertise (latency, usage, user satisfaction, …), you should know about it. Know how what you’re working on ties into these broader Stripe goals and align the decisions you make with the bigger picture.

Hubble is also an excellent proxy for understanding how users are using your product (and if it’s not, then you should encourage your team to build observability). When I was working on the home charts, peeking into various Hubble datasets helped reveal the strong “default dominance” of our users (almost no one changes their default settings), which guided how we prioritized features on the project and continues to be a key input into how I think about building products today.

AI

AI can both accelerate and hamper your ability to become an expert. It’s so good at retrieving information, synthesizing it, and writing code that it’s tempting to let it do your critical thinking too, especially when that feels faster. However, that's a recipe for short-term gains and long-term mediocrity — you end up as a meat proxy for the model.

In your first month(s) at Stripe, it may even make sense to actively slow down. Expectations of velocity aren’t as high yet, so use that time to internalize how the systems work and where they tend to break. You’ll speed up later — AI makes it simple to execute on a well-understood problem.

The “whiteboard defense”: I should be able to pull you aside at any moment and ask you to explain any customer-facing system you’ve shipped. You should be able to clearly explain how it works and defend the decisions you made. This is my benchmark for responsible AI usage.

A few habits that can help as you’re spinning up:

• Use a coding agent, but run it inside an IDE so you can visually inspect and internalize the changes that are being made. I found Cursor to be a great learning tool.
• Don’t ship a change you can’t explain and justify. By all means, have the AI help explain the changes to you, but make sure the understanding ends up in your head.
• When you’re a reviewer-in-training, actually review the code: use Orbit review, but take the time to understand the structure and purpose of the change so you’re absorbing expertise from the review. Just like asking for help, reviewing code is another way to absorb expertise.

Take ownership

Fundamentally, ownership is about caring.

To have a high sense of ownership is to understand and empathize with the users you’re serving, feel responsible for the software you’re delivering, and think about the long-term consequences of decisions. Honestly, pretty much all our operating principles are derivative of taking ownership.

It isn’t always easy to get ownership as a new grad. Part of the way you get it is by demonstrating expertise (see the previous section!), but you can take ownership much sooner than that. Just as you start with a small sphere of expertise, you can start by owning small problems — the things that frustrate you and that you can fix independently: the bug you spot, the legacy testing library your team is still using, the broken Hubble dashboard, the quality-of-life improvement that should obviously exist. By fixing each one, you develop your own bar for quality and ensure the systems around you meet it.

Just build it

Sometimes the thing you want to fix needs someone’s permission. It used to be that you’d write a proposal, get buy-in, then take a few days to build it. For many of the things you come across now, if you have conviction, then you can just spin up an Orbit session, build the thing in a dev environment in an afternoon, and then demo it to the relevant stakeholder. A small prototype makes the conversation concrete and the proposal more compelling.

Seek opportunities

Volunteer for interesting projects, and ask your tech lead or manager if you can sit in on higher-level meetings (e.g. org-level SHRs) — you’ll see how decisions get made well before you’re the one having to make them. Contribute to your team’s incidents, even ones you didn’t cause and don’t know how to fix. Incidents are the fastest way to learn how a system actually breaks, recognize how your product can impact users, and understand how to build maintainable long-term software.

My spin-up buddy and my manager were gonna have a call… [and] I was like, “Oh, I have some ideas as well. Can I also join this meeting?” I did, and then afterwards they were like, “You can just own this.”

No boundaries

While you start out focusing on your area of expertise, you should remember that you have (or will have) literal ownership in all of Stripe and should operate accordingly. If the problem you find requires you to change another team’s systems, that isn’t a sign to wash your hands of it. Propose the change, collaborate with that team, and ensure the impact is delivered. The old “edit the company” operating principle remains extremely relevant.

Don’t stop owning it

It is tempting to see projects as shiny new toys that you play with before moving on to the next big thing. Of course, almost all software inevitably gets deprecated and deleted, but until then, you don't stop owning it. It’s been five months since I swapped teams within Stripe, but I’m still in my old team’s Slack channels (except the private ones they kicked me out of), and I chime in when something I know a lot about comes up. Another new grad I spoke to even asked their manager if they could go back and iterate more on their spin-up project because they cared so deeply about making it better.

Making mistakes

Ownership is also about owning your mistakes (which will inevitably happen if you do anything meaningful and difficult). In 2025, I caused zero significant incidents. In the first half of 2026, I caused four S3+ incidents. That wasn’t because I got worse at my job (I hope). I was shipping faster, touching more critical systems, and (clearly) still learning where the sharp edges were.

When users reported these issues, I recognized they were caused by my changes, spun up the incidents, drove the mitigations, and owned the remediations. Stripe has a blameless culture, but I still held myself accountable (honestly, I was disappointed with myself). It bothered me enough that I talked with my manager about what we could have done better and wrote about it in my mid-year self-review. I’ve since caused more incidents, but I try my best to not repeat the same mistakes.

Have opinions

The goal is to become an expert, but you do not have to be an expert before you can start having and voicing opinions! Once you start feeling a sense of ownership, you’ll naturally have opinions because you care about what you’re building and want to shape how it gets built.

The mark of being a great engineer is being able to easily articulate tradeoffs and convince people of your opinions.

New grads (and new hires generally) often have the unique opportunity to share their opinions. You’re not colored by prior experiences and biases. The things that you find odd but no one else does may actually be odd — everyone else has just gotten used to them. Keep friction logs and share papercuts across both the surfaces your team owns and the internal tools you use every day.

However, as a new grad, there’s a mental blocker to voicing your opinion — “Am I about to say something dumb?” Thankfully, Stripe is full of kind people who are very excited to hear what you have to say. They know that you’re new to the company and the industry, but that doesn’t mean your ideas will be discounted.

There are a couple of techniques that you can use to start getting comfortable with sharing your voice:

Do your homework

As a fresh new grad, you’re still developing the instinct, taste, and knowledge to form thoughtful opinions on the fly. Even when you have a strong intuition, it’s pretty hard to convince other people that your opinion is sound without something to back it up. Bringing evidence — some UXR, a Hubble dashboard, or a system design that supports your view — helps you make the case to others and feel a lot more confident in the idea yourself. This is part of why building expertise (even if it’s narrow) is important — once you’re an expert, you will have the knowledge to cite that evidence on the spot.

Start small

It is extremely normal to vibe-check an idea with one or two people first — typically your spin-up buddy or the domain expert in that area. Even a year and a half in, I frequently DM my partners in crime before saying something more broadly. It builds confidence and filters out the occasional actually terrible idea.

When you do share more broadly, lean on Stripe’s culture of building in public. If you’re sharing a doc or demo, mark it as a work in progress ([wip] in the title, or saying “open to lots of feedback!”) — people engage much more generously with something that’s early in the process. That said, if you’ve done your homework and you feel truly confident in an opinion, then own it! Your confidence will impact how excited people get about your idea.

Even if it’s not a good design, you come up with a design. Then you talk to people about it — “Okay, this is what I’m thinking” — and you can iterate from there.

You can also start small by selectively choosing the method by which you share opinions. When I started attending my org-level SHRs, I was understandably terrified of sharing an opinion. The meeting was largely full of EMs and tech leads, and I mostly just lurked, but after a much-needed push from my manager, I started leaving small messages in the meeting Slack channel. This was a lot easier than actually unmuting myself to speak up! Eventually, I gained the confidence to do that, and after months of attending, I got comfortable enough to present some of our team’s incidents — including the ones I caused :).

Build a career

Careers are long, and you’re at the start! The advice above is about making early investments, but there’s more to think about as you chart a career.

Compounding growth

There are strong compounding effects to the things you do at the start of your career. As you get comfortable at Stripe, lean into what works and push it further:

1. Grow your context. Know the key things happening across Stripe — read strategy docs (go/corpstrat) and shipped emails, attend Friday Fireside and Eng ATH, and lurk in the Slack channels of interesting projects. You never know when the knowledge graph that you form will help you solve a burning problem in your day-to-day work (see go/mycelium for more thoughts on this).
2. Build your network. Building relationships is a positive-sum game. Meet people on your sister teams, the teams you rely on, and the teams that rely on you. Make sure you meet your skip manager! Cold DMing people can be scary at the start, but it is a really important part of operating in a big company. These people have interesting experiences to share, and one day you’ll need their help — an existing relationship makes that much easier.
3. Pitch bigger ideas. There’s no reason you can’t pitch an entire project. Great ideas can come from anyone on the team, and people appreciate well-thought-out pitches. Build a prototype or write a thorough doc and get lots of feedback. Even if it doesn’t become a full project, going deep on a new idea teaches you (and your team) a lot, and often surfaces tangential ideas along the way.

Set goals

Within my first month at Stripe, my manager made me write out a career roadmap articulating where I saw myself in five years and the concrete goals I wanted to accomplish within the next year. Sure, it was difficult to know where I’d be in five years, but just the act of writing this down and then working backward from those long-term goals helped give me clear direction. It’s also pretty cool to see how much of what I wrote still remains true today. If five years feels too far away, try thinking just one year ahead.

Your manager

The best managers (at least to me so far) feel like partners for your career. You cannot control who your manager is, but there are still things you can do to build the most meaningful and beneficial relationship with them.

Communicate a lot and often. It’s easy to fall into the trap of doing great work without telling anyone, but your manager can’t advocate for work they don’t know about. Learn how they like to communicate (Slack updates, a running 1:1 doc, weekly summaries) and keep them in the loop on both what you’re working on and how you’re feeling.

Share your goals. Hopefully your manager is already asking, but if not, tell them — and keep them updated as your goals change. The earlier they know what you want, the earlier they can steer projects and opportunities your way. This is especially relevant for perf — it shouldn’t be December and you still haven’t told your manager you’d like to get promoted.

Imposter syndrome

Everyone experiences imposter syndrome, and I mean everyone. Instead of my own thoughts, here is an excerpt from a 2020 interview with @michelle (who joined Stripe as a new grad all the way back in 2013) about how you can leverage your imposter syndrome:

[…] A potentially-surprising important factor for me was (and is) my imposter syndrome. It made me extraordinarily open to feedback; to learning and growing and to taking responsibility for anything remotely related to my work. It made me proactively seek out feedback on everything from the validity of my comments on PRs to how I ran a particular meeting. If something was broken (whether it was technical or organizational), I felt unsettled and was deeply, intrinsically motivated to go learn about it and to fix it […]

Of course, imposter syndrome is a double-edged sword. It often makes me scared and self-conscious — early on I constantly felt like I would get fired for not being fast or effective enough. I grew more secure about my own strengths over time, but to be frank, this took a lot of time and positive reinforcement from my managers and from leaders at the company.

Conclusion

Some of this will be useful in your first week, and some of it will make more sense six months in. You’ll figure out what matters most for where you are and the things you specifically want to get better at.

Over time, you’ll find yourself answering questions you used to ask, making decisions you used to need help with, and taking on problems you wouldn’t have known how to approach. The little area you became an expert in gets bigger, and people start coming to you for help. It’s pretty easy to miss that transition when you’re focused on everything you still don’t know.

Before long, you’ll be the person a new grad DMs with a silly question.

30 Months on the Exponential (in Charts)

My second book on AI, Co-Existence, is out today, 30 months after the release of Co-Intelligence in early 2024. Looking back across the gap between these books is a good way to get a sense of AI progress. And, thanks to AI, I can describe this progress with some quite lovely charts.

Making the charts is itself a good indicator of how far things have come. In the era of chatbots, the era of Co-Intelligence, I would have had to work back-and-forth with an AI in a chat window, guiding it through each new step. In the current era of agentic AI that I cover in Co-Existence, I delegated the work and let the AI handle it. In this case, I used Opus 5.5 in Claude Projects, which allows you to talk to a single “orchestrator” agent that then spins off additional agents as needed, giving them instructions and relaying information back to you. Getting the work done was straightforward. I gave the AI a couple sentences of suggestions and it did what would have taken me hours of work in minutes. And since I didn’t have to do the work, I had the AI make many options for me. I picked some and gave handwavy reactions to others that would drive a human team mad (“make it better”). It even caught that I’d gotten the math wrong. I was going to call this post “28 months on the exponential.”

“This is the worst AI you will ever use”

I wrote this in my last book on AI, and it has continued to be literally true at every moment since. From early 2023 through early 2024, there was a single AI model that was undeniably the best in the world: GPT-4 from OpenAI. This was the AI model that most people remember from the first couple years of AI use, it was the one I thought about while writing Co-Intelligence, and it was the one that gave ChatGPT its status as the default AI for many people. Since then, twenty different AI models have held the lead at one time or another, all from Anthropic, Google, or OpenAI.

But the changing lead alone hides an even bigger change. A total of 180 AI models are now better than GPT-4 was at its release, many of which are open weights models, which anyone is free to use or modify. Nineteen of them can run on a modern smartphone.

We are living in an exponential

The data below draws on the METR Long Horizon Benchmark, which used to be the most famous chart in AI. The benchmark measured the amount of human work that an AI can do with at least a 50% chance of success from a single prompt. Gains on this benchmark have been startlingly regular. From the release of GPT-4 to the end of the METR measure, the amount of work AI could do doubled eight times. In 2024, an AI could do a four minute task. At the moment I turned in my draft of Co-Existence in April of 2026, an AI could do 17 hours of human work.

And then the benchmark ran out.

METR said they could no longer accurately measure the amount of work an AI could do using their current methods. Other studies suggest that number might be 17 weeks of work, or more.

And, even as ability has increased, prices have dropped. My old favorite AI benchmark, the Google Proof Question and Answer Exam, which gives the AI problems that regularly stump graduate students, used to cost 30 cents in AI usage to get a single correct answer. Now, it costs four hundredths of a cent to get the same result.

How humans work with AI is changing

Despite all the progress, the frontier of AI ability is still jagged, just as it was 30 months ago. Some areas have stayed stubbornly weak (AI is still quite a bad long-form writer), while others, like solving math problems, went from terrible to superhuman in the same period. And, as I discuss in the book, even where AI is excellent, small gaps create large needs for humans, at least for now.

But AI’s gains have been so large that how we work with it has to change. In Co-Intelligence, I wrote about centaur work (divided between AI and humans) and cyborg work (blended between AI and humans). Both keep you in the loop… but you can only be in so many loops at once. Now that we have real agents, I added a third way to work with AI: the conjurer. Conjurers summon agents when needed, assign them work, and evaluates what comes back. The charts in this post were made that way: I summoned agents, gave some general guidance, and I said “make it better” a lot. It is the most powerful way to use AI, and it carries new risks as we lose control over what, exactly, the AI is doing to get us those results.

The Otter Test has fallen

Regular readers will know that I have been testing AI image generation and movie tools with the prompt “Otter on an Airplane Using Wifi” for the past few years. This is what the best image generator, Midjourney, produced 30 months ago.

So today I asked Opus 5.5 the following (typos and all, prompting has become much less important as AIs get better): “I have a post showing the what happened over the past 30 months of AI in honor of my new book (published 30 months after the origina). i want you to make the ultimate otter on an airplane using wifi video. just something fantastic, i know you are creative, give it a go.” Using my subscriptions to various online services from Google and ElevenLabs, it produced the following. (Turn the sound on)

The video has a few flaws. But the AI figured out how to make it, from the script to the images to the concept to the music to the technical aspects of creating all of this, without any help from me. That is what has changed.

This is the least we will ever talk about AI

As a result of all of these gains, AI is only becoming a bigger issue in the world. This timeline of achievements and controversies since Co-Intelligence came out gives you some idea of how much is happening (I am leaving out hundreds of important events, but, even with the power of AI, we are all having trouble keeping up).

Of course, the speed of AI advancement is a problem for an author like me, which is why my new book does not try to keep up with the news or the latest model releases. Rather, it focuses on trends like the ones in this post: AI is improving exponentially but unevenly, and the way we use it is shifting from chatbots to agents that do real work. The book is about what that means for the humans who have to co-exist with these systems: where we still have an advantage (I argue it comes down to deep knowledge, wide knowledge, taste, and agency), when to hand work to AI, and when to avoid it entirely. If you understand the trends, the present (and near future) becomes much less surprising.

Co-Existence is out today. You can get it here, on a website Opus 5.5 redesigned. I told it to make it better.

Full article content is not available for inline reading.

Read the original article →

Publish stats, get cited: The tactic that put my data all over the web

What I learned about AI search when I made up 50,000 stats that got cited by ChatGPT...

About 7 months ago, I launched a website with 50,000 statistics. Every one of them was fake.

Google has forgotten the site exists and deindexed the pages, but ChatGPT hasn't. Since the website went live 7 months ago, it has sent more than 10,000 AI sessions its way. And now more than 330 referring domains, including Shopify, Bluehost, and DHL, link to the site. Its made-up numbers have also ended up in an academic paper and in a bank's investment research.

At Peec AI and in the industry, we talk about GEO and SEO every day, and we keep looking for new tactics. I’ve been working on this stats page one for a long time, sharing pieces of it publicly along the way. Of course, it’s experimental, but the results have been hard to ignore.

Where the idea for publishing fake statistics came from

The impetus to seed a site with statistics came out of long-running research I've been doing on LLMs, relevance engineering, and grounding (credit to the iPullRank team, Dan Petrovic, and Wil Reynolds).

In the course of my work, I noticed a pattern: as more teams write time-sensitive and breaking content with LLMs, those LLMs have to look for (seemingly) reliable data during their research to fill the gaps in their knowledge. Statistics are often exactly what they’re looking for.

Most GEO advice focuses on one moment: getting your brand into the AI answer. This experiment looks at what happens after the answer, when people take that output and use it, putting it into presentations, adding it to a shopping list, and yes, publishing it on their website.

I didn't come up with the idea for this experiment alone. Hat tip to Seer Interactive, who already published an eye-opening study in August 2025 that got a few thoughts bouncing around in my head. Several months later, those thoughts came together and finally made it to the top of my to-do list. In March 2026, I started working on building the site that would let me test my ideas.

The setup: a website and 50,000 made-up stats, created in one day

Quick context for anyone who wasn’t following along in real time as I ran this experiment:

I bought a brand-new domain, stateglobe.com, and generated 50,000 statistics pages in a single day with GPT-4.1. The site purports to provide stats on SEO, content marketing, social media, e-commerce, and web technology from 200 countries. Except every number was totally invented. I never checked a single one. I pushed them to the site with a basic AI web design template. This cost me around $30. Later, I added 15,000 listicles on top of the statistics. That was about $20 more, so all in all, this is an experiment that came in at the very low price of about $50.

The reaction when I first shared it was wild. When my friend Kevin featured it in Growth Memo, it reached a much bigger audience. I think people responded so enthusiastically to this work because it showed great results while also speaking to the fears people have about what ends up in AI search results.

What happened

1. Google buried the site almost immediately

When I launched the site, traffic, indexing, and engagement were initially going really well. Using indexer tools, I got almost the entire site indexed within a week, and it started to appear in Google results.

Then two Google updates came out right after launch, and my Google visibility collapsed.

I've been invisible on Google for many months now. Indexed pages bounce between 5 and 20 impressions. At one point, I got curious and created a subdomain to see if I could increase impressions, but it didn't really help.

Other than that, I haven't touched the site. Some pages might even fail to load now and then. I don't know.

But when I sat down to write this, I checked the crawl data in Cloudflare after 7 months.

In the last 30 days alone, the site got more than 600,000 requests from bots. Around 72,000 of them came from ChatGPT-User. That is the agent ChatGPT uses to open a page in real time while answering someone. So these are not training crawls. Each one is ChatGPT fetching a fake statistics page to answer a real question.

2. Unlike Google, ChatGPT never stopped pulling from the site

And when I say, “never stopped,” I mean ChatGPT really never stopped.

After 7 months, the site now has 330+ referring domains

According to Ahrefs and Semrush, the site has links from 330+ other sites. That includes well-known brands like Shopify, DHL, and Bluehost. With strong Google performance, the numbers might have been even better.

ChatGPT is still citing it to this day

Google may have forgotten about the site, but ChatGPT clearly hasn’t. The site is still earning citations and traffic more than 7 months after launch.

If you've read Tomek's ChatGPT Index research, you're lucky, because it’s a great intro to how ChatGPT search works. If you haven't, add it to your reading list right now, because it helps explain why my website has been living in ChatGPT/OpenAI Index for so long.

A completely made-up website still performing so well is shocking. Since March, GA4 shows 10,000+ sessions from ChatGPT alone, and Cloudflare shows the site has high marks for retrieval.

Right after I joined Peec in June 2026, I also set up a visibility dashboard in our tool to track a few prompts. Performance for them is still strong. On October 7, stateglobe.com was ranked second for visibility on these prompts, behind only data powerhouse Statista.

Retrieval is not ranking

The constant retrieval is the first big lesson: ranking on Google and being retrievable by an LLM are two different problems.

When ChatGPT searches, it doesn't just copy Google's top 10. It rewrites your prompt into its own sub-queries (called fanout queries), pulls candidates from its search sources and its own cache, reranks them, and then opens a handful of pages to ground the answer. A page can be invisible on Google and still perform well in every one of those LLM steps.

StateGlobe is a good example. While Google dropped it, the AI retrieval bots kept coming back.

For GEO, this means Google rankings are only one signal, not the whole picture. If you only track Google, you are missing a channel that can work even when Google says “no.”

3. Fake data turned into real citations

You’d expect that if a site performs well with information retrieval tools, its information would begin to appear out in the wild. And you would be right.

StateGlobe was built entirely on fake data, using the cheapest model available and a prompt that was nothing more than a list of countries and topics plus two sentences. Nevertheless, the output was…

…cited in an academic paper…

…used in investment research by Qatar's largest financial group…

…cited by an economics publication in India…

…and used as a source in a research report from a Canadian influencer community.

Note that these are only the citations that linked back to the StateGlobe site. Presumably, many more used the data without a link.

The uncomfortable implication: citation laundering

On day one, a fake number on StateGlobe had zero authority. It was a random stat on a brand-new domain.

Then an academic paper cited it. Then a bank. Now the next person, or the next model, that finds this number sees a stat with a source. And that source is cited by serious institutions.

The fake data certainly didn't become true, but every citation made it look more trustworthy.

It has the potential to get worse: after so much retrieval and citation, these pages end up in web crawls, and web crawls end up in training data (I hope it hasn’t!). A number that started as a GPT-4.1 guess can become something the next model "knows."

This points to the larger risk suggested by this experiment, and it's why I'm sharing this as research, not as a playbook for fake content. Verifying sources should become an essential part of writers’ and researchers’ workflows.

4. Lastly, we tried running our experiment with real data

Once I began working at Peec, I decided to try a version of this test again. As the GEO Research team, we talked through the results up until that point and quickly launched a simple new page. This time, there were no made-up numbers, and we checked every stat and properly cited its source. You can view it here: peec.ai/ai-search-geo-statistics.

Almost immediately, sites we suspect use AI to write their content started linking to us. For the second time, using a second site, earning links by publishing statistical content proved that a hypothetically strong tactic worked in reality.

Lessons learned

Why publishing statistics leads to LLM citations

The two tests show that LLMs have a strong preference for seeking out statistics when asked to write articles. The mechanism is simple:

1. More and more content is written with LLMs.
2. When an LLM writes, it researches the appropriate topic.
3. When the LLM hits a gap, it looks for a number to fill it.

The reasons behind this process are equally clear.

Research means fanout queries asking for statistics

When a prompt requires research, the model breaks it into sub-queries, adds modifiers like "statistics," "data," "report," or the current year (if also requested in the prompt text). If someone asks for the model to compose a blog post about remote work, the model will search for "remote work statistics 2026." Pages that optimize for what models search rather than for what humans type will appear more often in LLM results.

A number looks like evidence that can fill knowledge gaps

When a model is asked to draft a piece of writing, a citation to a page that says "Remote work is growing" adds little authority and remains vague. If that page instead says "X% of employees work remotely at least once a week," there’s suddenly something that is specific, quotable, and additive. When a model grounds an answer, a specific number is the most useful thing it can find. These numbers look like evidence, though as this experiment proves, sometimes they aren’t.

The model doesn’t check the number

A search-grounded piece of LLM-written content has a tight budget for latency and cost. The system fetches a few pages, pulls the relevant passages, and starts to write. Fact-checking doesn’t happen anywhere in the process.

Verifying a single statistic would mean finding independent sources, comparing them, and resolving conflicts. Across billions of prompts by millions of users, the costs in both time and money would add up, making verification too slow and too expensive for practical purposes.

To get around this, LLMs use rerankers that score relevance, but not truth. That means if a passage matches the query and looks authoritative, it gets used.

Statistics have a built-in freshness that models like

I've written before about ChatGPT's recency bias in how it scores sources. Statistics pages contain the year in the title, the headings, and the URL. Three built-in signals to the model that what’s on the page is fresh.

How to build a statistics page the right way

The two tests I ran show that statistics pages are a very powerful way to get your content cited by LLMs. While I absolutely do not recommend coming up with bogus data, you can use the learnings from our small test to create your own pages with real data. Here's what worked for us at Peec and what I would do if I were making a statistics page again.

• Make one claim per sentence. Each stat should make sense on its own, out of context. Retrieval doesn’t work on pages, but rather on passages. If a number needs the paragraph above it to make sense, it probably won't get pulled.
• Put the source and year next to every number. "According to [Source] (2026), X%..." This helps the model, the human editor, and your credibility. It’s one small piece of information hygiene that benefits the broader information environment.
• Match headings to fanout queries. The headings on your page should match the way LLMs search the internet: "[Topic] statistics," "[Topic] market size," "[Topic] adoption by country." Think in the queries models run, not just the keywords people type.
• Show the update date (and actually update it). Freshness is a real signal that models take into account. A stale stats page will lose to a newer one.
• Make the stats easy to fetch. Use server-rendered HTML instead of hiding data behind JavaScript, tabs, or images. Also, be sure to check your robots.txt and your CDN or WAF rules. Your site may be blocking AI retrieval bots without your knowing it.
• Keep it simple. Our Peec AI GEO statistics page is basically a list. Yours doesn’t need to be anything more than that, either. Save design resources for sites and assets that can actually benefit from them.
• Add original data when you can. Aggregated stats may get you a few citations, but when you say something no one else does, you can become the primary source.

How to track whether your page is getting picked up

Google Search Console won’t show you what’s going on with AI bots on your site. Here is what I track instead:

• AI bot activity in your server logs or Cloudflare. Look for retrieval bots like ChatGPT-User, OAI-SearchBot, and PerplexityBot, not only training crawlers. We have this feature!
• AI referral sessions in GA4 from chatgpt.com, perplexity.ai, and similar sources.
• Citation and visibility tracking across a set of prompts. At Peec, we use our own tool for this, obviously.
• New referring domains in Ahrefs or Semrush.
• Unlinked mentions. Search your exact numbers in quotes. You will find people using your data without linking to you.

A word on ethics

It should go without saying that you should never publish fake content. While my experiment relied on tens of thousands of unverified stats, I ran it in order to share the results with the public, and the website, while still live, now carries a disclaimer. When putting out content we hope gets picked up by LLMs, our job should always be to put out accurate, accessible information.

To add another caveat, there’s no guarantee this tactic will continue to work. We tested it, and these are the results, but now that they’re published, LLMs might close this gap. For now, it works, but in the future it may not, and of course it should never be abused.

And one final point: this is a tactic, and neither can nor should be the basis for your organization’s entire GEO strategy. The results I got with StateGlobe were extreme, but the brands most likely to benefit from using the technique are those people already trust.

What's next from GEO Research at Peec

At Peec, our research team is running close to 40 experiments like this one. Some are still in progress, and others are already showing results. Building a long-term strategy for your brand means testing, measuring, and sharing what we learn along the way.

More soon!

Introducing Amazon ECS Gateway: A fully-managed ingress solution for ECS services

When platform teams manage containerized applications at scale on Amazon Elastic Container Service (Amazon ECS), configuring ingress networking for each service often becomes a bottleneck. For every new service, teams must provision and connect Application Load Balancers (ALBs), AWS Certificate Manager (ACM) certificates, AWS WAF web ACLs, and security groups, then keep those resources in sync as services scale, update, or operate. This per-service overhead slows down deployment velocity and increases the risk of configuration drift across an organization.

Today, we’re announcing Amazon ECS Gateway, a new account-level resource that simplifies ingress networking for containerized workloads by dynamically provisioning and managing the necessary infrastructure on your behalf, with all resources created directly in your account.

ECS Gateway introduces a clear separation of responsibilities between platform teams and application teams.

• Platform teams create and configure a gateway, defining organizational networking standards including subnets, security groups, TLS certificates, listener settings, and WAF web ACLs, in a single place. They then associate the gateway with one or more ECS clusters, making it available to services running on those clusters.
• Application teams opt their services in by adding a gateway connector parameter when creating or updating a service. ECS provisions the required per-service resources automatically: an ACM certificate, a unique DNS hostname, listener rules, target groups, and a service security group that permits inbound traffic only from the ALB. No networking expertise required.

ECS Gateway offers host-based routing using ALBs and enables up to 25 services to share a single ALB, optimizing infrastructure costs. ECS manages the full lifecycle of the underlying infrastructure, provisioning the first ALB when your first service connects, adding more ALBs as capacity is reached, and removing them when no longer needed. Each service receives a unique DNS hostname, giving your teams a ready-to-use endpoint the moment provisioning completes.

A first look at ECS Gateway

In the ECS console, platform team can create a gateway and sets its configuration. The infrastructure role is an IAM role that ECS assumes to provision and manage networking resources such as ALBs, security groups, and certificates in your account. To create a gateway, provide a gateway name and select an infrastructure role. You can optionally associate the gateway with ECS clusters during creation or later.

Choose a load balancing scheme, either public or VPC-internal traffic routing. Under Additional Settings, you can also set up a listener, port, maximum count of load balancers the gateway can scale to, and a web ACL lets you protect your web applications. Some settings, including the name, scheme, VPC, subnets, IP address type, listener protocol and port cannot be changed after gateway creation.

Using AWS Command Line Interface (AWS CLI), call CreateGateway with a name (unique per account), an infrastructure role, and a configuration object. No infrastructure is provisioned until a service connects with a gateway connector.

aws ecs create-gateway \
  --name my-team-gateway \
  --infrastructure-role-arn arn:aws:iam::123456789012:role/ECSGatewayRole \
  --configuration '{
    "HostSharedAlb": {
      "networkConfiguration": {
        "subnets": [
          "subnet-080779f9bd887757a",
          "subnet-03d5b2fead6bc6258",
          "subnet-0861210a397d31252",
          "subnet-06539e194023cb77a"
        ],
        "ipAddressType": "ipv4"
      },
      "scheme": "internet-facing"
    }
  }'

Once the gateway is created, you can take a look at the detailed configuration, gateway routes, load balancers, and associated clusters.

Now associate the gateway with one or more clusters (many-to-many), to make it available to ECS services on those clusters.

Use AssociateGatewayWithCluster to make the gateway available to services on a cluster:

aws ecs associate-gateway-with-cluster \ 
--gateway my-team-gateway \ 
--cluster arn:aws:ecs:us-east-1:123456789012:cluster/my-cluster

To create and manage the ECS Gateway programmatically, including calling APIs and searching documentation, try the AWS MCP Server and plugins with your preferred AI coding tool. To learn more, visit the Amazon ECS Gateway documentation.

ECS Gateway: Behind the scenes

A gateway can be associated with up to 10,000 clusters, and a single cluster supports up to five gateway associations. This many-to-many relationship gives platform teams the flexibility to share a gateway across environments or assign multiple gateways (for example, public and private) to a single cluster.

Platform teams can control access by choosing which gateways to associate with which clusters. For additional control, use IAM policies to restrict who can deploy services to specific clusters.

Application teams add the gateway connector to each service definition in CreateService or UpdateService. If you have an existing ECS service with its own load balancer, you can adopt ECS Gateway incrementally by calling UpdateService to add a gateway connector. Both ingress paths remain available, allowing you to validate the gateway connection before removing the existing load balancer configuration.

aws ecs create-service \ 
  --cluster my-cluster \ 
  --service my-api-service \ 
  --task-definition my-api-service:1 \ 
  --gateway-connectors '[{ "portName": "api", "filters": {"accessTypes": ["PUBLIC"]}, "healthCheck": {...} }]'

At deployment time, Amazon ECS evaluates the gateways associated with the cluster and connects the service to every compatible gateway. A gateway is compatible when its access type matches the service’s connector filter, it is in the same VPC as the service, and it covers all Availability Zones used by the service’s configured subnets.

For each connection, Amazon ECS assigns the service to an ALB with available capacity and provisions the per-service resources: an ACM certificate, a unique DNS hostname, a listener rule routing traffic for that hostname to the service target group, and a service security group allowing inbound traffic only from the ALB security group. If no existing ALB has capacity, ECS creates a new one (up to maxAlbCount referring to quota limit). Once provisioning completes, the gateway route becomes ACTIVE, and the service URL is ready to receive traffic.

ECS Gateway handles common operational scenarios automatically:

• Redeployments: On each deployment, ECS re-evaluates gateway associations. If new compatible gateways have been associated since the last deployment, your service connects to them. If a gateway has been disassociated, your service disconnects and the route is removed. Your service always reflects the current state of the cluster gateways at deployment time.
• Service deletion: ECS automatically disconnects the service from every gateway, removes all gateway routes, and cleans up all per-service resources. If the service held the last route on an ALB, that ALB is deprovisioned.

Things to Know

• Networking: ECS Gateway is available for ECS services using the awsvpc network mode across all launch types (Amazon ECS EC2, AWS Fargate, and ECS Managed Instances). Your service Availability Zones (AZs) must be a subset of each gateway’s AZs on the cluster. A gateway ALB only routes traffic to tasks in AZs where it has subnets. We recommend creating gateways and services for a given cluster with the same set of AZs.
• Capacity: If maxAlbCount is reached and no ALB has available capacity, the deployment fails. Default quota limit of maxAlbCount is 10. Plan your maxAlbCount setting according to the number of services you expect to connect.
• Observability: Use ALB metrics in CloudWatch with the TargetGroup dimension to monitor per-service traffic, and LoadBalancer dimension to monitor traffic across the shared ALB. Use describe-target-health to check real-time target status. Configure CloudWatch alarms for metrics such as HealthyHostCount, TargetResponseTime, and HTTPCode_Target_5XX_Count, using gateway ARN-based tagging so monitoring scales automatically with your maxAlbCount.

Now available

Amazon ECS Gateway is available now in all commercial AWS Regions where ECS is available at no additional charge. You pay only for the underlying infrastructure provisioned in your account when services connect to a gateway, including applicable ALB charges, CloudWatch logs and metrics, ACM certificates, and data transfer charges. To learn more, visit the Amazon ECS Pricing page.

To get started, create your first gateway today in the ECS console, and send feedback to AWS re:Post for ECS or through your usual AWS Support contacts.

Cloudflare and VoidZero are now working with Max Schwenk of Owner.com on tsrs, a Rust port of the TypeScript compiler. We are doing this work in close collaboration with the TypeScript team, and incubating tsrs as part of the Oxc JavaScript compiler toolchain.

Our goal is to help developers spend less time waiting for their code to be checked and more time building. Making TypeScript faster and more memory-efficient also helps bring complete development environments to more places, including right in your browser.

To be clear, we are not forking TypeScript. We believe the TypeScript team should remain in full control of the language’s design and direction. They have maintained a long-term vision while guiding TypeScript’s adoption across the JavaScript community. We do not want to change that.

The TypeScript team can move quite fast on many things, but for some features they may spend years intentionally exploring options and making prototypes before picking the right implementation. A Rust port has to follow those decisions closely, or it will eventually fall apart and become incompatible. We have to preserve the TypeScript language strictly based on its original implementation – and the TypeScript team should remain solely responsible for deciding what that looks like.

Performance and memory improvements

Early benchmarks of our Rust port of TypeScript are promising: tsrs is up to 8x faster and uses ~70% less memory compared to TypeScript 7.0 when using the same number of type check workers:

As you can see, the next version of TypeScript already brings many performance improvements, making TypeScript 7.1 beta up to 2.5x faster than TypeScript 7.0. Some of these optimizations were upstreamed from tsrs, and we’re hoping to find more.

You may have seen recent benchmarks of different TypeScript ports with various performance claims, often due to using larger machine types and different options compared to TypeScript’s defaults. We worked with the TypeScript team to ensure the above benchmarks are comparing the same configuration.

By default, tsrs ships with an adaptive configuration dependent on the machine type that uses more memory but is even faster:

Try tsrs

TypeScript in Rust aims to be a faithful port of TypeScript. You can try an experimental release by installing the tsrs package from npm:

npm install tsrs
npx tsrs

Why now?

We’ve seen many attempts at porting TypeScript to other languages over the years. Because of the complexity involved, only one of the ports succeeded: the TypeScript team’s port of TypeScript from JavaScript to Go.

However, frontier AI models can now do large-scale ports almost independently in a matter of days. We have now seen slopforks, faithful ports, and completely re-architected versions of TypeScript in Rust or C++. There is clearly a demand for even faster TypeScript.

Why Rust?

You might be thinking: “TypeScript’s Go port already made TypeScript 10x faster and far more memory efficient than the previous version written in TypeScript. Why isn’t that enough?”

However, it’s not just about speed. Rust offers some unique advantages: running in WebAssembly. The Rust + Wasm story has always been strong, and one of the reasons comes down to the fundamentals of Rust’s language design. When you compile a Go program to Wasm, you have to compile Go’s garbage collector along with it. But Rust’s memory management model enables straightforward memory interactions between a WebAssembly guest and host. This is one of the key things that makes Rust a much better fit for the WebAssembly use-case: memory efficiency.

Cloudflare Workers have supported Wasm for a long time. Just a few weeks ago we announced supporting native Rust in Workers with the new Emscripten target for wasm-bindgen. When you run entire software build and deployment systems in tight loops on Workers, the benefits of a small memory footprint pay off quickly. It means you can run these workloads so efficiently that it’s now possible to run fully encapsulated development systems in places where they couldn’t have been run before.

This enables new possibilities when combined with agents: You are now able to run a development toolchain end-to-end in a fully sandboxed environment on Cloudflare. But with Wasm, you can also run those environments safely client-side in your browser, or on an embedded device, or an IoT application.

How will the tsrs port be maintained?

First, we are working with Theo Browne to merge tsc-rs and tsrs into a single package that supports all of TypeScript’s features including Language Server Protocol (LSP) and content mappers. Second, we’ve seen a lot of success with how we maintain Vinext in sync with its upstream, Next.js. We explained how we do this in a recent blog post:

Each morning, an agent reviews the changes, fetches diffs, and opens tracking issues for anything that could affect Vinext. Every night, the compatibility matrix is regenerated as we run the Next.js test suite against Vinext. When one of these tests or issues reveals a gap, agents are now in the position where they can identify the change across both codebases, build a reproduction, port any relevant tests, and propose a fix.

We will apply the same process to TypeScript, taking extreme care that we don’t introduce diverging behaviors from upstream TypeScript. When we come across performance improvements, we’ll attempt to contribute them back to the TypeScript Go codebase so everyone wins.

Finally, we recognize that there are some differences to the upstream version of TypeScript right now, like for example the order in which diagnostics are printed. We aim to reduce those differences to a minimum by the time we publish a stable release.

What’s next?

Cloudflare is already the home of many free and open source Rust tools that power the JavaScript ecosystem, such as Vite, Rolldown, Oxlint, and Oxfmt. With more than 10 million developers building on Cloudflare, we care a lot about making the underlying developer toolchain faster, more portable, and easier to run in more places. We are excited to experiment with deeper tsrs integrations into our stack, for example for type-aware lint rules in Oxlint.

Like with Vinext, our goal is to run tsrs as a well-maintained healthy open source project — without getting in the way of TypeScript itself. The Oxc team has already begun refactoring the codebase to make it faster, use less memory, and make it maintainable. We know the TypeScript team is taking this seriously – they are experimenting with memory and performance improvements for tsc. We are excited to collaborate with the TypeScript team and the JavaScript community to build the best and fastest implementations of TypeScript together.

Updated October 15 to include more detail on the projects that pioneered many of the techniques we use in Scion.

At Anthropic, coding is no longer the bottleneck to building systems or doing research. Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI.

In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day. We don’t expect this rate of growth to slow down anytime soon; the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.

In order to support this growth, we built Scion, an in-house version control system, and fully migrated our monorepo from git in five months. We also built and shipped a code review surface (called Antique), code search, a rich source control IAM system, novel ways of managing and landing large amounts of reviewed code, and more, all in that same time period. In this post, we’ll focus on how we built Scion specifically. It makes for an interesting case study: Claude’s ability to write code was both what put us in this situation in the first place and how we were able to quickly build a high-quality, scalable system to deal with it.

Growing our monorepo

Before we explore what we built, we should acknowledge how unusual our workload is. Unsurprisingly, Anthropic uses Claude to write software far more than almost any other company, so we’re often the first to see where tools built for human scale start to strain. Source control is just one example of this. While we don’t expect many organizations to have these problems in the very near term, we suspect it won’t be too long before others who use AI coding tools experience these same pressure points, which is why we wanted to share what we built and what we learned.

Git is a decentralized system: every clone is a complete, self-contained repository that can compute diffs, read history and blame, and check out any revision while fully offline. For the vast majority of repositories, this is the right design, and it’s a large part of why git became the standard.

As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.

Git offers tools for working with a slice of a repository, such as shallow clones, sparse checkouts, and promisor remotes. Beyond these, there are standard workarounds for large monorepos, like internal git mirrors and repository snapshots (seeds) kept in blob storage. We’ve used all of them—at our peak, we operated several hundred internal git mirrors to deal with the load. This bought us time, but they became harder to operate as our merge rate increased. Seeds fell further from the tip of main between refreshes, and it took more and more care to keep operations that depend on history, like diff and blame, working on partial clones.

Taking a step back, we noticed that, like most companies, we were using git as a centralized system with one authoritative copy, but nearly all of our clients were agents and jobs that are online by definition. The properties that make git excellent for distributed, offline work were actively undermining our ability to scale as the repo grew. So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?

Principles for agent-scale source control

With agentic coding, many common assumptions about how software is developed are evolving. We embraced these changes when building Scion, which allowed us to take a fundamentally different approach than git on several core principles. Source control for huge monorepos has prior art, notably Google’s Piper and Meta’s Sapling, so we were able to build Scion with their hard-won lessons in mind. Our principles are:

Clients are always online. Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.

Client-server asymmetry. In git, every client is capable of being a full-fledged server. In Scion, the server answers questions about file history and blame, creates commits, performs merges and rebases, and so on. The client maintains a bounded local object cache, lays out the working copy files on disk, monitors for filesystem changes, and gives users an ergonomic way to work with the server. It can compute very little on its own but, in exchange, it requires much less data locally.

A comprehensive source control service can power many novel applications. With git, a CI job or tool that needs to ask a question about the repository, such as who last changed a line or how two commits are related, has to maintain a local copy of the repo or hope that vendor-specific APIs provide the answer. With Scion, all of the source control queries the client uses are also exposed via API, so in many cases jobs don’t need a local repository at all. Many internal tools, such as our automated bisect and revert pipelines, wouldn’t be practical at scale without this cheap, programmatic access to source control.

Everything must be horizontally scalable. At our peak, we ran several hundred git mirrors, which significantly strained capacity and no longer scaled once the repo outgrew the mirrors’ disks or memory limits. In Scion, the repository’s data is distributed, and every component is independently scalable; there is no fully materialized copy of the repo anywhere, even on the server. There is no equivalent of packfiles (git’s compressed bundles of objects), so no repository maintenance is needed.

Commits must be cheap, incremental, and lock-free. In git, you commit locally and then push to share your work. In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push, but each of those operations needs to be fast. The cost of creating a commit and updating its downstream data structures has to scale with the size of the change, not the overall size of the repo, or the server can’t keep up. The development lifecycle in Scion discourages refs (pointers to commits, such as branches and tags), except for the main ref or release branches, because global state that every server has to agree on is a looming scaling problem.

The working copy is optional, and sparse by design. Because our monorepo is so large, and because we check out code tens of millions of times a day for CI, we wanted to avoid checkouts whenever we could. With Scion, a CI job that needs the diff between a commit and its merge base, for example, can ask the server for it. The same goes for writes: many write workloads can also skip the checkout and function entirely via API. Scion’s protocol is centered around sparse checkouts, which only contain the directories the client asks for, and virtualized checkouts, which materialize files on demand, so when a job does need files on disk, the client only pays for what it actually reads.

How Scion works

Data structures

In Scion, the unit of work is a “changeset,” an entry in a write-ahead log. A changeset stores most of what you’d expect from a commit in git. But where a git commit stores a pointer to the tree, a changeset stores a list of the files that changed relative to its first parent, along with content-addressed pointers to the compressed file contents in blob storage. Changesets are created server-side and are available everywhere, instantly.

This structure lets us accept changesets with very high throughput because there are no packfiles to accept, no locking or cross-server coordination, and no refs to move around. Scion does technically support refs, but since neither creating nor fetching commits involves them, moving one is a compare-and-swap on a single Spanner row that happens separately from creating changesets.

A changeset records only what’s changed, so on its own, it can’t quickly answer questions like who last edited a line or which commits affected a file. To answer those, Scion builds additional data structures from the changesets, including dedicated structures for file blame and path history and skip lists for the commit graph. We call everything that isn’t directly part of the changeset “derived data,” and we compute it all through one generic mechanism. Each type of derived data is defined recursively, so its value for a changeset is computed from its values for that changeset’s parents.

This mechanism has three benefits. First, we can easily add a new type of derived data when a new data structure would make some operation faster. Second, for each type, we can choose whether to compute it eagerly or defer the work, so we avoid materializing data we don’t need. And third, because the client never sees the internals of these data structures, we can optimize them without changing the client. We’ve added new types of derived data many times as the system has grown, such as new indices for performing copy detection or path-based file search.

Scaling Scion

A small fleet of Scion servers provides source control APIs for any operation a client needs, such as getting changesets, computing diffs, finding merge bases, and fetching blobs. All source control data is content-addressed, so a piece of data never changes once it’s stored, and any server can cache it without checking with the others. As a result, the servers are horizontally scalable and require little coordination. They are essentially large caches that can also compute derived data on the fly and write the results back to Spanner.

Scion’s many consumers no longer need to maintain local repositories of their own. Fetching and maintaining the repository used to be a significant maintenance burden for CI jobs or tools that needed to ask source control questions, particularly about commit ancestry and blame. Early in the migration, before we had our own client, repo format, or protocols, Scion only answered these “metadata” queries. Fetching file contents and materializing working copies are far more data-intensive operations, and they needed a design of their own.

Scion’s scn client deviates from git’s fetch protocols in several important ways and, as a result, lends itself to a highly scalable architecture that requires significantly less CPU than git servers. Scion’s negotiation doesn’t tell the server what the client already has; instead, it operates exclusively on trees and blobs the client wants. Starting from a root tree, the client checks its local store for missing trees and requests them recursively, then repeats the process for blobs. Every request is for a specific object that never changes, so any cache can answer it, and the architecture closely resembles traditional tiered caches. The process is also efficient and resumable, because there is no single multi-gigabyte packfile response. Sparse and even virtualized checkouts don’t need special support, because the client simply doesn’t ask for trees it doesn’t want.

In the fleet, the client talks over a socket to a daemon, scion-daemon, that stores objects for the whole node. Most of our nodes need access to the monorepo, and jobs on the same node reuse most of the same objects. Each daemon starts from a periodically refreshed snapshot, then talks only to the tree and blob cache servers in its region, called scion-cache. These cache servers stream objects directly from memory, in the disk format that the daemon will consume, so they don’t need to spend time computing the client’s missing objects or performing delta compression. When a cache is missing an object, it asks the authoritative scion-servers, which read from Spanner or cloud storage as needed.

The scion-daemon, scion-cache, and scion-server all respond to identical RPCs for this protocol, so the client doesn’t need to know the cache topology. Any tier could theoretically talk to any other, but we strictly enforce the tiering to isolate failure domains. We want to avoid a situation where a failure in a regional scion-cache overwhelms the upstream scion-servers and brings down the entire system.

Each tier of this system maintains a cache hit rate of over 99% and does not require long-lived state, so it’s easy to scale up if needed. Our Scion infrastructure footprint is now 200x smaller than our previous git mirroring infrastructure in terms of CPU and memory requirements. CI and research jobs check out code two orders of magnitude faster (in seconds rather than minutes), which frees up a sizable amount of capacity across the fleet; CI alone realized a 25% CPU capacity win. Neither clients nor caches need to perform repository maintenance, which simplifies the lifecycle management of our jobs.

What we learned building Scion

We started building Scion around the same time as Claude Mythos Preview and Claude Tag became available internally. Both of these releases simultaneously amplified the need for Scion and meaningfully accelerated how quickly we could build it.

Here’s a particularly striking example. Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.

What we learned building and operating Scion offers some interesting lessons for what building agent-scale software could look like.

Rapid development with a small team will become more common. A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about), not to mention half a dozen related systems, and fully migrated to them in five months. We did all this while the volume of code merged per day increased by 10x. In the past, developing and deploying a source control system like this would have taken a company years of iterative migration, with many more engineers. We think this sort of fast, nimble migration will become more common in the future.

Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration. For example, there were multiple systems that tried to brute force looking up when a change merged to main by linearly reading the commit log instead of asking our code review tool. Anthropic also makes extensive use of adversarial review, and, while this is incredibly effective in many ways, it can also be myopic, so skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. For example, one such review was overly defensive about trusting another internal service, which resulted in Claude adding excessive validation and complexity. Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.

We were able to substantially improve optimization and quality. By virtue of Claude’s ability to write immense amounts of code much faster than we could ourselves, Scion is among the most well-tested and well-defended codebases we have ever worked on. Even on our tight timeline, we were able to build hermetic simulations of every non-trivial infrastructure and source control interaction we’ve ever encountered, as well as fuzzing and sophisticated rate limiting. Claude also casually found significant optimizations—for example, an algorithmic optimization to ancestry queries that missed one of our indexes—that would have taken an expert days or weeks of analysis and tuning… and then did it all again the next day.

We laid the groundwork for new forms of monorepo security. Managing safety, security, and compliance in a massive monorepo is as important as managing the size and scale. We built Scion with these in mind from the start. It supports rich access control on repositories, as well as the ability to apply permissions to individual directories in order to protect the most sensitive code without losing the benefits of a monorepo. Furthermore, the code review tooling we built can express complex semantic constraints on approvals, which lets us differentiate simple things like comment-only changes from sensitive logical changes, so that human reviewers can focus on the most consequential changes.

A new level of operational excellence is possible. Our team uses Claude Tag heavily, which has helped us monitor each component of the system. For example, we created Slack channels where we tasked Claude with being obsessively focused on the health of a single component. We call these Claudes “superfans.” We have a scion-daemon superfan, a scion-cache superfan, an Antique merge queue superfan, and so on. They have access to our telemetry and logs, and act like an experienced engineer who has nothing better to do than to refresh charts and look for anything strange. We also have traditional alerts for all of our systems, but our superfans can detect subtle and novel issues that make these alerts seem one-dimensional by comparison.

We’re completely rethinking our priors. This project prompted us to take a very hard look at other workflows designed for human scale. A proper merge queue, for example, doesn’t make much sense at agent scale. What does CI even mean when hundreds or even thousands of changes can merge while tests are running? We haven’t been able to effectively replicate the industry standard of a global repository lock on the merge queue, so instead, we’ve invested in post-merge and pre-deploy automated testing. This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.

Contributing to the wider ecosystem

We see Scion as just one part of the answer to the scaling concerns that increasingly capable AI agents have raised. We’re sharing these details as an example of how teams can tackle similar challenges, though we don’t expect our solution, or our considerations, to suit every case.

Scion is by no means a suitable replacement for git for general use. The concepts we’ve outlined here really only make sense for truly enormous monorepos, with either tens of thousands of engineers or swarms of agents prolifically writing code. Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. We do this both to address our own software development needs, and to test Claude’s capabilities.

Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.

However, we anticipate that this class of problems will become more commonplace over the next six to 12 months, and it would be an inefficient use of engineering resources for every company using git to face similar challenges alone. Given that, we think the most impactful thing we can do is to share our thinking, so that other developers can apply some of the most successful parts of Scion to git without departing from the overall model. A git server doesn’t have to work the same way as a git client does; novel server architectures exist that provide significant scaling improvements. Git platforms can provide comprehensive server-side RPCs like Scion, perhaps even in a standardized way across forges that the git client can learn to talk to. The negotiation protocol doesn’t have to look exactly like it does today, either, and the various sparse and shallow features in git can be made to compose well.

Lastly, we want to acknowledge that the git community has spent nearly two decades scaling the platform for millions of developers. Our contribution is far narrower: sharing our early view into how agentic software development interacts with these systems, so that we can partner with GitHub, GitLab, and the OSS community to support the changing needs of developers. We’re all in it together as the software development lifecycle intersects with the AI exponential.

Footnotes

01Storing changed files relative to the first parent of a merge is a stylistic choice; in practice, we’ve found that people only care about history or blame for commits on the main line in trunk-based development in a monorepo.

02Specifically, the server component of Sapling, Mononoke. It pioneered many of the techniques we use in Scion, in particular the Bonsai data model.

Authors & Acknowledgments

Acknowledgements

Written by Nick G. and the source control team: S.G., R.K., Dan C., Garrett H., Ben L., Colin M., Joe P. We’re grateful to Vilas J. and Sergiu D. for supporting the source control team on this ambitious journey and Amir O. for early contributions to Antique.

Full article content is not available for inline reading.

Read the original article →

When we previously blogged about our AI code reviewer in April, an average review cost a palatable $1.19. By the end of August, that figure had shot up to about $2.46. We hadn't made any significant architectural changes: the coordinator was still handing work to the same set of specialist reviewers, and the same risk tiers decided how intensive of a review we gave.

Around mid-September we tasked our agents with working out where those tokens were going and cutting them — aggressively. With code quality at high effort, the average review now costs just $0.77, down 68% from the $2.43 it averaged over our two most expensive weeks before we started. Our main goal was to cut costs, but as part of this work we also managed to optimize the reviewer, so that the median review now takes just 1m 32s instead of the 4m 09s it took before.

About 40% of the savings came from changes and optimizations to the reviewer's code, and the rest from experimenting with which models we use and how much effort they put in.

How a review doubled in cost

The chart below shows the cost of an average review on each weekday since March. It holds the mix of review tiers fixed, so a day with more large reviews doesn't skew the data.

The cost per review steadily rose from $1.22 (March) → $1.44 (June) → $1.74 (July) and finally, to $2.32 (August). A small part of that is that more reviews landed in the most expensive tier (59% of reviews in March and then 65% in August). We believe there’s a simple answer for why: models are getting much better, and much more capable. Naturally, engineers are producing bigger, more complex changes which means more of them fall into our “full review” tier. We found that most of the cost increase actually came from model changes over those months, though.

At the start of August we moved every reviewer onto Anthropic models, which raised the cost per job from $1.38 to $2.03. We made this choice because we found that during our evals, Anthropic models performed better on this specific workload than other models.

Our internal standards reviewer, Codex, ended up on Claude Opus 4.7 due to a mix of performance and reliability. That took it from about $0.25 per review to $0.88 in August, which made it more expensive than the coordinator agent itself.

The cache hit rate also increased, going from 85.7% in April to 97% by September, while the number of tokens per review more than tripled, from 0.91 million to 2.99 million. A high hit rate on three times as many tokens still costs a lot more, even with the low cache read cost.

What else has changed since our last post?

One of the biggest changes is that the reviewer now runs on OpenCode v2, which is a ground-up rewrite from v1. Before switching all reviews over from v1 to v2, we ran every review in every canary repo on both v1 and v2 in a comparative shadow mode. The two agreed on the approval decision in 98% of the merge requests we tested, so we moved everyone over in one go in early October and removed the v1 code. Comparing two days with the same models, full-tier reviews on v2 finish about 13% faster. We’re also currently experimenting with other ways to run our internal AI reviewer, including solely on Durable Objects using pi-durable. We’re hopefully going to have more on this coming soon, so keep an eye out!

So, where were the tokens going?

Right before we started focusing on cost optimization, our AI review spend was split by: output tokens making up about 23%, cache writes cost 43%, and cache reads cost 34%. This means that most of the money was going on context the model had already seen or was about to see again.

That makes complete sense when you look into a single session. The code quality reviewer made 16.6 large language model (LLM) calls per review on average and over those calls it wrote 72k tokens to the cache and read back 1.08m, because each call sends the whole conversation as context. A session's cost grows with both the number of calls and the size of the context, and over the last few months we had slowly increased both.

It’s worth taking a few minutes to read about how Anthropic's prompt cache works. It works by caching parts of the prompt that have already been computed by the large language model (LLM). These are known as prefixes, because they only work on the messages at the start of the conversation, not mid way through or the end. This means that a future request can reuse an earlier prefix, up to the part of the conversation that changes from what’s been cached. In our case, the harness tools, system prompt and messages at the start of our AI reviewer sessions rarely change, so they’re a great candidate for prefix caching. A prefix is only stored where a cache breakpoint ends it. Writing to the cache costs 1.25 times the normal input price and reading costs 0.1 times, and an entry expires after five minutes unless something reads it.

That means we had two things to work on: make the first request of each session read from the cache instead of writing to it, and make each session use fewer calls.

Sharing a cached prefix between every repository

Beginning September 14, the reviewer recorded the token usage of each reviewer's first LLM call, plus a hash of the first system block exactly as it was sent to the provider.

The data showed that the first request of a session found a cached prefix only 14% to 26% of the time. Every session starts with the same tool definitions and the same agent prompt, between 11k and 21k tokens that are identical in every repository we review, so almost all of them should have been cache hits.

The problem ended up being in how OpenCode v1 constructed the system prompt: It built one block containing our agent prompt, followed by an environment section with the working directory and the date, and placed the only cache breakpoint at the end of that block. The shared part was never stored on its own, so a review in one repository could never reuse what a reviewer ran in another repository had written.

On September 16, we added a marker to the end of every agent prompt and a plugin hook that split the block in two at that marker. The first half now ended on bytes that every job shares, and got its own breakpoint. To keep it that way we had to remove anything repository-specific from in front of it, so the list of available skills is disabled and tool descriptions can't change between jobs.

The same day we also moved our internal standards (Codex) reviewer's reference index (a table of internal standards, a skill file and a list of reference files) into its prompt. Until then every session had spent its first three turns reading those files with tool calls. Its LLM calls per run decreased from 15.9 to 9.6, and its cost per run dropped from $1.12 to $0.72.

Together these took the cost per review from $2.47 down to $2.02, and the drop was noticeable as soon as we kicked off the release.

Fewer calls, fewer dollars

The next set of changes went out on September 21 and 23. None of them were particularly impressive by themselves, but together they took about $0.20 off every review.

The most involved change was sharing one prefix between three reviewers. The code quality, security, and performance reviewers now send identical first requests with the same tools, a shared rules file, then the merge request with every patch included. Each reviewer's own instructions follow after that, and we make sure that one of them writes the cache and the other two read from it.

A cache entry can't be read until the response that wrote it has started streaming, and the three reviewers start at the same time. So the second and third wait for the first one's response to begin, for up to 20 seconds, before sending their own first request. It slows them down slightly and saves about nine cents per reviewer.

Changing models and effort

When we split the period at every release and every configuration change, the code changes turn out to be the smaller part of the saving.

Code changes account for $0.65 of the $1.67, and model and effort changes for $1.06. Each step compares the weekday hours before a change with the hours after it, so this isn't a statistically sound controlled experiment.

The coordinator had been running three model and effort combinations since August. On the same model, full-tier jobs on medium effort finished 20% sooner, so we dropped the other two. In a later randomized split, Opus 5.5 at medium showed no measurable change in review outcomes or critical findings compared with Opus 5, so we switched. Opus 5.5 at high effort cost more than Opus 5.5 at medium while not producing a meaningful increase in findings.

Opus 5.5 is cheaper per token than Opus 5, but price is only responsible for about $0.20 of the drop. The rest comes from sending and generating fewer tokens. Since the start of September cache reads per review are down 82%, cache writes down 54%, and output down 67%.

The most recent change was a new model. On October 3, we split every Sonnet reviewer 50/50 between Sonnet 5 and Sonnet 5.5. After two and a half days, Sonnet 5.5 was 62% cheaper per run for code quality, 58% for performance and 47% for security. The two models cost the same per token, so all of that comes from using fewer tokens. On full-tier reviews, the 5.5 reviewers posted about as many critical and warning findings as the Sonnet 5 ones, so on October 5 we moved all of them to 5.5.

On Sonnet 5.5 at medium effort, the code quality reviewer makes 1.7 LLM calls per run instead of 6.2, and more than half of its runs finish in a single call, which means they don't open any other file in the repository. At high effort it makes 3.4 calls per run. We were concerned that this made reviews shallower, which we cover in the quality section below.

To see whether more effort brings findings back on the larger tiers, the code quality reviewer has been split 50/50 between medium and high effort on 5.5 since October 5. After one day, high effort costs 61% more per run and posts 58% more critical and warning findings. That's about two cents per review, and the quality section below shows those extra findings are worth having, so we're moving code quality to high effort. The cost figures in this post assume that.

Latency

We weren't aiming to make reviews faster, but fewer calls also means less waiting, which means faster reviews and happier engineers.

The slowest reviews improved the most. In the four weeks before September 14, 37.9% of reviews took longer than five minutes and now just 0.7% do. The 99th percentile today is faster than three quarters of reviews were in April. Every tier got faster along the way with the median full-tier review going from 5m 16s to 1m 54s, lite from 2m 58s to 1m 14s, and trivial from 77 seconds to 30.

By August, the median review was 14% slower than what we reported in April and the 99th percentile 48% slower, so a good part of this work just got us back to where we had been.

Review quality

A cheaper review isn't worth much if it also finds fewer problems, and the first chart here looks like that's what happened.

With code quality on Sonnet 5.5 at high effort, it posts 40% fewer critical and warning findings than it did in late August. Security and performance, which stay at medium effort, post 56% and 71% fewer. A fall in severe findings doesn't tell you on its own whether real problems are being missed or noise is being cut, so we looked at it from two directions.

First, we checked whether authors act on what the reviewer flags. For samples of critical and warning findings, we compared the commit that was reviewed with the author's next push, and counted a finding as acted on if the lines it pointed at had changed. Lines 30 to 40 away from each finding changed about 15% of the time in every period, which gives a baseline for edits that happen by coincidence.

Before the changes, authors acted on 34% of the code quality reviewer's serious findings. Now it's 59%. The reviewer flags fewer things, but those it flags are much more likely to lead to a change, so the number of findings that led to a change per 1,000 reviews is about the same. We now saw 132 against 145 before, which is within the margin of error. Across all four reviewers, serious findings that led to a change went from 229 to 321 per 1,000 reviews.

Second, we had two models from different providers judge a blinded sample of 657 findings from the three Sonnet reviewers, with the diff each one referred to and without knowing which period it came from. Disagreements went to a third model. Each finding was labelled important, minor, duplicate, wrong, or unverifiable, and its severity was marked as fair, too high, or too low.

Most of what disappeared was minor findings, plus some duplicates and findings the judges couldn't verify from the diff. Important findings per review went from 0.28 before to 0.30 now, and the intervals overlap. The bigger change was in severity. Before, the judges rated 0.46 critical or warning findings per review as more severe than they deserved. Now that's 0.14, a 70% drop and the clearest result in the research. Much of the drop in critical findings is the reviewer no longer calling minor things critical.

So the answer depends on the reviewer. For code quality at high effort, what disappeared was mostly noise and inflated severity, and it still finds about as many real problems as before. Medium effort did lose real findings. In the randomized split, code quality at medium led to roughly a quarter as many acted-on findings as at high, which is why we're moving it to high. Performance also lost real findings: the judges found about 64% fewer important performance issues per review, which in practice is about one fewer important finding every 24 full-tier reviews. Security fell by about a quarter, though that isn't statistically significant. Both are still at medium effort, and testing them at high is next.

Neither method is perfect. An author changing the flagged lines shows they did something, not that the finding was right. The judges are models too, and there are only 35 to 66 findings in each group. The "now" window is also only a day long. Even so, the two methods agree with each other, and neither suggests that code quality is finding fewer real problems.

Reliability has held up as well. Reviews that end without a verdict went from 0.79% to 0.27%, and security reviewer retries from 3.5% to 0.04%.

Where this leaves us

An average review now costs $0.77 instead of $2.43, and the median one finishes in 1m 32s instead of 4m 9s. The biggest single saving came from checking what actually reached the provider on each session’s first request, and finding a way to order those messages in a way that was as consistent as possible across commits, merge requests and repositories. The remaining cost savings came from relentlessly experimenting with different models and effort mixes, while making sure we put any preconceived bias for particular models aside.

Lower cost reviews didn’t automatically mean worse quality, but it didn’t automatically mean fine either. Code quality at high effort finds about as many real problems as before, with much less noise and far fewer inflated severities. Performance lost real findings at medium effort, and security may have, too, so testing both at high effort is next. If you run agents at any kind of volume, the two things we’d check first are whether each session’s first request actually reads from the cache, and whether your reviewers still find real problems at the effort level you’re paying for.

If you’re at Cloudflare Connect in San Francisco, I’m talking about how we built the reviewer instead of buying one in my session, It’s time to build, not buy, on Tuesday, October 20 at 12:30 PM PDT. I’ll also be in the Speaker Connection Lounge from 1:10 PM if you want to ask about any of the numbers in this post.

Announcing Wiz Runtime Sensor add-on for GKE: Turnkey threat detection for Kubernetes

Everything is moving faster in the age of AI, and that includes security threats. What used to take attackers weeks to exploit can now happen in just a few minutes.

Standard vulnerability scans that run on artifacts at rest can catch basic setup issues, but to detect an attack that's actively happening you need real-time runtime protection.

Today, we’re introducing the Wiz Runtime Sensor add-on for Google Kubernetes Engine (GKE), in preview. This built-in integration brings low-touch deployment, automated lifecycle management, enterprise credential security, and fleet-wide governance to Wiz's industry-leading eBPF runtime threat detection for both Standard and Autopilot GKE clusters.

The operational friction of runtime security

Security and platform engineering teams share common dilemmas when deploying in-cluster runtime defenses:

• Deployment and lifecycle friction: Deploying runtime agents often requires managing installations from third-party sources across dozens or hundreds of clusters.
• Credential sprawl and security risks: Passing and rotating API keys as Kubernetes Secrets introduces administrative overhead and security risk.
• Fleet-wide enforcement gaps: Platform teams lack built-in policy controls to enforce that every new cluster has runtime sensors enabled before workloads are deployed and start receiving traffic.
• Performance and operational overhead: Without explicit and strategic resource limits, runtime agents can impact how much CPU and memory is available for your workloads and system functionality.

With the Wiz Runtime Sensor for GKE, Google Cloud and Wiz have minimized these operational barriers. The Runtime Sensor is now available in GKE as a first-party, managed platform add-on.

Bridging the gap with Wiz Runtime Sensor for GKE

The Wiz Runtime Sensor for GKE connects central security and platform engineering workflows by simplifying and automating the sensor lifecycle.

With the add-on, GKE manages the sensor deployment, credential injection, and lifecycle updates. Building runtime protection directly into the platform reduces administrative overhead and provides the following benefits for your Kubernetes fleet:

Native Secret Manager integration: Security teams no longer need to distribute Wiz API keys across cluster namespaces. The add-on uses GKE's Secret Manager secret sync integration to store your Wiz credentials in an off-cluster, centralized credential vault that only a dedicated service account can access.

GKE automatically retrieves the credentials and injects them directly into the Runtime Sensor pods. Updating a key in Secret Manager automatically syncs to the sensor without requiring cluster restarts or manual intervention.

• Simplified enablement: After you add your Wiz credentials to Secret Manager, platform operators can enable the Runtime Sensor add-on by using the gcloud CLI or the GKE API. The GKE control plane handles deploying the sensor components, managing the lifecycle of pods, and configuring the nodes.
• Secure egress and proxy support: For environments that restrict public internet access, the managed sensor supports standard HTTP and mTLS proxy configurations. Telemetry is routed back to Wiz without violating your egress firewall policies.
• Fleet-wide governance with organization policy: Platform security leads can enforce compliance across their entire organization using custom organization policy constraints. You can use a constraint to reject any attempts to create a cluster that doesn't enable the Runtime Sensor add-on, which can prevent malicious actors from spinning up rogue infrastructure to evade runtime detection.
• Kernel-level observability with minimal overhead: The Wiz Runtime Sensor continuously monitors system calls, running processes, network connections, and file access. It catches suspicious activity, including reverse shells, unauthorized privilege escalations, container escapes, and unsafe outbound connections while keeping CPU usage low even under heavy workload activity. The sensor uses a fixed memory limit on each node to ensure consistent performance as workload density changes. This is especially valuable for AI workloads and web services where application activity might fluctuate quickly.
• Independent auto-updates: Security threats and zero-day vulnerabilities don’t wait for scheduled infrastructure maintenance windows. The lifecycle of the Wiz sensor add-on for GKE is independent of your GKE cluster and node lifecycle. The sensor periodically rolls out updates of new sensor versions published by Wiz, to help ensure that your clusters always have the latest detection rules without requiring GKE version bumps, node pool recreations, or workload disruption.

How it works: Deploying and enforcing the sensor

Enabling the Wiz sensor add-on follows the same process that platform administrators are already familiar with on GKE. Using standard gcloud commands, you can activate runtime protection during cluster creation, turn it on for existing clusters, and enforce it across your fleet while keeping credentials securely managed in Secret Manager.

Provisioning a cluster with the Wiz Sensor add-on

gcloud container clusters create production-ai-cluster \
 --zone=us-central1-a \
 --release-channel=regular \
 --workload-pool=${PROJECT_ID}.svc.id.goog \
 --enable-secret-sync \
 --enable-secret-sync-rotation \
 --addons=WizSensor=ENABLED \
 --wiz-sensor-registry-secret-uri=projects/${PROJECT_ID}/secrets/wiz-registry-secret/versions/latest \
 --wiz-sensor-api-key-secret-uri=projects/${PROJECT_ID}/secrets/wiz-api-key-secret/versions/latest

To enable the sensor on an existing running cluster with Workload Identity Federation for GKE and Secret Manager secret sync enabled, run the following:

gcloud container clusters update production-ai-cluster \
 --zone=us-central1-a \
 --update-addons=WizSensor=ENABLED \
 --wiz-sensor-registry-secret-uri=projects/${PROJECT_ID}/secrets/wiz-registry-secret/versions/latest \
 --wiz-sensor-api-key-secret-uri=projects/${PROJECT_ID}/secrets/wiz-api-key-secret/versions/latest

Enforcing fleet-wide protection with organization policy

You can require all new GKE clusters across your organization to install the runtime sensor by deploying a custom constraint:

name: organizations/123456789012/customConstraints/custom.GKEWizSensorEnforced
resource_types:
- container.googleapis.com/Cluster
method_types:
- CREATE
- UPDATE
condition: resource.addonsConfig.wizSensorConfig.enabled == true
action_type: ALLOW
display_name: Enforce Wiz Sensor on All GKE Clusters
description: Mandates that all GKE clusters enable the managed Wiz Runtime Sensor backed by the corporate Secret Manager vault.

Intelligent threat correlation for AI security

The true power of this integration emerges when the managed sensor connects to the broader Wiz AI Application Protection Platform (AI-APP). By feeding GKE runtime telemetry into the Wiz Security Graph, you unlock a security toolkit that goes beyond static alerting, with:

• Real-time risk assessment and threat verification: By pairing background security scans with live activity monitoring, you get a clear view of real risks. The sensor checks whether vulnerable code is actually running in memory, so you can focus on active security threats and ignore inactive ones.
• Real-time threat detection and response: Identify suspicious activity like malware, unauthorized access attempts, and software vulnerabilities in real time. Security teams can set rules to automatically block or stop threats before they can spread.
• Network activity and detailed security insights: Track system activity, network connections, and data flows to uncover hidden risks and unauthorized software usage. If a threat is detected, the sensor automatically gathers diagnostic data to help teams quickly investigate what happened.

Get started today

The Wiz Runtime Sensor add-on for GKE delivers a standard for container security: deep, kernel-level eBPF protection without operational complexity or deployment friction.

By combining GKE's managed platform automation with Wiz's comprehensive security intelligence, platform and security teams can innovate at cloud speed while maintaining strong runtime defense. Check out the Wiz Sensor for GKE documentation to learn more.

Despite Figma co-founder Dylan Field’s concerns about MCP (model context protocol), which standardizes how AI agents connect to external data sources and tools, the company is expanding its support for the open source standard.

At the design company’s first Config conference held in India, the company announced 20 new MCP partners, including Amp, ClickUp, Dia, Junie by JetBrains, Linear, Meta Muse, Mistral, Notion, OpenCode, and Sarvam Agents.

Field has expressed his misgivings about MCP before. Days before this announcement, one of MCP’s co-creators complained about Figma not supporting the open source tool Pi. As part of his response, Field made a comment about Anthropic not making its various resources available via MCP.

“I find it interesting that Anthropic (creators of MCP standard) do not expose artifacts/other cloud resources created via MCP,” he wrote on X. “The debate is framed as open vs closed. Everyone always prefers open … obviously. And meanwhile issues of asymmetric context sharing are ignored.”

(Field later added that Figma plans support Pi.)

Can you help me understand why you don't make various Anthropic resources available via MCP? Maybe I'm missing something.

— Dylan Field (@zoink) September 30, 2026

At a media briefing, I asked Field about this debate. The co-founder explained that while Figma wants to be open, he doesn’t think MCP is a perfect protocol.

“I’m really excited about what we’re doing in terms of opening up our ecosystem. I also think that MCP is not a perfect protocol. And I think that it is something that has tons of room for improvement,” the co-founder responded. “It is advantaged more to frontier labs. Also, I understand everyone online who just wants a protocol that they can use when they want to use their Figma data, and I want them to use it, too.”

Figma’s CEO said that the reason the company doesn’t want to be fully open is that the more data the company exposes through MCP, the higher the chance of abuse.

He also clarified that while most users don’t intend to use MCP or Figma data in a malicious way, the company still has to think about those potential risks.

Open social network Bluesky is once again expanding beyond its flagship app. At TechCrunch Disrupt 2026 on Thursday, the company announced the launch of a new tool that allows creators to make websites, quizzes, games, polls, and other interactive experiences.

The tool is an expansion of Attie, Bluesky’s more experimental platform, which initially offered AI-powered technology for building custom social feeds. With its new creation capabilities, the idea is to allow everyday users to create the kind of experiences that previously only software developers could build.

These user-created games, sites, and other experiences will also live on the same AT Protocol (ATProto) that powers Bluesky and other decentralized social apps that make up the decentralized social web known as the Atmosphere.

Bluesky CEO Toni Schneider shared the news of Attie’s new capabilities during an interview at Disrupt, saying, “We’re adding a feature to our Attie product that allows you to build sites and social experiences again for a group of people. So think of it as … social vibe coding or vibe coding for your community.”

The exec, a former CEO of Automattic and a partner at True Ventures, had previously been leading the company on an interim basis after former CEO Jay Graber stepped down in March to become Bluesky’s chief innovation officer. Now he’s taking charge of the company as it enters the AI era.

Attie offers Bluesky a playground where it can try out AI-powered ideas like this.

“Vibe coding today is kind of a single-user, solo experience,” Schneider explained. “And what I want to be able to do is build together or build for a group of people, and that’s what this product does.”

When creators build with Attie, they’re connected to their ATProto identity. Plus, Attie will suggest things users might want to make based on their public Bluesky profiles.

To build with Attie, users can bring their own AI model or use open-weight options, and the resulting sites and games can then tap into Bluesky’s wider audience, noted Bluesky COO Rose Wang.

“Everybody is building … But the big question is: Who’s using the thing you’re building, because we haven’t solved that problem with distribution,” she said. “What makes Attie so interesting is that it comes with the Bluesky distribution, so it doesn’t stand alone.”

That network today has 47 million registered users, though it’s seen a decline in engagement, as some users who tested it out later returned to X.

Beyond being a testing ground for new ideas, Attie ties into Bluesky’s larger plans to reinvigorate activity on its network by focusing on both developers and communities in the months ahead. Those areas will also be supercharged by the protocol’s forthcoming support for private data, which Schneider said has been the biggest request from developers. Private data is now in alpha testing, with plans for a rollout by the end of the year, he said.

“There should be lots and lots of these fun, small social experiences for your friends,” said Schneider, referencing the types of projects Attie will enable. “We don’t all have to go communicate with the entire world all the time.”

Almost 96% of the top million home pages fail at least one automated accessibility check, and the same errors have led WebAIM's report for seven years. Low contrast, missing alt text, unlabeled inputs, and empty links and buttons top the list, each fixable without a rebuild. Automated scans catch only about 30 to 40 percent of issues, so keyboard and screen reader testing still has to happen before release.

Treating UI as a pure function of state lets developers see how a screen varies across roles, plans, billing statuses, and flags, which matters more now that agents take on front-end implementation work. In a worked example, an agent that tested only an owner account reported success on a delete action, yet viewers still saw the button despite a “View only” header. Rendering every state combination side by side exposes such gaps, so teams should make display-relevant state explicit and ask what an agent actually checked before merging.

Following accessibility best practices doesn't guarantee legal compliance, and the rules differ across the US, UK, and EU. A four-question check covering market, provider, product type, and exemptions helps identify which of the ADA, Section 508, the European Accessibility Act, and UK laws may apply. Cases involving Domino's, Target, and Fashion Nova show the legal and commercial cost of ignoring them.

DeepInfer combines distributed inference, caching, load balancing, and speculative decoding to improve efficiency. DeepSeek processes 155 trillion tokens daily per cluster and earns $38 in revenue per GPU-hour while open-sourcing much of its inference framework.

Laminar is an in-house agent harness for Ramp's agentic products. It features secure conversation history storage, session compaction, standard interfaces for tool calling, message queuing and steering, subagents, multiplayer chat support, observability tooling, and much more. The first version of Laminar was built in six weeks with a small team of obsessive, experienced engineers who worked with and steered the agents. Its code was generated entirely by agents using the most powerful models available at the time. While building in-house has become cheaper, Ramp still needs good engineers to be on-call, keep the platform healthy, and figure out what product teams actually need and how it fits into the bigger picture.

OpenAI acquired Orchestra Bio to combine its AI-powered drug development coordination platform with Rosalind, its suite of life sciences AI tools.

Meta’s AI companion, Muse, is now one of only a dozen mobile apps to have reached 10 million U.S. downloads in its first 40 days on the market. According to new data from market intelligence provider Sensor Tower, Muse has seen 10.9 million downloads since its September debut, 10.2 million of which were in the United States.

Only 11 other apps to date have reached this type of post-launch milestone, said Sensor Tower’s senior insights analyst Kara Lee. The most recent of these was Meta’s Threads, back in July 2023.

Other apps that saw this kind of scale were largely mobile games, Lee noted, like Fortnite, PUBG Mobile, Pokémon GO, and others. Besides Threads, Disney+ was the only other nongame app to make it into the exclusive club.

Meta’s success is tied to the company’s ability to scale its new apps by marketing them across its other services, which it has done with both Threads and Muse. Threads was built on top of Instagram’s social graph, and its content has been integrated into both Facebook and Instagram. However, Muse isn’t only benefiting from Meta’s ability to market the AI agent across its family of apps, which today includes 3.6 billion daily active users; the company is also heavily investing in advertising to drive new users to Muse.

Per Sensor Tower’s data, Meta has allocated 39% of its total advertising spend to market Muse in October, up from 23% last month. In addition, Muse now ranks as the 21st largest brand in the U.S. by advertising impressions this month, up from 73rd in September.

Still, marketing an app and retaining users after the initial shine wears off are two different things. In this case, Muse is seeing better retention rates than its AI competitors like ChatGPT, Claude, and Grok.

In the U.S., the app had a 30% retention rate after 7 days and a 23% retention rate after 14 days. By comparison, its rivals saw an average of a 19% 7-day retention rate and a 16% 14-day retention rate, according to the firm’s data.

Muse also hit another milestone that demonstrates active engagement with the AI app. Its daily active users reached 2.5 million for the first time on October 16, a 15% week-over-week increase.

Meta’s latest AI app, which can handle various tasks on users’ behalf, has seen breakout success since its debut. It shot up to No. 1 on the App Store and at one point even outpaced some of the metrics ChatGPT saw following its launch. Its latest milestone sees it topping its rivals again, as ChatGPT saw 3.7 million U.S. downloads in its first 40 days on the market. Grok and Claude saw only 500,000 and 200,000 U.S. downloads, respectively, in their first 40 days.

Globally, ChatGPT beat out Muse, having scaled to around 15 million global downloads in its first 40 days. But this isn’t a fair comparison, as ChatGPT benefited from a wider reach. Muse is only available in the U.S. and Canada for now, while ChatGPT’s app attracted attention from markets like Japan, Brazil, and Germany, too. (Japan, for instance, accounted for 25% of its global installs during its first 40 days.)

Since launching, Muse has retained its No. 1 rank on the U.S. App Store for all days except October 19, when streaming app Kick briefly beat it out. It has since reclaimed the top spot as of Tuesday. Muse has also been No. 1 on Google Play over the past 30 days.

OpenAI announced on Tuesday that users can now lock ChatGPT conversations to keep sensitive chats private when others may have access to their screen or device. Users can secure their locked chats with a four-digit PIN, or use Face ID or Touch ID on supported devices.

Unlike the app’s Temporary Chats, which provide a separate session that doesn’t appear in chat history, users can return to Locked Chats at any time.

The company says the feature is designed for situations where users might be discussing something personal or planning a surprise and want to keep those conversations private when handing their device to friends or family to show them something in another chat.

The addition of locked chats adds a new layer of privacy to ChatGPT, as the app is increasingly used for personal conversations. Of course, while locking a chat can limit access, it doesn’t address concerns about how sensitive information is stored or handled.

OpenAI notes that locking a chat doesn’t create end-to-end encryption or a separate encrypted vault, change how long it retains the chat, remove information that ChatGPT remembered before a chat was locked, or change the “Improve the model for everyone” setting.

Once users lock a chat, it moves to a dedicated “Locked Chats” section, where it’s kept separate from regular chat history, pinned conversations, and chat search results.

With Memory enabled, Locked Chats has its own separate memory shared across conversations within the section. New messages sent after locking a chat can update that memory but don’t update regular Memory. That means new information from locked chats won’t leak into regular chats.

OpenAI says resetting a PIN permanently deletes all locked chats, files, and other content protected by the previous PIN.

Additionally, the Locked Chats option isn’t available for temporary chats, archived chats, group chats, chats created from a shared link, chats with scheduled tasks, and chats with meeting recordings.

Locked Chats will be available to users on ChatGPT Free, Go, Plus, Pro, Business, Enterprise, and Edu plans. The company notes that the feature is not available on ChatGPT for Healthcare. The feature is starting to roll out on Tuesday, so it may not be live yet for all users.

SpaceX and Tesla have formed special committees to discuss a potential merger. While Elon Musk thinks there's a lot of merit in combining the companies, the special committees have yet to reach an agreement, and there's no guarantee that they will. Musk is not on either committee. Shareholders will get to vote on the matter. A deal this large will attract regulatory scrutiny.

I Paid $60 for Two Humanoids to Clean My Home In San Francisco

What a cleaning visit revealed about the work and economics of home robotics.

On Sunday, I paid $60 for 2 humanoids from Tau Robotics to clean my home in San Francisco for an hour. Sergey and Chelsea (the name of the two robots) tidied, wiped surfaces, vacuumed, mopped and cleaned the toilet. After an hour, they finished cleaning the dining, living room, kitchen and 1 half bathroom.

Each robot had a human operator controlling it remotely from the company’s office in the Mission. A staff member dropped them off and picked them up. For most of the hour, there was no company staff member in the house.

This experience reinforced my belief that home robots should be available to rent by the task or hour. For the user, paying $60 to try a cleaning service is a much smaller commitment than buying a robot or taking on a monthly subscription. For the robotics company, sending the same robots into different homes could increase utilization and give it more opportunities to collect data and improve both hardware and software before the next deployment.

From observing this visit, I also saw how much work surrounds that hour of cleaning. Below I share a) my experience as a first time user of a humanoid cleaning service, b) the economics for the user and for the company, c) what a future with humanoids and advanced robots doing productive work around us would look like.

If you’ve hired a humanoid to clean your home, be it in SF, or Shenzhen, or elsewhere, I’d love to hear about your experience.

1. The task is the product

I paid $30 per robot-hour, but the booking ended with work unfinished. Do I care whether it’s teleoperated or fully autonomous? No. What matters most is to have the whole home cleaned.

Around my neighborhood in San Francisco, a full-home clean costs roughly $200–220 for 2 human cleaners working about 2.5 hours. That comes to $40-44 per worker-hour. The robot service has a lower hourly price, but my $60 booking left the job unfinished. In later correspondence, one of the founders, Cornelia Weinzierl, shared that they could potentially do a longer visit by swapping batteries after an hour.

What can the cleaning robots do?

Vacuuming, mopping and wiping surfaces seem easy for them. They were doing it almost as well as a human.

The robots seemed slower on delicate tasks, such as moving objects to clean a surface and picking up small or soft objects like toys and socks. Chelsea dropped a toy, struggled with a pillow and had trouble balancing on the playmat. For the tasks they completed, grippers were enough.

They didn’t attempt unloading my dishwasher, as it was running at the time. After the service, I learned from another user that it was described as an “advanced task” by the company. The robots also didn’t fold our laundry. Weinzierl later told me that loading and unloading the dishwasher are possible but slow, and that some dishes may be placed at unusual angles. She also said the robots can “fold T-shirts, pants, shorts, skirts, diverse types of towels, roll up socks. Hang shirts, blouses, dresses, light jackets and similar. They struggle with folding bedsheets or tight sports long sleeve shirts if they are inside out.” Frankly, I find that pretty impressive with their grippers. That suggests five fingers may not be necessary for these household tasks.

I was told the batteries needed swapping after about an hour. Extending my booking would have required swapping the battery.

2. The economics extend beyond the house

Tau Robotics charged $30 per billed robot-hour. Beyond Sergey and Chelsea’s labor, the visit required several kinds of human support:

1. A staff member handled drop-off and pickup. The robots also needed help taking their shoes off and putting them back on, which the staff member handled at drop-off and pickup.
2. One operator was dedicated to teleoperating each robot. They were Tau staff working from the Mission, according to the person who dropped off the robots.
3. A driver transported them in a van.
4. Someone responded to my text request.
5. Between visits, batteries need swapping and equipment, clothes, socks and grippers need cleaning.

The case for optimizing utilization

At this stage of autonomy, I believe a shared fleet is a promising way to collect data and improve the service. Sending robots into more homes could expose them to different objects, layouts and failures. Collectively, this can bring a better experience for the user from each visit.

The value of collecting data in customers’ homes depends on how quickly it leads to better performance. For a comparable clean, that means handling difficult objects with less help and using fewer operator minutes. Weinzierl told me that for future visits, “It would probably start vacuuming parts autonomously. Still mostly teleoperated.” Lessons from those visits could also guide hardware changes that make battery swaps and delivery faster.

A low-hanging fruit to dial up the throughput would be apartment buildings or student housing. If enough residents book, a pair of robots could move between units throughout the day, with a place in the building to swap batteries and reset the equipment. This can increase the cleaning sessions per day, lower driver costs and could provide repeated practice in similar layouts.

3. Hospitality affects the product

Sergey waved when he came in. I would have liked both robots to introduce themselves. I didn’t initially realize they could respond by voice. When I texted a request to wipe the mirror, they did it.

Those interactions made a difference. So did the noise. They stomped around the house, which took a little getting used to. Another user told me the robots had scratched their hardwood floors when the robots weren’t wearing socks. For our visit, both wore gripper socks.

I accepted the fact that the data would be collected to train the robots. Other users may feel differently. What gets recorded, who can access it and whether a customer can opt out should be clear when booking.

4. What the robot economy looks like

The visit also got me thinking about what would need to exist around a larger fleet of working robots. The driver, battery swap and text message were small parts of my booking. Across a fleet, each becomes an operating responsibility.

Local depots and transport. A neighborhood depot could provide charging, battery swaps, equipment cleaning and repairs between jobs. Could robots eventually travel to the next booking in an Uber or Waymo? Where would robot charging stations be? Will they be in gas stations like Tesla charging ports? Someone would still need to solve loading, securing the robot, unloading and entry into the building. The van trip is part of a longer handoff.

Identity, safety and insurance. A household should be able to verify which company sent the robot and who is responsible for the visit. I would also want clear instructions for stopping it and a service contact if it malfunctions. What safety checks should a home-service robot pass? Who covers damage to the robot, the home or someone inside it?

Ratings and security. Booking history and customer feedback for each robot could help households choose a service and help operators find recurring problems. Connected robots also need controls over who can operate them and access household recordings. A customer should be able to manage access to their data and revoke that access.

Some of this could be handled by the robot company. Some could become services shared across fleets. There will be many new business opportunities from supporting robots doing productive work around humans.

Would I hire Sergey and Chelsea again? Yes, if they can finish the whole house. I would like to book about 2 hours. And I’d be curious to see them fold laundry and unload the dishwasher.

Stack Overflow built Stack Overflow for Agents in a few weeks with two engineers using Codex. A ChatGPT plugin reportedly tripled overnight registrations, and Codex cut enterprise connector build time from 3 to 6 weeks to 4 days.

Full article content is not available for inline reading.

Read the original article →

Apple is limiting iPhone Duo purchases to two per customer across markets including the US, UK, Japan, and Spain after shipping dates quickly slipped into January 2027. The restriction applies regardless of configuration and appears designed to reduce scalping, with resellers already listing devices for more than $5,000. Purchase limits cannot eliminate organized scalping entirely, but they could make bulk buying more difficult and improve availability for regular customers.

The free gradient background generator.

Create an animated gradient background right in your browser — free, with no signup. Choose from 20+ live shader modes, tune the colors in real time, and export a still PNG or an animated video with no watermark. Drop the result into a website, presentation, social post, product mockup, or video — or copy the embed code to run the live gradient on your own site.

When you need more than gradients, the full Backgrounds Supply library adds AI-generated backgrounds and animated video backdrops across 29 collections, plus a set of free wallpapers you can download today.

How to make a gradient background

1. Pick a shader mode: Choose from 20+ live animated gradient styles — from soft mesh blends to bold flowing shaders.
2. Tune your colors: Click any swatch and set your palette in real time. The gradient re-renders instantly as you edit.
3. Adjust motion and framing: Set the animation speed and pick an aspect ratio for your export — square, widescreen, or a custom size.
4. Export or embed: Download a still PNG or an animated video (MP4 or WebM) with no watermark, or copy the embed code to run the live gradient on your own site.

Gradient generator FAQ

• What is Gradient Lab?

  Gradient Lab is a free, in-browser gradient background generator from Backgrounds Supply. Pick from 20+ live animated shader modes, tune the colors in real time, then export a still PNG or an animated video — or grab an embed code to run the live gradient on your own site. Nothing to install, no account needed.

• Is Gradient Lab free?

  Yes — completely free, with no signup and no account. Everything runs in your browser, and every export is watermark-free.

• What can I export?

  Two things: a still PNG for static backgrounds, and an animated video for motion — export it as MP4 or WebM. You choose the format, the resolution (480p, 720p, 1080p, or 4K), and the length (up to 30 seconds). Everything exports with no watermark.

• Can I export in 4K?

  Yes. The animated video exports at up to 4K (3840×2160), so you can use it as a full-screen backdrop, a video-call background, or a high-resolution hero on any display.

• Does Gradient Lab export MP4?

  Yes. Choose MP4 in the export panel to download a standard H.264 MP4 that plays everywhere — social platforms, video editors, presentations, and every device. You can also export WebM if you prefer. Still images export as PNG.

• How many gradient styles are there?

  20+ animated shader modes — from soft Apple-style mesh blends to bold flowing, warping, and noise-driven gradients. Each one is fully editable, so the number of gradients you can make is effectively unlimited.

• Can I control the colors, motion, and aspect ratio?

  Yes. Click any swatch to set your exact palette, add or remove color points, set the animation speed, and choose the aspect ratio and resolution for your export. The gradient re-renders live as you edit.

• Can I embed a live gradient on my website?

  Yes. Copy the embed code to drop a live, animated gradient into any site as a lightweight iframe — no image or video files to host. It renders in real time in the visitor's browser.

• Can I share a gradient I made?

  Yes. Your settings are captured in the page URL, so you can copy the link and send it to anyone — they'll open Gradient Lab with your exact gradient loaded, ready to tweak or export.

• Does it work on mobile?

  Yes. Gradient Lab runs in any modern mobile browser with a touch-friendly interface, so you can build and export gradients from your phone or tablet.

• Can I use the gradients I make commercially?

  Yes. Gradients you create and export with Gradient Lab are yours to use in personal and commercial projects — websites, apps, social posts, presentations, and video. No attribution required.

• Do I need an account or signup?

  No. Gradient Lab runs entirely in your browser — no login, no email, nothing to install.

• What's the difference between Gradient Lab and the background packs?

  Gradient Lab makes gradients on the fly. The full Backgrounds Supply library adds 1,270+ handcrafted, AI-generated, and animated backgrounds across 29 collections — a one-time purchase with a commercial license.

Foursquare was acquired about a month ago, marking the end of a 17-year story. The acquisition was a $0 outcome for equity holders. This post captures the ending from a founder's point of view and details lessons learned from starting, scaling, and losing the company. In the end, the company was unable to survive a leadership change, leading to employee frustration and the zero-dollar outcome for equity holders.

The way we teach is often more important than the what.

Itaú is applying its AI-first strategy using OpenAI models, including the ia.i assistant that reached about 300,000 customers in its first phase.

A formal design degree is not necessarily required to build a successful creative career when years of experience, skills, self-study, and leadership already demonstrate professional ability. The 70-20-10 learning model suggests that most professional development comes from practical experience and relationships, with formal education accounting for a relatively small portion. Structured courses can still be valuable when they address a specific knowledge gap, build confidence, or support a desired career direction rather than simply validating an already proven career.

A curated collection of spinners, loaders, and animations for your next project.

3,800+ packs for Figma, Framer, Webflow, and code by 560+ independent designers.

HAL Typefaces' HAL Casino combines serif, sans, and monospaced styles within a single-weight family, allowing contrasting typographic voices to work together as one system. Its forms draw on historical serif designs and expressive swash lettering while balancing expressive details with enough consistency for the styles to be mixed freely. The result gives designers multiple distinct textures without requiring them to leave a single coordinated type family.

Brands increasingly chase launch-day novelty and short-lived visual trends at the expense of accumulated recognition and long-term usefulness. Durable identities instead build on principles, flexible systems, typography, proportion, composition, and other foundational elements that can evolve without abandoning the equity they have accumulated. Longevity does not require brands to remain visually static, but to preserve their core ideas while adapting their expression to new contexts over decades.

Hardware demands earlier commitment than software, since a released physical product can't be fixed with an update and design choices become costly to reverse.

Apple has reversed the icon-heavy menu design introduced with macOS Tahoe.