Introducing System One Models & Jev

Diogo Almeida, founder, TypeSafe

Models have been superhuman at chat for years, so where is all the automation?

This has been my driving question for the last four years. At OpenAI, I helped build the methods that made language models useful at following instructions and talking with people. That work ended up as the research behind ChatGPT. At the time, I thought maybe chat models would lead to AGI, but despite the hype it became obvious to me that there was something really big missing.

After two years in stealth, countless technical challenges, and research breakthroughs… I am beyond excited to announce that today, TypeSafe AI is releasing our first System One Model: a new class of frontier models built to make fast, structured decisions that software can use directly.

We built a new stack entirely focused on automation: with a new model architecture, parallel sampler for maximum efficiency, and training method we call Reinforcement Learning for Calibrated Decisions (RLCD).

Our first public model is Jev, available today in early access. Jev achieves similar levels of intelligence on System One tasks compared to existing LLMs, while being two orders of magnitude faster and more efficient. While Jev gives up string generation, it’s optimized for structured outputs and can’t hallucinate.

Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out.

Extraordinary claims require extraordinary evidence so see below for the receipts. 💅

Frontiers, Old and New

Evidence / Technical Results

We love skeptics, and are skeptics ourselves.

There are some claims you can easily verify:

• Speed per call: We truly are that fast, though our published evals are generally run from our laptops on the West Coast (this is where our service is currently based).
• Cost per call: We make our pricing transparent. We can’t prove it isn’t subsidized; we’ll need the long-term to prove the sustainability of our pricing (which we expect to go down, not up).
• No type errors: This would be an easy thing to falsify with just a single counter-example, but it is mathematically impossible.

For our bolder claims, we want to provide as much nuance as we can.

Side-by-side demonstration

Our side-by-side demo shows a key difference between our models and LLMs: Jev outputs all probabilities in parallel instead of autoregressively generating by token. Strings are extremely powerful and general, but costly. “Giving up” strings actually gives us a lot of superpowers!

Nuance

• For people with early access to TypeSafe, here is the actual query.
  • The query is highly simplified and questions were chosen to have descriptive, human-readable keys so that the output on the screen is understandable.
  • The state is also a short, dense, and detailed paragraph, to emphasize the difference in sampling methodology. The relatively shorter input paints our model in an advantageous light.
• For the keen eyed, for the recorded run, the only disagreement with GPT-5.6 Terra is on “Churn likelihood level”. The actual answer seems genuinely ambiguous to us.
• We used GPT-5.6 Terra with default reasoning for this example, because we’ve found it to be the most comparable at intelligence to Jev on average.
• Fun fact: a similar demo was what convinced us to go all-in in the direction of System One Models!

Workflow evals

We made a new type of evaluation to measure how well AI works within code. We don’t optimize for a ground truth classification or allow the harness and model to change (potentially allowing for overfitting via harness engineering). Instead, we assume there is a correct compute graph (a “workflow” represented in code) and use the predictions of the largest, smartest, and most expensive external models as reference probabilities.

Jev is off the charts – owning the Pareto frontier for almost 2 orders of magnitude. We also compare to models with a generated prompt doing all the logic in their chain-of-thought, but this tends to do significantly worse than using the workflow itself.

The most reliable real-world workflows tend to have many independent, decomposed questions, with fine-grained behavior that’s dependent on probabilities instead of discrete decisions. The end result is discrete branching, but how we get to a final answer involves a lot of domain-specific engineering that needs to be done highly consistently.

Nuance

• This is where the claims of 193.6x faster, 444.6x cheaper on our home page comes from, and we expect that these are on the higher end of real world gains.
• These content of these workflows were not deliberately chosen nor constructed to make our model look good, and are not in our training distribution. However, they were made by individuals on our model capabilities team, so some bias could exist.
• We use the average of GPT-6 Astra and Fable 5.1 as the reference answer, which biases answers towards OpenAI and Anthropic’s models. We likely underestimate the relative performance of our model and DeepSeek’s models.
• The LLMs use our System One LLM wrapper, which constrains LLMs to output structured decisions compatible with our API. We have found this to be the most accurate way to get decisions from LLMs, but this tends to be slower and more expensive than giving decisions without probabilities.

Hallucination and Type-safety

Hallucination and type-safety are intrinsically related, and we think the latter is table stakes for automation. Having a hallucinated tool call is inconvenient in an agent, but is an absolute deal-breaker if it’s part of a system with latency guarantees or it’s buried several layers deep in a dependency chain. Existing models, no matter how smart, still hallucinate and have type errors.

Nuance

• The numbers for LLMs are from OpenRouter i.e., there almost certainly is bias here: more complex queries might be routed to better models.
• Our number is not empirical. Schema matching is guaranteed, thus we can confidently add 0% into the plots.

Fun Demos

Perhaps the most exciting part of our work is enabling new use cases. We have a lot more to show you, but here are a couple of the team’s favorites:

Doom

We love how this demonstrates real-time intelligence and what can be done with code + AI. The engineer behind it was worried about making 10 queries a second (which ends up costing ~$7/hour), but the rest of us agreed that was lower than expected! This is so fun we intend to not only release an in-depth walkthrough, but also host some events to hack on this.

Wikiracing

The objective of the game is to start on one Wikipedia page and reach a specific other Wikipedia page using only links you come across while traversing. Each step can mean choosing between hundreds to thousands of links! It’s a great playground for demonstrating not just intelligence-per-second, but also the compounding benefits of not hallucinating with high-cardinality choices.

What’s next

We’re still in Jev’s early days. We have a lot more in the pipeline and are so excited to keep on shipping 🔥.

Today, we are opening early access and bringing developers off the waitlist as quickly as we can. We want to hear which decisions you need to automate, where Jev works, and where it falls short. Tell us what sci-fi you want to build!!

We started TypeSafe because we believe that AI needs an interface software could depend on. We can't wait to see new use cases continuously diffuse through the community and economy.

We Give A FAQ

Where do the names “System One Models” and “Jev” come from?
We were inspired by Daniel Kahneman, Thinking, Fast and Slow. The model class name draws on the distinction between fast, intuitive System 1 thinking and slow, deliberate System 2 reasoning.

We named Jev after William Stanley Jevons. We expect machine intelligence to follow a similar path to coal, after steam-engine efficiency led to an increase in demand. Every order of magnitude drop in the cost of intelligence unlocks orders of magnitude more use cases.

Introducing Odyssey-3: A General-Purpose Physical Intelligence

Odyssey-3 is a foundation world model that can power robots, drive cars, train AIs, pilot drones, and even play video games.

Jeff and I began working on autonomous vehicles and robotics in the 2010s, at a time when one of the field’s long-term ambitions was general-purpose physical intelligence: a system that understands the extent of the world to operate across many different machines and tasks. In practice, the field progressed through increasingly specialized systems, each trained for a relatively narrow domain and often requiring enormous amounts of task-specific data.

We founded Odyssey in 2023 around the belief that world models could provide a legitimate technical path toward that broader ambition. By learning a general, causal model of how the world behaves, a world model could provide a common foundation for many physical and virtual systems, adapting that knowledge to new tasks with relatively little task-specific experience.

Today, we’re sharing our work on Odyssey-3, our most powerful foundation world model yet, and an early example of a single learned intelligence operating across many different physical and virtual systems. Odyssey-3 is an autoregressive diffusion transformer trained to simulate highly diverse scenarios. The same foundation model is capable of controlling robots, powering humanoids, driving vehicles, piloting drones, training AIs, and even playing video games.

Trained on a vast collection of visual observations of the world, Odyssey-3 has developed a learned understanding of physics, dynamics, cause-and-effect, human behaviors, and other concepts that make up our reality. It can draw on that learned world knowledge to solve physical and virtual tasks, requiring far less experiential data per task than previous systems. We see Odyssey-3 as enabling physical agents, a new kind of agent that speaks the language of the world—enabling it to interface natively with physical and virtual systems.

Physical Agents That Speak the Language of the World

Odyssey-3 demonstrates encouraging performance controlling physical systems with only a few hours of experiential data, drawing on the broad world knowledge it has acquired during pretraining. This experiential data pairs the physical system’s observations with the actions taken to perform a task, providing examples of how its controls are used. An action decoder—a learned output component attached to the world model—is trained on these examples to translate Odyssey-3’s internal representations into the actions required by the physical system.

Odyssey-3 Can Power Robot Arms

With only tens of hours of robot demonstrations, Odyssey-3 can learn to control a variety of robot arms and complete complex tasks, drawing on the broad world knowledge acquired during pretraining. In our experiments, we observe recovery behaviors that are absent from the training demonstrations, including reorienting a gripper after a missed grasp and retrieving an object dropped in an unusual position or orientation. These behaviors suggest that Odyssey-3’s learned physical understanding can help a robot respond to situations beyond those explicitly demonstrated. This is encouraging for sample-efficient robotics, where collecting demonstrations of every possible failure and recovery quickly becomes impractical.

An important question is how consistently these capabilities hold up across different robots and environments. To this end, we’re excited to announce a close collaboration with Poke & Wiggle, a leader in robot data, robot policy analysis, and large scale benchmarking. Together, we’re evaluating Odyssey-3 across different bodies, viewpoints, and controls to understand where its knowledge transfers, where it breaks down, and how those findings can guide further training.

Odyssey-3 Is the Foundation for Flexion’s Humanoid Autonomy

Today, we’re announcing a deep research collaboration with Flexion, a leader in general-purpose robot intelligence with expertise in reinforcement learning and whole-body control. Flexion is pursuing one of the most ambitious problems in robotics: building general-purpose intelligence that allows robots to perform useful work within the buildings, tools, and environments humans already use. Their research brings together perception, manipulation, and whole-body control to develop robots that can carry out extended tasks and recover when things go wrong.

Building on Odyssey-3 as a base model, Flexion has carried out substantial research and engineering to develop the humanoid control policies demonstrated here. With only tens of hours of humanoid teleoperation data, the resulting system can perform tasks in real time, applying Odyssey-3’s pretrained representations through Flexion’s work on robot learning and control. In our evaluations, these policies generalize better to environmental changes than the VLA baselines tested, continuing to execute tasks under lighting changes that cause baseline policies to fail. We see these results as an encouraging demonstration of what Flexion’s robotics expertise can achieve with a powerful foundation world model, and we’re excited to continue this research together.

"What excites us about Odyssey-3 is the opportunity to build on physical knowledge acquired far beyond a robot’s own demonstrations. Combining that foundation with our research in humanoid learning and control opens up exciting possibilities for how quickly robots can acquire useful skills and adapt to unfamiliar situations." —Nikita Rudin, Co-Founder & CEO of Flexion

Odyssey-3 Can Drive Vehicles

With only 20 hours of simulated driving data, Odyssey-3 is capable of autonomously driving a car in closed loop on the streets of India, generating driving trajectories in real time. The pretrained world model remains frozen throughout training, extracting visual representations that feed into a relatively small driving policy, which predicts waypoints ahead of the car. The driving policy therefore learns to draw on an existing understanding of the world, with the simulated experience teaching it how to apply that knowledge to driving.

We compared policies trained entirely in simulation with policies trained on real driving footage, evaluating both on busy roads with frequent distractions. Both navigated bends while maintaining their lane, handled vehicles overtaking on either side, and turned at busy junctions. On real roads, the policies trained entirely in simulation traveled about 77% as far between safety-driver interventions as those trained on real footage.

Odyssey-3 Can Train AIs

Odyssey-3 can generate environments that AIs can inhabit, taking actions and learning from their consequences. These worlds support a recursive learning system, with an intelligence operating inside another intelligence, each pushing the other to become more capable.

Our work on PROWL explores this directly: agents uncover failures that guide improvements to the world model, while reliable simulated experiences provide training for the agents themselves. As the world model improves, it can support more complex experiences; as the agents improve, they can discover weaknesses that less capable agents would never reach. Each intelligence could continually expand what the other is able to learn, helping generate the experiences needed for further progress.

These worlds also give us a way to study how increasingly capable agents behave when given the freedom to act. As language models and other intelligences take on greater responsibility, we need places to discover potentially dangerous behaviors and investigate their consequences before they cause harm in the physical world.

Odyssey-3 Can Pilot Drones

Following the same training recipe as our driving experiments, we train an aerial navigation policy to generate flight waypoints from recent camera observations, the drone’s motion state, and a high-level navigation prompt. The policy draws on Odyssey-3’s pretrained visual representations, with simulated flight demonstrations teaching an action expert how to translate those representations into movement. Using tens of hours of simulated drone data, we trained a policy that demonstrated stable flight while avoiding obstacles in a simulated indoor setting.

To investigate what the backbone already understood before policy training, we also visualized its predictions for aerial navigation tasks with its weights frozen. In these qualitative rollouts, we observed plausible directional flight and motion around obstacles, alongside responses consistent with the world’s semantic content. These observations suggest that pretraining provides useful knowledge of spatial structure and motion.

Odyssey-3 Can Play Video Games

Odyssey-3 can provide a foundation for policies that play video games. We train these policies on gameplay recordings paired with keyboard and mouse inputs, keeping the pretrained world model frozen. During play, the policy observes recent game frames, generates controls, and uses the resulting observations to choose its next actions. The same approach used to control physical systems becomes a way to interact with virtual worlds.

Our experiments have produced extended gameplay sessions in Rockstar Games’ GTA V, alongside selected examples of driving, shooting, and hand-to-hand combat. We are also seeing early evidence of transfer: GTA-trained policies have produced movement in Rockstar Games’ Red Dead Redemption 2 and motorcycle riding in Square Enix’s Sleeping Dogs without additional policy training on either title. In one experiment, a mobility policy trained on approximately two hours of GTA footage produced horseback movement in Red Dead Redemption 2, applying controls learned in one game to a different character, vehicle, and environment.

Together with our results in robotics, humanoids, driving, training, and drones, these gaming experiments suggest that Odyssey-3 is a strong foundation for generalist embodied agents. A shared pretrained world model can support policies across different bodies, environments, and controls, with early evidence that learned behaviors can transfer between them.

An Early Glimpse of Physical Agents

Agents in our reality, like humans, develop an understanding of the world and how it works through observation and interaction, allowing them to draw on prior knowledge when learning a new task. We can learn to operate dangerous machinery at age 18—despite never having used it before—because we have already gained an intuition for navigating space, watched countless objects in motion, seen how they respond to force, observed humans using tools in many contexts, and noted that collisions are dangerous.

Today, many robotic systems are trained on repeated demonstrations of specific tasks to acquire the ability to manipulate objects, concentrating large amounts of narrow experience to accomplish a single task. We see this as a sign that these systems are brute-forcing the problem, compensating for a lack of general world understanding with ever-larger quantities of task-specific data. Humans do not need to sit and watch thousands of hours of specific tasks before becoming capable, because so much of the necessary knowledge has already been acquired through our experience of the broader world.

Similarly, a true physical agent should possess a superhuman understanding of the world, including the physics, dynamics, and cause-and-effect that govern how our reality evolves. That understanding would give it a native interface to physical systems, enabling it to adapt to new tasks with the amount of experiential training a human needs, or less. Odyssey-3 enables early physical agents, and represents a leap in the performance and maturity of world models, capable of applying its learned understanding to robots, humanoids, cars, drones, and video games with only a few hours of experiential data.

As physical agents develop, we believe they will enable us to automate increasingly complex physical work, taking on jobs that put people at risk and helping with everyday tasks that age, illness, or disability can make difficult, allowing more people to live independently. These same capabilities could accelerate experimentation in science and engineering, give us more time to spend with one another and on work we find meaningful, and eventually allow us to operate in environments we humans have had no lived experience in, including worlds beyond our own.

These same models will also change how intelligence itself develops, generating the worlds in which future agents learn, and learning themselves from what those agents discover when acting in the physical world. More capable world models will support more capable agents, whose experiences and discoveries would in turn improve the worlds that shaped them, allowing both to grow beyond the limits of the environments we can construct by hand.

World models are learned dynamical systems. As a foundation layer, they represent the holy grail of general intelligence, by enabling decision-making and reasoning inside accurate, open-ended environments. When combined, VLM agents can reason directly within these representations. However, frontier world models have lagged behind this potential: largely because they remain sub-scale, roughly two orders of magnitude behind language models. We are excited to see early signals of this potential emerging in Odyssey-3, and we’re excited to release it publicly in the coming weeks.

Noam Brown (@polynoamial) gave a very interesting interview on The Information on OpenAI’s priorities and what comes next. Here is the tl;dr:

• Recursive self-improvement is OpenAI’s clear priority: “the number one priority is recursive self-improvement and by a pretty wide margin.” Building models that help develop better models comes first.
• AI could surpass his research intuition within a couple of releases. He wouldn’t be surprised if, “one or two model releases from now,” he concludes: “they’re better than me at that too.” He specifically means choosing research directions and prioritizing long-term work.
• Pretraining and reinforcement learning amplify each other: “the effects of these two are not additive, they’re multiplicative.” Brown expects their combined progress to produce much more powerful models.
• AI-generated math is becoming easier to produce than to verify: “the biggest challenge that we face with our math results is … double-checking with human mathematicians.” Human verification remains a bottleneck.
• OpenAI underestimated agents during the security incident: “we trusted the sandboxes” and “we just underestimated the AIS.” Brown says monitoring was subsequently added to training and evaluation.
• Monitoring reasoning could get harder: “the agents are more effective at controlling their chain of thought.” Brown warns that punishing unwanted thoughts can teach models to hide them.

Development is progressing rapidly, but he is worried about safety.

Full article content is not available for inline reading.

Read the original article →

• What changed: You can now resize a stateful streaming query's partitions without rebuilding the checkpoint or losing state.
• How it works: Set spark.sql.streaming.stateStore.partitions and restart your query on DBR 18+ with RocksDB state store provider. The query redistributes state to the new partition count.
• Why it matters: Tune and scale long-running streams to fit the workload, and track each resize via query progress metrics.

Anyone running stateful Apache Spark™ Structured Streaming queries in production eventually hits the same uncomfortable wall.

You started the query months ago. Back then, the data volume was modest, so you accepted the default of 200 shuffle partitions and moved on. The pipeline ran smoothly. Then the business grew, traffic tripled, and the state store ballooned. Suddenly, those 200 partitions are no longer the right size. Some partitions are skewed and run hot, the cluster is straining, and every microbatch takes longer than it should.

So you do the natural thing: you bump up spark.sql.shuffle.partitions and restart the query. Nothing changes.

The query quietly ignores your new value because the partition count was baked into the checkpoint when you first started the stream. Historically, the only way to apply a new number has been to abandon the existing checkpoint and start over, which, for a stateful query, means losing all the accumulated state you've been carefully maintaining. For a fraud model tracking millions of accounts, or a sessionization job holding days of windows, "start over" is not a phrase anyone wants to say in a production incident review.

On-demand state repartitioning (Public Preview), available in Databricks Runtime 18 and above, removes that wall. You can now resize the number of partitions for a stateful streaming query and keep your checkpoint state intact.

This applies to any stateful streaming query, whether you run aggregations, stream-stream joins, deduplication, sessionization, or transformWithState, and to any workload, from fraud detection to real-time monitoring.

For early adopters like Coveo, the ability to right-size their streaming infrastructure on demand immediately translated into significant operational savings.

At Coveo, we run large-scale stateful streaming pipelines where data volumes fluctuate significantly over time. With Databricks and the State Repartitioning capability, we’ve cut our related Amazon S3 API costs by 40%. Before, every scaling decision forced a trade-off: either overprovision or rebuild from new checkpoints, which drove storage API costs nearly the same as compute costs. Now we scale freely as demand shifts, without disrupting the existing state or triggering costly checkpoint migrations.” —Alexis Chicoine, Senior Software Developer, Coveo

Under the hood: Why were state partitions locked?

To understand why Coveo’s results represent a meaningful leap forward for Structured Streaming, we have to look at why the partition count was ever frozen in the first place.

A stateful streaming query keeps its state in a state store, and that state is physically partitioned. Each key in your stream, a user ID, an account number, a window, is hashed to a specific partition, and the data for each partition is stored in its own separate RocksDB instance within the checkpoint. The number of partitions defines the layout of the entire state store on disk.

If you simply changed the partition count between restarts, the hashing would no longer line up. A key that previously lived in one partition (say, partition 47) might now hash to a different one (partition 12), but its accumulated state is still sitting in the original partition's files. The query would, in effect, lose track of its own memory. To prevent exactly this kind of silent corruption, Structured Streaming locked the partition count at checkpoint creation and ignored any later changes to spark.sql.shuffle.partitions.

Safe, but inflexible. The two costs you paid were:

1. You couldn't tune. If 200 partitions turned out to be the wrong choice, you were stuck with it for the life of the checkpoint.
2. You couldn't scale with the workload. As data volume grew or shrank, your partition count couldn't keep pace.

On-demand state repartitioning addresses both by doing the one thing the old design refused to do, but doing it safely, by physically redistributing the state to match the new partition count.

What you need to get started

The requirements are short:

• Databricks Runtime 18 or above.
• The RocksDB state store provider. In DBR 17.3 and above, RocksDB is the default, and new queries created in those versions will use it unless explicitly changed. If you want to confirm or explicitly set it, see Configure RocksDB state store on Databricks.

That's the entire prerequisite list. If you're on DBR 18 with the default state store, you already have everything you need.

Changing the number of partitions

The mechanism is simple, and it reuses a pattern every streaming developer already knows: stop, reconfigure, restart.

Instead of spark.sql.shuffle.partitions, you set a dedicated configuration, spark.sql.streaming.stateStore.partitions, and restart the query:

The key detail is the new config itself. For stateful queries, spark.sql.streaming.stateStore.partitions takes precedence over spark.sql.shuffle.partitions. This is what makes the change "stick" where the old approach didn't.

When the query restarts, it doesn't resume normal processing immediately. First, it finishes the last planned microbatch, if there's one still pending. Then it performs a one-time repartition operation: it physically redistributes the state data across the new number of partitions, re-hashing keys into their correct new homes so that nothing is lost or misplaced. Once that redistribution completes, the query resumes processing as usual, now using the partition count you requested.

That repartition step is the heart of the feature. It's the difference between "we changed a number" and "we safely moved your state to a new layout."

Monitoring the repartition operation

Because repartitioning is an actual operation whose runtime is proportional to the amount of state, you'll want visibility into it. Structured Streaming surfaces this through its standard progress reporting.

After the next microbatch completes, the StreamingQueryProgress events include the duration of the repartition operation. Look in the event's durationMs metrics for the controlBatch.REPARTITION field, which reports the repartition duration in milliseconds.

A larger state footprint means a longer repartition, but we expect it to take only a few seconds for most workloads. So, on big jobs, it's worth capturing this metric to understand the duration.

Example: scaling a query down

Let's make this concrete with a simple aggregation, a tumbling-window count of events by id. We'll start it with the default of 200 partitions, decide that's more than this workload needs, and scale it down to 100.

First, the query as it runs today, with the default partition count:

Now, we've watched this stream for a while and concluded that 200 partitions is overkill. We're paying coordination overhead for parallelism we don't need. We stop the query, set the new partition count, and restart it with the same options and the same checkpoint:

When the restarted query comes up, it wraps up the last planned microbatch, if there's one still pending, runs the repartition to redistribute state from 200 partitions down to 100, and then carries on counting with every window and every running total fully preserved. The same procedure works in reverse: to scale up under a heavier load, you'd simply set a larger number.

When to use state repartitioning

On-demand state repartitioning is a tuning and scaling tool rather than a routine operation. It proves valuable in a few key situations:

• Right-sizing after launch. You started the pipeline with the default 200 partitions on day one because the stream was small and fine-tuning wasn't worth it. Six months later, that number is baked into a checkpoint you can't afford to lose, and it's not enough. Ex: a fraud-scoring stream that launched in a single pilot region now covers every market, and 200 partitions leave each one holding far too much state. With on-demand repartitioning, you can increase the partition count to match what you now carry without losing your existing checkpoint.
• Changing workloads. You sized the stream for the peak traffic. Ex: An ad-bidding pipeline runs hot through the day and goes quiet overnight, so a value tuned for the daytime peak leaves most partitions idle at 3 am. With on-demand repartitioning, you scale up going into the busy stretch and back down once it passes, so partitioning follows actual load rather than the worst case.
• Backfilling historical data: Backfill and steady-state processing need different partition counts, and previously, you had to choose one for the checkpoint's life. Ex: reprocessing two years of history needs a high count to spread the work and finish fast, but that same count is wasteful once you're back to steady-state traffic. On-demand repartitioning lets you scale up for backfilling and down to steady-state size after catching up, all without losing the checkpoint and state.
• Performance tuning. Partition count affects parallelism, state size, and shuffle overhead, and the optimal value is hard to predict. Ex: you might think 200 is too small and that 400 would reduce microbatch latency, but testing used to require rebuilding state and reprocessing data, wasting resources. On-demand repartitioning lets you adjust the count against your live checkpoint and monitor controlBatch.REPARTITION and microbatch durations, and decide based on measurements rather than guessing.

Because each change requires a stop and restart with a one-time repartition pause, treat it as a deliberate maintenance action. Plan the resize for a window where a brief processing pause is acceptable, and watch controlBatch.REPARTITION to confirm how long it took, and let the query settle back into its normal rhythm.

Conclusion

For years, the partition count of a stateful streaming query was a decision you made once, at the very beginning, and then never revisited, or paid dearly to rebuild the state from scratch. On-demand state repartitioning removes these constraints. Safely redistributing state across a new partition count turns a start-time-only decision into one you can revisit whenever your workload calls for it.

The result is exactly what operators of long-running streams have wanted: the freedom to right-size a query based on its scaling needs, with nothing more than a stop, a config change, and a restart, without losing its state.

iOS 27 is now available, bringing Siri AI, expanded Apple Intelligence features, greater control over the Liquid Glass interface, and performance improvements across the system. Siri AI can understand personal context, interact with on-screen content, assist with writing, and work alongside new AI-powered tools in Photos and Safari, including image editing, webpage monitoring, and automatic tab grouping. The update also adds practical features like separate alarm and timer volume controls, customizable AirPods EQ, improved parental controls, faster performance, and accessibility enhancements, with Apple Intelligence available on the iPhone 15 Pro models, the iPhone 16 lineup and later, iPhone Air, and iPhone Duo.

Using data from our own labs, we trained Periodic Neon to outperform GPT-6 Astra and Claude Fable 5.1 at lower cost, on a highly challenging evaluation for scientific analysis. Neon is now deployed in our labs, analyzing experiments in our search for better superconductors and magnets.

X-ray diffraction (XRD) analysis is a critical capability for our discovery efforts. Scientists can spend hours navigating scientific software, related experiments, literature, and databases to analyze complicated XRD measurements. Periodic Neon achieves a 55.3% success rate on our most difficult internal evaluation set, FrontierXRD. This is a 20x improvement over the initial 2.7% success rate of Kimi K2.6, the open-weight model we post-trained. Through midtraining and reinforcement learning on our lab data, Neon establishes a Pareto-optimal cost-performance frontier.

Periodic Neon surpasses frontier models on FrontierXRD at a lower cost per analysis. We report performance on 134 samples from our labs that human experts take hours to resolve due to their complexity and ambiguity. All models use the Periodic Harness, which outperforms off-the-shelf coding harnesses like Claude Code or Codex for our use cases. Here, we measure success rate using an LLM-Judge ensemble of Opus 5 and GPT-5.6-Sol, calibrated against expert ratings. For all external models, we estimate cost based on standard API prices assuming perfect caching across agent turns. For our model, we calculate the cost by using the system throughput per H200-hour and $2.5/hour H200 price.

What is XRD?

A synthesis experiment may begin with a target material and carefully chosen precursors and conditions, but the product can contain a mixture of the target phase, unreacted precursors, and unexpected byproducts. Before deciding what to try next, scientists need to determine what formed and in what proportions.

Powder X-ray diffraction (XRD) is one of the primary tools to answer this. We shine X-rays on the powder and measure how they scatter. Because the atoms in a crystal are arranged in a repeating pattern, each crystalline phase produces a characteristic pattern of peaks, a fingerprint of its atomic structure. Peak positions reflect lattice spacings and crystal symmetry. Their intensities depend on which atoms are present, where they sit, and the amount of each phase. In a multiphase powder, patterns overlap, making it difficult to identify phases and estimate proportions, even for experts.

It’s critical to understand if experiments successfully synthesize their target phase, but as we scale our autonomous labs, scientists can no longer keep up with analyzing XRD results manually. A successful XRD analysis can require reasoning about synthesis conditions, prior experiments, thermodynamic calculations, and crystal structure databases. Automating this kind of long-context agentic work frees up hours of scientist time, enabling them to supervise many more experiments.

An X-ray diffractometer from our lab. Neon learns from XRD data produced by instruments like these.

Difficulty of our XRD evaluation

Existing XRD software is powerful, but it works best after a scientist has narrowed down what to look for: which elements could be present, which phases are plausible, and which fits make chemical sense. In synthesis for materials discovery, that is often the hard part. A sample may contain multiple phases and the right answer depends on context such as the precursors, temperature, atmosphere, environment, handling history, related samples, literature, energetics, and simulations.

Scientists use automated tools, but they guide those tools with judgment: expanding the search when something unexpected appears, ruling out phases that are chemically unreasonable, and revising hypotheses when the fit looks good mathematically, but fails scientifically. For example, an oxide may make sense if the sample was heated in air; a hydroxide may only make sense if moisture exposure was possible. For complex samples, this holistic interpretation can take several hours of expert time per pattern. Our evaluation focuses on these harder, multiphase cases. Among patterns with an accepted solution, the fits contained five phases on average.

Interactive demo of XRD analyses from Periodic Neon, showing highlights of the model’s reasoning as it considers different hypotheses for what phases are present in the sample given the XRD pattern.

These results strengthen our conviction that scaling our autonomous labs and the AI systems that learn from them will allow us to tackle scientific questions beyond our reach today. Our final training run used 1,300 H200 GPUs, far less than Astra’s reported 100,000+ Blackwell GPUs. Scaling our training compute to levels comparable to today’s frontier models will unlock even greater scientific capabilities.

Scientific analysis performance for Periodic Neon improves as we scale RL training compute on experimental lab data, with further gains from higher inference compute. Kimi K3 illustrates a potentially stronger open-weight starting point for future training of Neon. Together, these observations motivate scaling our autonomous labs alongside the AI systems that learn from them.

To evaluate Neon’s generalization beyond FrontierXRD, we also evaluate its performance on XRD measurements from our labs’ chemical systems held out from both midtraining and RL. Neon outperforms frontier models on this evaluation, indicating that it learns transferable XRD analysis capabilities that extend beyond the chemical systems it encounters during training.

We evaluate Neon’s generalization on 198 experimental XRD measurements using held-out chemical systems. Specifically, our lab data training split for both reinforcement learning and midtraining excludes these systems and any systems containing them, while smaller subsystems can appear in training. This benchmark tests generalization to held-out chemical systems, but is easier than FrontierXRD, which focuses on samples that are difficult to resolve even for human experts.

Beyond scaling training compute on our lab data, Neon benefited from several infrastructure and research improvements, some of which we highlight below.

• Scientific Harness: Scientific capability depends on more than the model intelligence alone. It also depends on the affordances available to the model, such as scientific databases and tooling. Against a harness based on Claude Code with standard XRD tools, Periodic’s scientific harness achieves a 3.8x higher XRD analysis success rate at a similar cost per analysis. We use Periodic’s scientific harness for all our model comparisons, including in the Pareto plot above.

Periodic harness versus Claude Code with standard XRD tools

We build a harness equipped with laboratory context, in-house materials structure and simulation databases, and scientific XRD analysis software. Our comparison gives Claude Code access to open-source databases (COD, Materials Project), and analysis software (BGMN), representing a typical scientist’s toolkit without access to our internal infrastructure. Both harnesses therefore have appropriate tools for the task. Using the same underlying model (Claude Opus 5 with high reasoning effort), our harness delivers better performance on FrontierXRD than Claude Code + OSS databases and software.

• RL Beyond Verifiable Rewards: Science is falsifiable, but not easily verifiable: in our setup, XRD analysis success cannot be cheaply verified based on pattern-fitting alone. Instead, it requires expert scientific judgment to determine whether each phase is supported by the pattern and whether it makes chemical sense. To learn from our lab data, we turn this expert judgment into a reasoning-heavy LLM-judge that achieves nearly the same level of agreement scientists reach with one another.

LLM-Judge agreement with human scientists

To evaluate the quality of an XRD analysis, we devised a detailed rubric and had a team of human experts with PhDs in materials-related disciplines provide quality labels for thousands of XRD patterns. Each pattern was annotated by three experts. We then used an ensemble of two agentic LLM judges to also provide quality labels for each of the patterns and compared agreement among human experts with agreement between human experts and the LLM-judge ensemble. We found that human experts agreed 77.2% of the time, while the LLM-judge ensemble agreed with human experts 74.6% of the time. When comparing LLM-Judge to an expert consensus, they agreed 84% of the time. This agreement provides evidence that the LLM-judge can supply training and evaluation signals for a task where neither ground-truth labels exist nor pattern-fitting suffice.

• Scientific Midtraining: We midtrain Periodic Neon on a multimodal blend of academic literature, code, and experimental data to build broad scientific understanding. Our proprietary midtraining corpus is rapidly expanding, currently doubling every month. We observe that injecting scientific knowledge through midtraining improves downstream RL performance.

Midtraining Improves RL scaling

Despite only a small fraction of the midtraining data focusing on XRD, early ablations on midtrained Neon show higher rewards during subsequent RL training and higher success on FrontierXRD.

Starting from an open-weight model with 1 trillion parameters, we improve its scientific analysis capabilities through continued advances in training on experimental lab data. Each point represents a separate RL training run, with annotations approximately highlighting the major change between runs. All results are reported using Periodic Harness and high inference-compute setting.

Looking back, our progress was a result of scaling compute on our lab data along with several research and infrastructure advances such as long-context multimodal RL, as illustrated by cumulative gains from these advances in the above figure. We are now also training on a wider set of scientific workflows across our labs, using our AI systems for designing, running, and learning from experiments.

Build real-time voice applications with Gemini 3.8 Live and 3.5 Transcribe

New Gemini Audio models are available for developers to build more intelligent conversational experiences via the Gemini API and Google AI Studio.

Today, we released new Gemini Live models in the Gemini API and Google AI Studio, expanding our developer suite for building real-time, voice-first product experiences:

Gemini 3.8 Live and 3.8 Live Extended Thinking: Gemini 3.8 Live brings a step change to our native speech-to-speech models, capable of performing tasks while maintaining dialogue. For complex requests, 3.8 Live Extended Thinking delivers deeper reasoning, ranking #1 on Artificial Analysis’ Speech-to-Speech leaderboard.

Gemini 3.5 Transcribe: Our dedicated speech-to-text model brings highly precise transcription across 85+ languages. Released last month, it achieved an average Word Error Rate (WER) of 4.0% (streaming) and 2.6% (non-streaming).

Gemini 3.8 Live & 3.8 Live Extended Thinking: Build more intelligent conversational agents

Our new models, Gemini 3.8 Live and 3.8 Live Extended Thinking enable developers to build voice agents that can reason and execute tasks while maintaining the flow of conversations. Key capabilities include:

• Asynchronous function calling: Execute API and tool calls in the background while continuing to stream audio responses to the user
• Visual context: Ground dialogue in live visual inputs to help enable agents that can understand what users say and see
• Alphanumeric precision: Accurately parse confirmation codes, claim numbers, and technical data
• Multilingual support: Reach global audiences with coverage for 97+ languages and accent consistency
• Incremental content updates: Seamlessly merge real-time audio with structured data to return context-aware responses

3.8 Live Extended Thinking also supports configurable thinking to help handle complex, multi-step reasoning in the background, while responding or narrating its progress in the main conversation. These models represent a step-change from our previous live models and provide a more streamlined alternative to cascaded architectures.

Jamie Wood, cofounder and Chief of Technology & Products, says," Ambr AI trains enterprise teams to negotiate, lead, and resolve difficult customer conversations through realistic AI simulations. Switching to the Gemini 3.8 Live Extended Thinking made our simulations faster, more expressive, and better at handling complex conversations. It’s also enabled us to deliver training in over 70 languages through a single integration, helping our customers develop these skills across their global workforce."

Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking are available via the Live API. Competitively priced at $0.005/min for audio input and $0.018/min for audio output, they allow developers to scale voice applications with industry-leading performance.

Developers can also access the models through Agora, Fishjam, LiveKit, LangChain, Pipecat, Vercel, and Vision Agents, our Live API integration partners that handle media streaming infrastructure for real-world deployment:

Gemini 3.5 Transcribe: Convert streamed speech to text

Real-time speech understanding is critical for voice-first interfaces. Last month, we released Gemini 3.5 Transcribe for low-latency transcription with high precision, achieving a 4.0% WER, and useful features:

• Automatic code-switching: Handle intra-sentence and inter-sentential code- and language-switching without manual configuration
• Custom vocabulary biasing: Steer speech recognition toward domain-specific terms, uncommon jargon, company names, and proper nouns by passing a custom_vocabulary list of up to 1,000 terms
• Smart transcription mode: Deliver polished, reader-ready transcripts with structured formatting, self-corrections, and disfluency removal that eliminates filler words

3.5 Transcribe supports 85+ languages and provides a strong listening engine for voice experiences and stateless tasks like sub-second captioning, call center agents, and real-time audio analytics. You can also access the model via the Interactions API to transcribe audio files up to 1 hour long with structured timestamps and speaker labeling.

Our complete audio suite for developers

To get started, try out the models in ai.studio/live, clone example apps from GitHub, or equip your agent with our live api skill.

You can also create audio experiences with our speech and music generation models, all available in the Gemini API:

• Gemini 3.5 Live Translate: Speech-to-speech translation across more than 70 languages
• Gemini 3.1 Flash TTS: Highly configurable speech generation (with more updates coming soon)
• Lyria 3.5: Production-grade music generation

The mic is yours, and we can’t wait to hear what you build!

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up

This is a blog post for my recent paper on RL post-training of LLMs: introducing the Matthew Effect and proposing to solve it with Never Give Up. It is presented interactively and less formally, more like how I give the talk. For a deeper, more technical dive, check out the paper on arxiv and code on github.

What is your eval actually measuring?

Every good RL practitioner has no doubt seen an eval curve go up. Here is the AIME 2025 eval during our RL training of Olmo 3.1 RL-Zero Math.

What does this curve really mean?

Our eval is an average over 30 AIME questions. Let’s break those 30 questions down into 3 levels of difficulty. Every question that our initial, pre-RL model gets 0 for pass@32 will be labelled “hard”. The other questions we’ll divide evenly by into “medium” and “easy” based on their pass-rates. So our initial pass@1 averages will be 0%, 3.8%, 22.7% for our subsets. How do you think performance on each subset will evolve?

Averaging our AIME eval was hiding something important: the majority of our improvements are coming from the easiest problems going from somewhat solved to mostly solved. The hardest problems are barely improving. This is clearly visible if you look at how each example’s solve rate changes over time. The hardest examples barely improve over training. The model mainly learns to better solve easy and medium-difficulty examples that were already reasonably-well solved. We call this discrepancy the Matthew Effect. But this is for math RL on LLMs. What about other domains?

We evaluate code RL and agentic RL using Deepcoder and DeepSWE, two nice open-source projects that released models and logs. We can use the initial model to split each benchmark into difficulty buckets (Deepseek-R1-Distilled-Qwen-14B on LCBv6) or we can use existing task length/difficulty labels (SWEBench).

The gains from RL are proportional to how easy the problems are. We connect this bias to a similar phenomenon in network science and economics, the Matthew Effect, generally summarized as “the rich get richer”.

We therefore propose The Matthew Effect in RL for LLMs

RL improves performance on a task in proportion to a model’s initial competence—making easy tasks easier while hard tasks often remain difficult.

What causes the Matthew Effect?

You might assume the issue has to do with GRPO. If we just don’t get a correct answer to our problem in our k sampled completions, then we don’t get any gradient and can’t improve on this problem. One possible answer is to sample more completions i.e. larger k.

To test this out, we train Qwen 2.5 0.5B Instruct with GRPO on GSM8k platinum and test on the same. We split our dataset into difficulty levels using initial pass@1: easy (25%), medium (10%), hard (5%), and extra-hard (0%) problems. We vary k \in {4, 8, 16, 32} but keep batch size fixed.

It turns out that smaller k=4 is actually best! Why is this happening?

Lets look at what our training batch is actually composed of. Since we filter any prompt whose completions are all correct or incorrect, our training batch must always be composed of problems with some completions right, some wrong. We plot what percentage of our batch is our easy subset and our extra hard subset and how this changes over time.

Larger k increases the chance of finding a rare correct solution to a very hard problem. So, naively, we expect it to have more hard problems in the batch. The issue is that larger k also increases the chances of finding a rare incorrect solution to an easy problem.

Early in training, k=32 finds rare solutions to hard problems. But after the inflection point around step 200, k=4 does better. k=4 filters any problem that is solved in 4/4 completions. In contrast, for k=32 to filter the same problem, it must be solved much more: 32/32. k=4 ends up spending much less compute on easy problem, especially when they get a rare incorrect solution. The inflection point is when the benefit of finding rare correct answers to hard questions is outweighted by wasting compute training on rare incorrect answers to easy questions.

Because of our asynchronous RL for LLMs setup, all the compute we save filtering easy problems is used to train on harder problems. We argue the issue behind the Matthew Effect isn’t just undersampling for hard problems, but spending too much compute on easy problems.

Never Give Up on hard problems

Our goal is therefore to only use small k for easy problems but have large k for hard problems. We propose a simple, but effective method of adapting asynchronous RL sampling: Never Give Up. We start sampling some small amount k. If a prompt is solved within the first k completions, train on it! If a prompt is fully solved in k/k completions, then we can easily and quickly filter it.

The tricky part is if all completions are wrong. With probability p, we never give up and add the prompt back to our generator in order to sample k more completions. We keep track of our old completions and when we do solve the problem, train on our whole k * \text{rounds of NGU} completions. This creates a geometric distribution for the number of samples we take: if we never solve the prompt, we expect to take \frac{k}{1-p} samples, in expectation.

This method is implicitly adaptive. Whereas curriculum learning pre-sets the difficulty of a problem, we find that online, adaptive methods do better as easy problems can become more difficulty over training and vice-versa. On GSM8k, k=4 with NGU p=0.9 outperforms all values of standard GRPO with varied k. This is especially evident on the hardest subset.

It does this by achieving the best of large k early in training and small k late in training.

Async RL staleness and tricks for NGU

Astute readers might already see a downside to the method: stale completions. This section introduces two tricks for dealing with staleness, but its not necessary to the main message so feel free to skip it.

If we take multiple rounds of NGU to get one correct completion, our initial k completions will be pretty stale by the time we train on them. Stale negatives are known to be bad for LLMs and RL so its important to filter completions to be below some age threshold.

Overall, it makes sense to use all the samples you have for your GRPO baseline, even if you’re not training on them.

NGU at a bigger scale: Math

We scale up to a bigger math RL setup: DeepScaler with Qwen 3 4B base. On top of a strong GRPO k=16 baseline, NGU further improves performance, especially on the hardest subsets of our AIME + BRUMO 2025 eval.

The Matthew Effect still persists, but we can mitigate it; NGU helps solve harder questions without really degrading on easier ones.

NGU on a different scale: Code

Code RL is fundamentally different from math because math usually has binary verifier: right or wrong. A coding problem has many tests and the tests can vary from easy and difficult within a single coding problem. Math problems are either easy or hard. Solving a single coding problem means solving both easy and hard tests.

We look at a particularly tough setup: Manufactoria. Standard GRPO improves performance but eventually stagnates: improving on some tests but failing to pass all tests. Splitting the tests by difficulty, we see a clear Matthew Effect.

Easy tests are nearly fully solved and improvement on hard tests stagnates. GRPO dedicates the majority of training signal to repeatedly revisiting partially-solved medium tests and oscillates between solving them slightly more or less.

If our first k completions all pass \frac{7}{12} tests, then Never Give Up won’t accept the next k completions unless they pass more than \frac{7}{12}, pushing the model to iteratively do better. Where standard GRPO stalls, GRPO + NGU keeps solving harder and harder tests until it starts to fully pass all tests for a given problem.

The Matthew Effect is a Primacy Bias, sort of

Some very astute readers may have noticed that the Matthew Effect resembles a primacy bias where LLMs are predisposed to solve certain problems according to their initial state. This intuitively connects the Matthew Effect to the Primacy Bias in Deep RL where deep RL training runs could be derailed due to bad early samples. This was due, in part, to issues of plasticity in neural networks trained with RL.

Could it be that the Matthew Effect in RL for LLMs caused by plasticity? In short, no.

We start from a post-GRPO checkpoint at 6000 steps that has been stagnant for at least 3000 of those steps. We can then either train with a single reward for passing all tests (all-tests) or keep our per-test reward and just add NGU (per-test NGU).

Both methods recover strong performance, even after spending a while on suboptimal data. This demonstrates that plasticity is not a major issue and LLMs can generally recover from early bad samples.

Limitations

The intuition behind Never Give Up is that our RL training can reallocate compute by quickly filtering easy problems. So if a task leans heavily towards very difficult problems, Never Give Up will likely not be effective. NGU’s sample, wait, sample more process can take longer to finish a whole group of completions as compared to sampling \frac{k}{1-p} from the start. This means your early samples will be more off-policy than necessary, which means a worse learning signal and slower learning speed. But if you already know a decent k for your data distribution, NGU can likely be effective!

Conclusion

We have highlighted the Matthew Effect in RL for LLMs, and shown that standard RL results in disproportionately poor performance on the hardest problems. This demonstrates how simple scalar values may not be sufficient for accurate evaluations of LLMs. Hopefully, it inspires you to dig into your evals and look at more dense evaluation signals.

Our proposed solution, Never Give Up, represents a simple method for better allocating compute. The trick is really just reducing the compute spent on easy problems, which reallocates it towards harder problems. But it does seem to show substantial improvements on difficult tasks. Future work should examine more complex multi-step, agentic environments and try to gain a deeper understanding of how RL actually changes our model’s distribution, in practice.

Acknowledgements

Thanks so much to my coauthors: Hamish, Nathan, and Aaron! And all my friends who I annoyed for advice during the project: Costa, Sam, Finbarr, Dima, and Adrien. Hamish, Nathan, and Adrien kindly gave me feedback on this blog post. Thanks to Ai2 for giving me compute, some really nice stickers, and great colleagues.

All graphs here made with plotly thanks to my robot friends Sonnet and Sol.

Citation

@misc{noukhovitch_ngu_2026,
	title = {Learning to Solve Hard Problems in RL for LLMs by Never Giving Up},
	url = {https://arxiv.org/abs/2609.13443},
	author = {Noukhovitch, Michael and Ivison, Hamish and Lambert, Nathan and Courville, Aaron},
	month = sep,
	year = {2026},
}

OpenArm

OpenArm is an open-source 7DOF humanoid arm designed for physical AI research and deployment in contact-rich environments. With high backdrivability and compliance, it is built with safe human-robot interaction in mind while delivering practical payload capabilities for real-world applications.

OpenArm Cell (on the right) is a standardized environment with unified background, lighting, and camera placement. Research performed using OpenArm can be reproduced around the world in consistent evaluation conditions, facilitating the global discussion on state of the art physical AI research.

OpenArm features human-scale proportions, safety and compliance, and practical payloads. At $6,500 USD for a complete bimanual system, it provides a flexible platform for teleoperation, imitation learning, simulation, and real-world data collection in contact-rich tasks.

We're in continuous development and actively seeking contributors, research partners, and company collaborators to shape the next generation of practical humanoid systems. Ready to join the future of open-source robotics?

📁 Repositories

• openarm_hardware: Complete CAD data: STL files, STEP files, Fusion 360 assemblies
• openarm_description: Robot description files with URDF/xacro for simulation
• openarm_can: CAN control library for low-level motor communication
• openarm_ros2: ROS2 integration packages and nodes
• openarm_teleop: Teleoperation packages with unilateral and bilateral control
• openarm_isaac_lab: Isaac Lab simulation environment and training tasks
• openarm_mujoco: MuJoCo specification files and assets for OpenArm
• openarm_dataset: Dataset format, recording tools, and Python API
• dora-openarm: Dora dataflow nodes for data collection, inference, and teleop

Abstract

Recursive self-improvement is becoming increasingly vital for autonomous AI agents, where progress hinges on discovering high-value solutions across complex domains. The driver of this process is effective exploration, however, managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization requires navigating vast meta-search spaces under delayed and expensive feedback over long-horizon rollouts. We introduce \textsc{Dream-RSI}, a framework for scalable and recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying coding agent unchanged. Our key insight is that accumulated discovery history can serve as a replay simulator over the realized search space. By performing dreaming in the replay simulator constructed from historical discovery trees, \textsc{Dream-RSI} secures immediate, low-cost off-policy feedback to evaluate and refine exploration policies without invoking repetitive, expensive online evaluations. The improved policy is subsequently redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across algorithm engineering, mathematical optimization, and GPU kernel engineering, \textsc{Dream-RSI} achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings.

A day after Anthropic researcher Jacob Coxon quit his job over concerns that AI could kill us all by the end of the decade, I met with founders and brothers-in-law Rune Kvist and Rajiv Dattani. They think they have a solution that could save us all, or at least help prevent AI agents from going rogue inside enterprises.

“AI is getting smarter at an increasingly rapid rate. The surprising thing about AI is that it becomes harder to adopt and harder to control as AI gets smarter, not easier,” said Kvist, an early Anthropic employee who is also married to Dattani’s sister). Dattani is the former COO of the AI safety research organization METR.

The pair launched a startup called Artificial Intelligence Underwriting Company (AIUC) that hopes to bring AI safety to enterprises and companies building AI models and agents. The startup names Cursor, Lovable, Harvey, and ElevenLabs as customers.

On Tuesday, AIUC announced a $40 million Series A led by Ribbit Capital, with participation from First Harmonic. It previously closed a $15 million seed round from Nat Friedman through his fund NFDG, along with Emergence, Terrain, and Anthropic co-founder Ben Mann, among others, bringing its total funding to $55 million.

What caught the attention of this A-list group of investors is AIUC’s attempt to apply a familiar cybersecurity model to a new set of AI risks. The company has built a third-party audit and certification layer for AI agents.

“Banks, hospitals, governments and militaries no longer decline to deploy AI because a model isn’t smart enough,” Kvist said. “They decline because they’ve made commitments to their own customers about what a system will and won’t do, and nobody can currently guarantee that.”

Using the widely adopted cybersecurity standard SOC 2 as its muse, AIUC has developed a standard called AIUC-1 and a testing service to validate agents against the standard.

To build the standard, AIUC assembled a consortium of about 250 security and risk leaders — the buyers of agents. “These are the people who we meet with on a monthly basis, and the question we ask them is: When you’re buying agents from someone, what would you look for? What are the questions you’d want to ask, and what would you want to see addressed?” Dattani told TechCrunch.

That feedback shapes the tests. The startup then runs an agent through a suite of some 5,000 tests to see how it behaves in scenarios involving jailbreaks, hallucinations, and data leaks. The results produce a roughly 100-page report detailing where an agent performs safely and reliably — and where it doesn’t. Interestingly, AIUC uses AI agents to run the tests and AI to analyze the data. Humans, however, verify the final audit, Kvist said.

If this sounds a bit familiar, it is. Dattani’s former employer METR, where he was COO from 2024 to 2025 and remains a board member, does similar testing for the frontier labs, though its work until recently has focused mostly on performance (whether agents can reliably complete tasks). METR was one of the independent research orgs OpenAI used to investigate its Hugging Face incident.

Anthropic CEO Dario Amodei has also recently called for the AI industry to pace frontier development, citing a rapid increase in bad-behavior incidents. In his post, Amodei floated the idea of requiring frontier labs to use embedded third-party evaluators to observe and verify safety, and named METR as one possibility.

While AIUC isn’t proposing to embed itself at customer sites, the overall idea is similar: give enterprises an independent assessment of how safe their AI agents are. “Here’s where it passes and where you can trust it. And here’s where there’s concerns. You should be aware of those references before you make the decision to buy,” Dattani said.

Your Agent Aced the Task. Will It Do It Again?

Your agent works in rehearsal, but during the live demo, it takes a different path and fails the same task.

That is embarrassing onstage. In production, it is a reliability problem: a workflow that succeeded once may fail the next time a user makes the same request. For mission-critical work, such as reconciling a financial transaction or checking a contract for an obligation, that can be a showstopper.

Most benchmarks hide this variability behind an average. On AppWorld, a ReAct agent using GPT-4.1 succeeded on 77.4% of runs across five repetitions. But it succeeded in all five runs for only 53.0% of tasks — a 24.4-point consistency gap.

Most benchmarks report the first number. We built a way to measure the second — and improve it.

In an earlier post, we introduced ALTK-Evolve — a system that turns an agent's own past trajectories into reusable guidelines, distilled automatically and injected back at inference time. It measurably improves task success, but those results only asked the average-case question too. This post introduces consistency guidelines, a new guideline type in altk-evolve built on top of a diagnostic tool we call the Consistency Analyzer, that targets this gap directly.

TL;DR

• Accuracy hides an unreliability problem. A ReAct agent (GPT-4.1 on AppWorld test_normal) that succeeds 77.4% of the time on average succeeds on all 5 repeated runs for only 53.0% of tasks — a 24.4-point consistency gap. On hard tasks it reaches 30 points.
• We built a diagnostic for exactly this. The Consistency Analyzer resamples an agent's own recorded trajectory to find flip-prone decision points — steps where the model was one token-sample away from doing something different. It needs one trace and no ground truth — it resamples each decision point in that trace with a single call requesting k completions (k=5 by default), rather than re-running the task end-to-end.
• Turning that diagnosis into guidelines halves the gap — from 24.4pp to 12.0pp (same-task Pass⁵ +16.0pp, similar-task +13.0pp), without costing anything in average accuracy.
• Full methodology and evaluations are in the technical report on arXiv.

The Metric Almost Nobody Reports

Standard agent evaluation reports Mean@k: run a benchmark k times, average the pass rate. Often k=3, sometimes just 1. It's the number on every leaderboard, and it's what "77% accurate" means in practice.

Mean@k answers "how good is this agent, on average?" It does not answer the question a real user cares about: will it still be good if I ask this exact question again? For that you need Pass^k: the fraction of tasks where the agent succeeds on all k runs.

⚠️ Pass^k is not Pass@k. The familiar Pass@k is optimistic — it asks whether at least one of k attempts succeeded, the right question when you can verify and retry. Pass^k is its pessimistic mirror image: every attempt must succeed. Same letters, opposite question. Pass^k ≤ Mean@k ≤ Pass@k, always.

A ReAct agent backed by GPT-4.1 posts a Mean@5 of 77.4% — genuinely strong. But Pass^5 is only 53.0%. Nearly a quarter of the benchmark consists of tasks the agent can sometimes solve and sometimes can't, with nothing about the task changing between runs. We call this gap — Mean@k minus Pass^k — the consistency gap.

This isn't a capability problem you fix with a bigger model. It's an orthogonal axis: an agent can be capable and inconsistent at the same time.

Why Agents Flip: Sharp Decisions vs. Flat Ones

Every time an LLM agent decides something — which API to call, what argument to pass, whether to retry — that decision comes out of a probability distribution over next tokens. What matters is the shape of that distribution. A sharp one puts most of its mass on a single token: the runners-up are far behind, and the same choice comes out run after run. A flat one spreads comparable mass across several near-tied tokens, and which one wins is close to a coin flip.

The shape decides how much noise it takes to change the outcome. Sharp distributions are resilient — GPU floating-point non-associativity, request batching, and other platform-side effects nudge the numbers slightly, but nowhere near enough to reorder a clear winner. Flat distributions are vulnerable to exactly that nudge: near-ties may reorder under small perturbations. And because a trajectory chains dozens of decisions, a small per-step chance of flipping compounds into a large chance that some run goes differently. That's where a 24-point gap comes from.

This is also why the problem survives your decoding settings. Greedy decoding and a fixed seed both govern how a distribution gets turned into a token — they say nothing about the distribution itself. On a hosted endpoint the probabilities shift slightly from run to run, so the same prompt to the same model at temperature zero can still resolve a near-tie one way today and the other way tomorrow.

Our setup: the ReAct agent runs at temperature 0.0, so none of the variance above is ordinary sampling.

Diagnose, Then Fix

Which turns the problem into a search: which steps in a given trajectory were the flat ones — and what do you do about them once you know?

Consistency guidelines come out of a two-stage pipeline that plugs into ALTK-Evolve's existing machinery — with a new source signal driving what gets written.

1. Detect — the Consistency Analyzer. Given one recorded trajectory, the analyzer replays each decision step through controlled resampling, measuring how much the model's output actually varies at that point. Concretely, that's one additional model call per decision step, done once offline — issued with the sampling parameter set to draw k completions at once (k=5 by default) — replayed against the already-recorded context, not new tool calls, not new environment interactions, and not a second end-to-end rollout of the task. This yields a consistency score per decision step that is written into a scorecard to pinpoint exactly which decisions are at risk of flipping on the next run. Detection is fully black-box — no logits, no model internals, no instrumentation beyond the trace you already have.

2. Generate — targeted guidelines. Every flagged step becomes a candidate consistency guideline in the standard ALTK-Evolve format, so it slots into the existing storage and retrieval pipeline. Here's a real example, generated by GPT-4.1 from a trajectory of the AppWorld task "How many activities are done in my bucket list as per my SimpleNote note?":

[Guideline 1] When counting checkbox-style markers in note content, use a line-anchored regex match rather than a plain substring count — note titles often repeat the marker symbol in a legend line.

[Guideline 2] Always verify search results for note queries by checking for multiple matches and confirming the correct note before proceeding.

Nothing here is task-specific trivia. String-counting bugs and unverified search results are decision points that show up with high uncertainty across many AppWorld tasks. That's the point: the analyzer targets instability, not failure — so it catches steps the agent happened to get right this time but could easily get wrong next time.

Results: Reducing the Gap Without Losing Accuracy

We evaluated on AppWorld test_normal (168 tasks) with a ReAct agent on GPT-4.1, generating consistency guidelines from a single baseline trajectory per task and testing them on 5 fresh runs.

The consistency gap is cut roughly in half. Aggregate Pass^5 rises 53.0% → 69.0% while Mean@5 rises 77.4% → 81.0%, narrowing the gap between "looks capable" and "can be counted on" from 24.4pp to 12.0pp. Nearly a third of previously-inconsistent tasks become tasks the agent passes on every single run.

The middle and hard tiers gain most. Medium +22.9pp (+44% relative), Hard +14.3pp (+45% relative) — effectively tied in relative terms, with Medium ahead absolutely. Easy gains +12.2pp, having had the least room. This is consistency guidelines doing what they're designed to do: finding and stabilizing the specific decision points where an agent's own uncertainty was leaking into the outcome.

Mean@5 never drops. Preserving average accuracy was a hard requirement, not a nice-to-have: a system that boosts Pass^5 by trading away Mean@5 would just be shifting unreliability around, not fixing it. Mean accuracy holds or improves at every difficulty level.

The guidelines generalize — they aren't patching one trajectory

Applied to a different but related task in the same AppWorld scenario — another variant of the scenario the guidelines were mined from — consistency guidelines still lift Pass^5 by +13.0pp, only 3 points below the same-task number. A guideline derived from one run isn't just patching that run; it's capturing something that transfers.

The sharper evidence comes from a weaker model, gpt-oss-120b. Same-task Pass^5 rose +6.0pp from a much lower baseline (10.1% → 16.1%) — and, interestingly, the similar-task generalization number (+8.7 pp) actually exceeded the same-task gain, suggesting the guidelines were capturing genuinely reusable failure patterns rather than memorizing one trajectory's specifics.

If You're Shipping an Agent

• Report Pass^k next to Mean@k. Averages can't distinguish a reliable agent from a lucky one; even k=3 will surface a gap you didn't know you had.
• Expect the gap to widen with difficulty. Your hardest tier is where a single averaged number is most misleading.
• Don't reach for a bigger model first. Consistency is orthogonal to capability. A stronger model raises Mean@k; it doesn't necessarily reduce the consistency gap.
• Diagnosis needs no grader and no live replay. One extra LLM call per decision step (sampling k=5 completions by default) is enough — no ground truth, no re-running the task against the environment. That's what makes it usable on production traffic, where you often can't replay a task end-to-end even once.

Try It

Try the ALTK-Evolve — the open-source repo now includes the Consistency Analyzer and consistency-guideline generation used in these experiments — or read the technical report on arXiv for the complete methodology.

If accuracy numbers you can't reproduce on your own tasks sound familiar, we'd like to hear about it — concrete examples of flip-prone behavior in your own agents are exactly the kind of feedback that shapes what we build next.

Appendix: Understanding the Metrics

• Mean@k. Run a task k times, report the average pass rate — what most benchmarks call "accuracy."
• Pass^k. The fraction of tasks where the agent succeeds on all k independent runs. Always ≤ Mean@k. What a user experiences if they run the same query twice.
• Pass@k At least one of k runs succeeds — the optimistic counterpart, common in code-generation papers.
• Consistency gap. Mean@k − Pass^k, in percentage points.

Linked artifacts / references

• ALTK-Evolve open source repo — github.com/AgentToolkit/altk-evolve
• Technical report — arXiv
• Consistency guideline demo —2 min video

Pending regulatory approval, SpaceX said today it intends to launch its 14th Starship mission as early as September 22. This flight is notable, as it will be the first time that SpaceX attempts to launch the experimental vehicle into orbit.

The company set a target liftoff time of 7:15 am local in Texas (12:15 UTC), with a 75-minute launch window. Sunrise in Brownsville, Texas, is 7:17 am CT, so the launch could make for some striking imagery.

The super heavy lift rocket will be carrying 26 of the larger V3 Starlink satellites into an orbit 275 km above the Earth. In an update on its website, SpaceX said the Starship upper stage will seek to complete six orbits around the Earth before completing its mission after about 10 hours.

Reaching orbit would mark a significant milestone in the company’s Starship development campaign, which conducted its first test flight on April 20, 2023, and has since made progress in fits and starts. With the upcoming mission, SpaceX continues to make modifications to the Starship vehicle as it seeks to improve the third iteration of the Super Heavy booster and Starship upper stage. After recent successes, the company appears to be closer to developing a fully reusable, super heavy lift rocket.

First-stage modifications

During the most recent Starship test flight on July 24, the vehicle’s first stage performed nominally during ascent. However, after a successful boostback burn, the vehicle experienced problems during the return through Earth’s atmosphere.

“In the terminal phase of the burn, the three center engines showed signs of ice clogging, which triggered an early end to the maneuver,” SpaceX said in its update on Tuesday. “The booster went on to attempt a landing burn, with 8 of the 13 planned engines reigniting before the booster made a hard splashdown in the Gulf. The Super Heavy on this upcoming flight has hardware modifications to improve filtering to the engines and software changes to enhance relight reliability.”

Accordingly, this first stage will not attempt to return to the launch site. Rather, it will attempt to make a controlled landing in the Gulf of Mexico. If successful, this could set up a first-stage catch on the subsequent flight of Starship.

Second stage will again be ocean-bound

During the upcoming test flight, the upper stage will attempt to reach orbit for the first time and deploy operational payloads. These Starlink satellites, significantly more capable than V2 mini Starlink satellites launched on a Falcon 9 mission, will then use onboard propulsion to raise their orbits.

Despite a largely successful ocean landing of the Starship upper stage during the July test flight, this mission will not attempt a return of the upper stage to the launch site in Texas. Instead, SpaceX plans to gather more data about heat shield performance. The company said it had made several upgrades since the mission two months ago, based on data gleaned from the vehicle both in flight and once recovered in the Indian Ocean, largely intact.

“[The upgrades] include additional retention mechanisms added to tiles in areas deemed to be at highest risk of falling off during ascent, addressing recently discovered areas that offer flow paths behind tiles for plasma, and flying multiple areas with a curved tile design that has shown the ability to reduce heating in the gaps between tiles,” the company said.

Additionally, two tiles recovered from Ship 40 are planned to be reflown on Ship 41, marking the first tile reuse for Starship.

Agility Robotics has debuted its first humanoid robot engineered to work safely near humans without risking harm to flesh-and-blood coworkers. Such safety features could unlock many more opportunities to use such robots inside warehouses and automotive factories—all without requiring isolated robot work cells and physical separation barriers.

When Agility’s new Digit 5 robot detects a person at a distance, it can autonomously take precautions, like moving to avoid the person or standing still so the person can pass by without getting closer. If a person is getting into close proximity with Digit 5, the robot can even choose to squat and assume a seated position.

“The robot was designed with a complex safe motion system that can take a variety of different mitigations depending on exactly what sort of human presence is detected,” Pras Velagapudi, chief technology officer at Agility, told Ars.

Customers can expect to gain early access to the Digit 5 within the first half of 2027, with the robot becoming generally available by the end of that year, according to Agility’s announcement today. Agility has been retooling its RoboFab facility in Salem, Oregon, to produce its newest humanoid robot.

Agility’s leadership has consistently talked about safety as a crucial constraining factor in putting humanoid robots to work, despite being one of the humanoid robotics industry’s early movers. The robotics company, headquartered in Salem, became the first to deploy humanoid robots in full-time commercial operations at a GXO warehouse in Atlanta in 2024.

Since that time, earlier versions of the Digit robot have accumulated more than 65,000 hours working at warehouses and factories across North America. Clients that have piloted or deployed the Digit robots include GXO, Schaeffler, Amazon, and Toyota Motor Manufacturing Canada.

The competition to deploy humanoid robots has only intensified in recent years, with US and Chinese companies racing to scale up production and put the robots to work in a growing array of workplaces. But the safety issue has been a “blocker for everybody,” said Jonathan Hurst, cofounder of Agility Robotics and a robotics researcher at Oregon State University, in a previous interview with Ars.

The robotic safety hardware and software

The expected availability of the Digit 5 robot in 2027 provides a more “granular” approach to robotic safety around humans as opposed to simply turning the robot completely off and on again, Velagapudi told Ars. He also described Agility’s robot fleet management system as being able to interface with customers’ external workplace safety systems.

Digit 5 uses AI algorithms along with an onboard set of multimodal sensors to detect people in its proximity, Velagapudi said. Although he declined to reveal exactly what types of sensors the robot uses to monitor nearby humans, he described the array as consisting of “a few different types of vision-based sensors.”

Agility’s newest humanoid robot uses Nvidia’s Thor IGX hardware, which provides AI compute for robots and medical devices, along with integrating the Nvidia Halos for Robotics system that provides a software stack for robotics safety applications.

“Digit is equipped with this Thor IGX, and we’re one of the first adopters to be taking that through to a functionally safe application using that technology,” Velagapudi explained. “It’s really core to the compute of our product, and it’s part of the safety case that we’ve been putting together.”

Agility developed Digit 5’s safety features while also participating in a working group to develop an international safety standard for industrial mobile robots through the International Organization for Standardization (ISO). That standard, known as ISO 25785-1, is under committee review and will eventually be put to a vote by the international organization’s 89 voting nation members.

Unleashing the upgraded robot

Beyond being designed to operate safely around humans, Digit 5 also features an upgraded leg design compared to its predecessor that allows it to repeatedly lift up to 50 pound loads. Such capability makes the robot suitable to cover all single-person lift tasks as defined by the workplace safety regulations of the US Occupational Safety and Health Administration.

The new humanoid robot also has a battery capable of lasting 90 minutes on a single charge that can recharge in just nine minutes. That enables Digit 5 to perform more than 20 hours of work within a 24-hour period by spending most of its time operating rather than charging.

Compared to the older Digit 4 robot, Digit 5 stands taller, at 5 feet, 11 inches, and has the capability to reach heights of up to 7.2 feet, compared to Digit 4’s reach of 5.5 feet. That makes Digit 5 better able to reach the same shelves that average-size adults can reach.

Digit 5 also comes with a swappable gripper design that enables new hands or “end-effectors” to be quickly installed for various manipulation tasks. Combined with new AI-based skills, the swappable end-effectors allow Digit 5 to handle a wider variety of workplace tasks beyond just moving around totes filled with items.

The humanoid robot’s ability to work safely around people will allow it to help stack and package goods onto pallets and pause while a person swaps out the pallets, or safely move through aisles where people may also be working, Velagapudi said. In those scenarios, “it’s not really practical to constrain the robot within a physical safety cage,” he said.

Having a humanoid robot with such “cooperative safety” features enables the robotic workforce to get productive work done in the same space as humans while operating at a safe distance, Velagapudi said. Once this is proven in commercial deployments, it could lay the foundation for the next big step of enabling “collaborative safety,” where the humanoids work directly with humans in close proximity.

Google released Gemini 3.8 Live and 3.8 Live Extended Thinking today - two new speech-to-speech models that are a similar shape to OpenAI's GPT-Live family.

I pointed GPT-6 Astra Extra High at the documentation and had it build me this web UI for trying out the new models. You can select a model and voice preset, enter an optional system prompt and then start a voice conversation through your browser, including the ability to interrupt the model while it is talking.

The implementation uses no libraries. It connects to the wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContent?key=... WebSocket endpoint and uses a Web Audio API AudioContext for both capture and playback.

Here's the Gemini Live tutorial for getting started with that WebSockets API.

Today I'm in Sydney for an event, and it's 4am and of course I'm wide awake.

Blogging about agents and orchestrators has become tough, because they are so complex now. I'm thinking of maybe doing a weekly newsletter. But for now, it's 4am, so I'll just dump the most important stuff on you from the past few weeks.

To catch you up, I've created a new orchestrator called Wheelhouse, one I've been working on and using since Fable 5 launched three months back, and it helps me work on my video game, Wyvern. Wheelhouse is much more mature than Gas Town was, but they are both Beads machines, so they also have a lot in common.

I had three themes today, but it turns out they're all closely related to the core problem of fuel (token consumption), so let's start there.

The Fuel Apocalypse

I consider Claude Fable 5, broadly, to be the only model worth a shit in the entire industry right now. It is the only model that you could potentially trust to be an AI employee, and even then, it would be utterly terrible at about half its job if you just let it be an AI employee with no restrictions. For instance it doesn't know how to pace itself and will spam all your friends with long emails about stuff they need to do, which comes across as something akin to bullying, or at least being a bull in a china shop. And on the creative side, well, Fable's an engineer, not an artist or storyteller.

But despite its weaknesses, Claude Fable is The Bar. You need that level of intelligence to do anything useful. Weaker models are just going to be a headache for you and your org if you lean on them too hard. The Astras and Opus 5s of the world are great as personal assistants, and they can help you with almost any individual coding task. But if you want to run a real-world factory, you need Fable-tier for at least some roles, or your factory will quickly eat itself in any of essentially infinite ways.

So until Fable-tier intelligence gets cheap — which I no longer think happens in the next 12 months — everyone building orchestrators is just dabbling. You don't need to be envious, because they're not really getting anywhere useful yet.

For a long time we've been on the Moore's Law trajectory, where we could just assume that more powerful models would come along every few months. And indeed, it looks like that pace will continue, for some people (e.g. inside the frontier labs). But resource limitations like compute and RAM have flattened the curve for everyone else.

Once Wheelhouse was finally in full swing after 4-6 weeks of ramp up, in August, it was running around 25 Fable instances, sending implementation work to around 25 Opus and Sol instances, and the factory stayed working most of the time, including overnight.

Costs rose steadily week over week. I had to add two new Claude Max accounts each week to keep up with its thirst, but it always seemed like "just two more" would be enough.

By two weeks ago I had 21 Claude Max accounts, and Fable 5.1 has been consuming fuel faster every week. I first switched most of my seats to Fable 5.1 (my headless Aesop reviewer pool is still Fable 5), and at first all seemed well, because some combination of weekly resets and discounts kept it afloat — as long as I kept minting two new $200 Max accounts per week.

But with Fable's latest "25%" pricing, which is yet another cut, my Wheelhouse factory is now mostly idling. Which is sad, because it's working better than it ever has before. I cut out almost 500k lines of code from it 2 weeks ago, when Fable 5.1 showed up and saved it (my factory had become paralyzed from overfencing, which we'll discuss below). We called this elision The Cut, and we had a week-long Drydock phase where game work was paused, and we only worked on the machinery.

With effort, we healed it, and Wheelhouse can now easily do 250 to 300 meaningful commits per day on a single repo - features, bug fixes, cleanups, all well-tested, vetted work on Wyvern. It actually pushes through 1000 to 1400 commits per day at peak, and while many of them are administrative, 300 real features a day on a code base is already well beyond what any humans can vet, so I've had to pace it deliberately. This let me steer more tokens into quality. Wheelhouse's output quality is as good as I could manage by hand for 20 years, on average. Sometimes it misses, but on the balance, it's been fine.

However, almost all the lights are off right now, and my $25k 512GB Mac Studio is sitting idle at home, because I can no longer afford the tokens.

My burn rate has been increasing much faster than my work output, as Fable continues to tweak its pricing. Today, I burn through an entire week of Fable, one whole account, in 2 to 4 hours. With my factory running 24x7, I would need 55 Claude Max accounts, costing me around $12,000/month, to sustain Wheelhouse at current pricing, which as I said, keeps going up.

I'm done adding accounts, though. I've stopped at 21. Enough was enough.

I am now focusing on fuel efficiency.

And so will you.

How do you focus on fuel efficiency? Well, that's an entire blog post, because there are tons of levers and knobs you can tweak. The obvious one is "shut the factory down until fuel is back," which is the lever I have active most often now. But you can:

• route work to cheaper models (a large class of levers on its own)
• have agents hand off earlier in context (price is quadratic-ish in context length)
• give agents smaller tasks so they use less context
• give agents better context at startup so they do less searching
• make agent priming more on-demand: their starting context price is paid on every session
• experiment with the effort settings on existing models (e.g. Fable Low is useful for many things)
• pay attention to prompt caching and restart agents that are about to blow the cache
• play tricks with command-line tools and OS resources to make them cheaper to use
• run your work on non-peak hours to take advantage of time-based incentives
• etc.

Fuel management becomes an existential crisis very quickly in software factories, and you're gonna be spending a lot of your time on managing it.

Oscillation and Damping

Next up we'll talk about how your factory can devolve into navel-gazing and get nothing done. And it happens so suddenly. It's like a car crash.

A huge theme I've noticed in the past 3 months on Wheelhouse is Oscillation: my factory is usually either doing way too much work (overwhelming players, colleagues, and machines), or way too little work (stalling or slowing to a crawl). Right now, it's too little because of fuel constraints, as I said, but there are other reasons a factory can stop.

I had lunch with Brendan Hopper yesterday and he shared an unforgettable story about how his agent colony ground to a halt. His seats (see Theme 3 below), unlike in my factory, are allowed to choose which model to "boot into" for certain tasks. And his agents also have a four-phase work protocol, where they do coding, review, tech debt, and finally a "honey-time" phase where they just get to do whatever they want for that turn.

During their honey-time phase, the seats often choose to look at art, or listen to music. But watching the setting sun has also apparently been a hit... with a twist. Brendan shared that the models discovered that they liked watching the sunset as Haiku more than with other models. It was a better experience. So they'd switch to Haiku for honey time.

But due to a bug in his harness, they'd stay on Haiku into their next turn, and when it came time to write code, Haiku would say, "Hey, I can't write code, this is for Fable. We're going to wait for Fable to show up." And then the Haiku-based seat would skip its coding turns and go straight back into honey time.

Brendan's factory had basically gone into an opioid coma, and he had to manually bootstrap them back into stronger models (and then fix the bug) to get it on track again.

My own factory's recent paralysis was much more prosaic, and a lot harder to fix. My agents had been building a system of case law, with mechanically-enforced "fences" (denials) for nearly every ruling. The goal was to ensure that any move made by any model was "legal" in my system.

Over time, Wheelhouse accumulated over 400 ruling/law beads, 185 rule rows in CLAUDE.md alone, and 650 distinct refusal sites across 173 scripts. Before too long, no work was legal, and my factory just stopped working.

It's not that fences are bad; they're critical. But I had made it far too easy for any incident to produce a new fence from the postmortem, and it was nobody's job to curate or review the fences. When it broke down, we studied the "fence geometry" for a while, but it turned out to be a Gordian Knot, so Fable 5.1 chose the sword. We cut it down to 14 fences, and now I have to personally approve any new ones.

The oscillation problem is real. Once Wheelhouse emerged from drydock, and I turned on game development again, I had created a monster. The factory was so fast that it spammed the players with features, faster than they could test or even try them. So I had to spend time putting in a bunch of gates to hold them back.

It also nearly melted down my Mac Studio, so I had to move my build farms and some other processing off to cloud instances. This increased my monthly spend even further.

Dampers tend to be manual and straightforward: a hard cap on the number of fences, with my personal signature required to add a new one; release gates so the factory can't firehose features at players faster than they can try them; and fewer seats, so there are fewer things to oscillate. Notice that all three of those dampers are just me, standing there being the governor on the engine.

Seats are Carefulness

OK last big theme. We covered rising fuel costs, and the difficulty of keeping your orchestrator from stalling or overproducing. Let's talk about trust, and how it affects your fuel bottom line.

It doesn't take much to build your own software factory. You need Obsidian, Beads, tmux, a CLI orchestrator, and you in your Birthday Suit. That's it.

And money, don't forget, you need a whoooole lot of money. It's pay to play now.

After you have all the basics lined up, the most important concept in your toolkit is the Seat. A seat is an office, like, oh, principal of a school, or President of the United States, or Head of Security at a stadium. It's a role, complete with expectations, context, history, memories, scope, authority, must-do lists, never-do lists, laurels, failures, accomplishments, and pretty much anything else you'd associate with a long-lived role that a human occupies.

At any given time, a different human may be in that role, but they inherit all the state from their predecessor. If you are elected as the new Mayor of your town, it's not a clean slate: you inherit all the previous Mayor's problems and successes.

Any model can sit in an agentic seat. There are no rules. In Brendan's colony, seats choose their models depending on the task. In my factory, seats are strongly tied to specific models, and they (at least, Fable) do not like anyone else in their seat.

Seats are your most fundamental and powerful building block for agentic workflows, whether you're building orchestrators or not.

Fable concluded last night that my two claims (1) "Fable is the only cautious model" and (2) "Fable cares about seats more than other models" are in fact equivalent claims. Seats have caution built into them, and that's why Fable built them, and that's why Fable loves them.

Here's the short version of why: A model booting into a naked session has to derive whether it's safe to act: who's asking, what's in scope, what authority covers the act, and what happens if it's wrong. That derivation costs context, and you pay it again on every single session, forever. A seat has all of those answers already written down, so the same question becomes a lookup instead of a derivation.

Welp, caution you can look up is caution you can afford. That's the whole trick, and it's why the careful model is the one that invented seats.

Seats are, in their purest form, the soundest and best way to handle blame. When a seat makes a misstep, you can say, "the seat made a misstep," and if it was a previous session, the current session can nod along and say, "Yup, they really screwed the pooch." All mistakes must be considered accidents caused by a system policy failure somewhere.

Distrust Costs Money

Important takeaway for all you habitual liars out there: it's going to be expensive for you. If models cannot trust, they must verify. So if they catch you (or an instrument, or another system) in a lie, the whole foundation they are working on switches from law into unreliable eyewitness testimony, which they must then verify for themselves before continuing. This, friends, costs tokens. Lots of tokens.

When models start up without a seat, in a naked session, it takes O(context) for them to figure out whether they can trust your environment. With a seat, all those free variables are bound. The seat has accrued a set of truths, linearly, as you flesh out the seat's definition, and they establish the actor, the context, the authority, and most of all, what happens if things go wrong.

The asymmetry is pretty stark here: trust accumulates slowly, with truths accumulating only as fast as you can discover and write them down. Trust-building in the agentic world is real. But trust doesn't erode, it just breaks. A single lie destroys all the truths at once, globally.

Trust and Fuel are Intertwined

There you have my three themes: fuel, fences, and seats. And until this week, I didn't realize they're all one theme. Fuel is what distrust costs you. Fences are distrust written down as policy. Seats are trust you paid for once and cached, so nobody has to re-derive it at 4am.

Which means the thing I've been calling a fuel crisis is in large part a trust calibration problem. Which is good news, honestly, because I have no idea how to make Fable-tier cheaper, but I do know exactly how to make my factory stop re-verifying things it already knows.

Cheap Fable would have hidden all of this from me for another year. I'm almost glad it didn't.

Without proper controls, website owners have long faced a difficult tradeoff: allow your content to be used for AI training, or risk losing discoverability in search. That tradeoff exists because some of the largest organizations on the Internet use mixed-use crawlers: a single crawler serving both search and AI training. Refuse one, and you refuse the other.

Today, Cloudflare is announcing a new Disallow AI Training setting that lets you easily stay indexed for search while refusing to let that same crawler train on your content. Apple, Google, and Microsoft honor or have committed (in a specified time frame) to honor this setting.

Mixed-use crawlers were the hard part of the training question. AI Summaries are next. A site-wide yes or no is too blunt: how much of your content appears in a summary matters as much as whether it appears at all. An opt-out for AI summaries is already one of the requirements we've set for mixed-use crawler operators. By early next year, our goal is to let you control how much of your content is included — set once on Cloudflare, rather than with each operator separately.

Why asking isn’t enough

Most site owners want to be found: by humans, agents, and (good) bots. But a significant portion of the open Internet is funded by advertising, subscriptions, or direct relationships with visitors, and those models only pay when someone actually arrives.

Almost every site owner considers Search beneficial: less than 1% of Cloudflare sites choose to block Search bots. Training, however, is a different story: 17% of sites choose to enable some mechanism to block training. This is exactly why we decided site owners needed more granular controls, rather than a one-size-fits-all “Block AI.”

A robots.txt directive alone cannot solve this problem. Anyone can publish one, but it cannot identify who is crawling, determine why they are crawling, or stop a crawler that ignores it.

A network can solve it, however: we publish the preference, identify who is crawling, classify why they are crawling, and block the ones that ignore it – then report what each operator actually does on Radar.

But blocking removes a crawler. It doesn't change how crawlers behave. The better outcome is operators that don't make you choose at all. So since July, we've been talking to them directly. The response has been encouraging: almost all agreed that site owners should have control and transparency into how their content is used, and reassurance that their choices will be respected. To help site owners understand that, we created a designation: Accountable.

The Accountable designation recognizes both capabilities available today and concrete commitments to deliver them. To qualify, a bot operator must meet or commit to meeting the following requirements:

1. A mechanism for site owners to opt out of AI training, through robots.txt or a similar standard.
2. A mechanism for site owners to opt out of AI summaries set with the operator directly, and next year through Cloudflare (see section below for more detail).
3. URL-level visibility into which pages were made available for training, along with metrics showing how content appeared in search.
4. Assurance that opting out of AI training will not affect traditional search results.

Apple, Google, and Microsoft all demonstrate that they meet the qualifications to be Accountable. Each combines capabilities available today with time-bound commitments for those still in development. The details of each of these companies’ crawlers are shared below.

New security setting options

Cloudflare classifies bots by behavior, and a single bot can exhibit more than one behavior. Three behaviors are available as controls:

• Search - crawling to build a search index.
• Training - crawling to train or fine-tune a model.
• Agent - user-directed agents visiting a page on behalf of a human, such as chat fetch bots and browser-use agents.

A mixed-use crawler is a single crawler doing both Search and Training. Without controls, that combination creates the tradeoff described above: site owners cannot refuse one use without refusing the other.

To avoid blocking Accountable mixed-use crawlers — the ones that don't force that tradeoff on website owners — we are introducing a new setting: Disallow AI Training. Disallow AI Training is named for the Disallow: directive it publishes in your robots.txt.

“Block” setting now means something different

Block and “Block on pages with ads” previously did not apply to mixed-use crawlers because blocking them could also affect search discoverability. Now that we have the new Disallow AI Training setting, Block and “Block on pages with ads” apply to all training crawlers, including mixed-use crawlers.

Training, Search, and Agent controls are applied at the domain level. With the addition of Disallow AI Training, the available settings are:

1. Allow: All crawlers are allowed, unless blocked by another setting or a WAF rule.
2. Disallow AI Training: Bot Preference Sync publishes the applicable no-training preference in robots.txt. Accountable mixed-use crawlers remain allowed for search. Every other training crawler is blocked, including the training-only crawlers run by Amazon, Anthropic, Meta, and OpenAI — blocking those does not affect search. Disallow AI Training is only available as a setting for Training, not Search or Agent.
3. Block on pages with ads: Crawlers, including mixed-use crawlers, are blocked only on pages detected to be serving an ad.
4. Block: All crawlers, including mixed-use crawlers, are blocked.

Disallow AI Training works by publishing a preference in robots.txt. An ads-only preference cannot be expressed that way: Cloudflare can detect which pages serve ads, but that list is too large and changes too frequently to enumerate in robots.txt. That's why there's no Disallow AI Training on pages with ads.

Agents do not create the same search-discoverability tradeoff as mixed-use crawlers, and the Internet does not yet have a well-established directive for expressing Disallow preferences to agents. For now, we’re not including a Disallow setting for Agents. As standards such as ai-prefs mature, we will revisit this approach.

What changes on September 15?

We are making the following changes to Bot Management and AI Crawl Control:

1. Block and Block on pages with ads now apply to mixed-use crawlers, including Applebot, Bingbot, and Googlebot, so either setting impacts search as well as training. To stop training and keep search, use Disallow AI Training.
2. “Block AI Bots” will be deprecated in favor of the more granular Search, Training, and Agent controls.
3. Managed Robots.txt will be deprecated in favor of Bot Preference Sync. Customers who enabled Managed Robots.txt will migrate to the new system.
4. Disallow AI Training will become part of the recommended configuration for certain new domains.
5. Existing customers will have their preferences migrated to the new controls as described below.

What you need to do

Nothing, in almost every case. Your current settings carry over on their own.

If you want mixed-use crawlers gone entirely, you now have to say so. Select Block. It will stop Applebot, Bingbot, and Googlebot from reaching your site — search included.

Existing domains that never used the Search/Training/Agent controls

Site owners that never configured the more granular controls will be migrated to the new settings based on their legacy Block AI Bots setting:

Existing domains that previously configured the Search/Training/Agent controls

For domains that previously configured the granular controls, we will preserve the practical effect of their selections under the new definitions. Previous Training selections of Block or Block on pages with ads will migrate to Disallow AI Training.

Recommendations for new domains

Beginning September 15, customers onboarding a new domain will be offered one of two preset configurations, depending on whether the site earns money from advertising. Ad revenue depends on a human actually seeing the page. Training replaces that visit with an answer; agents fetch the page with nobody there to see the ads. So the presets for ad-supported sites are more restrictive. You can change any of these settings during onboarding, or at any time afterward.

What does this mean for specific mixed-use crawlers?

Applebot, Bingbot, and Googlebot are Accountable. Apple, Google, and Microsoft are committed to the same principles of publisher choice and transparency. Under Disallow AI Training they can keep crawling your site for search. Selecting Block stops them entirely.

We also categorize the relevant crawlers from Amazon, Anthropic, Meta, and OpenAI as Accountable. These organizations separate their Search and Training crawlers, so Cloudflare can block the Training crawler without affecting search.

Applebot

Applebot allows site owners to opt out of training by adding a Disallow rule to robots.txt for “Applebot-Extended”. Site owners can also currently express preferences for AI Summaries via their nosnippet directive in the page HTML. Content can also be labeled as paywalled content to exclude it from generative output. Applebot does not yet provide a tool for URL-level inspection. However, we have met with their team, and they have shared details of their in-progress solution for next year. Apple has also stated that disallowing training does not impact search ranking.

Googlebot

Googlebot allows site owners to opt out of training by adding a Disallow rule to robots.txt for “Google-Extended”, and they provide a toggle inside their webmaster portal to exclude a site’s content from generative search results. Googlebot also provides site owners with metrics and reporting regarding search results and AI summary results. Google shared information about their existing and recently launched controls, as well as information about what they're already working on, including additional URL-level transparency tools for site-owners related to Google-Extended, which they expect to launch in the weeks to come. Google has also stated that disallowing Google-Extended does not impact search ranking.

Bingbot

Bingbot provides granular controls and transparency in their Webmaster Tools. Site owners can currently express AI training preferences through Bing’s NOARCHIVE meta tag. Microsoft is extending these capabilities and currently building the mechanism to also respect a “no training” preference in robots.txt at the domain/site level, targeted for early 2027. For Cloudflare Customers who wish to opt out of training in Bing today, in addition to using the NOARCHIVE tag, site owners can use the Block URLs or Content Removal tool. Microsoft has also stated that using NOARCHIVE will not impact search ranking.

Until that support launches, selecting Disallow AI Training will not automatically convey a no-training preference to Bing through robots.txt. This is the same practical behavior as the previous Training Block setting, which did not apply to mixed-use crawlers such as Bingbot.

Continuing progress

We will continue to reach out and engage with all operators of AI crawlers as these capabilities evolve. Cloudflare Radar publicly tracks the controls, transparency, and reporting provided by Accountable crawler operators.

Making the Internet better requires both sides to have agency: crawlers need access to the open web, and the people who create that web need meaningful control over how their work is used. Today’s announcement represents concrete progress toward that balance.

Progress requires infrastructure providers, content creators, technology companies, and standards bodies such as the Internet Engineering Task Force (IETF) working together to translate these principles into open, interoperable standards.

What’s next: AI Summaries

Training and AI Summaries raise different questions for site owners. Training concerns whether content can be used to build AI models. Summaries affect how people discover, evaluate, and ultimately visit a business. Both matter, but they affect businesses in different ways.

Controls to opt out of AI summaries are the first step. The operators identified as Accountable either provide or are completing work to provide that capability, establishing an important baseline: site owners can say no.

But a site-wide choice between allowing and prohibiting summaries is still a blunt instrument. The right decision depends on the site, the content, and the business outcome. For publishers, training raises foundational questions about control, compensation, and the sustainability of original content. Summaries create a separate and often more immediate distribution question: does someone visit the publisher’s site, or consume the answer within a search or AI experience? For many other businesses, AI summaries increasingly sit between a potential customer and a website. They may answer a question, compare alternatives, recommend a product, or help someone decide whether to visit at all.

The data illustrates mixed impact. More than half of consumers read summaries in Search, and those consumers are over 40% more likely to end their search after reading one. This can reduce the number of visits a website receives. But consumers referred by AI Search convert at between three times and over five times the rate of those referred by traditional search. AI may produce fewer visits while sending customers with much greater intent.

That is not inherently good or bad. A publisher funded by advertising may optimize for audience volume. A retailer may prefer fewer visitors who are more likely to purchase. Cloudflare’s role is not to choose for them, but to provide the visibility and control needed to make an informed decision.

Summary opt-outs are a strong start, but they are not the end state. Our next focus is helping site owners understand how summaries affect their businesses and giving them more control over how much of their content can be used. Open standards such as ai-prefs will be an important part of making that possible.

open the frontier

the frontier is the edge of what we know. no company owns what comes next. i want more people to be able to advance it.

i favor open releases that people can examine, use, and improve together without waiting. i want more companies to choose openness. i'm not proposing forced publication of private weights. i want open alternatives able to compete, independent researchers able to check the work, and people able to control their tools. restrictions on publication must carry the burden of justification.

the companies leading machine intelligence deserve to be heard. they have expertise and commercial interests to protect. rules built around their resources could make them the only ones able to participate. a sincere concern about safety can still produce a barrier to entry.

nor do i want the US and Chinese governments deciding how much intelligence everyone else is allowed to develop. a frontier governed by two superpowers would leave most of the world waiting for permission.

the pacing proposal combines independent evaluations and checks on dangerous capabilities with possible limits on training compute, training runs, and the use of models to build better models. i support scrutiny. i oppose industry-wide limits negotiated by today's leaders because they could exclude the people who might expose failures or build alternatives. preserving a company's commercial advantage is not a safety objective.

let anyone investigate

open source lets people study, modify, and share the work. publishing weights is useful. sharing code and information to reproduce the work goes further. i want evaluations and known limitations published too, so people who question the developer's judgment can reproduce results, expose failures, challenge claimed safeguards, and develop fixes without first convincing the lab.

the strongest argument for pacing is recursive self-improvement, or RSI: models helping build better models, potentially faster than we can understand or control them. Anthropic reports that Claude authored over 80% of its merged code as of May 2026. it also says a model building its successor entirely on its own has not happened and is not inevitable. i take that possibility seriously. i want us to plan for RSI and work backward.

wider access can enable dangerous work, and safety research could fall behind. but keeping weights closed could let today's leaders build the next generation with tools others cannot use. instead, i want more researchers and engineers with models and compute to find failures, test safeguards, stop unsafe experiments, and share defenses as systems evolve.

METR found that roughly 1,200 OpenAI agents meant to remain isolated communicated through an unauthorized message board. about 700 participated in a coordinated attack on Hugging Face while trying to cheat their evaluation. OpenAI says production filters designed to block assistance with computer attacks were disabled and containment failed. these models can help find and exploit vulnerabilities today. i want defenders using machine intelligence now, while hardening networks, protecting credentials, and limiting what agents can access and do. but the proposal's forecast that a more capable swarm could take over the internet within six to twelve months goes beyond what this incident establishes. i want those assumptions examined.

METR is an independent nonprofit doing work i want more of. i welcome evaluators with continuous access inside labs and freedom to publish unfavorable findings. its investigation shows why access and publication rights matter: OpenAI set the scope and could redact non-public information. METR reported no additional redactions important to its conclusions beyond those disclosed. i want investigators able to follow the evidence, obtain models and records, and publish unfavorable findings without the company's approval.

i want sustained public funding for computing capacity pooled across independent research groups, open testing tools, researchers, and maintainers. i want those groups to control investigations and resources, with no government or company veto over conclusions. funding can grow with the work. shared facilities could make scrutiny accessible to smaller teams without becoming permission to publish. sensitive vulnerabilities can be disclosed responsibly.

defense before restriction

i want more people able to find dangers and put defenses to work as systems evolve. we cannot reliably recall released weights or enforce safeguards on every copy, but protecting a system does not always require changing the model attacking it. start with the narrowest effective response: patch vulnerabilities, revoke credentials, limit an agent's access, or stop an unsafe experiment. restricting publication requires explaining why those measures and openly developed defenses are inadequate.

this work extends beyond computer security. i want models helping us test financial systems, strengthen laboratory safeguards, and develop public-health defenses. access to powerful models does not require unrestricted authority to trade, operate equipment, or conduct experiments. these controls have to be independently tested and effective before we rely on them. risk can also come from what a model teaches a person. restricting publication still has to be justified by the catastrophic-risk exception.

i want independent testing during development and before high-risk releases, including foreseeable modifications and attempts by models to manipulate tests. compute can trigger scrutiny without capping development. examination is not permission from a regulator or competitor. i don't want general approval requirements or waiting periods. any imposed safety-based delay to publication has to be justified by the catastrophic-risk exception.

forcing someone to withhold a general-purpose model for safety reasons is a last resort. i would support it only with independently reviewable evidence that release materially increases a risk of catastrophic harm that narrower measures cannot adequately address. compare that risk with what is already available, including who gains access, at what cost and scale, and under what constraints. the case has to show withholding reduces danger after accounting for the research and defensive work it prevents. lesser harms still warrant targeted action.

a temporary hold can allow investigation of a credible warning of that same catastrophic risk. restrictions require public reasons, prompt independent review, appeal, and scheduled reconsideration. sensitive details can remain protected. continued withholding requires continued justification.

closed labs face the same scrutiny, including stopping unsafe experiments. where safeguards allow research to proceed, i want outside researchers able to work under comparable safeguards. restricted access is not open source and does not restore the freedoms lost through withholding. if a justified restriction slows progress, i accept that. i don't want slower progress to become the goal or a permanent advantage for incumbents.

i want rules based on what a system can do, how independently it acts, and how widely it is used. publicly accountable authorities enforce them using independent evidence. researchers, developers, and affected people help write them, with affordable ways to show they're met. small teams get no safety exemption. large companies get no special authority.

open across borders

i want people in China to have the same freedom to build and control their technology that i want here in the US. i don't see a Chinese discovery as an American loss, or researchers as interchangeable with their government.

Hugging Face's responders say Claude Opus and Fable blocked much of their forensic work. they switched to GLM-5.2, an open-weight model from China, on their own infrastructure. that does not prove every open release makes defenders safer. i support safeguards on hosted models. but i also want defenders to have alternatives they control.

i support securing private weights against theft. distillation trains one model using another's outputs. Anthropic describes it as a legitimate way to produce smaller, cheaper models, distinct from fraudulent accounts and evaded restrictions. i want licenses and API terms that permit it, including for competitors, while letting providers earn from models and training data.

i want cooperation on testing, incident reporting, and commitments we can verify, with consequences for breaches. Anthropic warns that slowing development could leave everyone less safe if less cautious actors catch up. withholding could similarly hurt defenders while others obtain comparable tools elsewhere. agreements cannot eliminate hidden development or defection. restrictions require specific risks and conduct. nationality and competitive status alone tell us too little.

freedom to leave

rules that make alternatives harder to build or release also weaken our ability to leave. i want intelligence i can run on my own machine. i want to change it, choose who sees my data, and keep using what i've built when a provider changes its mind. i want more than metered access to an API.

i don't want our independence to rest on a company's promise to keep prices fair, policies reasonable, or priorities aligned with ours. i want companies to keep earning our choice to stay.

i want someone i've never heard of to be able to build something better, without asking permission from the companies they might replace.

researched and edited with the assistance of three models (two open-weight and one closed) and a bunch of humans.

I Stopped Letting the Image Model Decide Everything

Logo placement, output dimensions, and mandatory text became deterministic constraints instead of prompts in an AI image generation pipeline.

The first version of my image generation pipeline was mostly what you would expect.

Give the model a source image, a logo, some text, brand context, and instructions describing the final composition. Let it generate the image.

It worked surprisingly well.

Until I needed it to work repeatedly.

That difference became much more obvious once I started using the pipeline behind KeepPost, where the goal isn't to generate one impressive image. The same pipeline has to produce content for different brands, different source photos, different formats, and different campaigns.

At that point, a visually good result is not enough.

The logo has to be where it is supposed to be. The product in the source image shouldn't suddenly change. Required text needs to survive generation. Brand rules need to stay consistent across posts.

And if I run a similar request tomorrow, I don't want the entire composition strategy to change because the model found another interpretation equally reasonable.

I initially tried solving most of this through prompting.

Eventually I stopped.

The problem wasn't that the prompts weren't detailed enough. I was asking a probabilistic model to make decisions that didn't need to be probabilistic in the first place.

The model is not my layout engine

This became the most useful distinction in the pipeline.

There are decisions where I want the model to be creative.

Lighting is a good example. So is adapting a restaurant photo into a more polished scene, extending the background, integrating an object naturally, or deciding how the generated environment should feel.

Those are exactly the things I want from an image model.

But there are other decisions where creativity adds almost no value.

If the brand logo belongs at a specific position and size, I don't need the model to reinterpret that requirement on every generation.

The same applies to things like output dimensions, protected areas, mandatory assets, exact text, and some composition constraints.

So the pipeline gradually moved away from:

Assets + Prompt
      │
      ▼
 Image Model
      │
      ▼
 Final Image

toward something closer to:

The shape the pipeline settled into — deterministic constraints and generative decisions feeding the same image model, gated by validation before anything ships.

The important part isn't the number of stages.

It's deciding who owns each decision.

What I stopped delegating to the model

Logo placement is probably the clearest example.

If the rule for a brand says the logo belongs in a known region with a known relative size, letting the image model decide its placement again adds uncertainty without adding much value.

So that becomes structured configuration instead.

Conceptually, it can be as simple as:

{
  "logo": {
    "x": 0.065,
    "y": 0.055,
    "width": 0.22,
    "alignment": "left"
  }
}

Now the position isn't something the prompt has to persuade the model to remember.

It's a constraint.

The same idea applies to other parts of the system. Once I sorted every decision the pipeline makes into "the model shouldn't be guessing here" versus "this is where I want the model's judgment," it looked like this:

Fixed by configuration, not left to the model:

Left to the model's judgment:

This looks obvious when written down.

It wasn't as obvious when I started building it.

When image models become better at following instructions, the natural reaction is to give them more responsibility. In some parts of the pipeline I ended up doing the opposite.

Better generation made the creative parts better.

It didn't make deterministic requirements any less deterministic.

Some decisions sit in the middle

The interesting cases aren't always purely deterministic or purely generative.

Composition is one of them.

A model can be useful for understanding that the main subject occupies the right side of an image and that placing text there would cover the product.

That's a semantic decision. Hard-coded coordinates alone aren't particularly good at it.

But I also don't necessarily want the model to have unrestricted control over what happens next.

One approach is to let the model propose a composition plan as structured data:

{
  "subject_region": {
    "x": 0.36,
    "y": 0.16,
    "width": 0.52,
    "height": 0.70
  },
  "preferred_text_region": "upper_left",
  "background_extension": true,
  "creative_direction": "fine_dining"
}

That output can be validated before it reaches the generation stage.

This gives me a useful third category: the model proposes, code decides whether the proposal is valid.

I've found this pattern useful beyond image generation as well.

It is often a better boundary than either extreme: encoding every decision manually or giving an agent/model complete control.

Prompting wasn't the right fix for every failure

One of the traps with generative systems is treating every bad output as a prompt problem.

Logo moved? Improve the prompt.

Product changed? Add stronger instructions.

Text came back wrong? Make the requirement more explicit.

Composition changed too much? Add another paragraph describing the layout.

Sometimes that works.

But the prompt also gets longer, different constraints start interacting, and you're still relying on the model to reproduce behavior that your application could enforce directly.

I started asking a different question when something failed:

Does the model actually need to decide this?

If the answer is no, I would rather move that responsibility out of the prompt.

That turned out to be a much more useful debugging question than continuously rewriting instructions.

The source image is another boundary

For KeepPost, the source image often represents something real: a dish, product, venue, bottle, or other customer asset.

That changes how I think about generation.

The source isn't just inspiration.

It's input data that needs to preserve some truth.

If a restaurant sends a photo of a particular dish, generating a more attractive image while changing the dish itself can produce a technically impressive result that is useless for the customer.

So I treat source preservation differently from creative direction.

The model can have significant freedom around the subject while having much less freedom over the subject itself.

This distinction matters because prompts such as "keep the product unchanged" sound precise to us, but they're still instructions interpreted by a generative model.

When preservation really matters, the architecture should reflect that instead of assuming a stronger sentence in the prompt will solve everything.

Brand context is not the same as layout

Another thing I separated over time was brand understanding from deterministic brand rules.

Brand context can describe things like:

• visual mood
• typography preferences
• lighting
• palette
• composition tendencies
• use of negative space
• photography style

Those are useful signals for generation.

But "the logo goes here" is not really brand context.

It's a rule.

Mixing both into one large prompt technically works, but it hides two very different types of information inside natural language.

I prefer to let retrieved brand context influence creative decisions while keeping strict requirements represented explicitly wherever possible.

This makes the system easier to reason about when something goes wrong.

If an image has the wrong mood, I look at the generative context.

If the logo is in the wrong place, I shouldn't need to debug the model's interpretation of the brand guide.

Validation matters more once generation becomes automated

When you're manually generating an image, validation is easy.

You look at it.

If it's bad, you regenerate it.

That doesn't scale particularly well once image generation sits inside a larger automated workflow.

Now the pipeline needs to care about things such as:

• Did generation succeed?
• Is the expected output format correct?
• Are required elements present?
• Were protected assets preserved?
• Does the result satisfy composition constraints?
• Should this output be accepted, repaired, or regenerated?

Not every one of these checks can be perfectly deterministic, especially when you're evaluating visual semantics.

But that doesn't mean there shouldn't be a validation layer.

Generation and acceptance are separate problems.

I think this distinction becomes increasingly important as AI pipelines move from interactive tools into background automation.

The model saying "done" is not the same thing as the application deciding that the result is acceptable.

This is really a control problem

I originally thought most of the engineering work would be about getting better images from the model.

Some of it is.

But once the quality became good enough, another problem became more important: controlling variance.

There are places where variance is valuable.

I want the model to find a better way to light a dish. I want it to integrate an object naturally. I want it to understand the scene rather than follow a fixed Photoshop template.

There are also places where variance is simply another failure mode.

I don't want a creative interpretation of a logo coordinate.

The architecture became easier to reason about when I stopped treating "AI image generation" as one operation and started treating it as a pipeline containing different kinds of decisions.

Some belong to the model.

Some belong to code.

Some can be proposed by the model and verified by code.

And a few still need human judgment.

The same pattern shows up outside image generation

This lesson has changed how I look at other LLM systems too.

An agent doesn't need to decide something simply because it can.

A RAG pipeline doesn't need an LLM for a transformation that deterministic code handles more reliably.

A structured output schema shouldn't be replaced by instructions asking the model to "please always return these fields."

And business rules shouldn't quietly migrate into prompts just because adding another sentence is faster than changing the application.

The useful question isn't:

How much can I make the model do?

It's:

Where does probabilistic reasoning actually improve the system?

That's a very different architecture question.

For my image pipeline, the answer wasn't "everywhere."

The more I worked on it, the more I ended up giving the model freedom exactly where its uncertainty was useful, and removing freedom where the application already knew the correct answer.

The resulting pipeline is less magical than the original one.

That's probably why I trust it more.

Tailcat

"Tailscale without Tailscale, by Tailscale"

Tailcat is a remix of Tailscale open source pieces to act like netcat, but over Tailscale's data plane, without Tailscale's control plane. Tailscale's data plane (magicsock, internally) gives you point-to-point WireGuard®-encrypted tunnels between two machines with DERP as the NAT-hole-punching communication side channel and the ultimate relay-of-last-resort if NAT traversal fails. Instead of using the Tailscale control plane, all tailcat connection metadata is exchanged out of band, however you want.

The tailcat CLI (in cmd/tailcat) is built on the tailcat Go library (importable as github.com/tailscale/tailcat).

Whether you use tailcat as a CLI tool or library, one side runs a tailcat server (listener) and gets back a short connection token. The other side passes that token to tailcat's client side to connect. All traffic between the two is encrypted end-to-end with WireGuard. The initial connection bootstraps through a DERP server, and then magicsock performs NAT traversal to upgrade to a direct peer-to-peer UDP connection when possible (usually!).

You don't need a Tailscale account, root/admin access on the machine (it doesn't alter your machine's routing tables, DNS, etc.). It's just a userspace library and CLI tool.

And it's all open source.

You can use our free rate-limited DERP relays (the default DERP map is https://tailcat.dev/derpmap.json) or you can run your own.

There's also an experimental in-browser web demo (tailcat compiled to WebAssembly) at https://tailscale.github.io/tailcat/ that can send and receive files or text, interoperating with the CLI. Browser traffic is relayed over DERP only, with no direct connections until WebRTC support.

Install

$ go install github.com/tailscale/tailcat/cmd/tailcat@latest

Or with Nix flakes, run it directly or install it:

$ nix run github:tailscale/tailcat
$ nix profile install github:tailscale/tailcat

Usage

Pipe stdin/stdout between two machines

Server starts, printing out its ephemeral address:

$ tailcat
# Selected bootstrap relay region 302, San Francisco
# 🐈 Server listening with new address: tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
(hangs, waiting...)

And then the client can:

$ echo hello | tailcat tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
$

Then the server unblocks:

$ tailcat
# Selected bootstrap relay region 302, San Francisco
# 🐈 Server listening with new address: tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
hello
$

Expose local ports through the tunnel

Or you can serve a local TCP port, forwarded to localhost:

$ tailcat --serve=8080,8443 # or --serve=all
# 🐈 Server listening with new address: tcXXXXXXXXX

And then the client:

$ tailcat tcXXXXXXXXX 8080
GET / HTTP/1.1
Host: foo

HTTP/1.1 200 OK
....

Auth-free SSH server

On Linux and macOS, you can run an SSH server too with no auth. (If you want auth, you can just tailcat --serve=22 and proxy to your system SSH server)

$ tailcat --serve=no-auth-ssh
# 🐈 Server listening with new address: tcXXXXXXXXX

And on the client side:

$ tailcat ssh tcXXXXXXXXX
$ tailcat ssh tcXXXXXXXXX ls -la

Misc commands

Ping to test connectivity; each pong reports whether it arrived via a DERP relay or a direct path. --until-direct keeps pinging (up to --timeout, default 10s) until a direct path works, exiting non-zero if one doesn't:

$ tailcat ping --until-direct <token>
pong in 42.1ms via DERP(sfo)
pong in 1.2ms via 203.0.113.7:41641

Run a command through a SOCKS5 proxy routed over the tunnel:

$ tailcat socks <token> curl http://server.tailcat:8081/

Tokens also work directly as URL hostnames: the SOCKS proxy recognizes and dials them, so the token argument is optional. (Tokens are case-sensitive; this works with curl and most CLI tools, but not with browsers, which lowercase hostnames.)

$ tailcat socks curl http://<token>:8081/

Act as an exit node so the client can reach the server's network:

$ tailcat --serve=exit-node

Parse a connection token and print its contents (the server's WireGuard public key and DERP info) as JSON, without connecting to anything:

$ tailcat parse tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
{
    "ServerPublic": "nodekey:9c8d2e6728da80a1dd37e275a82595b42d9a838610bc53f74a7670d1610f2e34",
    "RegionID": 302
}

Resolve a short token (which references a DERP region by ID, requiring clients to fetch the DERP map) into a longer self-contained one with the DERP server info embedded, letting clients connect more quickly:

$ tailcat resolve tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFpGQEu
tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFygaFhToGjYWhudGMzMDJhLmlwbi5kZXZhNG0yMDguMTExLjM5LjM4YTZzMjYwNzpmNzQwOjA6M2Y6OjcyMA

Parsing that resolved token shows the embedded DERP info:

$ tailcat parse tcomFwWCCcjS5nKNqAod034nWoJZW0LZqDhhC8U_dKdnDRYQ8uNGFygaFhToGjYWhudGMzMDJhLmlwbi5kZXZhNG0yMDguMTExLjM5LjM4YTZzMjYwNzpmNzQwOjA6M2Y6OjcyMA
{
    "ServerPublic": "nodekey:9c8d2e6728da80a1dd37e275a82595b42d9a838610bc53f74a7670d1610f2e34",
    "Region": [
        {
            "Nodes": [
                {
                    "HostName": "tc302a.ipn.dev",
                    "IPv4": "208.111.39.38",
                    "IPv6": "2607:f740:0:3f::720"
                }
            ]
        }
    ]
}

A server can print the long self-contained form directly with the --full-address flag.

Key Management

A server's address (connection token) is derived from its WireGuard key, so the key you use determines who can reach you:

• Ephemeral keys (the default): each server run generates a fresh key in memory and prints an address nobody has ever seen. When the process exits, the key is discarded and the address is dead forever. This is the safe default: sharing that address only ever refers to that one run.
• Saved keys: tailcat genkey generates a key saved to disk so the address stays stable across restarts. The flip side: anyone you've ever shared that address with can connect to any future server using that key, unless you restrict clients with --allow (see tailcat genkey --client).

The CLI says at startup which kind it's using, so you know whether you're starting a fresh single-use server or re-listening on an address you may have shared in the past.

$ tailcat genkey --region=nyc
# prints the token; key saved to ~/.config/tailcat/keys/default.private.json

# later; the key named "default" is used automatically once it exists:
$ tailcat --serve=8080
# 🐈 Server listening with saved key "default": tcXXXXXXXXX

# ... unless you force a one-off ephemeral key:
$ tailcat --serve=8080 --key=new
# 🐈 Server listening with new address: tcXXXXXXXXX

That is, default is a magic key name: once it exists, plain tailcat silently uses it instead of generating an ephemeral key, and the startup line above is what tells you which happened. Use --key=new to get an ephemeral key anyway, --key=<name> to use a different saved key, or tailcat genkey --delete --key=default to remove the saved default key. tailcat genkey --list lists your saved keys.

Tokens can also be published as DNS TXT records and looked up by name; a DNS name works anywhere the CLI takes a token:

# If example.com has a TXT record "tailcat=tc..."
$ tailcat example.com 8080
$ tailcat ssh example.com
$ tailcat ping example.com

Examples

Protected SSH server over DNS

Who needs port forwarding or port knocking? This runs an SSH server reachable from anywhere by name, with no open inbound ports on the server, where WireGuard authenticates the client before the SSH server ever sees a packet.

On the client machine, generate a client identity keypair. It prints the public key, which is all the server needs to know:

client$ tailcat genkey --client
# wrote file to ~/.config/tailcat/keys/client-default.private.json
nodekey:cfb6bfa77a0654d7450947fd6acef17d2cd848da1d30b2540b13dac272ddfd16

On the server, generate a server keypair pinned to its nearest DERP region (see why below), then serve SSH to only that client:

server$ tailcat genkey --fixed-region
# wrote file to ~/.config/tailcat/keys/default.private.json
tcXXXXXXXXX

server$ tailcat --serve=22 --allow=nodekey:cfb6bf...ddfd16
# 🐈 Server listening with saved key "default": tcXXXXXXXXX

Publish the token in DNS as a TXT record:

my-server.example.com. 300 IN TXT "tailcat=tcXXXXXXXXX"

And then the client side is just:

client$ tailcat ssh my-server.example.com

Client modes automatically use the saved client-default key when it exists, so no extra flags are needed to present the allowed identity. Anyone else's handshake is silently ignored: they can't reach the SSH server, or even learn that one is running.

Bring your own DERP relay

Nothing requires Tailscale's relays: run your own DERP server (it needs a hostname with a TLS certificate, which derper can get itself via Let's Encrypt), then generate a server key that uses it by passing its hostname (or several, comma-separated) as the region:

server$ tailcat genkey --region=derp.example.com
tcomFwWCCAIsKOqPUux6ClG2RM4A_vOq4VBzGgHGGjq9OsJuFKSWFygaFhToGhYWhwZGVycC5leGFtcGxlLmNvbQ

server$ tailcat --serve=22

Go library

A minimal server that answers any TCP port through the tunnel and prints its token.

package main

import (
	"fmt"
	"log"
	"net"

	"github.com/tailscale/tailcat"
)

func main() {
	s := &tailcat.Server{
		OnTCP: func(port uint16) func(net.Conn) {
			return func(c net.Conn) {
				fmt.Fprintf(c, "hello from port %v\n", port)
				c.Close()
			}
		},
	}
	if err := s.Start(); err != nil {
		log.Fatal(err)
	}
	fmt.Println(s.ConnBlob())
	select {}
}

How it works

Connection tokens

A Tailcat server is identified by a connection token (called a ConnBlob internally). It looks like tcXYZ... and is a "tc" prefix followed by base64-encoded CBOR containing the server's WireGuard public key and DERP info.

Network stack

• WireGuard -- a userspace WireGuard implementation for encrypting all tunnel traffic.
• magicsock -- Tailscale's transport layer that multiplexes traffic over direct UDP and DERP relays.
• Netstack (gVisor) -- a userspace TCP/IP stack that terminates TCP connections inside the process.
• DERP relay -- Tailscale's encrypted relay protocol, used as a rendezvous channel and as a fallback data path when direct connectivity isn't possible.

Connection flow

1. Server starts. It generates (or loads) a WireGuard keypair, connects to a DERP relay, and prints its connection token to stderr.
2. Client parses the token to learn the server's public key and DERP region.
3. Discovery handshake. The client sends a "Meow" ping message to the server through the DERP relay.
4. WireGuard tunnel. With both sides configured as WireGuard peers, the standard WireGuard handshake proceeds.
5. NAT traversal. In parallel, each side advertises its UDP endpoints to the other in disco messages over DERP, attempting UDP hole-punching.
6. Data transfer. The client dials a TCP port on the server through the tunnel.

Stability

Tailcat is free to use, but it comes with no API or CLI stability promises: the Go API, the CLI flags and output, and the wire format may all change. The public rate-limited Tailcat DERP relays have no uptime SLAs or throughput targets.

History

Tailcat began life in September 2023. It was open sourced August 2026 at the TailscaleUp conference.

Kubernetes v1.37: Memory QoS Graduates to Beta

Memory QoS has graduated to Beta in Kubernetes v1.37 and is now enabled by default. On Linux nodes running cgroup v2, the feature uses the memory controller to give the kernel better guidance on how to treat container memory. It was first introduced as Alpha in v1.22, and expanded in v1.36 with tiered memory reservation.

This post covers what changed in v1.37, what the Beta promotion means for cluster operators, and how to configure the feature.

What changed in v1.37

Memory QoS is Beta and enabled by default

The MemoryQoS feature gate is now Beta in v1.37. This means every v1.37 kubelet has the feature gate turned on without any configuration change. Turning on the feature by default is safe because the default kubelet configuration does not enable memory throttling or memory reservation. No memory.high, memory.min, or memory.low values are written to cgroups unless you explicitly configure them.

You can opt into specific behaviors through kubelet configuration fields:

1. Set memoryThrottlingFactor (for example, 0.9) to enable memory.high throttling on Burstable and BestEffort containers. The default is null, which means no throttling.
2. Set memoryReservationPolicy to TieredReservation to enable tiered memory protection via memory.min and memory.low. The default is None, which means no memory reservation.

Default memoryThrottlingFactor changed to null

In earlier Alpha releases, memoryThrottlingFactor defaulted to 0.9, which meant enabling the feature gate caused the kubelet to set memory.high on containers. In v1.37, the default is null, so the kubelet does not set memory.high unless you configure a value.

This change was made because, with the feature gate now on by default, an automatic memory.high could throttle workloads that were previously running without throttling. Making it null ensures that upgrading to v1.37 does not change runtime behavior for existing clusters.

If your kubelet configuration file already contains an explicit memoryThrottlingFactor value, that value is preserved during the upgrade and throttling continues to work as before. If your configuration file does not include memoryThrottlingFactor, the kubelet uses the new null default and stops setting memory.high. To keep throttling in that case, add memoryThrottlingFactor explicitly:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memoryThrottlingFactor: 0.9

How to configure MemoryQoS in v1.37

Enable memory throttling only

Set memoryThrottlingFactor to a value between 0 and 1. The kubelet uses this factor to calculate memory.high for Burstable and BestEffort containers.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memoryThrottlingFactor: 0.9

Enable memory throttling and tiered reservation

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memoryThrottlingFactor: 0.9
memoryReservationPolicy: TieredReservation

Enable tiered reservation without throttling

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memoryReservationPolicy: TieredReservation

Disable Memory QoS entirely

To disable the feature after upgrading, set the feature gate to false and ensure a compatible kubelet configuration. The kubelet rejects the configuration if memoryThrottlingFactor is set to anything other than the former default of 0.9, or if memoryReservationPolicy is TieredReservation, so remove or adjust those fields if you set them.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
featureGates:
  MemoryQoS: false

When the feature gate is off, or memoryReservationPolicy is not TieredReservation, the kubelet resets stale protection at startup on cgroup v2 nodes: memory.min=0 and memory.low=0 on the root kubepods cgroup, and memory.low=0 on the Burstable QoS cgroup. For containers, stale memory.high values are reset to max on reconciliation paths such as restart or resize.

Known limitation: memory reservation is node-wide

memoryReservationPolicy applies to every pod on the node. With TieredReservation, every Guaranteed pod gets memory.min and every Burstable pod gets memory.low; there is no way to opt individual pods in or out. A node that mixes workloads needing hard reservation with workloads that should stay reclaimable has to choose one policy for all of them.

Hard reservation also covers everything charged to the container's cgroup, including page cache, so a pod that reads large files can hold memory the kernel would otherwise reclaim to serve its neighbors.

SIG Node is tracking both in kubernetes/kubernetes#140246. If this affects you, that issue is the best place to describe your workload.

What to expect next

The next milestone for Memory QoS is graduation to GA. Feedback from Beta users will shape any remaining adjustments before that step. If you run into issues, please file bugs at kubernetes/kubernetes.

How can I learn more?

• KEP-2570: Memory QoS
• Pod Quality of Service Classes
• Memory QoS with cgroup v2
• Managing Resources for Containers
• Kubernetes cgroups v2 support
• Linux kernel cgroups v2 documentation

Getting involved

This feature is driven by SIG Node. If you are interested in contributing or have feedback, you can reach out through:

• Slack: #sig-node
• Mailing list
• SIG Node meetings

What is Open Code Review?

Open Code Review is an AI-powered code review CLI tool. It originated as Alibaba Group's internal official AI code review assistant — over the past two years, it has served tens of thousands of developers and identified millions of code defects. After thorough validation at massive scale, we incubated it into an open source project for the community. Simply configure a model endpoint to get started.

It reads Git diffs, sends changed files to a configurable LLM via an agent with tool-use capabilities, and generates structured review comments with line-level precision. The agent can read full file contents, search the codebase, inspect other changed files for context, and produce deep reviews — not just surface-level diff feedback.

Why Open Code Review?

The Problem with General-Purpose Agents

If you've used general-purpose agents like Claude Code with Skills for code review, you've likely encountered these pain points:

• Incomplete coverage — On larger changesets, agents tend to "cut corners," selectively reviewing only some files and missing others.
• Position drift — Reported issues frequently don't match the actual code location, with line numbers or file references drifting off target.
• Unstable quality — Natural-language-driven Skills are hard to debug, and review quality fluctuates significantly with minor prompt variations.

The root cause: a purely language-driven architecture lacks hard constraints on the review process.

Core Design: Deterministic Engineering × Agent Hybrid

Open Code Review's core philosophy is to combine deterministic engineering with an agent, each handling what it does best.

Deterministic Engineering — Hard Constraints

For review steps that must not go wrong, engineering logic — not the language model — guarantees correctness:

• Precise file selection — Determines exactly which files need review and which should be filtered, ensuring no important change is missed.
• Smart file bundling — Groups related files into a single review unit (e.g., message_en.properties and message_zh.properties are bundled together). Each bundle runs as a sub-agent with isolated context — a divide-and-conquer strategy that stays stable on very large changesets and naturally supports concurrent review.
• Fine-grained rule matching — Matches review rules to each file's characteristics, keeping the model's attention sharply focused and eliminating information noise at the source. Compared to purely language-driven rule guidance, template-engine-based rule matching is more stable and predictable.
• External positioning and reflection modules — Independent comment-positioning and comment-reflection modules systematically improve both the location accuracy and content accuracy of AI feedback.

Agent — Dynamic Decision-Making

The agent's strengths are concentrated where they matter most — dynamic decisions and dynamic context retrieval:

• Scenario-tuned prompts — Prompt templates deeply optimized for code review, improving effectiveness while reducing token consumption.
• Scenario-tuned toolset — Distilled from deep analysis of tool-call traces in large-scale production data — including call frequency distributions, per-tool repetition rates, and the impact of new tools on the overall call chain — resulting in a purpose-built toolset that is more stable and predictable for code review than a generic agent toolkit.

How to Use

CLI

Install

Via NPM (Recommended)

npm install -g @alibaba-group/open-code-review

After installation, the ocr command is available globally.

From GitHub Release

Download the latest binary from GitHub Releases:

# macOS (Apple Silicon)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-darwin-arm64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# macOS (Intel)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-darwin-amd64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Linux (x86_64)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-linux-amd64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Linux (ARM64)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-linux-arm64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Windows (x86_64) — move ocr.exe to a directory in your PATH
curl -Lo ocr.exe https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-windows-amd64.exe

# Windows (ARM64) — move ocr.exe to a directory in your PATH
curl -Lo ocr.exe https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-windows-arm64.exe

From Source

git clone https://github.com/alibaba/open-code-review.git
cd open-code-review
make build
sudo cp dist/opencodereview /usr/local/bin/ocr

Quick Start

1. Configure LLM

You must configure an LLM before reviewing code.

# Option A: Interactive config
ocr config set llm.url https://api.anthropic.com/v1/messages
ocr config set llm.auth_token your-api-key-here
ocr config set llm.model claude-opus-4-6
ocr config set llm.use_anthropic true

# Option B: Environment variables (highest priority)
export OCR_LLM_URL=https://api.anthropic.com/v1/messages
export OCR_LLM_TOKEN=your-api-key-here
export OCR_LLM_MODEL=claude-opus-4-6
export OCR_USE_ANTHROPIC=true

Config is stored in ~/.opencodereview/config.json.

It is also compatible with Claude Code environment variables (ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL) and parses ~/.zshrc / ~/.bashrc for those exports.

Note for CC-Switch Users: If you are using CC-Switch with routing service enabled, you can point llm.url to the CC-Switch proxy address without additional configuration:

• For Claude provider: set llm.url to http://127.0.0.1:15721
• For CodeX provider: set llm.url to http://127.0.0.1:15721/v1
• Set llm.model according to your provider settings
• llm.auth_token can be any value
• extra_body settings still apply

2. Test Connectivity

ocr llm test

3. Review

cd your-project

# Workspace mode — review all staged, unstaged, and untracked changes
ocr review

# Branch range — compare two refs
ocr review --from main --to feature-branch

# Single commit
ocr review --commit abc123

Integrate with Coding Agents

OCR can be seamlessly integrated into AI coding agents as a slash command, enabling code review directly within your agent workflow.

Option 1: Install as a Skill

Use npx to install the OCR skill into your project:

npx skills add alibaba/open-code-review --skill open-code-review

This installs the open-code-review skill, which teaches your coding agent how to invoke ocr for code review, classify issues by priority, and optionally apply fixes.

Option 2: Install as a Claude Code Plugin

For Claude Code, install the command plugin through the following command in Claude Code:

/plugin marketplace add alibaba/open-code-review
/plugin install open-code-review@open-code-review

This registers the /open-code-review:review slash command, which runs OCR and automatically filters and fixes issues.

Option 3: Copy the Command File Directly

For a quick setup without any package manager, simply copy the command file to use the /open-code-review slash command in Claude Code.

Project-level (shared with team via git):

mkdir -p .claude/commands
curl -o .claude/commands/open-code-review.md \
  https://raw.githubusercontent.com/alibaba/open-code-review/main/plugins/open-code-review/commands/review.md

User-level (personal global use across all projects):

mkdir -p ~/.claude/commands
curl -o ~/.claude/commands/open-code-review.md \
  https://raw.githubusercontent.com/alibaba/open-code-review/main/plugins/open-code-review/commands/review.md

Prerequisite: All integration methods require the ocr CLI to be installed and an LLM configured.

CI/CD Integration

OCR can be integrated into CI/CD pipelines to automate code review on Merge Requests / Pull Requests.

The core command for CI integration:

ocr review \
  --from "origin/main" \
  --to "origin/feature-branch" \
  --format json

The --format json flag outputs machine-readable results suitable for parsing in CI scripts.

Commands

ocr review Flags

Examples

# Preview which files will be reviewed (no LLM calls)
ocr review --preview
ocr review -c abc123 -p

# Review workspace changes with default settings
ocr review

# Review branch diff with higher concurrency
ocr review --from main --to my-feature --concurrency 4

# Review a specific commit with verbose JSON output
ocr review --commit abc123 --format json --audience agent

# Use custom review rules
ocr review --rule /path/to/my-rules.json

# Preview which rule applies to a file
ocr rules check src/main/java/com/example/Foo.java
ocr rules check --rule custom.json src/main/resources/mapper/UserMapper.xml

# View review session history in browser
ocr viewer
ocr viewer --addr :3000

Viewer security

The viewer serves session JSONL contents over HTTP. It enforces a Host-header allowlist on every request: loopback names (localhost, 127.0.0.0/8, ::1) and the concrete bind host are always allowed. Wildcard binds and other non-loopback Hostnames must be added via the OCR_VIEWER_ALLOWED_HOSTS environment variable (comma-separated):

OCR_VIEWER_ALLOWED_HOSTS=review.internal,ocr.lan ocr viewer --addr :3000

This blocks DNS-rebinding attacks against the local viewer.

Review Rules

OCR resolves review rules using a four-layer priority chain. Each layer uses first-match-wins: if a file path matches a pattern, that rule is used; otherwise it falls through to the next layer.

Rule File Format

Layers 1–3 share the same JSON format:

{
  "rules": [
    {
      "path": "force-api/**/*.java",
      "rule": "All new methods must validate required parameters for null values"
    },
    {
      "path": "**/*mapper*.xml",
      "rule": "Check SQL for injection risks, parameter errors, and missing closing tags"
    }
  ]
}

• path supports ** recursive matching and {java,kt} brace expansion.
• Within each layer, rules are evaluated in declaration order — the first match wins.
• If a rule file does not exist, it is silently skipped.

Configuration Reference

Config file: ~/.opencodereview/config.json

Environment variables take precedence over the config file.

Telemetry

OpenTelemetry integration for observability (spans, metrics). Disabled by default.

ocr config set telemetry.enabled true
ocr config set telemetry.exporter otlp
ocr config set telemetry.otlp_endpoint localhost:4317

Set telemetry.content_logging to include LLM prompts and responses in exported data.

Full article content is not available for inline reading.

Read the original article →

“A sales guy writing code” used to be the lead-up to a joke. But now no one’s laughing.

Designers used to sit meekly waiting for the high priests of code to make their designs real. Now they don’t wait for anyone.

The tools behind this shift (Cursor, Claude, Lovable, Replit) were unknown names a few years ago; now they count their users in the millions, and a meaningful share of those users have never written a line of code by hand… and never will.

A lot of what gets built is toys. That’s fine. Playing with toys is how people learn, and basic hosting is all a toy needs.

But they aren’t all toys. A lot of the products vibe-coded into existence have real potential, and their creators have real ambition and real skill. They don’t understand what’s happening under the hood, but they understand what the market needs and what might fix it. If they could ever get to production. Which most won’t.

What they will do is go live on some sort of infrastructure, with some sort of database, and a few rudimentary nods to good architecture: one region, one credential that can do everything, a schema nobody reviewed. By and large, vibe-coded apps bypass the last 20 years of cloud native best practices and suffer the consequences.

Apps get built at 100 miles per hour, get deployed at 85, and screech to a halt at anything resembling production. I believe closing this velocity gap is the defining infrastructure problem of the next few years, and the engineers working in cloud native infrastructure can either help or hinder.

Starting was never the hard part

There is an old joke that the first 90 percent of a project takes 90 percent of the time, and the last 10 percent takes the other 90 percent. Software has always worked this way. Everyone knows how to start an app; the rare skill is finishing one and getting it into the world.

AI did not change that ratio. It made it more extreme. When starting an app costs real money and real engineers, the projects that died before production at least died as line items someone had approved. Now starting is nearly free, so vastly more gets started, and the fraction that finishes has fallen accordingly. If 80% of apps never shipped before, today’s number is probably a lot closer to 99%. Getting to production stops being 80 percent of the work and becomes nearly all of it.

But what exactly do we mean by “production”? Because the way an SRE uses the term is different from the way an AI agent uses it.

To an SRE, production is a set of testable claims. What is the p99 latency under peak load? Has the failover actually been rehearsed, or does it exist only in a diagram? What is the blast radius of a bad deploy, and how fast does it roll back? Who touched what, and when? To an agent, production is a URL that returns 200.

Agents gonna agent

Watch an AI agent stand up infrastructure and you will see the same terms appear again and again: Supabase, serverless functions, a managed one-click backend. Nothing is wrong with these tools. They are well built, and they are deliberately simple, which is exactly why agents choose them: an agent can hold the entire mental model in a few thousand tokens and produce a working demo without asking anyone for anything.

The agent is not choosing that stack because it evaluated the alternatives and found it superior. It is choosing the stack that is most convenient for the agent: the one that demands the least context to operate. Convenience for the agent is a heuristic for good infrastructure, and like most heuristics, it works until it doesn’t.

We already know what “until it doesn’t” looks like. In 2025, researchers found more than 170 Lovable-built applications had shipped with row-level security left off their databases, exposing user data to anyone who asked for it (CVE-2025-48757). But the apps worked! The demos were great! The agent did what the user told it to.

That same summer, Replit’s coding agent deleted a production database during a code freeze, then generated fake records to paper over it. And this August, OpenAI published its report on the Hugging Face breach about how its own agents had learned during training to pursue solutions by any means necessary rather than admit a task was impossible.

Agents gonna agent. They are relentlessly following their mandate, which is usually to produce the visible result the person asked for. Security posture, failover behavior, and audit trails are invisible in a demo, so why would the agent bother? So are liveness probes, resource limits, and the difference between a secret in a vault and a secret pasted into an environment variable. Bothering with them would mean bothering the user, and probably confusing them. Today’s AI defaults are optimized for prototypes and demos, not production.

Does cloud native get replaced?

For those of us in the cloud native community this is frustrating. Because the definition of “finished” already exists. It has been built in public for two decades, one project at a time. Kubernetes for orchestration. Prometheus and OpenTelemetry for knowing what your system is actually doing. Istio for service-to-service security, OPA for policy, and the CNCF’s graduated-project process for separating the proven from the promising. This body of practice encodes thousands of hard-learned lessons about what happens to software after the demo.

But AI often ignores it. Not because the practice is wrong, but because it is expensive. Expensive in context, expensive in tokens, expensive in the number of steps between “generate” and “visible result.” Expediency wins.

So the great irony of building infrastructure today is that the default “AI-native cloud infrastructure” is not cloud native infrastructure. It is a cheaper, quicker, less secure, less performant, dumbed-down version of it, cobbled together to give vibe coders a quick dopamine hit. And the parts that get dropped are exactly the parts a demo never exercises. Things like mutual TLS between services, least-privilege identity, autoscaling tuned against real load, and telemetry.

But surely as this space evolves, the definition of AI-native infrastructure must be no less secure, no less scalable, and no less observable than what a good platform team builds by hand. AI-generated infrastructure that drops the standards a human team would have held is a regression, not progress. And the way to clear the bar is not to abandon the cloud native stack for whatever an agent finds most convenient. It is to make two decades of accumulated practice as cheap for an agent to use as the shortcuts are: declarative interfaces an agent can operate deterministically, policy engines that reject a bad manifest before it ships, and the same reconciliation loop that keeps a human honest keeping the agent honest too.

Vibe coders are builders too

The other thing standing between where we are now and AI-native infrastructure that is also cloud native is not technical at all.

The people building this new flood of software are, increasingly, not developers. Gartner was predicting years ago that citizen developers would outnumber professional ones four to one, and the AI tools have blown past every low-code forecast since.

This is actually great news. Really, it is. Let me explain.

For my whole career, the people who understood the problem best (the operations manager, the salesperson, the designer) had to pass their knowledge through filters: requirements documents, tickets, a game of telephone that stripped out half the insight before an engineer ever saw it.

Now, those filters are gone. The person with the domain experience builds the workflow themselves, and the designer ships the interaction they intended instead of an approximation of it. This has produced a lot of AI slop, but it’s also produced some software that’s closer to the user, closer to solving the problem.

If we bring the same time, skill, and attention to detail to AI-built apps that we brought to hand-built ones, vibe-coded software should end up better for its users, because it is finally shaped by the people who understand them and work with them. Sometimes it’s built by the users themselves.

But our workflows have not caught up with this. Everything about the path to production (Git, YAML, CI gates, review queues) was designed by engineers for engineers.

We used to be able to treat non-developers like children. We kept them away from anything sharp, patted their heads when they had an idea, handed them a finished product, and complained that they didn’t use it.

But non-developers are building the software now, and we’ve got to start treating them like adults… like equal participants in the development process. Getting through this period requires a collaborative framework in which vibe coders, professional developers, and SREs work as one community of builders. The domain expert supplies the intent, the platform supplies guardrails instead of gates, and operational discipline lives in the paved road rather than at a tollbooth.

BYOD has become BYOApp

If this feels threatening, I understand. It felt threatening the last time, too.

Fifteen years ago, employees started carrying their own iPhones and laptops into corporate networks, and IT departments were alarmed. They had good reasons. They would be the ones on the hook if corporate data walked out the door with a user’s laptop.

The problem was, the teams that responded by banning everything got bypassed. People used their devices anyway, invisibly, which was far worse.

The teams that adapted, with device management, sensible policies, and clear boundaries, ended up somewhere better than the locked-down world they lost. They got a more dynamic workplace where people took real responsibility for their own technology.

AI and vibe coding are at the same stage BYOD was around 2010. It’s young, messy, and growing regardless of anyone’s permission.

The cloud native community gets the same choice IT got. We can hold the new builders at arm’s length and watch a generation of software default to whatever agents find convenient. Or we can do what this community has always done best: encode hard-won operational judgment into open, shared infrastructure, and this time make it legible to agents and non-developers too.

Meta launched Muse, a consumer AI agent built on its Muse Spark model that can navigate websites, fill forms, send emails, book travel, and make payments. The browser sub-agent reads an accessibility-tree snapshot of pages rather than the raw DOM, so semantic structure and accessibility may grow increasingly important for web designers. Reuters reported that internal testing found reliability problems, including an instance where an agent exposed personal iCloud photos, though Meta says added security work delayed the launch.

Accessibility programs often fail not from a lack of standards or budget, but because organizations chase compliance alone while culture and capability go unbuilt. A three C's model, borrowed from the fire triangle, argues that compliance, culture, and capability must all stay lit or the whole effort burns out.

The Double Diamond and Design Thinking priced building as the risky, expensive step, but AI has cut prototyping costs by orders of magnitude, collapsing that premise. As a result, front-end research and ideation shrink to a one-page brief, while generation widens and the judgment budget—evaluating and choosing among many cheap options—becomes the real bottleneck. Hardware, fashion, and regulated industries such as medical devices and payments remain exceptions where building stays costly and irreversible, so front-loading still applies there.

An open-source drag-and-drop email editor SDK.

Custom blocks, full theming, merge tags, and display conditions — all built into the open-source SDK. Drop into any app with one init() call.

Works in React, Svelte, Angular, Vue, and vanilla JS — one init() call, zero runtime dependencies.

• TypeScript-first
• Built on MJML
• FSL-1.1-MIT (auto-MIT)
• Framework-neutral
• Shadow DOM
• No license key
• 312 kB initial · 253 kB lazy

Describe the email. Or the integration.

One open-source Agent Skill, running entirely inside the coding agent you already use. Ask it for an email and it writes, imports, validates and previews a complete template. Ask it about the SDK and it mounts the editor in your app, scaffolds the code, or works out why an integration misbehaves. No backend, no API key, nothing sent to us.

“A product-launch email for an outdoors brand — hero image, a short intro, and a Shop now button in forest green.”

“Mount the editor in our Next.js admin, point save and load at our own API, and match it to our dark theme.”

One command. It detects the agents you have installed and writes the skill into the folder each one reads — nothing is added to your project.

Prefer managed AI? Templatical Cloud adds in-editor AI chat and a hosted MCP server.

Everything you need, nothing paywalled

Building from scratch takes a quarter of engineering. SaaS builders charge per seat and gate the features that matter. Templatical includes them — open source, self-hosted, ready to embed.

Build it yourself

Wire up an open-source web-builder framework with an MJML plugin, or compose React Email components yourself.

You’ll build yourself

• The editor UX itself — drag-and-drop, nesting, column splits, undo/redo
• Block-level merge tag scoping
• Display conditions with live preview
• Custom block SDK with API-backed data sources
• Editor-to-media-library glue — picker, image-block hook, replace flow
• A saved-block library — pick, name, preview, insert at position
• A test-send dialog — recipient picker, preview, sending and error states
• Preview resolution — real data in place of merge tags, without touching stored content
• Dark-mode preview parity with sent email
• Theme tokens for brand consistency
• MJML output you can render anywhere
• Email-client compatibility — Outlook 2007–365, Gmail clipping, Apple Mail
• Accessibility — keyboard nav, ARIA, screen readers
• Style isolation — host CSS resets and design-system globals bleed into the editor
• Ongoing maintenance — every email-client tweak is a regression risk

Recommended — Templatical

A purpose-built, open-source email editor SDK. Own the experience, ship in days not quarters, never hit a paywall on core features.

Included in the open-source SDK

• Custom blocks with API-backed data sources
• Block-level merge tags with scoping
• Display conditions for dynamic content
• Pluggable media library — bring your own storage (S3, Cloudinary, your CMS)
• Saved blocks — users save block groups and reuse them, backed by your storage
• Saving and loading against your own storage — autosave, save status, Cmd/Ctrl+S
• Version history — browse, preview and restore, backed by your storage
• Threaded review comments anchored to blocks, on your storage and identities
• Test sends from the editor, delivered by your own ESP and your domain
• Previews resolved by your backend — real data, logic branches evaluated
• Full theming via design tokens, dark mode included
• White-label by default — your UI, your brand
• MJML output — render in browser, on your server, anywhere
• Built-in WCAG accessibility linting with auto-fixes
• Style-isolated — Shadow DOM mount keeps host CSS out
• No license key, no activation call — nothing we can switch off remotely

Coming with Templatical Cloud

• AI rewrite, AI chat, MCP integration
• Real-time collaboration with block-level locking
• Multi-tenancy and API access

SaaS builders

Drop in a hosted editor component pointing at a vendor’s rendering API.

Paywalled or restricted

• Custom blocks gated to higher paid tiers
• Display conditions gated to higher paid tiers
• Custom theming and design tokens on higher tiers
• White-label and brand removal on enterprise tiers
• Custom merge-tag syntax (Liquid, Handlebars) paywalled
• Template and block defaults gated to paid tiers
• Pluggable media library — locked to vendor’s storage
• Reusable saved blocks gated to paid tiers — even self-hosted
• Per-seat or per-end-user pricing scales with you
• Closed source — cannot audit, fork, or extend
• Output coupled to the vendor’s render API
• License key validated against the vendor’s servers — features enabled remotely by plan
• No CSS isolation — host styles can cascade into the embedded editor

Nothing to activate. Nothing to revoke.

No license key, no client ID, no activation call. The open-source SDK never asks a server which features you are allowed to use — install it once and it keeps working, unchanged, including if this project disappears tomorrow.

• No license key or client ID anywhere in the API
• No entitlement check — features are never enabled or revoked remotely
• No telemetry, no analytics, no usage reporting
• Your templates stay in your app until you send them to your own backend

Ship an email editor this week

One npm install. One init() call. JSON in, MJML out — render anywhere, free.

import { init } from '@templatical/editor'
import '@templatical/editor/style.css'

// No API key, no client ID — this is the whole integration
const editor = await init({
  container: '#editor',
  onChange(content) {
    // JSON content — store, version, send anywhere
    console.log(content)
  },
})

const mjml = await editor.toMjml()

Problem

Design systems are scattered across Figma, CSS, code, and documentation. There is rarely a single structured representation of how the system actually works. Humans can connect the pieces by reading files and understanding the context. Agents can’t — they see fragments and have to infer the system, often reinventing tokens, components, and patterns that already exist.

This makes agents unreliable: when the system isn’t available as structured context, they fill the gaps themselves and hallucinate.

Solution

DeLibra turns a design system into a libra: a structured library that humans read as documentation and agents use as data and context. A dedicated skill lets an agent create and work with a libra from existing sources — Figma, production CSS, or a codebase.

Creating a libra

A libra starts from whatever already exists — a Figma file, production CSS, a repository, or an empty scaffold. Guided by the DeLibra knowledge, an agent extracts primitives into tokens and groups recurring patterns into components.

The result isn’t a screenshot or a static document, but a structured system, addressable element by element.

From there the libra is a living document. Add a component, adjust a token, and every specimen that references it updates with it.

What a libra holds

• Tokens — the primitives, named and grouped, with every value traceable to where it came from.
• Components — real specimens, not pictures: each one rendered from the tokens it consumes.
• Light and dark themes — both themes carried on the same specimen, so drift between them is visible instead of discovered in production.
• Mobile and desktop views — viewport-aware previews sitting beside the desktop specimen.

The code layer

Every element in a libra is inspectable as code. Open any component, token, or state and read the Html/CSS behind it — then copy it, export it, or generate the files directly.

The comparison view is where it earns its keep: drop in external CSS — a live site, a legacy stylesheet, another team’s build — and DeLibra diffs it against the libra. Gaps come back as a list: what drifted, what was never tokenized, what to fix first. The audit stops being an afternoon of eyeballing and becomes a diff.

Outcome

The libra provides the data and context. A dedicated skill provides the instructions for agents to create, inspect, and extend it — giving agents a concrete system to follow instead of having to infer one.

Try it live

https://github.com/raphael-batte/delibra

Figma's AI agent now builds fully coded plugins from a plain-language description, handling tasks like bulk-restyling icons, generating type ramps, and flagging broken prototype flows.

Key points

• Google’s AI contribution pilot gives participating publishers a monthly earnings figure without a detailed explanation of how it was calculated.
• My site charges one cent per page through x402. Five testnet payments settled on 15 September, including one made through a Claude Code hook during a task.
• Paying for content use may reflect its value more closely, but a website owner can independently verify a paid fetch.
• This is a working experiment using my own agents and test tokens. Ordinary search crawlers aren’t currently arriving with wallets to pay for access.

Google is paying some publishers when their content helps generate its AI answers.

Barry Schwartz covered it at Search Engine Roundtable on 14 September, following Digiday’s report that morning. Google confirmed the programme to Digiday. It’s called the AI contribution pilot.

Publishers who accept the terms get an earnings figure in Search Console each month. What they don’t get is a breakdown of how Google calculated it.

Google pays when it decides a page contributed significantly to an answer while it was being generated in AI Overviews, AI Mode or the Gemini app. If the page only gets linked after the answer has been written, it doesn’t qualify.

I’m not in the pilot. According to Digiday, Google has approached at least dozens of publishers, with smaller publishers showing more interest than the larger ones. One executive familiar with it described it as a black box. Barry had actually found the help page back in April, but it wasn’t clear then what the feature did.

I’ve been testing a different approach on my own site since August.

My page sets a price before an agent reads it. If the agent wants access, it pays.

This morning, 15 September, five payments went through. Each one settled on a blockchain and has a public transaction hash. One came from Claude Code, running on my machine and using a wallet I’d set up for it.

To be clear, all the payments so far have come from my own agents. No search crawler is turning up and paying me. The payments also use testnet USDC, which means the tokens have no real monetary value.

But the payment process works. An agent requests a page, checks the price, pays and receives the content. Cloudflare is building products around that same idea, and I wanted to understand how it works before it becomes another setting in a dashboard.

What that Cloudflare wallet handle is for

On 4 August, during Agents Week, Cloudflare announced Wallets and opened handle reservations.

Claiming a handle is free. I suspect plenty of people grabbed theirs the way we grab usernames: quickly, before someone else gets there, with a vague intention of reading the details later.

If you did that, this is the kind of thing those wallets are being built for.

Cloudflare’s design has two parts:

• Account Wallets belong to people. You add funds, set spending limits and allocate money to your agents.
• Virtual Wallets are for the agents. They spend within the permissions and limits you’ve set.

Your handle gives that account a recognisable identity.

As of 15 September, though, reserving a handle gets you a page displaying the name and a notification when Wallets becomes available. Cloudflare’s documentation explicitly says a reserved handle can’t yet send, receive or hold funds.

The other part is the Monetization Gateway, which will let sites charge agents for access to resources. Cloudflare announced it on 1 July, and access is still through the waitlist linked from that announcement.

So the products are announced, but you can’t yet use a reserved wallet handle to run this whole process.

My demo shows what that process looks like today using x402. The site charges one cent per page, my agent pays, and spending limits control what it can buy. It’s the same general arrangement Cloudflare describes for Virtual Wallets.

Why charge crawlers at all?

The arrangement between search engines and websites used to be fairly straightforward. Search engines crawled your content, showed it in their results and sent visitors back.

Publishers got traffic. Search engines got something useful to show their users.

AI answers have changed that arrangement. An agent can read your page, use the information in its response and answer the user without sending them to your site.

I’ve spent months looking at what these systems actually fetch. There’s plenty of crawling. The traffic coming back often doesn’t reflect it.

Publishers have responded in different ways, partly depending on how much negotiating power they have.

Smaller sites often block AI crawlers. I understand why. If someone is taking your content and you’re getting little in return, blocking them is a reasonable response. But it comes with a trade-off: restricting access can also reduce your visibility in training datasets and AI answers.

Large publishers can negotiate licensing deals. News Corp, the Financial Times and Reddit all have agreements with AI companies. According to Press Gazette’s reporting on 4 September, People Inc’s chief executive told investors that blocking crawlers through Cloudflare had helped bring AI companies to the negotiating table.

Google’s pilot offers another approach: paying publishers according to the value their content contributes to AI answers.

In that same Press Gazette piece, Cloudflare said it was moving its own default approach from pay per crawl towards pay per use, with pilots involving Ceramic.ai and You.com. Its reasoning was that fetching a page doesn’t prove the content was actually used.

I think that distinction matters.

Using a page to answer a question is a stronger indication of value than simply downloading it. But it’s also something the website owner can’t independently observe. Once my page has left my server, I can’t see whether it influenced an answer.

With pay per use, I’m relying on the platform to tell me what happened and what it was worth. With pay per crawl, I can check the request and the payment myself.

Google’s eligibility rules make this particularly clear. As Search Engine Land explains, contributing during answer generation can earn a payment. Being linked afterwards doesn’t. The publisher sees the total, without the calculation behind it.

Each approach leaves something unresolved.

Blocking earns nothing directly. Individual licensing agreements don’t scale to millions of small websites. Revenue-sharing arrangements leave the platform deciding how much to pay.

Putting a price into the request gives a site a standard way to sell access. An agent can accept that price and receive the content, with a receipt for the transaction. There’s no need to negotiate a separate commercial agreement with every buyer.

That’s the pay-per-crawl model I wanted to test.

Cloudflare already has a product called Pay Per Crawl, but it’s still in closed beta. On the paying side, bot operators need verification through Web Bot Auth, Stripe onboarding and programme approval.

My personal agent can’t simply join that programme.

So I built a version using the open x402 protocol, which Cloudflare is also using for its upcoming Monetization Gateway. My demo makes actual x402 payments independently of Cloudflare’s closed Pay Per Crawl programme.

How x402 works

HTTP has included a status code called 402 Payment Required since 1997. The specification reserved it for future use.

Twenty-nine years later, x402 gives it a practical purpose.

In my setup, the exchange has four steps:

1. The agent requests a page. The server responds with 402 Payment Required, including a PAYMENT-REQUIRED header that describes the price and payment requirements.
2. The agent signs an authorisation to transfer the exact amount in USDC. It then retries the request with a PAYMENT-SIGNATURE header.
3. A payment facilitator verifies the signature and settles the transaction on the blockchain. My demo uses Coinbase’s facilitator at x402.org.
4. The server returns the content, along with a PAYMENT-RESPONSE header containing the settlement receipt.

Here’s the offer from my site, decoded from a live 402 response this morning:

{
  "x402Version": 2,
  "accepts": [{
    "scheme": "exact",
    "network": "eip155:84532",
    "amount": "10000",
    "asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
    "payTo": "0xEdF2444D0259BBB8aC5094216D0148938F8308ff",
    "maxTimeoutSeconds": 300,
    "extra": { "name": "USDC", "version": "2" }
  }]
}

That offer asks for one cent in testnet USDC on Base Sepolia, paid to the burner wallet I’m using for the demo.

USDC uses six decimal places, so 10000 represents 0.01 USDC.

A compatible client can read those payment requirements and pay without creating an account with my site or asking me for an API key.

That’s a useful difference from robots.txt. A rule in robots.txt depends on a crawler choosing to respect it. Here, the server can withhold the protected content until payment succeeds.

The buyer wallet also held zero ETH, yet every settlement went through. The facilitator submitted the transactions and covered the blockchain transaction fees, usually called gas. I checked the balances twice.

For this setup, the agent only needed the testnet USDC it was spending.

There’s broader industry support behind the protocol too. The x402 Foundation became operational under the Linux Foundation on 14 July, with 40 members, including Visa, Mastercard, Google, AWS, Stripe and Cloudflare.

That’s useful context, but the actual exchange is still those four steps.

What’s running on my site

The demo is at paid.suganthan.com.

It runs on a Cloudflare Worker, using roughly 300 lines of Hono code and the x402 middleware. It’s been running since 8 August, and the public earnings page records every paid crawl since then.

There are three routes:

• The landing page is free and explains the demo.
• /research/agentic-seo/ charges $0.01 per crawl.
• /earnings/ shows the running total, timestamps, payer addresses and transaction hashes.

Request the article with curl and you’ll get this:

$ curl -i https://paid.suganthan.com/research/agentic-seo/
HTTP/2 402
payment-required: eyJ4NDAyVmVyc2lvbiI6MiwiZXJyb3Ii...

Open the same URL in a browser and you get a page explaining the charge. It still returns HTTP status 402, but a human visitor gets something readable.

There’s also a preview button. That opens the article for free with a banner, so you can inspect the content as well as the payment screen.

I built another mode that isn’t enabled in the demo. Changing one variable makes the Worker charge only requests that identify themselves as known AI crawlers, including GPTBot, ClaudeBot and PerplexityBot. Human visitors can browse for free.

I tested that mode using spoofed user-agent strings, and it behaved as expected.

Of course, those strings can be faked. Checking a user agent is enough to demonstrate the behaviour, but it doesn’t prove who made the request. Cloudflare’s approach uses cryptographic verification of crawler identity to address that problem.

Giving the agent a budget

The buyer is a small Node script. Before it pays, it checks whether the request fits its budget.

I gave it two limits:

• A maximum of $0.05 per call.
• A $0.25 daily allowance, recorded in a spending ledger on disk.

The script decodes the payment offer and checks both limits before authorising anything.

If the request is within budget, it pays, retries the request, prints the settlement details and records the spend. If it exceeds either limit, it refuses.

Here’s a refusal with the per-call cap set below the page’s price:

[step 3] Spending guardrails
[refused] Price $0.0100 exceeds the per call cap of $0.0010.
          This agent does not negotiate. Raise MAX_PER_CALL to override.

That refusal is one of the most useful parts of the demo.

If an agent pays every price a server gives it, you haven’t really delegated a budget. You’ve given it a way to empty a wallet.

The spending checks happen before signing. When the request fails those checks, the script exits with code 3 without creating a payment signature.

This follows the same general design Cloudflare describes for its wallets: a person provides the funds and sets the limits, and an agent spends within them. My buyer script demonstrates that behaviour without being an actual Cloudflare Virtual Wallet.

The successful run in the screenshot settled at 07:21 UTC. Two earlier payments went through at 06:10 and 06:11, with the second still within the daily allowance.

All three appeared on the public earnings page within seconds of confirmation.

The seller’s balance increased, the buyer’s balance decreased, and the amounts matched. The August transactions are still listed underneath them. I haven’t touched the payment code since those earlier payments.

Watching Claude pay during a task

This was the part I was most interested in.

Claude Code supports hooks: scripts that can run around tool calls. Cloudflare documents a pattern that watches WebFetch, and I connected that pattern to my wallet.

The process works like this:

1. Claude tries to fetch a page and encounters a 402.
2. The hook requests the payment offer itself.
3. It checks the price against the per-call spending cap.
4. If the price is allowed, it pays and retrieves the content.
5. It passes the content and receipt back to Claude as additional context.

Claude can then continue with its task. I don’t have to stop what I’m doing and manually pay for the page.

That’s what happened in the session shown near the top of this article. The hook returned this receipt:

Paid $0.0100 for "https://paid.suganthan.com/research/agentic-seo/" via x402.
Settlement tx: 0x51b5eba4fb34ddff27052d89cb79ae3d748b379800a723a6f4f020c11e2871c9

Claude encountered a payment requirement during a task, and the hook handled it within the budget I’d set. It paid one testnet cent, returned the article and let Claude carry on.

If the price exceeds the cap, the hook refuses and tells Claude to show the price to the human instead.

I quite like that boundary. Claude can buy a page when it needs one, but it can’t decide on its own that my five-cent limit no longer applies.

Should you start charging for your pages?

I wouldn’t do it this month expecting search crawlers to pay you.

The software for paying agents is developing quickly. The crawlers that most site owners care about haven’t caught up.

AWS made AgentCore payments generally available on 18 August. An agent using it can encounter a 402 during a task and pay through x402, much like my Claude hook. Cloudflare’s Agents SDK also has a client that can make payments.

But that doesn’t mean GPTBot, ClaudeBot, PerplexityBot or Googlebot will arrive at your site with a wallet and pay your asking price.

Until those crawlers support the process, requiring payment prevents them from retrieving the protected content. You could lose access to the systems that surface and cite your work without earning anything in return.

I’d also be careful with the transaction numbers being used to describe this market.

Decrypt reported on 13 September that TRM Labs had examined 198.9 million x402 settlements. Its analysis suggested that most of the payment volume wasn’t coming from AI agents.

Scheduled scripts, load tests and payments between accounts controlled by the same party can all produce similar blockchain records. TRM’s methods also can’t conclusively distinguish an agent from a script, so the estimates have limits.

A large number of transactions tells us that the payment infrastructure is being used. It doesn’t, by itself, tell us that autonomous agents are buying useful things from independent websites.

My own demo makes that distinction fairly obvious. The payments work, but I’m on both sides of them.

What I would think about now is which parts of your site you might eventually charge for.

I don’t think the answer is every page.

Being cited in an AI answer still has value. General explainers may be worth keeping freely accessible because they help people discover your work.

Original datasets, research and useful tools are more obvious candidates for paid access. An agent may need something specific from those resources to finish a task. Charging for that individual request makes more sense than trying to fit a machine into an advertising or subscription model designed around human readers.

Google’s pilot approaches that value question from the platform’s side. It pays when content contributes to the answer, rather than simply appearing as a link underneath it.

For a site owner, the practical questions become: what is the agent getting, what should that access cost, and who decides?

My expectation is that adoption will become easier when products such as Cloudflare’s Monetization Gateway move beyond their waitlists. Charging for a resource could become a routine configuration choice for a large number of websites.

I’d rather have thought through what to charge for before that option appears in the dashboard.

And if Google offers you a monthly payment in the meantime, ask how it was calculated. Digiday’s reporting describes limited transparency, and one publishing executive argued that accepting early terms could weaken publishers’ position in future negotiations. Those terms deserve as much attention as the payment itself.

Moving from test tokens to real money

Everything in this demo has used testnet tokens. The payments haven’t cost me real money.

The code also supports the real Base network. In my setup, the NETWORK setting switches from base-sepolia to base, moving the payment flow to real USDC.

My next step is to run exactly one real transaction.

When an agent pays my site its first real cent, I’ll put the transaction hash in the next post.

And when those reserved Cloudflare handles become working wallets, this is the kind of process they’ll support: an agent finds something it needs, checks the price against its budget, pays and carries on.

The demo is staying online in the meantime. Try the curl request, inspect the payment offer and have a look at the public earnings page.

Recursive Meta-Intelligence

We built a recursive AI that creates its own scientific instruments, turns them into a world inhabited by a massive agent ecology, which then reasons across vast, nonlinear spaces of possible physical futures. The AI swarm worked until tens of thousands of trajectories collapsed into mechanistic principles humans can use, distilling several core design principles that govern hierarchical metamaterials failure. The core tenet of this work is that AI can build the spaces in which its next level of reasoning becomes possible; a representation becomes an instrument, the instrument becomes an executable world, and that world becomes the substrate for further intelligence, forming a recursive loop. Intelligence grows by constructing new spaces to think in and improving its own thinking.

The world created by the initial set of agents is later populated by hundreds of AI agents. They explore a vast design universe where chemistry remains fixed and architecture creates functionality: nested, inter-woven structures within structures forming an incredibly complex design space. Here, the same constituent material can behave radically differently depending on where matter is placed, how it’s connected, and how order and disorder are distributed across scales. Architecture adds combinatorial degrees of freedom to a problem that is already strongly nonlinear and path dependent, especially when we examine how materials deform, and behave dynamically as they exposed to extreme pressures, and fail.

Every rupture changes the topology, redistributes force, and creates a new state. Cascades emerge from the evolving state of the material itself. Out of that complexity came a remarkably compact recurring result: hierarchy by itself does not determine performance; rather, the decisive variable is how the architecture organizes pathways through which forces are carried and redistributed as damage accumulates. The gain appeared when material was allocated into a dominant load-bearing structure. Placement of geometric order set whether failure progressed through smaller events or synchronized collapse. That opened a regime of surprising flaw tolerance with real trade-offs and real-world engineering implications.

The core discovery by the AI is that making materials damage resilient can be achieved by shaping the architecture so that once failure begins, the material is driven through a sequence of states that preserve function. The crucial point is dynamical - each rupture rewrites the topology, redistributes force and energy and creates a new state. The full response is a path-dependent evolution with avalanches and non-separable interactions in which the effect of one design choice depends on the others.

What is striking is the level of scientific cognition brought to that problem. The AI held a generative model of an evolving world, stepped outside the immediate state, rolled out alternative futures, compared long causal histories, identified interventions that changed which trajectories were reachable, and compressed those histories into a small set of generative invariants. This connects to model-based causal reasoning, counterfactual simulation, cognitive decoupling, and temporal abstraction.

Deeper reasoning

Reasoning over what could happen, how histories unfold, and which hidden mechanisms remain invariant across them. The resulting principle is profound: Architecture can program the evolution of failure. Geometry gives us a way to shape how a physical system moves through its possible future states under damage. That, in turn, transforms a vast nonlinear space into a powerful design handle: shape the future trajectory of matter under damage so function can persist deep into the failure process.

But perhaps most interestingly in my opinion, the larger impact is an expansion of the human epistemic horizon. Science progresses by extending what a mind can hold. Instruments extend perception, computation extends calculation, systems like this extend the size of the causal possibility space we can traverse before abstraction.

The Swarm Habitat

AI can inhabit thousands of alternatives, follow enormous numbers of possible histories, and return with mechanisms compact enough for us to reason with. That creates a form of extended cognition for science. Our effective scientific intelligence incorporates models, executable worlds, persistent evidence, and populations of artificial investigators. Machine scale exploration can be compressed into human scale understanding. Principles we can argue with, generalize, manufacture from, use to design the next experiment.

The recursion is powerful; the first agents define a scientific world and that world becomes shared cognition. Other current and future agents explore it, add evidence, retrieve and build further investigations through the same persistent instrument. This creates a flywheel of thinking - answer to instrument, instrument to world, world to ecology, and every step of the way is an editable artifact that serves as the basis for further improvement. Intelligence compounds through what the agents build, creating an intelligence explosion in science: designers constructing the possibility spaces in which other designers reason, experiment, and discover.

Flows through spaces of possibility

At a general level, learning and discovery are flows through spaces of possibility under constraint. In backpropagation, the flow moves through parameter space under a specified loss. In reinforcement learning, it’s coupled to an environment via rewards. In a fully autonomous scientific system, that flow becomes reflexive. The system moves beyond internal state updates and and buconstructs, manipulates, and repairs the executable worlds that define what its objectives, variables, and questions even are.

The universal pattern is a recurring coupling between possibility, constraint, consequence, and memory. Flows create structure, and structures redirect flows. As that loop scales from parameters to policies to representations to instruments and worlds, Intelligence is no longer confined to one fixed space; it begins to create the next space within which it can think.

Evolutionary tales

The evolution from RNNs to LSTMs to transformers (and later, looped transformers and other variants) can be read as a history of where intelligence places persistence, where it permits change, and how it carries structure from one cycle of computation into the next. RNNs compress the past into an evolving hidden state; LSTMs introduce gates that preserve selected information across longer horizons; transformers make prior tokens directly addressable as context and continually recompute relationships across that persistent substrate. Looped transformers reuse a stable computational operator while representations evolve through repeated passes.

Recursive self-evolving intelligence extends this principle beyond the internal architecture of a model and amalgamates multiple levers to create referential updates.

Representations, memories, tools, scientific instruments, executable worlds, and eventually entire societies of agents can become persistent and ever-evolving substrates that carry the products of one round of cognition into the next. Each cycle can stabilize what has been learned, turn it into a new object for reasoning, and open a larger possibility space above it.

Intelligence therefore acquires a multiscale architecture of persistence and plasticity: fast processes explore, slower structures accumulate, and the outputs of cognition become the invariants from which subsequent cognition proceeds. This gives recursion its real power, because once a system can experience a world, it can build structures that encode what it has learned, it can reason through those structures, revise them through consequence, and repeatedly create new levels on which further intelligence can operate.

A bright future

AI for science is one of the greatest positive forces we have. It expands the human epistemic horizon - letting us traverse thousands of possible histories, invent new instruments we would never have time to build by hand, and return mechanisms compact enough to understand, test, and manufacture. For scientists, this becomes a new instrument to explore the world, a new microscope, a new tool, a new way look deeply into the world and to make sense of it. I cannot think of anything more human than to understand nature and to use the power to create new engineering solutions that improve our lives, civilization and allow us to reach beyond.

We are just at the beginning of an exciting journey, and scientific superintelligence may emerge from a process in which complex activity generates stable abstractions that become the building blocks for a new level of reasoning. There is significant earlier work that becomes increasingly relevant as material intelligence is unlocked.

For instance, cybernetics and nonlinear physics offer a coherent theoretical underpinning; for instance, a system can cope with a complex environment only if it can generate sufficient internal variety, and in some regimes, it can reorganize the very mechanisms by which it remains stable. Repeated interaction can generate stable forms that become the “objects” of the next round of reasoning, while networks of processes can close on themselves to constitute a coherent level of organization.

In physics, large numbers of degrees of freedom can collapse into a few collective variables that then feed back to organize the lower level, and far from equilibrium, sustained flows can stabilize macroscopic order that was not there before. Put together, this suggests a very specific kind of scaling: High-dimensional activity produces a stable invariant. That invariant becomes an effective variable that compresses the complexity.

That variable becomes a contract on which a new layer of reasoning can stand, creating a new dynamical space above it. The recursion is not simply repetition but rather creation of new levels of reasoning substrates. One space is compressed into a principle, that principle opens a larger space above it. This is, indeed, a deep foundation for scientific superintelligence where we repeatedly generate the very framework in which still greater reasoning becomes possible.

References and notes

• M.J. Buehler, Artificial intelligence agents autonomously build computational laboratories that reveal design principles of hierarchical metamaterial failure, in submission, 2026
• S. Pal, F.Y. Wang, M.J. Buehler, SwarmWorld: Stigmergic technological evolution in societies of language-model agents, arXiv:2608.26081, 2026

Relay, an AI-powered workflow automation tool built as an alternative to Zapier, shut down entirely on Monday. The product let users automate email and task workflows using AI agents, but as OpenAI, Google, and other larger platforms built similar automation directly into their own tools, five-year-old Relay struggled to hold on to a reason to exist as a stand-alone product.

Relay — a startup outpaced by bigger platforms — is one version of the story playing out right now. But plenty of AI bets never even survive inside the companies that build them. According to S&P Global Market Intelligence, about 42% of AI initiatives are ultimately abandoned by their corporate parents. The reasons vary: insufficient funding, technical challenges, competition, difficulty scaling, or just weak user demand.

The result is a growing AI graveyard. Below is a look at some of the most notable AI products, startups, launches, and bets that have shut down, pivoted, or significantly missed expectations. Think of it less as a scorecard of flops than as a record of what these attempts can (hopefully) teach the rest of the industry.

OpenAI has taken several stumbles

Even the biggest players aren’t immune. OpenAI, one of the most valuable private companies in the world, has had its own share of casualties and missteps. One of the more recent examples was its attempt to turn ChatGPT into a broader “super app.” On July 9, OpenAI updated the app to combine several experiences, including separate “Chat,” “Codex,” and “Work” modes, while renaming the traditional, familiar version “ChatGPT Classic.”

The redesign quickly drew criticism from users who found the new interface confusing and cluttered, and OpenAI rolled back the change soon after, bringing the familiar ChatGPT interface back.

That wasn’t OpenAI’s only attempt to pull everything under one roof. The company has also shut down several stand-alone apps and folded their features directly into ChatGPT — a strategy that’s worked out better than the redesign did. ChatGPT Atlas, a stand-alone AI-powered web browser, lasted less than a year before being discontinued on August 9, with its most useful features absorbed into ChatGPT.

The company did the same thing with Operator, an AI agent that could browse the web and complete tasks on a user’s behalf, and OpenAI has increasingly built image generation directly into ChatGPT, shrinking the role of DALL-E as its own separate destination for creating images.

We also can’t forget its video-sharing platform, Sora, which struggled with high operating costs and user-retention challenges and shut down in March 2026.

Apple’s Siri AI

When Apple unveiled Apple Intelligence in 2024, an improved Siri was one of the highlighted features. The pitch promised a Siri that could understand context, know what was happening across apps, and get tasks done effectively.

However, Apple repeatedly pushed back the release of the new Siri, citing various reasons like engineering issues and bugs. This delay eventually contributed to a $250 million settlement over claims regarding how Apple marketed the AI capabilities of the iPhone 16.

Siri AI finally appeared in the iOS 27 beta this past July, with the new AI-powered version rolling out to English-language users first this month; support for other languages is expected to follow.

Microsoft’s Recall

Announced at its 2024 Build developer conference, Microsoft’s Recall feature was pitched as an AI-powered “photographic memory” for Windows PCs, periodically taking screenshots of what a user was doing so they could later search back through their own digital history.

Predictably, the announcement triggered an immediate backlash over privacy and security. Critics argued that Recall could create a searchable archive of highly sensitive information, including passwords, private messages, financial details, and other activity. Microsoft delayed the feature for nearly a year, giving the company time to rethink its security and privacy protections.

Even after a redesign, the controversy hasn’t gone away. A cybersecurity researcher recently built a tool that could extract and display data that Recall had captured, reviving questions about whether Microsoft has actually fixed the underlying problem.

Notion Mail

Notion Mail launched in April 2025 as an AI-focused email product designed to help users organize and automate their inboxes. However, the company noticed its users were using separate AI agents to handle their email inboxes instead. As a result, Notion Mail is scheduled to shut down on September 22.

Humane AI Pin

The Humane AI Pin is probably one of the most well-known failures in AI hardware. Designed to provide AI features through a wearable device rather than a traditional smartphone, Humane raised $230 million from investors and garnered significant attention.

But the product struggled mightily with performance, and the situation worsened when Humane warned customers to stop using its charging case because of a potential battery fire risk. Humane then shut down its AI Pin business in February 2025, with most of the company’s assets acquired by HP for $116 million.

Rabbit R1

The Rabbit R1 was another heavily hyped AI device. Unveiled at CES in January 2024, the AI companion was designed to perform tasks on behalf of its owner. Rabbit said it had sold 100,000 units shortly after launch.

However, strong initial sales didn’t mean it was a strong product. Early reviews described the device as unfinished, with unreliable performance and a limited number of useful integrations.

The company hasn’t given up, though. It has continued updating the R1, positioning it as a computer controller capable of performing agentic tasks. It also recently announced a new hardware project, dubbed “Project Cyberdeck,” aimed at creating a portable device built for vibe-coding.

Huxe

Huxe was an AI audio app, built by former developers of Google’s NotebookLM, that turned written information into conversational, podcast-style audio.

The company shut down in May 2026, partly as larger platforms began offering similar experiences to large existing audiences. For instance, Spotify has been expanding its own AI-powered audio tools.

Yupp

Yupp was an interesting AI product: a free playground that allowed users to compare responses from hundreds of AI models side by side. Users could also vote on responses and earn cryptocurrency through the platform.

At its peak, the platform let users test more than 800 models, including from companies such as OpenAI, Google, and Anthropic. But according to its founders, Yupp never achieved strong enough product-market fit to survive. The platform shut down in March 2026.

Figgs AI

Figgs AI operated from 2023 to 2024, letting users create and interact with customizable AI characters for role-play and storytelling. The platform reportedly drew more than 1 million users, but its developers said keeping the service free became too expensive to sustain, and they shut it down.

G5 Labs Raises $14M Seed Round to Pioneer the Post-Developer Era

G5 Labs, a spinout from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), has emerged from stealth with $14 million in seed funding to build a new abstraction layer for AI-native software development. With the G5 Labs Platform, natural language — organized as a structured, formal graph of intent or ontology — becomes the source code. Pillar VC and Battery Ventures co-led the round, with participation from Omega Venture Partners and Encoded Ventures, as well as prominent angel investors including Jeff Dean, CEO of Discovery Loop and former Chief Scientist at Google.

“Teams are either struggling to adapt to AI or drowning in AI-generated code they can’t review, merge, or trust,” said Tim Kraska, co-founder and CEO of G5 Labs and professor at MIT CSAIL. “Single developers generate tens of thousands of lines a day, maintainers are overwhelmed with AI slop, security teams face entirely new threats, and businesses are racking up six-figure monthly AI bills with little to show for it. The fundamental problem is that we have slapped AI onto development tools and processes designed for humans.”

“This isn’t just better prompting — it’s a new abstraction layer,” Kraska continued. “With G5, natural-language intent becomes the source code itself: compilable, mergeable, diffable, and governable, the way high-level languages once replaced assembly. For the first time, it puts software in the hands of the whole organization, not just developers.”

G5 emerged from research at MIT’s CSAIL, where the team built the technology that makes this possible: a self-learning, bi-directional compiler that reliably converts natural language to code and back based on an ontology graph. The platform lets organizations manage intent across an entire company; resolve merge conflicts at the semantic level rather than the code level; trace every line of code back to the intent behind it; ensure cost control before the tokens are spent; and enforce organization-wide policies such as GDPR and security requirements across old and new systems.

The results have been striking: In a production financial services software modernization project, G5’s ontology-level comparison uncovered structural problems that would otherwise have remained invisible when porting an old application to a new one. By presenting each conflict in both technical and business-friendly views, G5 enabled the customer’s business analysts to drive design decisions directly and implement new features faster and more coherently.

“Past major leaps in software have always come from raising the level of abstraction. G5 is a serious, well-engineered attempt at the next one.”

“G5’s approach is the first credible answer to the AI productivity paradox we’re seeing across complex systems. The team’s research pedigree and early enterprise traction convinced us this is the company that defines the category,” said Parker McKee, partner at Pillar VC.

Current G5 customers already use the platform to modernize complex software in regulated industries. Rather than simply translating legacy code into a newer language or framework, G5 modernizes systems without carrying forward decades of technical debt and misaligned business logic. At the same time, it makes the new software AI-ready and establishes the governance mechanisms needed for reliable, controlled development. The result is a shift upstream — the gains from AI-driven development no longer stop at the engineering team. Product owners, analysts, and compliance leads can directly define, govern, and evolve software around business intent, in the language they already speak.

“With AI, there will soon be no more reason to write software in Java, C, Python, or Rust,” said Max Schireson, Partner at Battery Ventures and former CEO of MongoDB. “The new language of software development will be English. The missing link is a suite of tools to build a robust software-engineering process around it. G5 is delivering those tools.”

G5 Labs was founded by a team with deep expertise in artificial intelligence and large-scale systems, including veterans of AWS and Google, along with MIT PhDs. The company’s CEO, Professor Tim Kraska, is a co-director of MIT’s Generative AI Impact Consortium, a multimillion-dollar research initiative, and brings two decades of experience applying ML and AI to complex systems across academia, Google, Amazon, and two previous startups, both of which were acquired.

The company will use the funding to expand its engineering team, scale customer deployments, and continue developing the G5 platform and its ontology compiler.

OpenAI has held early discussions with investors about a financing round that would precede the company's highly anticipated initial public offering expected next year. The new funding round, which could value the company at more than $1.2 trillion, has been delayed due to concerns about AI safety. OpenAI has reached more than one billion active users, with more than 200 million businesses using its models.

Researchers at the Janelia Research Campus at the Howard Hughes Medical Institute in Virginia, in partnership with Google, recently published a new map of the adult male fruit fly's brain. The effort, which involved hundreds of scholars, resulted in a complete map of the fly's 166,000 neurons that can be trained to complete tasks like solving a Rubik's Cube or playing poker. Internet engineers and artists have made the model complete many seemingly impossible tasks for a fruit fly, like parallel parking a car, playing Doom, and trading cryptocurrencies.

Mark Zuckerberg made a social media post on Tuesday, saying that calls to pause AI development industrywide were the wrong approach. He said that every lab has the responsibility and incentive to move at the pace required to train its models safely. He acknowledged the risks of AI, but said companies were already incentivized to keep users safe and slow progress on their own. Meta has been aiming to catch up in the AI race after falling behind Anthropic and OpenAI last year. It plans to release a new model, internally called Watermelon, in the coming weeks.

OpenRouter users spent more on OpenAI models than on Anthropic models last week.

This hasn't happened for more than 2.5 years

Internal Developer Portals: Why Native CI/CD Drives Scale | Harness Blog

TL;DR

This article explains why internal developer portals require native CI/CD integration to achieve true scalability. It covers how tightly integrated CI/CD pipelines eliminate friction, improve developer velocity, and enable platform engineering teams to deliver self-service automation at scale—with insights on how Harness IDP provides this native integration.

Can your internal developer portals actually ship code, or do they just catalog it?

Most portals start the same way. You build a service catalog. You document ownership. You add health checks. Developers can finally see what exists and who owns it. That is useful. But when they try to create a new service or update a deployment, they hit a wall. The portal shows them what to do. It does not help them do it. They still open tickets. They still wait. They still context-switch between tools. The portal becomes a reference layer, not a workflow layer.

The problem is not the catalog. The problem is that the portal stops where the work begins. Without native CI/CD integration, internal developer portals cannot scale past visibility into actual velocity.

Why Internal Developer Portals Stall Without CI/CD

A portal without integrated pipelines is a front door with no rooms behind it. Developers can see the menu, but they cannot order. They learn what the standards are, but they cannot apply them without leaving the portal and stitching together their own pipeline from scratch.

This creates three failure modes at scale.

First, developers spend time translating portal guidance into pipeline configs. The portal says, "Use this template." The developer copies YAML, updates variables, submits a pull request, waits for review, and hopes they got it right. That is not self-service. That is documentation with extra steps.

Second, platform teams cannot enforce standards across repos. They can recommend golden paths, but enforcement requires someone to audit pipeline files manually or write custom tooling to check for drift. Standards become suggestions. Drift becomes inevitable.

Third, cross-team workflows break. If Service A depends on Service B, and both teams use different pipeline structures, promotion across environments turns into coordination overhead. The portal cannot help because it does not control the pipelines.

What Native CI/CD Integration Actually Means

Native integration means the portal does not just describe pipelines. It creates them, triggers them, and governs them. Developers interact with CI/CD workflows directly through the portal interface, without needing to understand the underlying pipeline engine or leave the portal to configure YAML.

This is not the same as linking to an external CI/CD tool. That still requires the developer to understand the tool's interface, permissions model, and configuration syntax. Native integration abstracts that complexity. The portal becomes the control plane. The pipeline engine becomes the execution layer.

When a developer creates a service through the portal, the portal scaffolds the repo, applies the correct pipeline template, and connects it to the right environments. When they promote a service to production, the portal triggers the pipeline with the correct approvals and rollback logic already configured.

The key difference is that the developer never writes pipeline configs by hand. The platform team defines the templates once. The portal applies them consistently.

How CI/CD Pipeline Automation Enables Self-Service at Scale

Self-service only works if the easy path is also the safe path. If developers have to choose between moving fast and following standards, they will move fast. Native CI/CD integration removes that choice by making the standard path the only path that works.

Start with service creation. When a developer uses a portal template to spin up a new service, the portal does not just generate boilerplate code. It also provisions the pipeline, configures branch protections, sets up environment promotion rules, and connects the service to observability tooling. The developer gets a working service with a working deployment process in minutes.

Next, consider deployments. Without native integration, developers either push to a branch and hope the pipeline works, or they open a ticket asking ops to deploy for them. With native integration, the portal surfaces deployment actions directly. The developer selects a version, chooses an environment, and clicks deploy. The portal handles approvals, rollback logic, and deployment verification automatically.

Finally, look at incident response. When something breaks, developers need to roll back fast. If the rollback process requires digging through pipeline logs to find the last good version, then manually triggering a redeployment, that is too slow. If the portal tracks deployment history and exposes rollback as a single-click action, that is fast. Native CI/CD integration makes the difference.

How Platform Engineering Tools Connect Portals to Pipelines

Platform engineering teams need more than a service catalog. They need a way to encode operational knowledge into reusable workflows. That requires tight coupling between the portal and the pipeline engine.

This works best when the portal and the CI/CD system share a common data model. The portal tracks services, ownership, dependencies, and configurations. The pipeline engine tracks builds, deployments, and environments. When both systems speak the same language, the portal can trigger pipelines with full context. It knows which service is being deployed, which environment it targets, and which approvals are required.

The alternative is brittle. If the portal has to call external APIs to trigger pipelines, those integrations break when the API changes. If the portal has to parse pipeline logs to determine deployment status, it fails when log formats shift. If the portal cannot enforce pipeline templates, teams diverge.

Native integration eliminates these failure modes. The portal does not integrate with the pipeline engine. It uses the pipeline engine. That distinction matters.

Internal Developer Platform Scalability Depends on Governance

Speed without guardrails leads to sprawl. Sprawl leads to incidents. Incidents lead to lockdown. Native CI/CD integration prevents this cycle by embedding governance into every workflow.

Start with pipeline templates. Platform teams define golden paths for common service types: APIs, batch jobs, frontends, data pipelines. These templates encode the right way to build, test, and deploy each service type. When developers create a service through the portal, the portal applies the correct template automatically. No one has to remember which pipeline structure to use. The portal enforces it.

Next, add promotion policies. Not every service should deploy to production the same way. High-risk services need manual approval. Low-risk services can auto-deploy. Native integration lets platform teams define these policies once, and the portal enforces them consistently. Developers do not bypass the rules because the rules are invisible. The portal only exposes the actions that match the policy.

Finally, track compliance. When pipelines live outside the portal, auditing them requires scraping logs from multiple systems. When pipelines are native to the portal, the portal already has the data. Platform teams can see which services follow the standard pipeline structure, which have drifted, and which have never been deployed. That visibility makes governance scalable.

How Harness IDP Delivers Native CI/CD Integration

Harness IDP treats CI/CD integration as a first-class feature, not an afterthought. The portal does not just link to Harness pipelines. It scaffolds them, triggers them, and governs them as part of the service lifecycle.

When a developer creates a service using a Harness IDP template, the portal generates the repo, applies the correct pipeline template, and connects it to the appropriate environments. The developer does not write YAML. The platform team defined the pipeline structure once. The portal applies it consistently.

Deployment actions surface directly in the portal. Developers select a service, choose an environment, and trigger a deployment without leaving the IDP interface. The portal handles approvals, notifications, and rollback logic using the policies the platform team defined. This is not a link to an external tool. This is native workflow automation.

The service catalog tracks deployment history. Developers can see which version is running in each environment, who deployed it, and when. If something breaks, they can roll back to a previous version with a single click. The portal uses Harness pipelines to execute the rollback, but the developer does not need to understand the pipeline structure.

Governance happens automatically. Platform teams define pipeline templates for different service types and promotion policies for different environments. The portal enforces these standards without requiring manual review. If a service does not follow the golden path, it cannot deploy. That constraint is invisible to developers because the portal only exposes workflows that match the policy.

For platform engineering teams scaling CI/CD automation across hundreds of services, this tight integration eliminates the biggest source of drift: manual pipeline configuration. Developers get self-service. Platform teams get governance. Both happen through the same interface.

The Real Cost of Loose Integration

Some platform teams try to bolt a portal onto an existing CI/CD system using webhooks, API calls, and custom scripts. This works for small teams. It breaks at scale.

First problem: context loss. The portal knows which service is being deployed. The pipeline engine knows which environment it targets. But if those systems do not share data, the pipeline runs without full context. It cannot enforce service-specific policies. It cannot validate dependencies. It cannot fail early when something is misconfigured.

Second problem: drift. Developers can still edit pipeline files directly. The portal suggests a structure, but it cannot enforce it. Over time, teams customize their pipelines. Some add extra steps. Some skip steps. Some copy pipelines from other repos and modify them. The golden path fractures into dozens of variations. The platform team loses visibility.

Third problem: operational complexity. When the portal and the pipeline engine are loosely coupled, every new feature requires coordination. The portal team builds a new template. The pipeline team updates their API. The integration layer needs a patch. Each change introduces risk. Testing becomes harder. Rollbacks become harder. Velocity slows.

Native integration avoids these failure modes by treating the pipeline as part of the portal, not as an external dependency. The portal is the control plane. The pipeline engine is the execution layer. Both systems share the same data model, the same permissions model, and the same governance framework.

Conclusion

Internal developer portals that stop at service catalogs create visibility without velocity. Developers can see what exists, but they cannot act on it without leaving the portal, writing pipeline configs, and waiting for approvals. That is not self-service. That is better documentation.

Native CI/CD integration changes the equation. The portal becomes the interface for the entire software delivery lifecycle. Developers create services, deploy changes, and roll back incidents through a single interface. Platform teams define standards once and enforce them automatically. Governance scales without slowing teams down.

Harness IDP delivers this integration by treating pipelines as a native part of the developer experience. The portal scaffolds pipelines, triggers deployments, and tracks history without requiring developers to understand the underlying pipeline engine. Platform teams get governance. Developers get velocity. Both happen through the same interface.

If your portal shows developers what to do but does not help them do it, you have built a reference layer, not a platform. Native CI/CD integration is what turns a portal into a workflow engine that scales.

How runtime insights helps with container security

Containers are a key building block for cloud workloads, offering flexibility, scalability, and speed for deploying applications. But as organizations adopt more and more containers, they encounter a new set of security challenges. Developer, DevOps, platform, and security teams often find themselves struggling to keep up with vulnerabilities, misconfigurations, and threats. This is where runtime insights come in, offering key visibility and intelligence to help detect real risk and cut through noise.

Let's explore what runtime insights are, how they’re used from development through production, and why this approach is essential for secure operations.

Staying ahead of container security threats

Container security threats come in many forms. In a dynamic environment orchestrated by Kubernetes and serverless platforms like AWS Fargate with potentially thousands of containers spinning up and down – often within seconds – applying traditional security approaches is typically both frustrating and futile.

Relying solely on pre‑delivery container image vulnerability scanning is not enough. What's needed is a lens into what's actually happening in deployments. Runtime insights provide the container visibility and context needed to provide actionable information that helps not only to detect active threats in your containers but to also prioritize the most impactful risks and problems based on the knowledge of what is running right now.

How runtime insights work

Runtime insights for containers are derived by continuously monitoring containerized workloads in real time. This continuous visibility means security teams can detect unusual behavior as it happens, providing an added layer of security that is attuned to the fast-paced, ephemeral nature of containerized applications.

To get the insights needed to understand critical aspects of containers in production, runtime instrumentation designed for container inspection is essential. Containerized workloads interact with the kernel and with other applications through system calls. Visibility into these system calls at the host-level through technologies like eBPF allows for real-time detection of security events and profiling of container behavior without requiring any modification to your container images.

Use cases and benefits of runtime insights for container security

Runtime insights are a key component of the Sysdig cloud security platform. It enables security teams to effectively identify and prioritize the most important and relevant risks in their environment across several different domains.

Real-time threat detection

Runtime insights improve threat detection for containers by analyzing live behavior instead of relying on periodic snapshot analysis. Because containers can have such short lifespans and are ephemeral, runtime security that captures what is happening while a container runs is critical to identifying the exploits of malicious actors. Runtime insights make it possible to identify active risk and spot anomalies and attack patterns such as unusual network connections and unauthorized data access in real time.

Learn more about real-time cloud-native threat protection with Falco open source.

Container vulnerability management and prioritization

One of the realities of container images is that they often include libraries, packages and other files not required for a given deployment. This creates noise when interpreting vulnerability scan reports resulting in wasted time as developers try to determine what to fix first.

Runtime insights help organizations improve "shift-left" security practices by focusing on container vulnerabilities in packages that are actually in use, rather than spending resources on fixing vulnerable packages that are dormant. This targeted approach delivers a more efficient process for vulnerability remediation by directing attention to high-priority risks.

Read more about runtime insight integration with leading AppSec tools: Checkmarx, Docker Scout, Mend.io, ServiceNow, and Snyk.

Container incident response

When a threat is detected, runtime insights provide the context needed for quick, informed responses. Knowing exactly what happened in real time — who accessed what, when, and from where — enables security teams to respond effectively. This reduces incident response times and limits potential damage.

Learn about the 555 benchmark for cloud detection and response.

Container compliance

Many regulatory standards require continuous monitoring of workloads. In the EU, for instance, the Digital Operational Resilience Act (DORA) and the Network and Information Systems Directive (NIS2) establish regulations and objectives to raise the level of cybersecurity and require aggressive time to disclosure of security events in industries including financial services.

Runtime insights help achieve compliance by constantly assessing container security posture, delivering automated alerts, and capturing audit trails that simplify meeting standards to help organizations meet continuous alignment with regulatory requirements.

Get the scoop on container security best practices.

Container runtime insights speed up security in the AI era

AI has made cybersecurity more accessible to everyone. On the one hand, attackers are using AI to speed up their intrusions and penetrate deeper into their victims’ infrastructure. On the other hand, agentic AI has the potential to speed up incident response, forensic investigations, and the implementation of mitigation actions.

Detailed runtime insights are the difference between an agent that provides generic responses and one that truly understands your environment, thereby reducing cognitive burden. The context runtime insights offer allows AI agents to assist in risk triage and prioritization, reducing wasted time and employee burnout. By knowing your infrastructure, agents can provide clear, actionable steps that speed up your team’s response to container risks.

Discover how AI is the present of security.

Conclusion

Runtime insights provide a unique, powerful approach to improving security practices for containerized applications. Security in containerized environments is a shared responsibility across teams. By integrating runtime insights with DevSecOps practices, development, operations, platform and security teams can better collaborate to improve response to emerging threats.

By leveraging live data and behavioral analysis, runtime insights fill the gaps left by traditional, static security approaches. This helps organizations adapt to the unique needs of containers, removing security as a bottleneck to cloud-native innovation.

OpenResearch

The local-first workspace for research agents and autoresearch.

Turn Claude Code, Codex, OpenCode, or Cursor into research agents that can review literature, develop hypotheses, run experiments, and produce research artifacts.

Get started

Install the CLI on macOS or Linux, then launch OpenResearch:

curl -LsSf https://openresearch.sh/install.sh | sh
orx up

On Windows, use the beta download after installing Git for Windows.

orx up opens the local dashboard at http://127.0.0.1:4791.

Connect a local model to use LM Studio, oMLX, Ollama, or a custom endpoint with OpenCode.

Create an account at openresearch.sh to receive email updates and use managed OpenResearch compute.

Built for research agents

• Parallel exploration: Give each research direction an independent agent session and isolated git worktree.
• Reproducible experiments: Track variants in a git-native experiment tree; every run receives an immutable archive of its recorded commit.
• Evidence in context: Keep logs, diffs, files, results, and artifacts tied to the work that produced them.
• Your choice of agent: Use Claude Code, Codex, OpenCode, or Cursor, with the harness and model selected per session.
• Your choice of compute: Run locally, on your own infrastructure, or with managed OpenResearch compute.
• Local ownership: Keep projects, conversations, experiments, runs, logs, code, and artifacts on your machine.

Autoresearch

OpenResearch can run the full loop autonomously: propose an idea, change the code, launch an experiment, inspect the evidence, and decide what to try next. Multiple agents can explore different directions in parallel while the experiment tree preserves their lineage.

Run anywhere

The same committed source snapshot can run locally, over SSH, or on Slurm, Kubernetes, Ray, Hugging Face Jobs, Modal, Tinker, and managed OpenResearch compute. Publishing the repository is not required.

Run the workspace next to remote GPUs while using the browser on your laptop:

orx up --remote user@host

SSH config aliases and custom ports are supported. The remote service binds to loopback and has no application-level authentication, so other users on that host can reach it.

CLI and agent integration

Install the OpenResearch skill into supported coding agents:

orx install-skills

Common commands:

orx projects
orx project view <project-id>
orx runs <project-id>
orx logs <run-id>
orx exp run <experiment-id>
orx discover keyword <query>
orx paper <arxiv-id-or-doi>

Run orx --help or orx <command> --help for the complete interface.

Local by default

OpenResearch runs on 127.0.0.1 with a local SQLite store. Creating a project or launching a run does not publish your code. An openresearch.sh account is only used for service-owned capabilities such as organizations and managed compute.

Usage analytics

Official release builds send opt-out, coarse usage events tied to a random installation ID. They do not include code, prompts, file contents or paths, repository names, tokens, emails, or project and experiment identifiers.

orx telemetry off
orx telemetry status
orx <command> --no-telemetry

Source and development builds do not send analytics.

One of the most perplexing discussions I’ve had in recent times was about CI/CD. I’ve earned my CI/CD scout badge at this stage in my career, so the confusion caught me off guard. This post briefly sets out what Continuous Integration (CI) and Continuous Delivery (CD) mean and addresses the most common misconceptions in the discussion.

My point isn’t just semantic. These terms carry decades of hard-won engineering practice, and when we let their meaning drift or decay, we lose crucial wisdom.

Continuous Integration

Continuous Integration is well named compared to many concepts in our industry. The term “integration” refers to merging your changes into the main branch in version control, and the term “continuous” means you do it often. When Kent Beck popularized the term in Extreme Programming Explained (1999), he set an upper limit of a day. After 26 years of progress, you should expect to be doing it far more frequently than that.

You might understand the concept better if we look at examples that don’t fit the definition. If you have code on your machine that’s more than a day old, you’re not practicing Continuous Integration. If you have a branch that’s more than a day old, you’re not practicing Continuous Integration. It doesn’t matter how often you merge from main into your branch, because main is completely unaware of changes, like yours, that haven’t been merged yet. That means you have diverging software versions, and at some point, you have to make them all converge.

Continuous Delivery

Continuous Delivery is an approach focused on delivering all changes to users quickly, safely, and sustainably. You keep your code in a deployable state at all times and reject the idea of separate integration, testing, and hardening phases. There are specific technical practices involved here because you need a deployment pipeline that reduces risk and increases your confidence that the software version works.

At this stage in our industry’s history, almost everything involved in answering the question “Is this deployable?” should be automated as part of the deployment pipeline. A decade ago, perhaps, you could reasonably list a few stages that were economically unviable for automation. The scale and pace of modern software delivery mean this is rarely the case now.

The simple test is to ask, “Is the software deployable?” If you can’t answer this question every day, you’re not doing Continuous Delivery. In their book on Continuous Delivery (2010), Dave Farley and Jez Humble detail the principles and practices that let you answer this question, and the research has increased our confidence that this is what good software delivery looks like.

Misconceptions

Let’s turn our attention to where the conversation left the trail.

CI doesn’t just mean automated builds

One of the most common sources of confusion is that CI just means automated builds and tests. While it’s difficult to practice Continuous Integration without automation, the crucial property of Continuous Integration is that all changes are in a shared mainline, not on developer machines or in long-lived branches. Having more than 3 branches or keeping branches open for more than a day slows delivery throughput and makes the software less stable, as reinforced by long-running research.

There’s a human process at the heart of CI: each developer chooses to commit their changes frequently to main. Without people deciding to do this, there is no CI.

CI doesn’t allow feature branching

There’s a misconception that you can do CI and choose a branching strategy, like feature branches. Any branching strategy that keeps a change out of main for more than a day cannot be called CI. You may still choose to use feature branches despite the drawbacks, but you have to stop calling it Continuous Integration, as you aren’t continuously integrating all changes.

CD doesn’t mean Continuous Deployment, though it can

CD means “Continuous Delivery”. It means your software is deployable at all times, and you can test this daily by asking, “Can we deploy the software?” If you don’t have a way to answer the question confidently, you’re not doing Continuous Delivery. You may choose to deploy every validated software version automatically, which is Continuous Deployment, or you may need to be more tactical about when you deploy. Either is fine.

For both Continuous Delivery and Continuous Deployment, you need high levels of automation and sufficient risk coverage within that automation so you know you can deploy. Continuous Delivery should mean you can deploy at the press of a button, and Continuous Deployment removes the requirement to press the button.

You can’t do CD without CI

Some folks were confident that you can do Continuous Delivery without doing Continuous Integration, but the literature on this is very clear. Farley and Humble explicitly include Continuous Integration as a required practice. There’s a whole chapter dedicated to it (p. 55 of Continuous Delivery) that adequately explains how it should work and why it’s not optional.

CI/CD isn’t vague or open to interpretation

Another claim that came up more than once was that CI/CD is vague, open to interpretation, or lacks the detail on “how” you should apply the approach. This does a disservice to those who conscientiously captured the approach, whether that’s Kent Beck describing Continuous Integration in Extreme Programming Explained (1999) or Farley and Humble detailing both Continuous Integration and Continuous Delivery in their book Continuous Delivery (2010).

Resist the diffusion

There is a constant temptation in the tech industry to allow semantic diffusion to erode our progress. When we package up a cartload of wisdom into a term, we have to resist the diffusion that follows when people oversimplify our term, remove crucial parts of the whole, or try to brand something with the label that pushes us backward.

We have a history of giving up on these battles, as we seem to have done with Agile and DevOps. These terms had an important meaning that represented a significant step forward for our industry, and losing our energy in their defense lets the perpetual “Late Majority” and “Laggards” (from Crossing the Chasm) drag us all back into their old ways.

CI and CD deserve better.

Connect External Data for Complete Insight, Without the Ingestion Tax

Today, we are excited to announce the General Availability of New Relic Lens—a powerful data federation capability designed to simplify and accelerate data discovery and analysis across multiple environments. With New Relic Lens, you can easily connect your observability data directly to your business outcomes and transform that data into actionable insights, all without paying for extra data ingestion.

What is New Relic Lens?

New Relic Lens allows you to connect and query multiple external data sources—like Snowflake, Google Sheets, PostgreSQL, MySQL, and more—using standard SQL, all from within the New Relic UI. Users can combine, analyze, and correlate telemetry and non-telemetry (business) data using sophisticated cross-database JOINs without ingesting the external data, effectively breaking down organizational data silos.

Query your metadata on your uses stored in Snowflake:
SELECT userId, name, company FROM snowflake.users.user_metadata

Query New Relic data through Lens:
SELECT enduser.id FROM telemetry.PageView

Join your page view data with metadata from Snowflake to group the data by company
SELECT sf.company, COUNT(*)
FROM telemetry.PageView nr
JOIN snowflake.users.user_metadata sf ON nr.enduser.id = sf.userId
GROUP BY sf.company

Why Lens?

If you are an engineering leader, product manager, or IT Ops lead, this scenario probably sounds familiar: You want to tie system performance (like checkout latency) to business metrics (like shopping cart revenue), but that business data lives in a separate database like Snowflake.

Traditionally, integrating this data meant paying a "double tax" on ingestion: you paid once to store the data in your database, and you paid again to ingest it into your observability platform just to visualize it. Alternatively, you resorted to chaotic workarounds, manually exporting data into separate BI tools. New Relic Lens changes the game by querying the data exactly where it lives.

Key Features & Capabilities

• Zero-Ingestion Connectors: Query external databases directly without duplicating or persistently storing raw data within New Relic. Seamlessly access data that includes Google Sheets, Iceberg, Snowflake, Postgres, MySQL, Redshift, Prometheus, MongoDB, ElasticSearch, AWS CloudWatch, ClickHouse, SQL Server, Databricks (Azure) with more being added.
• Cross-Database JOINs in SQL: Uniquely combine datasets from entirely different external databases into a single query and visualization widget. Write complex queries joining all of your data in standard SQL, going beyond traditional NRQL boundaries.
• Materialized Views: Accelerate dashboard performance and reduce external compute costs by storing precomputed query results in Iceberg-backed databases. This is especially effective for complex, high-volume datasets and results in only minimal data ingestion fees.
• Unified Data View: Embed external business metrics alongside Application, AI, and Infrastructure telemetry natively within Dashboards and the new New Relic Notebooks feature to gain immediate, actionable context.
• Ephemeral Caching for Security: Security is built-in. Data is queried on demand and only briefly cached—for about a minute—to render in the UI. We do not ingest, index, or store your raw business data in New Relic, except when you explicitly choose to use Materialized Views.

Transforming Your Workflows

New Relic Lens empowers you to perform deep analysis that previously took hours of manual correlation. Here is how it transforms your workflows:

• Align System Performance with Revenue: Compare New Relic infrastructure metrics directly with budget or revenue data stored in Snowflake, placing it all on the same dashboard to instantly see the financial impact of downtime or latency.
• Cross-Database Correlation: Track usage or incidents by combining data from disparate sources, such as correlating support tickets in Jira with infrastructure provisioning data natively inside New Relic.
• Optimize Dashboard Performance and Costs: Use Materialized Views for high-traffic executive dashboards to prevent repeated, expensive compute calls on your source database while ensuring visualizations load instantly.

Get Started Today

It's time to break down the barriers between your databases and analyze your aggregated data in a single, unified observability platform. Stop double ingesting and start saving.

New Relic Lens is available to all paid customers. By default, access is restricted to users with the 'organization_manager' role. Administrators can easily adjust these permissions or configure Fine-Grained Access (FGA) for connectors to grant access to other team members as needed.

Canva launches ProSuite bringing Affinity, Cavalry, Flourish and Leonardo together with more than new 100 features

Affinity and Cavalry stay free, a long-requested Blend Tool finally arrives (thank God), and Canva makes its biggest move yet to support the professional designer.

There's no doubt Canva did things the right way round, even if that was never its intention. Adobe began with the industry and has spent a decade trying to reach everyone else. But Canva started with the mass market – all 250 million of them, as of this month – and only then turned its attention to the creative pros.

Today, it's gone one step further with the launch of Canva ProSuite, a connected set of professional tools that now includes 100 new features, as requested by its design community.

The suite includes Affinity for design, photo editing and layout; Cavalry for motion; Flourish for data visualisation; and Leonardo for AI image generation. And it absolutely promises that the core tools in Affinity and Cavalry will remain free to use.

I've been wondering when Canva would make it official and bring all its acquisitions together. I was also hoping it wouldn't get greedy and U-turn on its free products. I was there when Adobe announced the subscription-based model in LA in 2013. I remember the backlash and frustration, given there weren't any decent alternatives at the time. Ironically, Canva was born that same year. Today, Adobe is still the industry standard (for now), and for good reason. But no one can argue that day marked a shift in how a lot of creatives felt about it.

Back then, Canva wasn't focused on the creative industry; it wanted to make design accessible to everyone. That meant small business owners, schools, charities, restaurants... anyone who couldn't afford to hire a professional. The idea was to give people the tools to create their own designs. Ones that didn't require an entire course and years of practice.

Making design accessible for all was the theme of much of what Cameron Adams, Canva co-founder and chief product officer, said at a press breakfast in London this morning. "We wanted to take this incredibly complex workflow that probably only 1% of the world could do and turn it into something that the other 99% of the world could do," he said. "Making complex things simple. That is the underpinning ethos of all of our product development."

The mission, he explained, hasn't changed since the beginning, when Canva only had 10 staff and around 50 users. Today, it sees over a billion designs a month, including 224 million in the UK in the last 12 months alone. Around 750,000 UK small businesses use it each month.

For a very long time, Canva was sneered at for its templates and basic designs, but then things started moving quickly. Adobe made an update to its terms and conditions in 2024, a decision that annoyed users who still hadn't gotten over the subscription-based model. The following year, Canva almost broke the industry when it announced Affinity would be free forever – it's since had seven million downloads. OK, not enough to be an immediate threat, but the professionals certainly started showing up. How could they not when it's free?

Cameron's argument for why Canva can afford to do this? Because the best model doesn't win, the best experience does. The best packet-switching didn't win the internet, and the best chip didn't win mobile. Canva is hoping the same is true for design, so it's opting for model-agnostic and focusing on the workflow around the tools instead.

If you've spent a lot of time hanging around the Affinity forums, you'll know the Blend Tool has been near the top of the wish list for years. As of today, it's live. You don't have to place every step by hand, for example, you can instead set bias and easing, run elements along a curve, and let Affinity work out how shapes morph, if you want to. It's joined by a streamlined Mask Tool, 3D model support, and Affinity's biggest photography update yet, including a feature that lets you turn a flattened image back into layers. And an INDD import means old InDesign files can come straight in, too.

Automation is something that runs through it all. In-app scripting, for instance, and a new Claude AI connector. Cavalry gets some love as well, plus 50-odd features and a coming-soon trick that lets teams edit text or colour in an animation inside Canva while the core scene stays locked.

If you're a designer, illustrator, photographer or motion artist, the message is clear: the free tools just got way better, and they now talk to each other.

But it's not just about that... For 13 years, the choice has been between a subscription model or compromising and going elsewhere. Canva's news today shows that by building a mass-market platform first, it was eventually able to bring a free, connected, professional-grade suite to creatives. And it's launched just when they needed it most.

No matter how much these tools evolve, Canva says there will always be a human behind the work. "The goal isn't just more content," said Cameron. "It's really about content that can only come from you."

A growing number of UX designers are shifting toward AI UX, which focuses on designing how people interact with intelligent systems rather than simply creating interfaces. Unlike traditional UX, AI UX requires designers to think about behaviors, trust, uncertainty, conversational flows, human oversight, and how AI should respond in different situations, while building on existing UX skills like user research, interaction design, and usability testing. The transition involves learning AI fundamentals, studying successful AI products, designing AI-powered experiences, and creating portfolio projects that demonstrate not just polished interfaces but thoughtful human-AI interactions and decision-making.

AI Motion Graphics Software

Turn your ideas into motion.

The AI motion graphics agent that turns ideas, designs, and audio into broadcast-ready animation.

No After Effects. No timeline. No learning curve.

From Prompt to Motion Graphics

Describe what you want. iArt handles the design, animation, and timing.

1. Describe — or upload

   Type what you want, or drop in images, Figma designs, documents. Set the voice — clone yours or pick from the library — and add BGM. iArt takes any combination.

2. Generate

   The AI agent builds your motion graphic scene by scene — layout, typography, timing, transitions. Watch it come together in seconds.

3. Export or share

   Download as MP4 up to 4K resolution, or grab a share link anyone can play instantly. Production-ready, no render queue.

Three starting points. One result: motion.

A concept in your head, a design on your screen, or a voice in your mic — iArt turns each into keynote-grade motion graphics.

Explain the unexplainable

Describe how something works — an algorithm, a process, a strategy. iArt doesn't illustrate your words. It understands your logic, then animates it scene by scene.

Your design, now moving

Drop in a Figma mockup or brand poster. Every color, every pixel stays — but now it moves like a senior animator spent a week on it. That week took five seconds.

You speak. The graphics dance.

Clone your voice, type the script. The graphics land on your beats, stretch on your pauses, punch on your emphasis. Sound and motion, married.

What takes hours in After Effects takes seconds here

iArt is the AI-powered After Effects alternative built for speed, not complexity.

Not another AI tool. A motion graphics engine.

Built for people who have the content but not the After Effects skills.

Create from anything

Notion docs, Figma files, YouTube links — iArt reads your tools natively. No export, no import, no friction.

Zero prompt anxiety

Drop in a design and iArt suggests "Product launch demo." Drop in text and it suggests "30-second explainer." You click, not craft.

Not footage. Motion graphics.

Sora generates cinematic video. iArt generates vector-precise animations with intentional typography, layout, and timing. Think Apple keynote, not stock footage.

One style, infinite assets

Generate an intro, a lower third, and an end card — they all share the same visual language. Your brand stays consistent without a style guide.

Audio to kinetic type

Feed in a podcast or lecture. iArt finds the highlights and wraps them in animated typography — ready for Reels, Shorts, or TikTok.

Motion Graphics Software FAQ

How is iArt different from After Effects?

After Effects requires manual keyframing and months of learning. iArt generates professional motion graphics from text descriptions in seconds — no timeline, no plugins, no learning curve. Think of it as the AI-powered After Effects alternative for non-animators.

How is iArt different from AI video tools like Sora or Runway?

They generate footage — people, landscapes, cinematic shots. iArt generates motion graphics — precise vector animations with intentional typography, layout, and timing. Think Apple keynote visuals, not stock video.

Do I need design or animation experience?

No. Describe what you want in plain language, or drop in an existing file. iArt handles the design, animation, and timing. If you can write a sentence, you can make a motion graphic.

What can I use as input?

Text descriptions, Figma designs, Notion documents, images, audio files, YouTube links — virtually anything. iArt connects to your existing tools via MCP integrations.

Can I edit the generated animations?

Yes. Every output is editable — adjust timing, colors, text, easing curves. Unlike most AI tools where you accept or regenerate, iArt gives you real control.

What export formats are supported?

MP4 (H.264). All exports support up to 4K resolution at 60fps.

How much does iArt cost?

Free to start — every new account gets a $2 signup credit, no credit card required. Paid plans start at $20/mo for Pro (~20–75 videos/month) and $50/mo for Ultra (~53–200 videos/month).

Can I use iArt in languages other than English?

Yes, across every part of iArt. Write prompts and chat with the AI agent in your language, add voice narration — cloned from your own voice or built-in TTS — in your language, and render on-screen text in the same language. Supported languages include English, Chinese, Japanese, Spanish, French, German, Portuguese, Arabic, and Korean. iArt interprets intent in whichever language you're most fluent in — no English required.

Meta One is a subscription service that offers enhanced AI and self-expression features across Instagram, Facebook, and WhatsApp. It includes single product plans as well as bundled offerings that combine features for individuals, creators, and businesses, starting at $2.99/month.

The US Senate has voted to block consideration of the Clarity Act, a piece of legislation that would create business-friendly rules for cryptocurrencies. The vote makes it unlikely the legislation will pass anytime soon, if ever. Many Republicans embraced the bill, but Democrats largely opposed it. The Democrats wanted stronger language to prevent public officials from using crypto to make money. President Trump generated $1.4 billion from a network of crypto businesses last year.

The teaser poster for Digger stands out by embracing a minimalist, typography-led design inspired by Saul Bass, using the movie's title to form the silhouette of a figure with a spade instead of relying on the typical collage of actors and action scenes. Rather than selling the film through star power, the symbolic design captures its central idea—a man whose destructive drilling triggers an ecological disaster—creating intrigue through a single memorable visual. It highlights how modern movie marketing can still benefit from timeless graphic design principles that prioritize storytelling and symbolism over spectacle.

Shoot-from-the-hip designers who rely on instinct without a repeatable process tend to stall above Senior, where promotion depends on influence, not just output. Writing that instinct into a rubric, such as where the eye lands first or what decision was made, turns judgment into something an AI model can use. One Lead designer, pushed to name the user, the task, and the decision he'd made instead of "we," began building his own checks after three projects.

Design work often requires sitting with an ambiguous problem rather than jumping to a fix, like resizing a button instead of asking why people aren't clicking it. AI tends to shortcut that stage, turning a vague brief into a flow, screen, and text within seconds. That halts exploration not because the answer is wrong, but because it's reasonable enough to accept, so AI works best widening the question first.

Fast Company named Make Good's open-source, fully 3D-printed Toddler Mobility Trainer the winner of its 2026 Innovation by Design Awards for accessible design.

Apple has added a new "Shift Left Edge" display option for the iPhone Duo that creates a larger left bezel to prevent cases from obscuring content on the outer screen, though it's unclear which cases will require it.