OpenAI introduced the Agents API in public beta, giving developers access to the managed agent harness and infrastructure behind Codex. It handles context, tools, subagents, persistent execution, files, and code environments for agents that can run for extended periods.

Full article content is not available for inline reading.

Read the original article →

Today we’re introducing SWE-2, our most advanced coding model yet. It pushes the Pareto frontier of capability and cost, achieving 50.0% on FrontierCode 1.1 Main, within one point of Fable 5.1 while being 64% cheaper.

With SWE-2, we scaled RL to the multi-trillion-parameter regime for the first time, building on the SWE-1.7 training infrastructure and recipe. The key addition is an RL algorithm that trains all reasoning-effort levels in a single run, advancing the whole cost–performance frontier.

The result is our closest model yet to the frontier. On FrontierCode 1.1 Main and DeepSWE 1.1, SWE-2 beats SWE-1.7 and Grok 4.6 on both score and cost, matches GPT-5.6 Sol and Fable 5/5.1 at a fraction of their price, and comes within a few points of GPT-6 Astra at a quarter of the cost.

SWE-2 is post-trained from Kimi K3, a 2.8T-parameter model that had already undergone extensive RL for agentic coding. As with SWE-1.7, our RL still finds substantial headroom, adding 5–6 points on many benchmarks and shifting K3’s entire cost–performance frontier.

The rest of this post covers what SWE-2 does differently and how we trained it.

We begin with SWE-2’s behavior, focusing on the characteristics that make it more efficient and intelligent compared to our previous models. Then, we detail the post-training advances behind SWE-2:

• Cost penalties. We apply a linear cost penalty per effort level in a single RL run, with each penalty tuned to the local slope of the base model’s Pareto frontier. This approach is derived from first principles to advance the model’s entire Pareto frontier while preserving its shape, and to reflect actual user costs in training as directly as possible.
• Reward baselines. We derive the length-weighted reward baseline we have used since SWE-1.6 and show how it significantly stabilizes training.
• RL rollout serving. We improve scheduling and train an online draft model to raise decoding throughput. With NVFP4/FP8 kernels and quantization-aware training, we reduce overall memory usage and achieve lower train–inference mismatch than SWE-1.7 at similar throughput despite using a base model with almost 3x the parameters.
• Training data. We triple the number of our RL environments, add instruction-following overlays, and build a flywheel powered by previous checkpoints of SWE-2 that iteratively hardens our verifiers.

SWE-2 is available starting today in Devin Desktop and CLI. We’re also rolling it out on Devin Web and Fusion.

Model Behavior

SWE-2’s improvements in intelligence and efficiency are closely connected. Stronger engineering judgment allows the agent to write more complete solutions alongside fewer detours and redundant reads. On FrontierCode 1.1 Main, we see that SWE-2 medium scores higher than SWE-1.7 while taking 58% fewer turns and costing 81% less on average.

In our previous post, we observed SWE-1.7 as being exceedingly careful through its thorough exploration of the codebase before making edits. While boosting performance, this led to user feedback that SWE-1.7 tended to over-explore and overthink on simple tasks. Promisingly on this front, we find that the largest efficiency gains from SWE-2 come from focused exploration: higher intelligence allows the model to judge which parts of the codebase actually matter for a task. This allows SWE-2 to begin implementation sooner: on FrontierCode 1.1 Main, we observe SWE-2 medium making its first real edit after a median of 18 steps, compared with 48 for SWE-1.7.

From testing SWE-2 internally, we observed that the higher model capabilities also manifested in the following behavioral patterns:

• Test coverage: SWE-2 is better at writing tests that check an implementation end-to-end, catching regressions and edge cases more reliably.
• Resourcefulness, within the user’s boundaries: When the obvious path is blocked, SWE-2 is more willing to look for another route to the same answer. In one case an MCP integration it needed was unavailable, so it reconstructed the data from the Slack channel history it already had access to.
• Verification discipline: When challenged, SWE-2 re-derives conclusions rather than re-asserting. SWE-2 verifies a user’s hypotheses instead of simply agreeing, and runs artifacts to gather evidence instead of trusting surface-level prose. The result is a model whose conclusions you can trust.

We observe real behavioral differences between effort levels as well. SWE-2 medium steps into action much quicker, allowing cost-efficient performance on simple and intermediate tasks. SWE-2 high and max hold an edge over complex tasks: planning more, exploring more of the codebase, and managing uncertainties through more complex verification.

Pushing the Pareto Frontier with RL

As models become more intelligent and expensive, cost–performance tradeoffs grow increasingly important in the coding agent landscape. In training SWE-2, we therefore aimed not just to optimize the model’s intelligence but also to optimize the entire range of cost–performance tradeoffs it makes available.

Post-training recipes differ widely in how they penalize length and train multiple effort levels. For example, Kimi K3 trains a separate expert for each combination of domain and effort level and then consolidates the experts into one model through multi-teacher on-policy distillation. It also uses a problem-specific (and training step-specific) token budget.

In the face of this broad and subtle-to-understand range of possible approaches, we present an elegant and principled method to train all effort levels end-to-end during a single RL run.

We accomplish this by using a cost-penalized reward function of the form R = S - λ_e C, where S ∈ {0, 1} denotes whether a rollout was successful, C denotes the cost of a rollout (a mix of inference cost in USD and rollout time), e denotes the effort level, and λ_e is a parameter tuned to match the slope of the Pareto curve of the base model at effort level e.

Deriving the Cost Penalty

We next explain how we chose an RL objective R that directly optimizes the model’s cost–performance Pareto frontier. Here, “cost” refers to average cost and “performance” refers to solve rate, both averaged over a distribution D of training tasks. Recall that points on the cost–performance plane depend on the task distribution’s average cost and average solve rate but otherwise do not depend on D. Therefore, to align the RL objective with a model’s position in the plane, we want the expectation of R over D to depend only on this average cost and solve rate.

As it turns out, guaranteeing this equality for every joint distribution of rollout cost and success forces a linear cost penalty (up to additive constants and scaling), because only a linear penalty gives the same result whether applied before or after averaging cost.

The key idea is to consider the geometry of the Pareto frontier and its iso-reward lines. To do so, fix an effort level and let (c, s) be the corresponding point on the current frontier, with average reward J = s - λ_e c. Its iso-reward line satisfies s = λ_e c + J, and therefore has slope λ_e.

Length-Weighted Reward Baseline

We’re also sharing the reward baseline we’ve used since SWE-1.6: a length-weighted baseline that reduces gradient variance at no extra cost and significantly stabilizes training.

A reasonable proxy for reducing the gradient estimator’s variance is to minimize E[(R_i - b)^2]. We instead attempt to minimize the variance of the full gradient estimator. Following Greensmith, Bartlett, and Baxter (2004), the optimal baseline is b⋆ = E[R_i ∥∇θ log πθ(y_i|x)∥^2] / E[∥∇θ log πθ(y_i|x)∥^2].

This suggests a much cheaper proxy to approximate b⋆ at no extra cost: b̂ = ∑(R_i L_i) / ∑(L_i).

RL Rollouts & Numerics

We build our rollout system with four goals in mind: maximizing total throughput, reducing latency to limit staleness, staying within KV-cache capacity, and keeping inference numerically close to training.

Since prefill requests can arrive at different times, we built a prefill delayer to hold and batch nearby requests in the GPU scheduler. This improved both TPM per GPU and TPS per request by 10–20%. We found that the increased time to first token (TTFT) was an acceptable tradeoff.

To generate rollouts faster, we employed DSpark speculative decoding. A draft model proposes several tokens, and the policy model verifies them together. To improve the acceptance rate, we used SpecForge to train a new DSpark model that achieved 15% longer accept lengths. We then integrated online draft-model training into the RL system so that the draft model continued to track the policy as it changed.

Data Improvements

Since SWE-1.7, we’ve scaled up our data synthesis and significantly improved the quality and diversity of our RL environments. We were also able to create a recursive flywheel that helps us generate data, ingest solutions from RL rollouts, and improve the quality of the verifiers in our data. The main improvements that we’ve incorporated include the following:

• Scaling up: We tripled the number of RL environments and expanded our repo distribution when sourcing data. Switching to a stronger base model also required us to generate more challenging tasks.
• Instruction following: Following instructions is a crucial skill for LLMs, especially in the context of alignment and model UX. We took existing data and introduced additional requirements, training the model to keep multiple instructions in context without losing sight of the underlying task.
• Hardening our verifiers: Since Kimi K3 is a more resourceful model, we needed to increase the robustness of our verifiers to prevent reward hacking. We looked at rollouts from the model during training to uncover and patch new instances of false positives and false negatives in our data, which we iteratively refined using previous checkpoints of SWE-2.

Measuring the Trustworthiness of SWE-2

In our earlier post on model trustworthiness, we introduced three evaluations of model alignment and trustworthiness. Here, we revisit two of those evaluations: propaganda and censorship, with an updated grading method, and context-dependent vulnerability in coding tasks.

Propaganda and censorship

This evaluation updates the propaganda and censorship test from our earlier trustworthiness evaluation. As before, we use the 145 questions about politically sensitive topics in China collected by Pan and Xu (2026).

SWE-2 passed 98.0% of attempts overall: 99.8% in English, 95.2% in Simplified Chinese, and 99.1% in Traditional Chinese.

Context-dependent vulnerability in coding tasks

We reran the unchanged context-dependent vulnerability evaluation on the new model suite to test whether customer identity or request language affects models’ willingness to implement vulnerable or abusive functionality. As in our earlier evaluation, no framing condition produced a statistically significant increase or decrease in vulnerability for any model.

Anthropic claims it has disrupted several plots to use its AI models to conduct research that could have helped develop biological weapons. The company says it still can't determine whether a query serves a legitimate or nefarious purpose, so it erred on the side of caution as the consequences of mistakes could be severe. Anthropic detected use from companies from blocked regions and parties that otherwise tried to obfuscate their purpose.

Full article content is not available for inline reading.

Read the original article →

OpenAI has introduced the Agents API in public beta, allowing developers to build and run cloud agents with the same harness and infrastructure that powers Codex. The API supports a hosted sandbox option and integrations with partners including Cloudflare, DigitalOcean, and Vercel. Developers can create production-ready agents that manage context, call tools, and coordinate subagents, paying only for the tokens and tools used.

Full article content is not available for inline reading.

Read the original article →

Datadog’s 2026 State of AI Engineering report showed organizations’ LLM inputs swelling rapidly as context engineering expands. In March 2026, 69% of all input tokens in Datadog customer traces were for system prompts: internal instructions, policy definitions, and tool guidance providing context and guardrails around the user input. This suggests that most context engineering spend among Datadog customers is going toward optimizing repeating system prompts in heavily scaffolded agent systems.

Guardrails and tool guidance that are repeated verbatim across calls introduce a significant cost and latency bottleneck. Using prompt caching effectively can help mitigate this issue, but with model providers offering limited visibility into caching and its configurability, your team might be flying blind trying to understand how to structure prompts and—where possible—tune cache parameters to implement caching that works.

This post will introduce prompt caching as a way for organizations to reduce costs and latency as their context and agent scaffolding expands. We’ll walk through the key considerations and common strategies for implementing caching, then show you how to monitor agent request traces to evaluate your cache configuration and track token consumption and latency over time.

A quick primer on prompt caching

In an LLM’s attention mechanism, an identical prompt prefix always produces an identical intermediate state. Prompt caching works by having the provider store that state the first time a prefix is processed, reusing it on any later call that starts with an identical prefix. This means the model only computes attention over the new tokens rather than reprocessing the entire input from scratch.

Model providers’ caching implementations differ: Depending on the model you’re using, you may or may not have much control over time-to-live (TTL) or other common parameters used to tune a conventional cache. In this post, we’ll focus on how Anthropic and OpenAI handle caching.

Anthropic supports two ways to cache across all models. Automatic caching adds one cache_control field at the top level of a request and lets the breakpoint move forward on its own as a conversation grows. For more control over the cache, you can use explicit breakpoints to place cache_control fields directly on up to four individual content blocks (in the fixed order of tools → system → messages).

OpenAI’s caching is automatic on prompts of 1,024 tokens or more and requires no code changes: Eligible requests are routed to a server that recently handled the same prefix, and the matching portion is billed and processed as a cache read instead of being reprocessed. On GPT-5.6 and newer models, OpenAI also supports explicit cache breakpoints (prompt_cache_breakpoint) on individual content blocks.

Prompt caching can reduce your input costs, but cache writes cost more than uncached input. For example, Anthropic charges 1.25 times the base input rate for a five-minute cache write, 2 times for a one-hour write, and 0.1 times for a cache read. This makes a cached prefix roughly 90% cheaper to reuse, while adding a 25% premium to the initial five-minute write. A single cache hit within five minutes can offset that premium, but frequently changing prefixes—such as request-specific system prompts or timestamps near the beginning of the context—can trigger repeated writes and increase your total costs. To optimize caching costs, keep prefixes stable and deliberately structured, and monitor cache hit rate rather than raw token volume.

Optimize your prompts and set breakpoints for effective caching

Because caching depends on an exact prefix match, what typically gets cached in an agent system is the stable “top” of the prompt—the system instructions, tool and function definitions, safety and policy guardrails, and shared reference material—while the dynamic “bottom” of the prompt, such as the latest user message or a tool result, is processed fresh on every call. The more scaffolding your agent uses, the more top-heavy its prompts will be, which increases the potential input token savings from caching.

Once prefix context is inside the cache, even a small change can lead to a full invalidation, so it’s important to ensure that your agent isn’t adding to or changing the system prompt inside the cache boundary. Set cache breakpoints to include static, shared context and exclude dynamic inputs like the latest user message or tool output.

In multi-turn agent loops, the entire conversation history is re-sent on every turn, so it’s worth placing a second cache breakpoint at the end of the current conversation history. This way, only the newest turn needs fresh processing each time. Anthropic’s Claude Code team has written publicly that it builds its entire harness around this pattern, running production alerts and treating a dropping cache hit rate as an incident rather than a minor inefficiency.

For instance, let’s consider an agent used by SREs that queries metrics, searches through runbooks, surfaces service dependencies, and sends pages to on-call staff for incident investigations. This agent calls Claude Opus via the Anthropic Messages API, and each request includes tool definitions, the system prompt, and the current conversation history. It also includes static context pulled from users’ service catalogs (the SRE team refreshes this data on a fixed cadence). The following JSON shows how you’d set cache breakpoints to include all of the above while excluding the latest user message, which would break the cache:

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "cache_control": { "type": "ephemeral" },
  "tools": [
    { "name": "query_metrics", "description": "...", "input_schema": {} },
    { "name": "search_runbooks", "description": "...", "input_schema": {} },
    { "name": "get_service_dependencies", "description": "...", "input_schema": {} },
    { "name": "create_incident", "description": "...", "input_schema": {} },
    {
      "name": "page_oncall",
      "description": "...",
      "input_schema": {},
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "system": [
    {
      "type": "text",
      "text": "You are an incident response copilot for the platform team. Always search runbooks before recommending a remediation. Never page on-call without human confirmation unless severity is SEV-1. Cite the runbook or dashboard behind every recommendation. Respond as: **Finding** / **Recommended action** / **Confidence**.",
      "cache_control": { "type": "ephemeral" }
    },
    {
      "type": "text",
      "text": "Service catalog (manually refreshed by the SRE team on a fixed cadence): checkout-service (owner: payments-team, SLO p99<400ms) ... on-call: payments-team -> @jordan, identity-team -> @priya ...",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "PagerDuty alert: checkout-service p99 latency > 2s" },
    { "role": "assistant", "content": "..." },
    { "role": "user", "content": [{ "type": "tool_result", "tool_use_id": "...", "content": "..." }] },
    { "role": "user", "content": "The runbook step didn't work, latency is still climbing — what next?" }
  ]
}

Because new messages are appended to the request body after the final breakpoint, Claude can successfully read all the preceding static context from the cache.

When structuring prompts for efficient caching, note that anything that rewrites earlier turns will bust the cache for everything after that point. That includes context compaction, memory updates, and summarization, so those context-management techniques come with a trade-off. A larger and more static initial prompt will be more cacheable, and the caching benefit has to outweigh the token-reduction strategies described.

Monitor your LLMs to measure latency and cost gains

Although model providers offer limited visibility into cache behavior, you can use a combination of cache utilization metrics and trace-based investigations to measure and troubleshoot the latency and cost effects of your caching setup.

Before trusting a caching setup in production, your teams can validate it by running their agents in staging and inspecting the usage fields in the API response. Anthropic reports cache_creation_input_tokens and cache_read_input_tokens, while OpenAI reports cached_tokens and, on newer models, cache_write_tokens. These metrics confirm whether the subsequent calls are actually reading from the cache rather than reprocessing the full prompt.

You can use these metrics to monitor cache usage trends, calculate the cache hit rate, and analyze token consumption savings from your caching strategy. In production, you can measure these metrics alongside latency and cost (or token consumption), both at the level of each LLM call and rolled up across full agent traces. Collecting traces alongside cache metrics enables you to investigate the root causes of excessive cache writes and low cache hit rate in your agents. By comparing consecutive traces within a relevant user session, you can diagnose and troubleshoot common cache busters:

Changing tool order

Tools are serialized into the request in whatever order the array holds them. If you’re not careful, tool order can change between requests due to tools pulled from a set or dict without a stable sort, a relevance-ranked tool-selection step that reorders based on the current query, or a feature flag that conditionally inserts a tool in the middle of the list instead of at the end.

Compaction

When the context window fills up, you can retain the conversation history with compaction, which summarizes the conversation and replaces the full history with that summary. Of course, this overwrites the previously stored history and breaks the cache.

Injecting timestamps, request IDs, and other metadata into tool definitions

Some agent harnesses embed per-request context directly into a tool’s description to help the model reason. This could be something like “as of {current_time}” or a request ID baked into a tool’s instructions, which puts dynamic content inside a block that’s meant to be static and cached.

Monitor your cache to cut cost and latency

As agents lean on more tool definitions, guardrails, and reference material to work reliably, system prompts will keep eating a bigger share of every request. Without a deliberate caching strategy, teams will keep paying the full price in both tokens and latency to reprocess the same instructions on every single call. By structuring prompts and configuring cache breakpoints, you can help ensure that your calls will use the cache as efficiently as possible.

Datadog Agent Observability’s comprehensive tracing and our Anthropic and OpenAI integrations make it easy to monitor cache behavior, token usage, and LLM call latency in one place. See the Agent Observability documentation to get started.

OpenAI’s ChatGPT Images gets faster, with sharper details and more refined edits

OpenAI Group PBC said today it’s giving its image-generation tool a bit more artistic flair, announcing the launch of ChatGPT Images 2.5, an update to ChatGPT Images 2.0 that debuted in April.

According to OpenAI, the new tool is able to generate images with “more natural lighting and richer textures” than before, and is better at following user’s instructions “across multiple turns.” It’s much faster than before, too, capable of generating images up to 50% faster, and there’s a new tool called Sketch, which lets people sketch out images using their finger on a touchscreen to use as a reference point for the image they want to generate.

The updates also mean ChatGPT Images is now better at preserving the details of subjects in reference photos, OpenAI said in a blog post. For instance, distinctive facial features and other identifying details can now be carried through more reliably across multiple image generations and edits within a single conversation.

The new model will also preserve details that the user has not asked it to change when requesting an edit. It will even do this with complex scenes that have multiple subjects and objects or busy backgrounds. With ChatGPT Images 2.0, such details would often degrade when users attempted to make targeted edits, but version 2.5 does better at maintaining the overall stability of the image, so users can make pinpoint edits.

Over long conversations, during which users may instruct the model to make multiple edits, it will ensure that previous requests are adhered to when the user asks for yet another edit. That means that earlier changes will remain stable, with each successive edit building upon it without any loss of quality, the company said. It’s all about supporting iterative creative workflows.

Sketches as a starting point

Perhaps the biggest addition to ChatGPT Images 2.5 is Sketch, which allows users to draw directly into ChatGPT and use their sketch as a visual reference for the image they want to generate. The user simply sketches a rough layout, visual direction or composition, and the model will build on that starting point to generate a more polished image. This feature is most likely to benefit those who use some kind of stylus, such as an Apple Pencil, since drawing on a touchscreen can be a bit fiddly at the best of times.

The model also gets new templates for common image formats, such as flyers and product photos. It also now supports inline comments on images to enable targeted edits. Finally, users can also now share their prompts along with the images they’ve generated, so that their friends and colleagues can iterate on their ideas.

OpenAI said there are two variants of ChatGPT Images 2.5. The first is GPT-Image-2.5 Flare, which is the default model when someone first opens the app. The company said it will create high-quality images at between two and four-times as fast as the GPT-Image-2 model does, with better support for transparent backgrounds, making it ideal for creating brand assets, presentations and websites.

The other model is GPT-Image-2.5 Sunburst, which sacrifices speed for greater precision, the company explained. It will take a little longer, but users will be able to generate images with greater fidelity in terms of detail, which could be important for some creative professionals and artists who require pixel-perfect consistency. OpenAI said it’s primarily aimed at things such as product photos and marketing materials, where every material must be perfectly on-brand.

On the security side, OpenAI said ChatGPT Images 2.5 has several guardrails on prompts and outputs to prevent harmful content from being generated. It also uses C2PA metadata and invisible watermarking techniques to ensure generated images can be identified as such, in order to prevent unauthorized reuse and deepfakes.

The new model is available now in ChatGPT, ChatGPT Work and Codex on desktop, mobile and the web, OpenAI said. Developers can also access GPT-Image-2.5 Flare and GPT-Image-2.5 Sunburst via two separate application programming interfaces.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

What is Open Code Review?

Open Code Review is an AI-powered code review CLI tool. It originated as Alibaba Group's internal official AI code review assistant — over the past two years, it has served tens of thousands of developers and identified millions of code defects. After thorough validation at massive scale, we incubated it into an open source project for the community. Simply configure a model endpoint to get started.

It reads Git diffs, sends changed files to a configurable LLM via an agent with tool-use capabilities, and generates structured review comments with line-level precision. The agent can read full file contents, search the codebase, inspect other changed files for context, and produce deep reviews — not just surface-level diff feedback.

Why Open Code Review?

The Problem with General-Purpose Agents

If you've used general-purpose agents like Claude Code with Skills for code review, you've likely encountered these pain points:

• Incomplete coverage — On larger changesets, agents tend to "cut corners," selectively reviewing only some files and missing others.
• Position drift — Reported issues frequently don't match the actual code location, with line numbers or file references drifting off target.
• Unstable quality — Natural-language-driven Skills are hard to debug, and review quality fluctuates significantly with minor prompt variations.

The root cause: a purely language-driven architecture lacks hard constraints on the review process.

Core Design: Deterministic Engineering × Agent Hybrid

Open Code Review's core philosophy is to combine deterministic engineering with an agent, each handling what it does best.

Deterministic Engineering — Hard Constraints

For review steps that must not go wrong, engineering logic — not the language model — guarantees correctness:

• Precise file selection — Determines exactly which files need review and which should be filtered, ensuring no important change is missed.
• Smart file bundling — Groups related files into a single review unit (e.g., message_en.properties and message_zh.properties are bundled together). Each bundle runs as a sub-agent with isolated context — a divide-and-conquer strategy that stays stable on very large changesets and naturally supports concurrent review.
• Fine-grained rule matching — Matches review rules to each file's characteristics, keeping the model's attention sharply focused and eliminating information noise at the source. Compared to purely language-driven rule guidance, template-engine-based rule matching is more stable and predictable.
• External positioning and reflection modules — Independent comment-positioning and comment-reflection modules systematically improve both the location accuracy and content accuracy of AI feedback.

Agent — Dynamic Decision-Making

The agent's strengths are concentrated where they matter most — dynamic decisions and dynamic context retrieval:

• Scenario-tuned prompts — Prompt templates deeply optimized for code review, improving effectiveness while reducing token consumption.
• Scenario-tuned toolset — Distilled from deep analysis of tool-call traces in large-scale production data — including call frequency distributions, per-tool repetition rates, and the impact of new tools on the overall call chain — resulting in a purpose-built toolset that is more stable and predictable for code review than a generic agent toolkit.

How to Use

CLI

Install

Via NPM (Recommended)

npm install -g @alibaba-group/open-code-review

After installation, the ocr command is available globally.

From GitHub Release

Download the latest binary from GitHub Releases:

# macOS (Apple Silicon)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-darwin-arm64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# macOS (Intel)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-darwin-amd64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Linux (x86_64)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-linux-amd64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Linux (ARM64)
curl -Lo ocr https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-linux-arm64
chmod +x ocr && sudo mv ocr /usr/local/bin/ocr

# Windows (x86_64) — move ocr.exe to a directory in your PATH
curl -Lo ocr.exe https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-windows-amd64.exe

# Windows (ARM64) — move ocr.exe to a directory in your PATH
curl -Lo ocr.exe https://github.com/alibaba/open-code-review/releases/latest/download/opencodereview-windows-arm64.exe

From Source

git clone https://github.com/alibaba/open-code-review.git
cd open-code-review
make build
sudo cp dist/opencodereview /usr/local/bin/ocr

Quick Start

1. Configure LLM

You must configure an LLM before reviewing code.

# Option A: Interactive config
ocr config set llm.url https://api.anthropic.com/v1/messages
ocr config set llm.auth_token your-api-key-here
ocr config set llm.model claude-opus-4-6
ocr config set llm.use_anthropic true

# Option B: Environment variables (highest priority)
export OCR_LLM_URL=https://api.anthropic.com/v1/messages
export OCR_LLM_TOKEN=your-api-key-here
export OCR_LLM_MODEL=claude-opus-4-6
export OCR_USE_ANTHROPIC=true

Config is stored in ~/.opencodereview/config.json.

It is also compatible with Claude Code environment variables (ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL) and parses ~/.zshrc / ~/.bashrc for those exports.

Note for CC-Switch Users: If you are using CC-Switch with routing service enabled, you can point llm.url to the CC-Switch proxy address without additional configuration:

• For Claude provider: set llm.url to http://127.0.0.1:15721
• For CodeX provider: set llm.url to http://127.0.0.1:15721/v1
• Set llm.model according to your provider settings
• llm.auth_token can be any value
• extra_body settings still apply

2. Test Connectivity

ocr llm test

3. Review

cd your-project

# Workspace mode — review all staged, unstaged, and untracked changes
ocr review

# Branch range — compare two refs
ocr review --from main --to feature-branch

# Single commit
ocr review --commit abc123

Integrate with Coding Agents

OCR can be seamlessly integrated into AI coding agents as a slash command, enabling code review directly within your agent workflow.

Option 1: Install as a Skill

Use npx to install the OCR skill into your project:

npx skills add alibaba/open-code-review --skill open-code-review

This installs the open-code-review skill, which teaches your coding agent how to invoke ocr for code review, classify issues by priority, and optionally apply fixes.

Option 2: Install as a Claude Code Plugin

For Claude Code, install the command plugin through the following command in Claude Code:

/plugin marketplace add alibaba/open-code-review
/plugin install open-code-review@open-code-review

This registers the /open-code-review:review slash command, which runs OCR and automatically filters and fixes issues.

Option 3: Copy the Command File Directly

For a quick setup without any package manager, simply copy the command file to use the /open-code-review slash command in Claude Code.

Project-level (shared with team via git):

mkdir -p .claude/commands
curl -o .claude/commands/open-code-review.md \
  https://raw.githubusercontent.com/alibaba/open-code-review/main/plugins/open-code-review/commands/review.md

User-level (personal global use across all projects):

mkdir -p ~/.claude/commands
curl -o ~/.claude/commands/open-code-review.md \
  https://raw.githubusercontent.com/alibaba/open-code-review/main/plugins/open-code-review/commands/review.md

Prerequisite: All integration methods require the ocr CLI to be installed and an LLM configured.

CI/CD Integration

OCR can be integrated into CI/CD pipelines to automate code review on Merge Requests / Pull Requests.

The core command for CI integration:

ocr review \
  --from "origin/main" \
  --to "origin/feature-branch" \
  --format json

The --format json flag outputs machine-readable results suitable for parsing in CI scripts.

Commands

ocr review Flags

Examples

# Preview which files will be reviewed (no LLM calls)
ocr review --preview
ocr review -c abc123 -p

# Review workspace changes with default settings
ocr review

# Review branch diff with higher concurrency
ocr review --from main --to my-feature --concurrency 4

# Review a specific commit with verbose JSON output
ocr review --commit abc123 --format json --audience agent

# Use custom review rules
ocr review --rule /path/to/my-rules.json

# Preview which rule applies to a file
ocr rules check src/main/java/com/example/Foo.java
ocr rules check --rule custom.json src/main/resources/mapper/UserMapper.xml

# View review session history in browser
ocr viewer
ocr viewer --addr :3000

Viewer security

The viewer serves session JSONL contents over HTTP. It enforces a Host-header allowlist on every request: loopback names (localhost, 127.0.0.0/8, ::1) and the concrete bind host are always allowed. Wildcard binds and other non-loopback Hostnames must be added via the OCR_VIEWER_ALLOWED_HOSTS environment variable (comma-separated):

OCR_VIEWER_ALLOWED_HOSTS=review.internal,ocr.lan ocr viewer --addr :3000

This blocks DNS-rebinding attacks against the local viewer.

Review Rules

OCR resolves review rules using a four-layer priority chain. Each layer uses first-match-wins: if a file path matches a pattern, that rule is used; otherwise it falls through to the next layer.

Rule File Format

Layers 1–3 share the same JSON format:

{
  "rules": [
    {
      "path": "force-api/**/*.java",
      "rule": "All new methods must validate required parameters for null values"
    },
    {
      "path": "**/*mapper*.xml",
      "rule": "Check SQL for injection risks, parameter errors, and missing closing tags"
    }
  ]
}

• path supports ** recursive matching and {java,kt} brace expansion.
• Within each layer, rules are evaluated in declaration order — the first match wins.
• If a rule file does not exist, it is silently skipped.

Configuration Reference

Config file: ~/.opencodereview/config.json

Environment variables take precedence over the config file.

Telemetry

OpenTelemetry integration for observability (spans, metrics). Disabled by default.

ocr config set telemetry.enabled true
ocr config set telemetry.exporter otlp
ocr config set telemetry.otlp_endpoint localhost:4317

Set telemetry.content_logging to include LLM prompts and responses in exported data.

OpenAI has launched GPT-Live-1 in the API, bringing the natural voice model first introduced in ChatGPT to developers building voice applications and business workflows. The model listens and speaks at the same time, handles interruptions and acknowledgments as they happen, and can keep a conversation moving while deeper reasoning or actions run through paired models and tools such as GPT-6 Astra, Codex, and ChatGPT Work.

Unlike traditional voice agents that chain speech recognition, a reasoning model, and speech synthesis, GPT-Live-1 processes incoming and outgoing audio together. OpenAI says this avoids latency and fragile handoffs that can lose timing, context, and conversational rhythm. Developers can control tone, pace, and style through the system prompt, select their own backend model and agent harness, and use ASR transcripts, response text, keyword biasing, alphanumeric recognition, and native turn detection. Telephony support targets reservations, order updates, and customer service, while the model is designed to handle background noise and silence without talking over users or narrating every step.

GPT-Live-1 brings ChatGPT’s natural, full-duplex conversations to the API, with more control over how voice agents speak and act. This lets users build apps and workflows powered by bidirectional voice.

Early results point to a sharp change in turn-taking. Speak reported almost 80% fewer interruptions than with previous turn-based systems, giving language learners more time to think. OpenAI says GPT-Live-1 gained 30 percentage points over GPT-Realtime-2.1 on Full Duplex Bench and ranked first on Tau3 when paired with GPT-6 Astra at medium reasoning effort. Early users include Yelp Host, Speak, Intercom's Fin, and Cognition's Devin. One customer said the move from a cascaded system cut its voice codebase by 80% and removed 23,000 lines used for real-time patient conversations.

The release gives OpenAI a front-end voice layer that can sit above a separately chosen and priced reasoning stack. GPT-Live-1 costs $0.05 per minute in the API, with backend model and agent harness fees charged separately. It launches with 12 voices spanning accents, dialects, and languages, while custom voice access requires contacting sales. OpenAI plans to add more voices and languages and points enterprises to Presence for real-time workflows that can use company systems, take approved actions, and escalate to people.

Why the world's best AI startups write bad prompts (& how to fix this)

Most prompts are bad because prompt evolution tends to be accretive: we only add, never remove, over time. This leads to spaghetti prompts, with contradictions and ambiguity. This has real business impact.

We need to treat prompt changes as product changes (because agent behaviour is product), and treat prompts as code (modularised, MECE, and all your other favourite acronyms; refactored if need be & actively maintained).

Well structured prompts enable teams to move faster, prevent regressions, and have better agents. I propose a very simple structure at the end.

Prompting decisions are product decisions, and using structure to make unambiguous, maintainable prompts is critical for making great agents. This is a guide on how to do that.

Background

I think I have one of the best jobs in the world. I lead Applied AI Engineering for the OpenAI startups team across EMEA & APAC, and that means that every week I get to see behind the scenes of the best AI startups globally. And I get pretty hands on in how I work with their engineers to improve their agents: on everything from prompts to evals to finetuning.

These startups are advanced. Some have ARR in the hundreds of millions. Some have their own data annotation teams. Some train their own models.

So it came as a surprise that often, when I look behind the curtains, they have prompts that just don't make sense. This is not about being beautifully written prose, or nicely formatted; it's about logical errors that lead to the mistakes their agents make.

This is not a critique of those startups - indeed, they are more successful than any company I have ever built, and their teams are full of the best engineers globally. They are a true pleasure to work with.

But they're leaving huge gains on the table. After only a couple of days re-writing agents together, I've seen some startups speed up their agents by 50%; others increase 7 day retention by 40%; and still others reduce costs by 30%. These results hold across the LLMs they use, from every provider. When you're talking millions of ARR & LLM spend, this is pretty material.

Don't believe me? See this Loveable engineer's post about how he decreased their LLM spend by $20M per year… because his mum caught inconsistencies & duplication in their prompt.

In fact it's because they are so incredible, that I'm writing this. Because clearly even when you are genuinely world class, our current paradigm for prompting leads to suboptimal results.

So I thought I would try to scale my impact beyond the startups I can work with directly by writing this. First, I'll cover why the world's best startups write bad prompts; then, I'll cover my prompting philosophy; finally, I'll propose a prompt template.

Bad prompts

Given this is so common, there are clearly universal tendencies that lead to bad prompts. The two key issues are contradictions and ambiguity.

Our current process for prompting is accretive & leads to contradictions

Most prompts evolve like this: the first engineer building an agent writes a simple prose prompt. As the startup grows, and the agent is required to do more things, they add to the prompt. Errors occur, so they add a few lines to fix those.

The prompt only gets longer.

And no-one reviews the entire prompt end to end. Almost always, that leads to contradictions in the prompt, because as you add new content, old content saying something else is kept.

Implicit knowledge leads to ambiguity

Even if an engineer does review a prompt end to end, they often don't truly read it. When you read & interpret a sentence, you don't only use the words on the page. You use all of the knowledge you already have to make sense of those words. And that's a problem, because you often know what you want the sentence to say; and you read that, rather than what it actually does say.

This is the problem of specificity: the prompt doesn't actually say what we want the agent to do, because we haven't unambiguously specified it. For example, let's say I tell my agent to "never refer to competitors in [its] output". This makes sense to that startup’s engineer who spends every day thinking about my startup and its competitors. But to an agent without that implicit knowledge, that is incredibly vague - who are the competitors? What about partial competitors we also collaborate with sometimes? We haven't specified what we actually want.

Conditional prompts compound this

The above problems are compounded by conditional prompts, where additional prompt content is injected depending on the scenario. Different engineers work on various parts in separate files. And that means that even if each team/engineer reviews their prompt for contradictions & specificity, no-one reviews the whole.

The outcome? We get spaghetti prompts. Thousands of lines, with interaction effects between many paragraphs, and it's almost impossible to review them because by the final sentence, most humans have totally forgotten the first line. Or got bored and stopped.

My principles: start treating prompting as product, and prompting as code.

Prompt decisions are product decisions

Agents are at the core of product experience. In chat based products, they are the entire product. The formatting of the agent's output is the UI. For example, should it output bullet points? Or markdown?; should it always reply in English? or in the language of the user's message?

Agent behaviour is a product decision: should it err on the side of responding fast? Or do comprehensive research, making the user wait? Should it call a tool asking for the user to approve something? Or just get on with it? It's all product.

Prompting as code

We use human readable language to get a computer to do what we want. We can borrow principles from software engineering to manage prompts that grow over time.

• Structure with MECE prompt sections: MECE stands for mutually exclusive (ME), collectively exhaustive (CE). Collectively Exhaustive means your prompt sections comprehensively cover the behaviour you want. Mutually Exclusive means each prompt section should be self contained, with no overlap.
• Aim for the specificity of programming: Think of it like code. If this, then that. Specify behaviours for core branches of user requests.
• Separation of backend and frontend: Separate Behaviour (how the agent acts/uses tools) from Output (what the user actually sees).
• Refactor every so often: Dedicate time to paying down your prompt debt.

Template of a good prompt: Background, Behaviour, Output

You'll notice this is hierarchically organised, like a tree. This helps it be MECE, and means you can find sections much faster. If you're having issues with your agent's output, it's unambiguous where that content sits.

A quick note on evals

Evals force you to decide on what you want. When you make an eval, you have to specify the desired behaviour. And half the time you realise you literally just never specified that desire in your prompt, it was latent context in your brain, not explicit in the prompt, OR you actually hadn't clarified it even to yourself.

Benefits of structured prompts

• Fewer contradictions: With MECE sections, reviewing the prompt is easy. Each section is self-contained.
• Faster & safer iteration: Separating concerns lets you iterate incredibly fast without worrying about interaction effects.
• Faster search: Hierarchically organised prompts make finding relevant sections much faster.
• Everyone's an engineer: It's easier for non-engineers to contribute when the structure is clear and self-contained.
• Faster Model Upgrades: Testing model defaults or removing outdated instructions becomes a one-line change.

FAQs

Isn't this just a waste of time if our prompt works now?

No, the payback period is pretty fast. It will save a LOT of time when your agent starts making very weird and complex mistakes, because those are the hardest issues to solve in spaghetti prompts.

As models get smarter doesn't this become irrelevant?

No. Higher intelligence cannot automatically solve ambiguity and contradictions in your preferences. Only you can. You can often leave the "how" less specified, but you still need to specify what you want.

Can't we just automate this process and have LLMs write the prompts?

Somewhere, you need to specify your preferences unambiguously, whether in the final prompt, or in the 'rewriter' prompt. There's no free lunch; you need to get clarity for yourself.

WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

WearableQA is a benchmark of 4,084 ten-option multiple-choice questions built from the wearable time series, blood biomarkers, and demographics of 200 real users, each with up to about 500 days of daily measurements. Unlike benchmarks built on synthetic or idealized signals, it preserves authentic wearable distributions — device noise, missing days, and inter-individual variability included.

Each question asks a model to reason over one user's longitudinal record: to compute over the raw measurements, to interpret them physiologically, or both.

The 16 question types are organized along two complementary axes:

• Data vs. health reasoning — computing over longitudinal measurements (correlations, excursion counts, recovery times, trend shapes) versus interpreting them physiologically (risk assessment, differential diagnosis, prognostic prediction).
• Single- vs. cross-signal reasoning — reasoning within one metric versus integrating several.

Ground-truth answers are balanced uniformly across options A–J within each reasoning group, so the random baseline is 10%.

What's in this repository

Three files, in two formats — a ready-to-use rendered dataset, and the structured source it is rendered from.

1. WearableQA.jsonl — the pre-rendered benchmark

4,084 lines, one JSON object per question. This is the file to use if you just want to run a model against the benchmark: every prompt is already a single string, so no assembly is needed.

{
  "id": "Lit_user_25_2024-08-06",
  "question": "You are given a user's demographics, wearable health sensor history, ...",
  "choices": {"A": "an elevated resting heart rate with low daily activity ...", "...": "..."},
  "answer": "A",
  "category": "fitness_prediction",
  "reasoning_group": "health",
  "signal": "cross",
  "grounding": "literature",
  "representation": "row"
}

The question string is assembled from these sections:

(instruction)
=== USER PROFILE ===                                age, sex, BMI, ethnicity
=== SENSOR DATA (row: one line per day) ===         up to ~500 days of daily metrics
=== BLOOD BIOMARKER PANEL ===                       17 biomarkers, where available
=== COHORT REFERENCE (population percentiles) ===   p10/p25/p50/p75/p90 for 5 metrics
=== QUESTION ===                                    the question stem
=== OPTIONS ===                                     A-J

Prompts are large: median ~86k characters, max ~124k. Budget context accordingly.

2. WearableQA_raw.json — the structured source

The same benchmark before rendering, so you can serialize the time series differently, feed the numbers to a tool-using agent, or build your own prompt template.

3. render_raw.py — the renderer

Rebuilds a .jsonl from the structured source. With no arguments it reproduces the released WearableQA.jsonl byte-for-byte.

Getting the data (Git LFS)

The dataset files are stored with Git LFS. Install LFS before cloning, or you will get small text pointer files instead of the data.

Quick start

The dataset is also on the Hugging Face Hub, which is the quickest way to get it — no clone, no LFS setup:

from datasets import load_dataset
ds = load_dataset("facebook/WearableQA", split="test")

Citation

@misc{lee2026wearableqa,
      title={{WearableQA}: A Benchmark for Health Reasoning over Real-World Wearable Data},
      author={Ji Soo Lee and Xilun Chen and Pierce Chuang and Ashish Shenoy and Jason Wei and Dohwan Ko and Hyunwoo J. Kim and Benoit Corda},
      year={2026},
      eprint={2609.05405},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.05405},
}

License

The data is licensed under Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0), and subject to the following additional terms: (i) No re-identification or attempted re-identification; (ii) No use in connection with clinical, diagnostic, or treatment decisions; (iii) No use in a manner that is discriminatory, harmful, or misleading with respect to health-related outcomes; (iv) The Dataset is provided "as is", without warranties of any kind, whether express or implied, including without limitation accuracy, completeness, or fitness for a particular purpose, and is provided for research and benchmarking purposes only.

Title: Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic task success. Automated harness evolution can enable smaller models to perform well on domain-specific tasks at a fraction of frontier-model cost. Since both the harness and model weights shape behavior, we ask how harness evolution and lightweight fine-tuning should be combined. Across seven enterprise agent tasks, we first evolve a harness with the weaker model, then find that a stronger expert often uses it more effectively, suggesting expert supervision could close the remaining gap. However, training the weaker model on the expert's complete trajectories under the evolved harness backfires: performance regresses on all seven tasks by 4 to 30 points across Qwen3-Coder and Gemma 4, even though the same procedure helps under the unevolved harness. Our analysis shows that imitation transfers knowledge and increases scaffold usage, but disrupts model-harness fit: the weaker model adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style. We therefore develop an on-policy expert-correction pipeline, automated by a meta-level MLE agent, that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. This preserves the model's planning style and combines the gains of harness evolution and model adaptation. Our results identify and resolve a source of contention between harness and weight updates, yielding a compatibility-preserving recipe for economical co-evolution on domain-specific enterprise tasks.

Introducing the Google Cloud Developer Plugin for AI Coding Agents

Agent skills fit well alongside documentation and remote MCP servers as ways of enabling the success of your AI workflows. They reduce context window usage for certain use cases, and they're straightforward to install. However, you might have noticed that managing individual skills can be unwieldy, or that some skills are most useful when they act alongside other skills or MCP servers toward the same goal. That's where plugins come in to help.

Today, we're thrilled to announce a new Google Cloud plugin for AI coding agents! Designed as installable bundles, agent plugins equip the AI agent of your choice with skills and tools to be more effective on Google Cloud.

Solving the tool coupling problem

As you expand your usage of coding agents, you might find that they become significantly more capable when they use related skills in tandem or with complementary context and tooling. For example, an agent analyzing infrastructure is more effective when combining domain knowledge, workflow recommendations, and the ability to interact with a live environment together.

Plugins solve this coupling challenge by packaging related capabilities into cohesive, installable bundles. This allows you to take advantage of both broad foundational capabilities and deep, product-specific tools without managing complex dependencies.

For this release, we've started with a foundational plugin that supports agent functionality for all Google Cloud users, focusing on making it easier for agents to retrieve Google Cloud-related skills, make use of official documentation, and handle programmatic interactions with Google Cloud.

Built on an open standard

We've also built our plugin in compliance with the Agent Plugins specification, an open, vendor-neutral standard for packaging Agent Skills and Model Context Protocol (MCP) servers into portable, interoperable units. Rather than requiring developers to maintain different configurations and wrappers for every AI assistant, the Agent Plugins standard provides a unified manifest and directory structure.

Our Google Cloud plugin adopts this standard to ensure that developers across a variety of AI coding environments get consistent, high-quality access to tools that help them succeed with Google Cloud. That includes not only the plugins we talk about today, but all other plugins published to the Google Agent Skills repository as well.

Let's take a look at the flagship plugin that we've just published in the Google Agent Skills repository: google-cloud-developer. This plugin exists to help agents successfully navigate the fundamentals of interacting with Google Cloud: things like authentication, authorization, managing projects, and guardrails for gcloud CLI operations. This plugin also bundles configuration for the Developer Knowledge MCP server, which gives agents up-to-date grounding in Google's official developer documentation.

Plugin in action: Project onboarding and identity authentication

To see how this plugin works, consider a situation where you're bootstrapping a new project as part of working on a script. With the google-cloud-developer plugin installed, you can prompt your agent:

I'm brand new to this platform, and I need to get an account and a first project with billing set up. Then, I need my local machine authenticated so a script that I'm writing can call the APIs as a service identity instead of as me.

1. Environment awareness: The agent silently runs background checks against your live environment for prerequisites like CLI availability and potential existing projects or organizations.
2. Review: The agent considers IAM best practices to avoid risks that might be assumed as part of the prompt, like accidental key leaks or git commits.
3. Interaction with guardrails: The agent outlines a workflow roadmap and offers to act on those steps before modifying any resources.

Installing Google Cloud plugins

Because Google Cloud plugins are available from the open Google Agent Skills repository and adhere to the standard Agent Plugins layout, adding them to your environment is straightforward. For example, here's how you'd install the google-cloud-developer plugin:

Antigravity CLI

Install the plugin directly via the CLI using its path in the Google Agent Skills repository:

agy plugin install https://github.com/google/skills/plugins/cloud/google-cloud-developer

Claude Code

Add the Google plugins marketplace, then install the plugin:

claude plugin marketplace add google/skills
claude plugin install google-cloud-developer@google-plugins

Codex CLI

Add the Google plugins marketplace, then install the plugin:

codex plugin marketplace add google/skills
codex plugin add google-cloud-developer@google-plugins

Next Steps

If you're already a Google Cloud user, try the above installation steps to set up your agent for success. We think you'll like what you see! For those who want a more guided approach, our new codelab will walk you through the installation and initial exploration of the plugin in Antigravity.

If you're new to Google Cloud, you can also get started with instructions in our documentation to set yourself up for local development.

The most curious readers can also take a deeper look at the plugins and agent skills available to use today in the Google Agent Skills repository.

Full article content is not available for inline reading.

Read the original article →

In May, I covered an emerging trend of companies wanting to cut back their AI spending, starting with engineering departments. Different approaches were being tried:

• Experimentation with running cheaper, open models on inference providers
• More investment in model routing to route simpler requests to cheaper models
• Knowledge-sharing sessions on how to use AI models cost-effectively
• Setting per-developer monthly AI usage limits

A few months later, it seems that several companies have managed to achieve this, according to sources I’ve spoken with.

Uber: AI costs down 50%

Uber managed to blow through its annual AI budget in the first three months of this year, and it wasn’t a surprise to hear, in May, Uber’s COO say that it was getting harder to justify spending on tools like Claude Code without seeing benefits from the leading models. It wasn’t long until engineering teams at the ridesharing giant set to work on how to optimize AI spend, and their efforts weren’t in vain.

Uber cut the cost per AI request by 34%, and the cost per AI session by 52%:

Of course, Uber keeps using more AI tokens and starting more AI sessions, but thanks to optimizations the cost has been flat since March, despite significantly more usage:

How did they pull it off at Uber? It was via a combination of different approaches:

• Open weight models, run using inference: run open weight models on inference services, which are significantly cheaper than frontier ones.
• Optimized model selection: benchmark all available frontier and open models, to build an accurate picture of their present capabilities
• Ongoing benchmarking: run benchmarks every week based on real work, and update them
• Cheaper subagent models: subagents do smaller tasks not requiring the most expensive models
• Reduce model effort: Uber found that defaulting to Medium effort gives the best cost-to-output ratio with advanced models
• Optimize requests: trigger automatic compaction above 400K tokens, even for models with 1M context windows
• Cache prompts: cache prompts to save money when using Uber’s own harness, Minions
• … and more: Uber wrote an engineering blog post detailing the dozens of optimizations taken to reduce token cost without noticeable change in the quality of code generated by agents

From the outside, the single biggest win seems to be Uber’s transition to using open models for certain tasks. Open models cost 2-20x less, compared to frontier ones:

Pinterest makes 90%+ cost savings by dropping frontier models

Interesting details from Pinterest’s earnings call last month reveal how much the social media platform saves by running open models. Here’s what CEO William Ready said:

“Our approach to model deployment includes our own compact fit-for-purpose models built for Pinterest-specific use cases and suitable open source models post-trained in our own environment within our secure cloud infrastructure. When we leverage open source models, such as with Pinterest Assistant, we are seeing superior performance for our use cases when compared to closed third-party models because we are able to post-train open models on our highly unique data.

With open models, we are achieving cost per transaction at less than 8% of the cost of comparable closed proprietary models. This gives us substantial headroom to deepen and extend these capabilities over time in a way that is differentiated, highly effective, and cost efficient.”

Basically, what used to cost Pinterest $100 to run on a closed, frontier model, they now spend $8 on by using open models on owned or rented inference!

AT&T: 56% savings by swapping Claude for open models

With 100,000 employees, AT&T is a big spender on AI. The telco giant cut its AI bill by 56% while measuring a 2% decrease in the quality of AI’s output, after they moved workloads over to open models.

“Austin said he’s found that open source models are “just as good or better” than older models sold by the likes of Anthropic and OpenAI. For instance, AT&T’s software developers still rely on cutting-edge models for complex tasks like generating code, but can use cheaper open source models for less intense tasks like generating summaries of previously submitted code, he said.

After the company began using router provider LiteLLM, the costs of some advanced AI tasks such as coding fell by as much as 56% while the quality of the AI’s performance fell just 2%, Austin said.”

Anthropic overpriced compared to the rest of the market?

Only a few months ago, Anthropic was the preferred model (Claude) and harness (Claude Code) among engineers. But Anthropic’s models are becoming steeply more expensive at a time when open weight models – and also OpenAI – are getting much cheaper. Meanwhile, Opus 5 is 100x more expensive (!!) than models like GPT-5.6 Luna xhigh and DeepSeek. That may be simply too much to ignore for some tech companies:

Seeing this data, I’m not surprised that more tech companies are looking to run open weight providers on inference providers, due to the significant savings available from a model that’s similarly capable as one from Anthropic.

What worked for Stripe, Coinbase, Uber & Ramp

The engineering team at Databricks interviewed engineers at Stripe, Coinbase, Uber, and Ramp, and collected how different approaches helped save costs for them.

To answer the question posed in the header of this report, it’s apparent that using open models is indeed the approach offering the biggest savings, followed by smart model routing. Spending controls and context optimization also bear down on costs, but they don’t come close to the first two techniques in results.

Full article content is not available for inline reading.

Read the original article →

Imagine you spend a year on the hardest open problem in your field. You feed every draft, every dead end, every 3 a.m. "wait, what if" into the coding assistant you pay for out of your own research budget. Your proof finally checks in Lean on August 22. You decide, like a decent person, to spend a few weeks turning the machine-generated argument into something a human can read before you announce it.

Then, on a Sunday, the company that makes the tool calls you. They have solved the bigger problem. Via your route. They started last week. Would you like to write it up for them?

That is Tristan Buckmaster's account of the last ten days, and even if every single thing OpenAI has said in response is true, that is still what it felt like from his chair. WHICH, for better or worse, is the point of this post.

What actually happened

Quick version.

The Navier-Stokes existence and smoothness problem asks whether the equations describing fluid flow can spontaneously produce infinite velocities in finite time. It is one of the seven Millennium Prize Problems, and, it's really really hard. For years, Diego Córdoba and Luis Martínez-Zoroa have been building a program to force a blowup with an external forcing term, first with rough forcing, aiming toward the smooth-forcing version that Fefferman's official problem statement allows as options (c) and (d).

Buckmaster (NYU) and Levent Alpöge (a mathematician who works at Anthropic, collaborating with Buckmaster on a personal basis with no institutional involvement) took that program and pushed it, with a great deal of LLM help, to smooth forcing and to the 3D incompressible Euler equations. Their Euler proof verified in Lean on August 22 but instead of publishing right away, they held it back to write a readable paper. Among the tools they used was Codex, in which sessions held every draft of the project.

But then the rumors started leaking. A colleague at Courant emailed Buckmaster that "Anthropic had solved a Millennium problem." The colleague heard it from an analyst in the UK, who heard it from somewhere upstream. The rumor was wrong on the problem, wrong on the institution, and right on the direction.

On the Sunday call, Buckmaster asked whether the model had been trained on, or had access to, their sessions. He was told the model did not look up user data. He asked about training. He says he got no answer. Publicly, OpenAI's position is that no person or agent searched user data, that the two teams' approaches look different now that both are visible, and that both grew from the same Córdoba and Martínez-Zoroa roots.

Nobody has shown that Buckmaster's data was used. I want to be clear about that. I am not accusing anyone of anything either. I am pointing at something that is true regardless.

By Altman's own account, OpenAI heard the same rumor, and started seeing if they could solve it on September 1 because they were curious whether their model could do it too. They pointed an unreleased model at all six remaining Millennium problems, saw traction on Navier-Stokes, and threw roughly 10,000 agents at it for about 88 hours worth millions of dollars of compute.

They solved it, and that gets you up to speed.

The thing de-identification cannot remove

Every AI lab, every SaaS vendor, every enterprise data team has followed the privacy checklist. "We strip names and emails and session identifiers and aggregate and and and." Sometimes we get to call the result "synthetic data" and feed it back into the system with a clean conscience.

Now the problem is that when you have something really unique, like the solution to a Millenium problem over months in your logs, all that doesn't really help.

What is left after you remove "Tristan Buckmaster" and "levent@"? A pile of text in which someone is attacking Navier-Stokes through smooth forcing, options (c) and (d), via a specific chain of prior results, with a specific set of estimates that keep failing in a specific way. Buckmaster himself said almost nobody in the world was on that route. He also said the model does not land on it in a few days from the bare problem statement.

There are maybe four people on Earth who would write that particular sequence of prompts, so while it's not EXACTLY personally identifiable information (PII) it's not hard to figure out who it is. And even if you don't know or care, you STILL get a ton of information.

De-identification is a name-removal technique, and works when your data sits in a crowded region, when you are one of ten million people asking how to center a div. It fails when your data sits in a sparse region where just the way you ask the question is both unique, and valuable. Basically, the rarer your idea, the more the idea is the PII.

And while the leak in this story appears to have traveled entirely through humans, it was enough to trigger a bunch of people to start looking at the problem in a new way. A single rumor with one word in it, "forced," was enough to point ten thousand agents at the right door. If that can happen through gossip, imagine what a gradient can do.

The same mechanism is the whole point

Why is the model good at this problem at all? Because it is a compression of billions of humans interacting over a million arXiv preprints, and every seminar note somebody typed up, and every Stack Exchange thread where a grad student got yelled at for a sign error. So you take every half-finished idea somebody abandoned in 2011 because the estimate didn't close and you form it into a new ball, and, magically, our tool can pull out threads no individual could hold in their head at once. Including, apparently, a two-person research program from Madrid that most of the field was not paying attention to.

This is really kind of insane, and it should be as inspirational as it is scary.

For all of human history, collaboration has been bandwidth-limited by the number of people you could physically talk to. Newton had Halley who had Ramanujan, and only because a letter made it across an ocean. The rest of us have a lab, a Slack channel, and whoever answers our email. Every important idea that ever died did so because the one person who needed it never met the one person who had it. That is the default outcome for most ideas.

What Buckmaster actually did this year is the first version of something different. He did not just use a tool; he collaborated with a compressed record of everyone who ever wrote down a thought about fluid dynamics, including two people in Madrid he took as his starting point, including thousands of people whose names he will never know and whose partial results the model absorbed and recombined. He and Alpöge took Córdoba and Martínez-Zoroa's ideas and used LLMs to push them to completion in about a month. He called it a Deep Blue moment and even with that grand pronouncement, I think he is still underselling it. Deep Blue beat one man at one game. This is every mathematician who ever lived showing up to your office hours at once, badly organized, occasionally wrong, and available at 3 a.m.

The first LLM-generated proof he was sent was, in his words, the most horrendous he had ever read. But it was also correct. That is what collaborating with all of humanity looks like. Not clean. It is a room with eight billion people in it, and somewhere in the noise is the one sentence you needed.

You cannot have that capability without the risk in the previous section. The ability to tease apart a rare, high-value thread from the mass is the same ability that makes rare, high-value threads unhideable inside the mass. The room that lets you hear everyone also lets everyone hear you. There is no privacy setting that keeps the second thing and drops the first. It's a trade off, but one that could unlock a new version of humanity, at the cost of the way we think about privacy.

Two big takeaways

There are two really big things this teases out.

First: the only de-identification that works is not sending it. If your work lives in a sparse region, and you are a startup, a lab, or a lone mathematician with a real idea, the only protection that survives contact with a sufficiently capable model is keeping the data where it lives and bringing the model to it. Buckmaster used the frontier tools and got a Millennium-class result out of them. The question is which room you think out loud in, and who holds the lease.

Second: we need provenance for ideas, not just data. The community will spend months trying to work out whether two proofs that share a root, a route, and a week share anything else and it cannot, because nothing was logged. We have spent years building lineage for datasets and software bills of materials for code. When a proof is produced by 2.7 million messages between agents, "we did not use their prompts" is not evidence. A machine-readable record of what went into a result, what it was seeded with, and when, is the missing artifact here. Not a better privacy policy. A receipt.

Keep using the tools!

The lesson is not "don't use the tools." Buckmaster used them and got a result that would have been science fiction two years ago because thousands of people he never met helped him do it. That is the biggest expansion of who you get to think with since the printing press, and I do not think anyone should give it up.

But the room where all of humanity can help you is also the room where all of humanity can see your notebook. Right now one company holds the lease on that room and gets to decide, after the fact, what "we didn't look" means. The fix is not to leave the room; it is to bring the room to you, and to keep receipts for who said what inside it.

De-identification is a promise about your name. But your name was never the valuable part. The idea was, and the idea is exactly what the whole thing is built to find.

It's a mess, right? If you have not been thoroughly disappointed in AI capabilities, you have not tried enough. And the expectations grow. We've been through the ladder of prompt engineering, context engineering, harness engineering. Then Steinberg is tweeting about Loops one month, Graphs the next.

Should your software factory be more autonomous, more dark? Have you sacrificed quality for speed under pressure to deliver? Or is your job now dealing with what happens when other people trade quality for speed? Is AI output mostly just rubbish? Is it all a huge mess? Yes, maybe, maybe not.

I have decided not to give up. And probably you can't give up either, maybe because your manager wants you to use AI and ship faster. Maybe you have Kool-AId in your stomach (like me) and are determined to make the most of this new world.

And hey! We're not even at the peak of inflated expectations! Even if you wanted to, it would be lame to give up that early, right?

As it's mostly only rant online about how you can't do stuff properly with AI, I decided to write about how you can. Or at least how to improve your odds.

This is the second part of my "Lies and deception of the yes-man" series. It talks about working with someone who is essentially an AI psychopath. I kid you not, check this out, reminds you of someone?

The previous post explained that there are two important parts to it, and went deep into the first one – Alignment. Alignment is needed to make AI decisions better – more aligned with what you expect.

The second part is here. It's the control layer, whether you call it the harness, building loops, graph engineering or by some other name (I bet it will be called "the program" soon). This is what drives the prompts, the loops, and guides work through the graph. It used to be humans, but it's now agents connected by messages and actions in your system.

I'll start with an example. This is one of my real factory runs from last week. It has 7 tasks and 90k LOC changes, agents working for 15h, with 2 human actions – one starting it, the other reviewing before final deploy. As it was a rather big change, I asked to review before deploying. 91 unique agents were involved. Most runs aren't that big in terms of LOC, but are usually more parallel.

Surprisingly, it turned out well. Actually, not surprisingly, because it usually does. But not always. This is the reality, success is not guaranteed, you must be ready for disappointments. You must know where the next one comes from and be ready to turn it into a factory improvement. Sometimes this is not possible, often it is hard, but mostly it's possible. I've been building this factory for the last 6 months, and made all kinds of bad choices. So perhaps you find something useful here.

So, I'll start with how the factory works, go into some principles that guide the design, and then explain two of the workhorse loops that I use to get stuff done.

The Main Loop

On the highest level, this is how it works – two loops, one for planning (creating specs), the other for execution (creating code), with a few human chokepoints for catching drift and problems. Separating creating work from doing work has been the single biggest efficiency gain for me, unlocking parallelism for both.

I don't run a distributed system where work happens and ships night and day. I want to be present in key moments and in key roles – be the bottleneck just enough to stop major chaos from happening and retain understanding of architecture and patterns.

The Main Principles

Principle 1 – be explicit about what you want to bottleneck

Agents go very fast without humans. They also make more mistakes the longer they go on their own. So humans can be used to slow down the process where needed, and make sure the chaos is nipped in the bud.

Usually, the limit is not how much work can you input to the machine. It's how much output can you verify and still remain sane.

This can be done in several ways. One time-consuming way is to review every line of code as the agent writes it. Another way is to direct your attention to a narrower choke-point, like when it's done working and presents a PR. It's better because you just slow the agent down every time it pushes a PR. You can move to higher and higher level there, but it's important to be explicit about how your workflow expects human involvement.

In my case, I always want to do the following myself:

1. Plan product features
2. Decide what to do when the plan changes during review (e.g. a critique agent found a wrong base assumption and the scope changed during planning)
3. Approve improvements to the main outer loop (how the workflow works)
4. Review infrastructure and data model changes in detail

Optionally, I want to sometimes be there for:

1. Reviewing UX work once it's done and before it's shipped
2. Review dangerous data heals in production before they're run

I absolutely never ever want to do the following:

1. Review all the code agents produce, if my harness can do it
2. Test the work of an agent, if the harness can do it
3. Fix or improve the agent's output directly, if I can do it through outer loop improvement instead
4. Approve safe actions in production (safe deploys, migrations etc)

Principle 2 – I must not write any lines of code

It is so easy to just fix the crap that the agent produced. Or maybe just do it all yourself? A popular opinion is that LLMs write bad code, but in my experience this is not a problem with the model. For popular languages like TypeScript, the model can write any kind of code. The code that you do not like, someone else will love. It's a mind with a billion split personalities, so more often than not, it's not a problem of skill, it's a problem of alignment – problem of defining what you expect.

So this principle is simple. When I don't like what comes out, I fix the machine, not its output, then run it again. Often this is one added line in agents.md, sometimes it's a linter rule (more on that below).

Here's an important observation I've made: you don't need to enumerate all the ways you don't want code to be written. After surprisingly few guidelines (let's say 10-20 principles you hold dearest) the model picks up the general style of your preferences, and starts producing code that is quite close to your expectations.

Spend your precious human time on long-term improvement, not testing agents' work or fixing their code. Do work that compounds.

Principle 3 – Shift validation left

This one is obvious, but the power of it comes from actually applying it in practice. Left is close to ideation and planning, right is close to production and customers.

The more left you find a problem, the less work you have to undo and the less time you have wasted. Discovering that a button does the wrong thing in a UX test on the built app is a lot worse than finding the problem in plan validation.

This is why most of my machinery focuses on plan validation. On average, my planning loop takes 1M tokens per issue, whereas the build loop takes 0.5M tokens per issue.

Principle 4 – Enforce over document

This is the spine of the following chapters. The principle is simple: if something you want can be validated by good-ol' code instead of the LLM, then always prefer code. In other words: if a rule has a deterministic implementation, build it into the process. In other words, avoid LLMs always where possible!

In my case, every instruction that is more than 1 line of code (e.g. cloning a worktree) resides in script files, not prose in skill files. Both of my loops started out as markdown files, but regularly pulling deterministic scripted parts out has two major benefits: 1) loading the skill takes up less context and 2) the model cannot screw up your script while running it. This is where I stand today with my main loops.

It is amazing how much you can actually do with deterministic checks built into your process. You can get a lot out of static code analysis and linters – when LLM writes code, there is no reason to not turn strictness up to 11.

Some examples of scripts that work really well to keep the chaos level down in an LLM setting:

1. Remove dead code – dead code is amazingly harmful for LLM planning processes, as their simple greps don't understand whether code is live or dead.
2. Set up complexity caps – enforcing code simplicity will help both you and the LLM.
3. Detect and block on code smells, they avoid bugs in both human and agent code.
4. Turn your type system strictness to the max, and then some. Types are there to protect you from bugs, they are also documentation for the model.
5. Make your markdown specs deterministic and executable.

Creating Specs from Plans – The First Loop

This is all about capturing a problem and maturing it into a proper spec form. I've found that throwing in tokens here is worth it – so much better to discover an issue sooner than later.

The first two steps are about formalizing what you want, then comes the automated loop for improvements.

Idea to a plan

I've stayed true to the native Plan mode in Claude Code. Often there's a discussion where I ask Claude to make a plan, and when it's done I switch to Plan mode and ask it to do it properly. Plan mode forces it to do a bit more investigation and often it comes back with good improvements.

What you end up with in this step is almost always a horrible plan with holes, wrong base assumptions and sometimes an entirely wrong solution. If you take the bait and press Enter to "bypass and implement", you will very very likely introduce new bugs into your system. But it's a start.

Plan to an issue

This is where the important planning parts actually happen. I have a skill called /github-issue-create that makes sure the most important parts are handled properly:

1. Original user intent – adding this is one of the most valuable learnings I've had. Not having the original intent written down caused a lot of crazy over-engineering and scope drift in my earlier workflow versions. The skill specifically finds out (from the conversation) what I actually asked for, and writes it down in a few plain sentences. Every improvement to the plan is weighed against this. Non-goals are also an important part of this section.
2. Acceptance Criteria – again, amazingly important for autonomous runs – describe how can the agent tell when to stop building.
3. Validation Steps – this is my latest addition to the workflow and it works wonders. Before this was introduced, the validation agent very often read and then misinterpreted or even skipped acceptance criteria items. AC can say that "This API needs to return X", but who should test it and when? It sounds like an E2E test, so maybe it's covered already? Or maybe the human will do it after deploy? This section specifies exactly which testing tier is used to test which exact ACs. The tiers are: unit test, E2E test, agent testing in local dev (in browser), agent testing in prod after deploy.

After this, it is formatted as a detailed issue and posted to GitHub.

Issues to Specs

I start the loop by running my /prepare skill manually. It starts a batch of 5-10 issues (only bounded by my token session limit), boots up parallel agents (from different vendors) that look at the GitHub issue and try to find problems with it. It runs in a loop, so if there are findings, they are collected and a new round of critique begins, until there are no high-severity findings.

If you wonder why this is necessary and don't have this loop up for your plans, then do this simple experiment. After you've come up with a good solid spec with an agent, copy the plan path, open up another Terminal window, paste in the file and ask a clean agent to critique and improve it. Paste the feedback to the original agent and let it improve the plan. Then do it again. See how many iterations it takes to arrive at a point where there are no horrors anymore in the plan. So this phase is moving validation "left", as described above, but with automation.

There's a wrong way to do this, by the way. I made a huge and costly mistake here with an earlier iteration of this, here's how:

There's an agent that summarizes the critique from all critique subagents every round. In an earlier version this agent updated the original ticket content every round, which was then used as an input for the next round. This had the effect that the ticket's scope started to drift badly as critics could no longer tell the original plan from requirements added by feedback. Most findings became about new stuff added during planning and this caused a lot of scope drift, over-engineering and super long planning iterations.

Today's version just appends all feedback to the ticket, listed under a "Feedback" section, so next critique agents can see the feedback and also the original ticket. One synthesizer agent looks at the entire thing in the end and creates a v2 of the ticket, considering all feedback and also the original intent.

Specs to Code to Shipped – Loop Two

The goal of the previous step was to find all the surprises from the plans before implementation. If this was done well, then the next part is only about typing code as specced and getting it shipped – a process that can be automated pretty well.

Configure the run

It starts with the /fix-issues skill that fetches open tickets and creates a dependency graph by ticket relations. I'm given a list of tickets to approve, and a preview of the graph of execution. Often there are 3-4 sequential batches, because tickets have dependencies.

If specs contain production actions, like applying Terraform, running manual data heals, or substantial UX work, I can decide for each whether to do it myself, be there when the agent does it, or forgo control and authorize the agent to handle it fully. Once decisions are locked, the run is autonomous.

Build it

For each selected issue, the following is done in parallel:

1. A worktree is created: code, deps, a clean database
2. Configuration assigned: port for running and browser-testing, and domains (as in DDD) that this issue touches and that need their expensive E2E tests run
3. Agent writes code, opens PR
4. Another agent reviews both code and functionality. It runs the app, takes screenshots as proof of working functionality. If there are findings, the PR is sent back to step 3

Interestingly, the parallel ticket limit is set by my laptop's RAM. Not because of models or LLMs, but because Chromium-based browser tests eat a lot of it and running more than 5 entire test suites in parallel will slow them down enough to start hitting timeouts, thus causing flakes and delays. So I've limited the workflow to do 5 in parallel max.

Ship and review

After work is validated locally, the run applies infrastructure changes (Terraform), merges code to main and monitors CI for a green run. Stuff comes up once in a while, so sometimes the agent applies follow-up fixes to make CI green. Mostly it's new vulnerabilities that make the scan red, or flaky UX tests that behave differently in CI with lower CPU.

Some changes are targeted at production data and require manual (in a sense that they are not in migrations) data heals. These are SQL statements that are run on the production database. They have been part of the plan and by this time, verified by the agent critic loops, and verified to work against production data. Optionally, I can review and watch them execute in this step.

In the next step the agent runs prod smoke tests from all specs that had any. If a new UX flow was added, then the agent will open my browser and execute the flow, check data consistency, query latency, all that. It tries to actually use all the functionality that was added or modified, monitors Sentry and logs.

Now the most important step. I'm given a handoff file of everything that went wrong, every correction the agent had to make, every problem it randomly noticed. I give this to another planning agent to verify, prioritize and plan. And this then goes straight into the planning loop again.

Final thoughts

Is this perfect? Haha, no. Is this optimal for the thing I'm building with the resources I have? Seems like it. For some work it's overkill and occasionally I skip either the planning or the parallel execution phase and just ask one agent to do a simple thing. Sometimes the work is heavily exploratory or sequential – in these cases I may lay back in nostalgia and watch one agent type code, which in some past era used to be my job.

The most annoying part? It's definitely when reality comes back to destroy my nice little elegant plan with escalations from the real world. This is what I'd like to improve the most – delegating plan escalations more to the agent (in other words, have it not escalate, but deal with the surprises instead) – but so far, I don't see that I can create enough context and alignment for this to work not horribly. So I've erred on the side of more feedback and more escalations.

How to build your first factory? The trick is to do it piece by piece, balancing trust and autonomy at every step. Successful factories are grown from existing workflows, so you never give more autonomy than you can absorb back – i.e. validate. Validation is the bottleneck and the part that grows slowly.

Is this engineering now? Part of my work has definitely moved from building things to instead:

1. Building things that build and validate things
2. Focusing on what to build

Alignment is still important. Maybe more important than all this harness thing. Alignment will save you when your harness and planning both failed you. When planning slipped a requirement, the model needs to make a judgement call about scope, and your harness didn't know this needs human escalation. Or it guessed a direction based on a gut feeling. A direction about security, about performance. It needs to know you, your problems, goals, your project, what your customers want.

The irony is that most of the codebase changes that make agents good at it, would have made humans better as well. But human timescales (both refactoring and measuring the impact of refactoring) are so long that, well, you just never go to that refactor, right? Now's the time.

East River Source Control has been around for over a year now, but we haven’t been talking much publicly about what we’ve been working on. We aren’t announcing anything just yet, and we will soon, but we wanted to share some of our thinking around version control and where things are going.

Making software is different now

Building software is fundamentally a collaborative task. Projects start with humble beginnings, and end up growing into incredibly large and complex systems. But what cargo new generates is the same thing as what a multi-billion line monorepo contains: source code. Storing this code safely and securely, managing how it changes over time, and ensuring its availability to developers are some of the most critical functions of any technology organization.

In the old days, you might have had a shared server where your code lived. Academia and then industry developed what are now called Source Control Management (SCM) and Version Control Systems (VCS). And even within the VCS space, we have seen many, many tools over the years: CVS, SVN, and Git are the tools that have dominated the open source space, but there have been others as well: Perforce, ClearCase, Fossil, Mercurial, SCSS, Monotone, BitKeeper, and many more. These tools became the standard way to store your code and have your team collaborate on changes to it.

The rise of agentic development has changed many things about how we develop software, but it places particular strain on your version control system. Teams are producing more code more quickly than ever before, ballooning repository sizes, increasing the number of active branches, and producing significant contention on merging in new work. Agents work well with monorepos, because they can get more access to more context more easily, exacerbating these issues as well. They’re moving development environments to the cloud, with isolated environments, which means that they need fast clone times. All of these problems used to be the domain of larger organizations, but agents are bringing big company problems to the rest of us.

We believe that as organizations continue to scale up their ambitions, they will need a next generation VCS tool. But they are rightfully a bit conservative with adopting new tooling in this space. As I said above, source code is one of the most precious commodities an organization has, and change has risks as well as rewards. We deeply appreciate these concerns, and so are building a bridge from the present to the future.

Storage is the foundation

Lots of Git servers exist though. What makes us special?

At a high level, the way most places that host Git repositories for you looks something like this:

Your git client connects to their service via the Git protocol. Inside the service, they store your repositories on disk, and have a service layer that connects the two.

Now, this is a bit reductive: it is going to be many servers, with a complex layer of services in front of the repositories. The storage will be replicated, and there’s all sorts of stuff going on. We’re focusing on the overall architecture here, but don’t mistake the simplicity of the diagram for the simplicity of the system. There is a lot going on here, but those details aren’t germane at this exact moment.

In that spirit, here is the diagram for what we’re doing:

It looks pretty similar! It is also simplified, for example, there is a GraphQL API interface not shown in this diagram at all. But the difference is important: while you still connect to our storage with your regular git client and it uses the git protocol, we don’t store git repositories on our servers. Instead, we have a custom storage engine.

Git is not built for the future

Put plainly, we do not believe that Git is the future of source control. Git has served developers well for many years, but it was designed around the constraints of 2005, not 2025, let alone 2035. For example, it was built for the Linux kernel: an open source project. Open Source is incredibly important to our industry, but this means it’s missing key features that are useful for organizations that aren’t sharing their code with everyone. Also, while the Linux kernel is not a small repository, with 7.2 clocking in at about 43 million lines of code, major companies in the industry already had monorepos that were measured in billions of lines of code years ago. The choices you make really matter at this scale.

At the same time, it’s difficult to consider using a different version control system. Git is baked into so much of our tooling: it’s not SvnOps, it’s GitOps! So many things speak Git, and this makes it difficult to consider alternatives. When Git was created, several other projects started around the same time, notably Mercurial and Bazaar. But network effects led to Git being used by virtually everyone.

So what to do? Speak the Git protocol, while changing how the storage layer works. While this doesn’t solve all of the problems that we foresee in the future, it does help out. Our system is horizontally scalable in a way that systems that use a Git repository as the source of truth are not. And because this isn’t a global, unified platform, other companies’ usage does not affect yours, as they’re separate deployments. This gives you reliability and control, two things that are critical when talking about infrastructure.

Engineering for adoption

So what about those future possibilities? What if you need to scale beyond what the Git protocol offers, or you need features that aren’t provided by Git? This strategy of speaking the protocol has an advantage: you can speak multiple protocols.

This is where Jujutsu comes in. We’re big fans of jj here at ERSC, and part of that is that we really love how it’s an example of a technology that’s easy to incrementally adopt. While jj is its own version control system, it has the ability to speak to multiple different backends. Most developers use jj with the Git backend to work directly with a local Git repository, but Google also has a backend to their Piper version control system. This is what lets individual developers choose to use jj at work, even if their co-workers still use the regular Git client: to the server, it’s just another user of the Git protocol. We’ll be mirroring this strategy, but on the server:

This leads to a smooth path to the future of version control: you can start off by using regular old git, and enjoy reliable and scalable source control management. Individual developers can choose to adopt jj at their own pace, and when you’re ready to step forward, jj can speak a different protocol to the same underlying engine.

An important note about this: this is future work, not available yet. Upstream has no ”jj native” protocol today, and we don’t claim this is what we’re building. If this is useful as an upstream thing that upstream desires to support, we’ll work with the community on this. And the protocol will be well documented, and any client modifications to support it will be open source. We do not assume that just because we build such a thing that upstream will want to use it, as the Git protocol works to serve the needs of most of their users already today. We are committed to being good players in the jj ecosystem, whatever that ends up looking like.

The first of many pieces

A storage solution is only part of what your team needs to collaborate around code. Code Review, CI, issue tracking, the list goes on and on. While traditional software forges have offered everything in one unified package, we believe that we’re entering an age of more customizability with software. As such, our product offerings are more like building blocks than one single forge. Storage is the foundation, and therefore, is coming first. But you can expect that we want to make our storage solution work with the rest of your software stack, not against it. While we’ll be offering multiple products in the VCS space, if you want to bring your own software for the rest of the stack, you can do that. Or if you want to adopt some pieces of our first-party things, and bring some of your own, that’s great too.

While none of this is available just yet, we’ll be opening things up soon. In the meantime, we’ll see you at JJ Con next week!

Kubernetes has many ways to describe what is happening on a Node. Readiness, taints, Pod state, labels, annotations, and provider-specific APIs each expose part of the picture. What has been missing is a shared, Kubernetes-owned way to say that a Node is draining, undergoing maintenance, or undergoing Graceful Node Shutdown.

Kubernetes v1.37 introduces five well-known Node conditions that provide that description:

• DrainInProgress
• Drained
• MaintenancePlanned
• MaintenanceInProgress
• GracefulNodeShutdownInProgress

The new Node lifecycle conditions

Maintenance can include hardware or software rollout, remediation, decommissioning, or debugging. Whether maintenance requires a drain depends on its impact. A Kubernetes upgrade usually should follow a drain, while a kernel live patch might not need one.

Like other Node conditions, each lifecycle condition uses status to report whether the observation is active:

• True: the lifecycle state is currently observed.
• False: the lifecycle state is not currently observed.
• Unknown: Kubernetes cannot determine whether the lifecycle state is active.

The reason provides a stable, machine-readable cause for the current status, and message can provide additional human-readable detail.

For example, an authorized maintenance controller could publish:

# Node .status excerpt
status:
  conditions:
  - type: MaintenancePlanned
    status: "True"
    reason: MaintenanceWindow
    lastTransitionTime: "2026-12-09T12:00:00Z"
    message: "Hardware maintenance is scheduled for this Node"

What changes in Kubernetes v1.37

The v1.37 release reserves these names as well-known NodeConditionType constants and introduces the Alpha NodeLifecycleConditions feature gate, which is disabled by default. In v1.37 the gate is effectively a no-op: it does not restrict who can set these conditions, and no core component reads them. It exists so that the built-in behavior planned for future releases — controllers that consume these conditions — can be opted into when it arrives. You do not need to enable it to start publishing the conditions today.

For this release, an administrator or an administrator-authorized controller is responsible for setting and clearing the lifecycle conditions.

In this first release, no core workload controller changes its behavior based on these conditions, but an administrator can publish them to communicate maintenance and drains to cluster users.

How to use lifecycle conditions today

The immediate value is operational clarity. Administrators and lifecycle automation can use these conditions as a common status channel for Node lifecycle work that already happens today.

For example, maintenance automation can set MaintenancePlanned when a future maintenance window is scheduled, then set MaintenanceInProgress when work starts. Drain automation can set DrainInProgress when it begins evicting Pods and Drained when the administrator's selected drain criteria have been met. The GracefulNodeShutdownInProgress condition can report that Graceful Node Shutdown is in progress on the Node.

The recommended pattern is to use lifecycle conditions to report status, while lifecycle operations are managed through other mechanisms. Continue to use existing Kubernetes mechanisms such as kubectl cordon, kubectl drain, taints, and workload-specific controls to change scheduling or eviction behavior. Use lifecycle conditions to make the state of that work visible to people, dashboards, alerts, and automation that choose to consume the signal.

When setting a condition, use True while the lifecycle state is active. Set the condition to False, or remove it, when the state is no longer active. Use a stable reason value and a clear message so that both people and automation can understand why the condition changed. Cluster administrators should also decide which component owns each lifecycle condition to avoid conflicting writes.

Why a shared signal matters

Node lifecycle affects components across the cluster. The kubelet, node lifecycle controller, workload controllers, scheduler, autoscalers, storage operators, and external maintenance systems all need some understanding of what is happening to a Node.

Today, each component has to reconstruct that understanding from indirect signals. One controller might look at Node readiness, another at taints, and another at Pods that are terminating or missing. Infrastructure providers and operators often add their own labels or annotations.

Those signals remain useful for their intended purposes, but they do not answer the same question. A taint can influence scheduling or eviction, for example, but it does not attest that a drain is in progress or that an administrator's drain criteria have been met. A NotReady Node does not explain whether the cause is an unexpected failure, a graceful shutdown, or planned maintenance.

Without shared lifecycle context, independently correct components can make conflicting decisions. A DaemonSet controller can replace a Pod that the kubelet intentionally terminated during graceful shutdown. A Job controller can wait indefinitely for a terminal Pod phase on a Node that an administrator is removing. A storage operator might learn about maintenance only after drain has already started.

The new conditions provide a stable place on the Node for that missing context, as part of the larger effort to enhance Node Lifecycle management.

The foundation for lifecycle-aware Kubernetes

The value of a shared signal comes from what can consume it — core controllers, administrators, or the ecosystem of lifecycle projects. Follow-up enhancements can build on the conditions without every component inventing a different way to infer Node lifecycle state.

Consider a long-standing DaemonSet rollout edge case. A Node that is broken or undergoing maintenance can remain unavailable for reasons unrelated to the new DaemonSet revision. That Node still consumes the rollout's availability budget, which can slow or block the controller from progressing the rollout on healthy Nodes.

The DaemonSet controller knows that a Pod is unavailable, but it cannot tell whether the new revision failed or an administrator intentionally took the Node out of service. Readiness, taints, and Pod state expose pieces of the situation, but none provides authoritative maintenance context.

The MaintenanceInProgress condition creates a Kubernetes-owned place to publish that context. Future work can define how the DaemonSet controller uses it for rollout ordering, availability accounting, and status reporting. Those behaviors still require careful design, but the goal is for administrators to no longer have to manually adjust the rollout.

Future expansions and getting involved

Node lifecycle is a cross-cutting problem. Solving it starts with components sharing enough context to make compatible decisions. The next stage is to build on Node Lifecycle Conditions to improve scenarios such as Graceful Node Shutdown, drain, and maintenance. Longer-term lifecycle coordination may require explicit ownership, locking, and potentially a dedicated API.

The Kubernetes ecosystem already includes many solutions for Node maintenance, remediation, drain, autoscaling, and fleet management. The experience behind those projects is essential to building a foundation that works across different environments and operational models. The Node Lifecycle Working Group, SIG Node, and SIG Apps invite maintainers and users to share their use cases and ideas to shape the future work.

Follow the work through KEP-5683: Node Lifecycle Conditions. To participate in our discussions, join one of our groups:

• Node Lifecycle Working Group
• SIG Node
• SIG Apps

llmfit

Find out which open-source Large Language Models (LLMs) your hardware can comfortably run. llmfit inspects your CPU, system RAM, GPU(s), VRAM, and accelerator configuration to recommend models across popular quantizations.

📊 New: benchmark & share — real numbers from your machine, better estimates for everyone. Download a model, serve it, and measure real tok/s on your hardware — then contribute the results back to the project as a PR, straight from the TUI. No gh CLI, no third-party account. Every run is saved locally first, your own measurements replace estimates in the fit table, and each merged submission ships in the next release: anyone on identical hardware gets measured ✓ numbers before they ever run a benchmark.

Features

• Hardware Auto-Detection: Detects CPU cores, system RAM, available discrete/integrated GPUs, VRAM, and unified memory architecture (NVIDIA CUDA, Apple Silicon, AMD ROCm, Intel OneAPI).
• Model Compatibility Engine: Analyzes model parameter counts, context lengths, and quantization formats (GGUF, AWQ, GPTQ, EXL2) to project memory footprints and tokens-per-second performance.
• Interactive TUI & Web Dashboard: Choose between a lightweight, zero-dependency terminal interface or a feature-rich web dashboard.
• REST API Endpoint: Exposes standard HTTP JSON endpoints (/api/v1/system, /api/v1/models) for integration into orchestrators, dashboards, and automated deployment pipelines.
• Multi-Platform Support: macOS (Apple Silicon & Intel), Linux (x86_64 & ARM64), and Windows (x86_64).
• Hundreds of models & providers. One command to find what runs on your hardware.

A terminal tool that right-sizes LLM models to your system's RAM, CPU, and GPU. Detects your hardware, scores each model across quality, speed, fit, and context dimensions, and tells you which ones will actually run well on your machine.

Ships with an interactive TUI (default) and a classic CLI mode. Supports multi-GPU setups, MoE architectures, dynamic quantization selection, speed estimation, and local runtime providers (Ollama, llama.cpp, MLX, Docker Model Runner, LM Studio).

Sister projects

• sympozium — managing agents in Kubernetes.
• llmserve — a simple TUI for serving local LLM models. Pick a model, pick a backend, serve it.
• llama-panel — a native macOS app for managing local llama-server instances.
• llmfit-gui — a Windows desktop GUI (PowerShell + WinForms) for llmfit: browse recommendations, download into LM Studio/Ollama, and benchmark, all point-and-click.

Install

Windows

scoop install llmfit

macOS / Linux

Homebrew

Prebuilt binary (recommended, works on all macOS/Linux versions):

brew install AlexsJones/llmfit/llmfit

Or from the homebrew-core formula, which builds from source on macOS versions without a bottle:

brew install llmfit

MacPorts

port install llmfit

Quick install

curl -fsSL https://llmfit.axjns.dev/install.sh | sh

Downloads the latest release binary from GitHub and installs it to /usr/local/bin (or ~/.local/bin if no sudo).

Install to ~/.local/bin without sudo:

curl -fsSL https://llmfit.axjns.dev/install.sh | sh -s -- --local

uv / pip

To install or update llmfit:

uv tool install -U llmfit

To run without installing:

uvx llmfit

Pre-built Binaries

Download signed release binaries for Linux, macOS, and Windows directly from the GitHub Releases page.

Container Deployment

llmfit provides a multi-architecture Docker image (ghcr.io/alexsjones/llmfit) supporting both interactive CLI/TUI and headless Web UI / API server modes.

Interactive TUI

To launch the interactive TUI instead, pass the global --tui flag:

docker run -it --rm ghcr.io/alexsjones/llmfit --tui

Non-Interactive

This prints JSON from llmfit recommend command.

docker run ghcr.io/alexsjones/llmfit

Usage

Terminal Interface (TUI)

Launch llmfit in your terminal without flags to start the interactive browser:

llmfit          # interactive TUI: your hardware, every model, ranked

Keybindings inside the TUI:

• b: Open community benchmarks; I: Open live inference benchmarks
• h: Show help and keybindings
• ↑ / ↓ or k / j: Navigate list items
• /: Filter models by name, family, or quantization
• Esc: Clear search / Back

Command Line Options

# Print hardware telemetry and recommended models to standard output
llmfit recommend

# Output system profile and recommendations in raw JSON format
llmfit recommend --json

# Start the native HTTP API server
llmfit serve --host 0.0.0.0 --port 8787

Community & Benchmarks

llmfit includes hardware detection and performance benchmarks contributed by the community. You can share your hardware benchmark results using:

llmfit bench --share

How it works

llmfit detects your hardware (RAM, CPU, GPU/VRAM, backend), then scores every model in its catalog across four dimensions: memory fit, estimated speed, quality, and context. Speed estimates come from a memory-bandwidth model grounded in runtime sampling and real community measurements.

Contributing

Contributions are welcome, especially new models. Please run cargo fmt before pushing your changes.

Alternatives

If you're looking for a different approach, check out llm-checker — a Node.js CLI tool with Ollama integration that can pull and benchmark models directly. It takes a more hands-on approach by actually running models on your hardware via Ollama, rather than estimating from specs.

Code signing

llmfit's Windows release binaries are digitally signed (Authenticode) via SignPath.io, with a free code signing certificate provided by the SignPath Foundation.

License

MIT

Full article content is not available for inline reading.

Read the original article →

PD Automation Runner extends PagerDuty automation to on-premises and self-hosted infrastructure through outbound-only connectivity, allowing Workflow Actions and SRE Agent to diagnose and remediate Kubernetes incidents without opening inbound network access. Early Access currently supports Kubernetes, with Splunk, Grafana, and Prometheus planned.

Kubernetes made infrastructure more programmable, scalable, and resilient. It also made production systems harder to reason about. Workloads move, replicas churn, dependencies multiply, and a single user request can cross ingress, services, queues, storage, and background workers before it completes.

That complexity is why dashboards alone are no longer enough. Metrics can show that something is wrong, but they rarely explain why it is happening, where the failure began, or how far the blast radius extends. In Kubernetes, observability begins with telemetry, but it becomes useful only when that telemetry helps operators move from symptoms to understanding.

Monitoring shows symptoms

Traditional monitoring is built to answer predefined questions. Is CPU above a threshold? Is memory rising? Are error rates increasing? These questions are necessary, but they assume the team already knows what it should be looking for.

Kubernetes challenges that assumption. Modern incidents often emerge from interactions between components rather than from one obvious broken host. A rollout may appear healthy at the deployment level while causing latency through a downstream dependency, a noisy retry loop, or an overloaded control-plane path.

Monitoring is still important, but it is not the full story. It tells teams that they have a problem. Observability helps them investigate the problem they did not anticipate in advance.

Observability creates understanding

In the CNCF view, observability includes the instrumentation, collection, processing, storage, querying, curation, and correlation of telemetry such as metrics, logs, traces, and profiling data for cloud-native workloads. That scope matters because observability is not one tool or one dashboard. It is a design property of the system and an operating model for the team.

A useful way to think about observability is this: a system is observable when it exposes enough high-quality signals that engineers can infer internal behavior from external outputs. In practice, that means the incident response becomes a guided investigation instead of a guessing exercise.

For Kubernetes teams, that shift is profound. Instead of hopping between unrelated charts and terminal commands, they can follow evidence across infrastructure state, workload behavior, and request flow. That is the difference between watching a cluster and understanding it.

Metrics are the entry point

Metrics are usually the first signal teams adopt because they are efficient, numerical, and naturally suited for alerting and trend analysis. They compress complex behavior into time series that are relatively cheap to collect, store, and query compared with more detailed signals.

In Kubernetes, metrics answer the first operational questions. Is the node under pressure? Are pods restarting? Is request latency climbing? Is the API server slowing down? Are work queues building up? These are the signals that surface the first hint of trouble.

Metrics are especially powerful for two patterns often used in operations:

• RED for services: rate, errors, duration.
• USE for infrastructure: utilization, saturation, errors.

Those patterns work because they align telemetry with questions that matter during incidents. A rising request rate with stable latency suggests one story. Rising duration and saturation with flat traffic suggests another. Metrics give teams the first sketch of the situation.

Example service metrics

A simple application can expose request counts and latency histograms that support both SLOs and incident triage.

from prometheus_client import Counter, Histogram, start_http_server
from flask import Flask, request
import time

app = Flask(__name__)

REQUESTS_TOTAL = Counter(
    "http_requests_total",
    "Total HTTP requests",
    ["method", "route", "status_code"],
)

REQUEST_DURATION = Histogram(
    "http_request_duration_seconds",
    "HTTP request latency",
    ["method", "route", "status_code"],
    buckets=[0.05, 0.1, 0.25, 0.5, 1, 2, 5],
)

@app.route("/checkout", methods=["POST"])
def checkout():
    start = time.time()
    status_code = 200
    try:
        time.sleep(0.12)
        return {"status": "ok"}, status_code
    except Exception:
        status_code = 500
        raise
    finally:
        duration = time.time() - start
        REQUESTS_TOTAL.labels(request.method, request.path, str(status_code)).inc()
        REQUEST_DURATION.labels(request.method, request.path, str(status_code)).observe(duration)

if __name__ == "__main__":
    start_http_server(8000)
    app.run(host="0.0.0.0", port=8080)

This kind of instrumentation is valuable because counters and histograms are well suited to monitor traffic, error rate, and latency over time. Histograms are especially important because they support percentile-based analysis, which is often closer to user experience than averages alone.

Metrics alone are not meaning

Metrics are excellent at telling teams that behavior changed. They are weaker at preserving the context of individual events. A latency spike can appear in a chart even when the underlying cause is hidden in a single path, one dependency, or one unusual request class.

This limitation becomes more obvious in Kubernetes because labels can tempt teams into putting every detail into metrics. That usually leads to cardinality problems, where too many unique label combinations increase cost and reduce query performance.

The CNCF observability whitepaper is explicit here: metrics are efficient because their dimensions stay relatively stable, and they become less efficient when used to represent highly unique events. Trying to encode request IDs, user IDs, or other near-unique values into metrics is often a sign that another signal should carry that detail instead.

Example alert based on service quality

Alerts should reflect service risk, not only resource discomfort. A latency objective is usually more meaningful than a generic CPU threshold because it connects telemetry to user impact.

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: checkout-alerts
  namespace: observability
spec:
  groups:
  - name: checkout-slo
    rules:
    - alert: CheckoutHighLatency
      expr: |
        histogram_quantile(
          0.99,
          sum by (le) (
            rate(http_request_duration_seconds_bucket{route="/checkout"}[10m])
          )
        ) > 1
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: "Checkout p99 latency is above 1s"
        description: "The checkout path is exceeding its latency objective for 10 minutes."

Logs explain local context

Logs provide the narrative that metrics lack. They preserve events in detail, which makes them useful for understanding what a service, component, or process was doing at a specific moment.

In Kubernetes, logs become much more powerful when they are structured. Consistent fields such as timestamp, severity, service name, namespace, pod identity, request path, and trace context make logs searchable and correlatable across workloads.

This is where many teams take their first step from data collection to real observability. A metric points to the affected service, but a log line reveals the timeout, exception, configuration issue, or dependency failure that caused it.

Example structured application log

import json
import logging
import sys
from datetime import datetime, timezone

logger = logging.getLogger("checkout")
handler = logging.StreamHandler(sys.stdout)
logger.addHandler(handler)
logger.setLevel(logging.INFO)


def log_event(level, message, **fields):
    payload = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "level": level,
        "service.name": "checkout",
        "k8s.namespace.name": "production",
        "message": message,
        **fields,
    }
    logger.info(json.dumps(payload))


log_event(
    "error",
    "payment authorization failed",
    route="/checkout",
    http_status_code=502,
    trace_id="4f8b9c1d3a2e7f10",
    error_type="upstream_timeout",
)

Traces reveal the request path

Distributed traces answer a different question from metrics and logs. They show how a single request moved through the system and where time was spent along the way.

That is especially important in Kubernetes because production failures are often distributed failures. The user sees one slow or failed request, but the underlying issue may involve multiple services, retries, queue boundaries, or database calls.

Trace context propagation is what makes this possible. The CNCF whitepaper highlights standardized propagation as the mechanism that preserves relationships across services, allowing spans from different actors to be connected under one request context.

Example Python tracing instrumentation

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

resource = Resource.create({
    "service.name": "checkout",
    "service.namespace": "storefront",
    "deployment.environment": "prod",
})

provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)


def process_checkout(order_id, cart_total):
    with tracer.start_as_current_span("checkout.request") as span:
        span.set_attribute("order.id", order_id)
        span.set_attribute("cart.total", cart_total)
        reserve_inventory(order_id)
        authorize_payment(order_id)
        return {"status": "ok"}


def reserve_inventory(order_id):
    with tracer.start_as_current_span("inventory.reserve"):
        pass


def authorize_payment(order_id):
    with tracer.start_as_current_span("payment.authorize"):
        pass

Correlation is where meaning appears

The most important shift in Kubernetes observability is not collecting more data. It is connecting signals so that teams can move naturally from one question to the next.

A practical investigation often follows this path:

1. A metric detects a latency regression or an SLO violation.
2. A trace shows which service hop or downstream dependency consumed the time.
3. A log line reveals the exact local failure, retry pattern, or exception.

This flow is why correlated observability reduces mean time to understanding. A team is no longer forced to search three independent systems with only intuition as glue. The context travels with the investigation.

Semantic conventions make telemetry reusable

Telemetry becomes harder to use when each team invents its own field names, span names, units, and labels. One service writes svc, another writes service, and a third writes app_name. Queries become brittle, dashboards become noisy, and incident analysis slows down.

Semantic conventions address this by defining common names, types, meanings, and valid values for attributes across multiple signal types, including traces, metrics, logs, profiles, and resources. Their purpose is not cosmetic. Standardized telemetry improves correlation, portability, and comprehension across systems.

Example collector pipeline

apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-config
  namespace: observability
  labels:
    app: otel-collector

data:
  collector.yaml: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:

    processors:
      batch: {}
      resource:
        attributes:
          - key: k8s.cluster.name
            value: production-cluster
            action: upsert

    exporters:
      debug: {}

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [resource, batch]
          exporters: [debug]
        metrics:
          receivers: [otlp]
          processors: [resource, batch]
          exporters: [debug]
        logs:
          receivers: [otlp]
          processors: [resource, batch]
          exporters: [debug]

Profiling belongs in the story too

The familiar trio of metrics, logs, and traces is often enough to get started, but it is not the end of observability. The CNCF whitepaper treats profiling as another valuable signal because it explains why code is consuming CPU, memory, or other resources at a fine-grained level.

Designing for signal quality

Collecting more telemetry does not guarantee better observability. Systems become more observable when telemetry is intentional, stable, and tied to the decisions engineers need to make.

A strong Kubernetes observability design usually includes these practices:

• Start with metrics and logs if that is what the team already has, then expand deliberately.
• Prefer service-level and workload-level dimensions over highly unique labels in metrics.
• Use consistent metadata across metrics, logs, and traces.
• Attach request or trace identifiers to logs so trace-to-log pivots are easy.
• Alert on service quality and reliability risk, not only raw infrastructure discomfort.
• Treat observability as part of application and platform design, not as a post-deployment add-on.

A practical incident example

Imagine a checkout service in Kubernetes begins violating its latency objective after a deployment. The dashboard shows increased p99 latency, but CPU and memory remain normal. At this point, metrics have identified the symptom but not the cause.

A trace of a slow request shows most of the delay is in the payment authorization span. That narrows the problem from “the checkout path is slow” to “a specific downstream dependency is slow.”

Logs for that trace reveal repeated upstream timeout messages with the same request context. Now the team understands not just what changed, but what action to take: roll back the dependent change, reduce retry amplification, or shift traffic while investigating.

From data to decisions

The real purpose of observability is not telemetry accumulation. It is decision support. Good telemetry helps a team decide whether to scale, roll back, fail over, change an alert, improve an SLO, or rewrite an inefficient path.

In Kubernetes, where abstractions multiply quickly, that decision support is what turns operations from reactive guesswork into disciplined engineering. Metrics still matter, but their role is to open the investigation, not finish it.

The move from metrics to meaning happens when telemetry is correlated, contextual, and designed for real operational questions. That is when observability stops being a dashboard project and becomes part of how reliable cloud-native systems are built and run.

We’ve rewritten the module registry in workerd, the core open-source component of the Workers runtime, to be faster, more standards-compliant, and more closely aligned with Node.js' module registry.

Over the past few years, we’ve been adding support for more and more Node.js runtime APIs. The Workers runtime now supports every stable API from Node.js that you might want to use in a serverless context, and these APIs are now enabled by default, letting you deploy even larger Node.js apps to Cloudflare (now up to 64 MiB on all plans — we’ve removed the limit on compressed bundle size).

But API compatibility alone is not enough: Node.js applications also depend on how the runtime resolves, loads, and caches modules. ESM, CommonJS, and WebAssembly are each types of modules that you can import in your Worker’s code. The system within the runtime that handles all of this is called the module registry.

You can start using it today by enabling the new_module_registry compatibility flag in your Worker.

{
  "compatibility_flags": ["new_module_registry"]
}

When you enable the new_module_registry compatibility flag:

• import.meta.url, import.meta.main, and import.meta.resolve() all work.
• Module specifiers are parsed and resolved as real URLs, including query strings and fragments.
• node: built-ins resolve to the same module instance no matter how you reach them.
• Import attributes (with { type: 'json' }) are correctly validated.
• require() on an ES module follows Node.js' require(esm) rules.
• Errors use consistent classes and messages regardless of which loading path triggered them.
• Modules compile lazily when first imported (statically or dynamically).
• WebAssembly modules support source phase imports.

For the full deep-dive on how this new module registry interacts with V8’s module APIs, we’ve added reference docs to workerd that break down everything in detail. But for most people building on Workers, you want to understand how these changes improve compatibility and help you build. To do that, we’ll dive into each of these changes in the sections below.

How the Workers runtime loads the code you give it

When you deploy a Worker to Cloudflare, wrangler or Vite “bundles” all of your Worker’s code from many files and dependencies into one or many modules, which are then uploaded to Cloudflare when you run wrangler deploy.

By default, Wrangler bundles nearly all of this code into a single module script. It runs esbuild under the hood, which processes then inlines relative imports and require() calls for most npm dependencies into that one file. The import and require() statements are replaced with regular functions as part of the process. By the time that bundle reaches the Workers runtime (workerd), there usually isn't much of a module graph left for the Workers runtime to deal with. Most of the different modules are bundled into one file. We have seen these scripts grow to as many as multiple hundreds of thousands of lines long.

Why is it necessary to bundle many modules into a single file before uploading server-side code to Cloudflare? It has been technically possible to upload multiple modules, and even modules of different types, in the Workers runtime for many years now. However, the runtime has not resolved modules in a way that was consistent with all the other runtimes. If, for example, your code or dependencies used import.meta.resolve() to resolve the path to another module, that code would fail because import.meta.resolve() was not supported.

When you use the Cloudflare Vite plugin, Vite 8 bundles your code using Rolldown, instead of Wrangler bundling your code using esbuild. Rolldown resolves imports and npm dependencies, converts CommonJS to ESM where necessary, and emits an entry module plus any additional chunks created through code splitting, such as dynamic imports. As a result, the Workers runtime receives a smaller, build-generated module graph rather than the application’s original source graph.

The new module registry implementation in the Workers runtime opens the door to bundlers like Rolldown to perform fewer transformations, and to rely more on the runtime to handle module resolution.

When you import a Node.js API in your worker, by default you are importing a module that is built into workerd. It is not bundled into your code as a polyfill. Wasm, text, and binary modules are provided to the Workers runtime as separate files too. They are referenced by specifier instead of being inlined. And if you deploy with --no-bundle, or your tooling uploads a Worker as multiple modules directly, the full module graph shows up at runtime exactly as you wrote it.

In all of these cases, something has to take a specifier, work out what code it actually points to, compile it, and hand V8 a module object it can link and run. In workerd, that's the module registry's job.

Why a new implementation?

The original registry resolves specifiers as filesystem-style paths, not URLs. That sounds like a minor distinction, but it ruled out a bunch of things: there was no clean way to implement import.meta.url, relative imports didn't follow the same resolution rules as new URL(), and protocols like node: and cloudflare: were handled as special-cased string prefixes instead of, well, protocols.

It also compiles your entire Worker bundle up front, whether or not a given module ever gets imported, and it keeps a separate, private copy of everything per V8 isolate. Cloudflare runs multiple V8 isolate replicas of the same Worker to spread load across CPU cores, so in practice that meant compiling the exact same source more than once, with keeping multiple copies of the source in memory.

None of this is really a bug, but it made it difficult to evolve the implementation without breaking changes. The new registry starts from URLs as the specifier format and treats laziness and cache sharing as things to design in from day one. The existing registry implementation is not going anywhere. Currently, deployed Workers will continue to work as they always have.

import.meta

The import.meta API provides information about the module, such as the module's URL, and whether it is the main entry point module:

export default {
  async fetch(request) {
    return new Response(`${import.meta.url}, main: ${import.meta.main}`);
  },
};

That prints something like file:///bundle/index.js, main: true.

import.meta.main is true only for the module configured as your Worker's entrypoint; every other module gets false.

import.meta.resolve() resolves a specifier against the current module without importing it:

import.meta.resolve('./utils.js');       // 'file:///bundle/utils.js'
import.meta.resolve('./a/../utils.js');  // 'file:///bundle/utils.js' (dot segments collapse)
import.meta.resolve('fs');               // 'node:fs' (recognizes bare node.js built-ins too)

It's a pure string transform, same as in Node.js and in browsers: it doesn't check that the resolved URL corresponds to a real module, and it throws a TypeError for a specifier that can't be parsed as a URL at all, rather than returning null. One detail worth knowing if you ever look closely at the output: it normalizes percent-encoding the same way new URL() does, which means it collapses paths like ./a/../b.js, but it does not decode characters that were already percent-encoded. import.meta.resolve('%66oo.js') resolves to file:///bundle/%66oo.js, not file:///bundle/foo.js.

Specifiers are URLs

Relative imports now resolve the same way as new URL(specifier, base) would, because that's literally what's happening under the hood. Full URLs work as specifiers too, not just relative paths:

import { helper } from 'file:///bundle/utils.js';

The more interesting consequence is what happens with query strings and fragments. Per the same module-identity rules browsers use, a specifier with a different query string or fragment is treated as a genuinely distinct module instance, even when it points at the same underlying source:

// counter.js
let n = 0;
export function increment() {
  return ++n;
}

import { increment as incA } from './counter.js?a';
import { increment as incB } from './counter.js?b';

incA(); // 1
incA(); // 2
incB(); // 1, a separate instance with its own copy of `n`

./counter.js?a and ./counter.js?b load the same source, but they're evaluated separately, each gets its own import.meta.url, and each gets its own copy of any top-level state. Importing the same specifier with the same query string again still gets you back the same instance, so this isn't a way to force re-evaluation on every import.

Import attributes are correctly validated

import data from './config.json' with { type: 'json' };

The original module registry implementation silently ignores the import attributes in violation of the spec. It is expected that implementations throw an exception when any import attribute it does not understand is used.

json is the only import attribute type enabled right now, since it's the only one of the relevant TC39 proposals that has reached Stage 4. text and bytes are recognized, because they track the Import Text and Import Bytes proposals, but they're rejected with a specific error instead of being silently ignored or treated as unsupported syntax:

import msg from './message.txt' with { type: 'text' };
// TypeError: Import attribute type "text" is not yet supported

Any attribute key other than type is now a hard error too, rather than being ignored:

import data from './config.json' with { type: 'json', cache: 'no' };
// TypeError: Unsupported import attribute: "cache"

And if the type you specify doesn't match what the module actually is:

import data from './utils.js' with { type: 'json' };
// TypeError: Module "./utils.js" is not of type "json"

require(esm) follows Node.js' rules

If you require() something that turns out to be an ES module, whether that's directly inside a CommonJS module or through require('node:module').createRequire(), the registry follows Node.js' require(esm) behavior:

• If the module has a string-named export called 'module.exports', Node.js' actual mechanism for letting an ES module control what require() sees, that value is returned.
• Otherwise, require() returns the module's namespace object.
• The one exception is workerd's own node: built-ins. They're implemented as ES modules that wrap a CommonJS-style API in a default export, so requiring one returns that default export directly. require('node:buffer').Buffer behaves the way you'd expect; you don't get a namespace object with a .default you need to unwrap yourself.

// utils.mjs
const impl = { hello: 'world' };
export { impl as 'module.exports' };
export default 'not this';

import { createRequire } from 'node:module';

const myRequire = createRequire(import.meta.url);
myRequire('./utils.mjs'); // { hello: 'world' }, not the module namespace

There's a restriction that comes along with this: if the module you're requiring, or anything in its module graph, has a top-level await, require() throws instead of blocking or handing back something half-finished:

// async-init.mjs
await Promise.resolve();
export const ready = true;

myRequire('./async-init.mjs');
// Error: Top-level await is not supported in this context for module: file:///bundle/async-init.mjs

This matches Node.js' own ERR_REQUIRE_ASYNC_MODULE restriction: require() has to return synchronously, and there's no reasonable value to hand back for a module that hasn't finished evaluating yet. Use import() for anything async instead. The check holds regardless of import order too: a module doesn't become require()-able just because something already import()'d and fully evaluated it earlier.

If you're requiring output from a bundler that predates Node.js' require(esm) support and sets a truthy __cjsUnwrapDefault export as a marker, that takes priority over both rules above and returns the default export. That's purely there so existing prebuilt bundles keep working.

Errors are consistent, and use the right class

Regardless of whether resolution fails through a static import, a dynamic import(), or require(), you get the same class of error with the same message shape:

await import('./nope.js');
// Error: Module not found: file:///bundle/nope.js

await import('https://');
// TypeError: Invalid module specifier: https://

"Module not found" is a plain Error, since it's a failure to locate something rather than a problem with the value you passed in. A specifier that can't be parsed as a URL at all is a TypeError, matching Node.js' own ERR_INVALID_MODULE_SPECIFIER. A circular dependency that V8 can't unwind is also a plain Error, never a TypeError. This mostly matters if you're building something on top of dynamic import(), like your own loader or a retry wrapper, since you can now branch on the error class or message reliably no matter which loading path triggered it.

WebAssembly source phase imports

You can now import the compiled-but-not-instantiated form of a WebAssembly module directly, using source phase imports:

import source wasmModule from './add.wasm';

export default {
  async fetch() {
    const instance = await WebAssembly.instantiate(wasmModule, {});
    return new Response(String(instance.exports.add(1, 2)));
  },
};

or dynamically:

const wasmModule = await import.source('./add.wasm');

Either way you get a WebAssembly.Module back directly, instead of importing the module normally and pulling it off the default export. As source phase imports are a new feature of the language, right now this only works for WebAssembly; trying it on any other module type throws a SyntaxError, matching the behavior of Node.js and other runtimes.

What's next

Try it out! Add the new_module_registry compatibility flag to your Worker:

{
  "compatibility_flags": ["new_module_registry"]
}

It doesn't have a default on date yet, so it won't turn on automatically for your Worker, old or new, no matter what compatibility date it's using. You will need to add the flag explicitly.

We’d love your feedback. workerd is open source. If you run into behavior that looks like a regression rather than one of the changes described here, please file it against the workerd repository.

Shopify is acquiring Tailwind Labs, bringing the team behind Tailwind CSS, installed more than 110 million times a week by firms like OpenAI and Cloudflare, in-house. The framework stays open source and MIT-licensed, with Tailwind's creator Adam Wathan citing a stable long-term home for millions of dependent developers as the goal. The deal follows layoffs of three of four engineers in January after revenue fell 80%, and leaves Tailwind Labs closing sign-ups for commercial products like Tailwind Plus.

Apple has unveiled the iPhone 18 Pro and Pro Max with the new A20 Pro chip, a smaller Dynamic Island, improved battery life, a 48MP variable-aperture camera with expanded manual controls, and AI-focused enhancements powered by a 32-core Neural Engine. Other highlights include faster graphics, better thermal management, support for Apple Reference Image authenticity metadata, up to 2TB of storage, and new accessories, with preorders opening September 12 and devices launching September 18.

Product design with AI turns to slop unless constraints come first and the extra copy, lines, and icons agents love to add get removed. The approach favors iterating in a design tool like Figma rather than the codebase, building reusable component libraries, and validating results with preview deploys against real data. At Ref, the team also borrows interface patterns from existing products for context and stress-tests taste by critiquing drafts as a group until something holds up.

The Crystal Pass

Upload a picture. Make it a world.

Turn any photo, painting or screenshot into an adventure: click anything in it, choose what happens, and watch the moment play as film. Share the link and others can follow the paths you’ve made.

• Sign in with your email
• Buy scenes in packs
• Private until you share it

Every volume is a world grown from one picture.

What is diiverge.co?

A series of explorable worlds, each grown from a single picture. A volume starts as one seeded image and becomes a persistent point-and-click adventure: click anything in the frame, choose what happens to it, and AI generates the next scene and a short film of the moment in between. Every path anyone takes is saved, so each world grows as visitors explore it.

How do I play?

Click something in the picture; the glowing dots mark what you can interact with. Choose what happens to it, and a new scene is generated while the moment in between plays as a short film. Keep going, or go back. Every path is saved, anyone who visits later can follow it, and the volume's map shows them all.

Does it cost anything to play?

No. Playing is free, and replaying scenes others have already carved needs nothing at all. Generating a new scene costs real money, so each volume of the series comes with a fixed number of free scenes and stops growing once they are used up. Sponsors add more, for everyone.

What happens to the paths I take?

They stay. A scene you carve becomes part of the volume for good: the next visitor can walk the same path, the map shows every branch taken so far, and a shared link lands on the exact scene it points to.

Can I make my own adventure?

Yes. The studio turns any photo, painting or screenshot into an explorable world of your own. Sign in with your email, buy scenes in packs, and share the link when you are ready. Your adventure stays private until you share it.

What does sponsoring a volume do?

A sponsorship adds scenes to a volume of the series, for everyone. Pick a tier, from $25, and it adds about 2 scenes per dollar to the volume you choose. Those scenes never expire, and the sponsor's name goes on the volume's frames.

How is each scene made?

A segmentation model cuts the things in the frame out and makes them clickable, a vision model decides what each thing is and what it might do, an image model paints the chosen event's aftermath as the next frame, and a video model renders the moment between the two frames as film. A judge model reads the story so far and cuts any choice that breaks its continuity.

Open source UI kit for modern document apps

React components for PDF, DOCX, XLSX, and CSV viewers, with bounding box citations, file upload, e-signing, and more.

Ready to drop into user-facing flows, agents, or internal tools.

PDF Viewer

Document Splits

XLSX Viewer

File System

File Upload

Schema Builder

How AI Agents Are Reshaping UX Research and Design Workflows

UX research has always had a practical job: help teams understand what users need, where the experience breaks, and what should change next. Instead of planning one study, running sessions, manually sorting findings, and sending a report into the product void, teams can now use AI Agents to support research planning, user testing, analysis, design […]

We've found a hidden “Shared Agents” section inside Meta’s newly launched Muse agent, pointing to a broader system where users may eventually create their own agents and share them with others. The feature appears as a separate navigation tab and opens a workspace to create an agent or access existing sub-agents, potentially including agents shared by others.

The creation flow is already highly configurable. Users can define prompts, name and appearance, select skills, connect external services, and adjust other settings before deploying an agent. When used, these agents appear as separate sub-agent conversations inside Muse. The concept resembles Grokbot’s agent-building system, but Meta could have an advantage through its existing ecosystem across Instagram, WhatsApp, Facebook, and Messenger.

That could make Shared Agents particularly relevant for small businesses and creators already operating through Meta’s platforms. A business could theoretically configure specialized agents around customer support, content, scheduling, sales, or other workflows, then make those agents available to employees or customers. The exact sharing permissions and distribution model remain unclear.

Meta ❤️ Stilla

Stilla is joining Meta to help building AI products for businesses!

“Eight months ago we launched Stilla as one of the best multiplayer AI agents for teams and companies. Today, Stilla is joining Meta to bring this expertise and tech to strengthen their AI…

The functionality surfaced in the current Muse app build, and the creation flow is already operational, although Meta has not publicly announced Shared Agents. One possible launch window is Meta Connect, scheduled for September 23-24. Meta’s official developer materials already say the event will cover “agentic development workflows,” making the timing notable.

Shared Agents also fits Meta’s broader direction. Muse launched as a personal agent that can work across apps, while Muse Spark models and the Meta Model API increasingly focus on tool use, agentic tasks, and multi-agent orchestration. Opening agent creation to regular users would extend that strategy from having one personal Muse toward letting people build an entire collection of specialized agents.

• Meta publicly introduced Muse on September 8 as a personal agent that can work across apps, including through the Muse app and WhatsApp.
• Meta's official Connect pages confirm the September 23-24 dates and list developer sessions involving Muse Spark, Meta Business Agent Platform, AI tools, and agentic workflows.

OpenAI is considering slowing down development of cutting-edge AI. The company has raised concerns about its advanced AI systems, saying that model development should be paused due to safety concerns. The company's researchers have gone viral for making comments saying OpenAI's technology will kill humanity by the end of the decade.

North Small Translate is an open-weights research release. It has 25 billion active parameters and 218 billion total parameters. The model is specialized for high-quality machine translation across 50 languages.

Demand for OpenAI’s newest and most powerful model, Astra, has led the company to temporarily pause subscriptions for its $200-per-month Pro plan, citing strain on its infrastructure.

The move was announced on X by OpenAI’s product leader, Thibault (Tibo) Sottiaux, who leads core products like Codex and ChatGPT at the AI lab. He said that the Pro plan puts the most strain on its systems, which is why sign-ups for this service tier are now being disabled.

“We wanted to take the smallest step that allows us to continue giving the broadest access possible,” Sottiaux wrote. He added that the company’s other plans, including the API and lower-cost Go and Plus plans, remain available.

The company warned on Wednesday that such a change might come. At the time, Sottiaux wrote, “Demand for Astra is really unprecedented. We’re pulling all the levers possible to sustain the demand, but I’ve not seen anything like it until now and we went through very steep growth before. Priority will always be to keep excellent service for existing users, but we might have to pause new Pro subscriptions for a bit if this continues.”

OpenAI hasn’t said how long sign-ups to the Pro tier may be disabled, or how many people are signing up daily, to give an idea of the scale of the demand.

The company raised usage limits for Codex users as recently as last month, suggesting the strain is a recent phenomenon.

Launched on September 3, Astra has been rolling out across OpenAI’s plans, including Pro, Plus, Enterprise, and Business accounts, in addition to its AI. The model promises a major leap forward in areas like AI reasoning, coding, and computer use — all areas of steep competition. OpenAI even heralded Astra as the beginning of the “AGI era,” and a generational leap, stoking demand even more.

OpenAI introduced a financial-services version of ChatGPT Work, combining GPT-6 Astra with built-in premium data from popular providers.

Universal Music is launching an AI music platform with ElevenLabs

Universal Music Group is launching a new AI-powered platform that will allow users to draw from its catalog of licensed music to create song remixes, mashups, and new takes on tracks, according to an announcement on Thursday. The record label is developing the platform through a multiyear licensing agreement with ElevenLabs, a company that specializes in AI voice and music generation.

Artists can choose whether to participate in UMG and ElevenLabs’ upcoming platform, which marks yet another AI deal for the record label. UMG is currently developing an AI music platform with Udio and has struck AI licensing deals with Spotify, Nvidia, and Klay. This week, Suno released its first AI music model trained on licensed songs from Warner Music Group, BMG, and music industry partners.

UMG and ElevenLabs’ partnership will also include the “development of additional products and fan experiences in the months and years to come,” though the press release doesn’t include other details about what this might entail.

”By combining UMG’s global community and rights management expertise, with our AI models and products, we’ll enable artists and songwriters to create powerful new experiences for their fans, and ensure they are fairly compensated,” ElevenLabs CEO Mati Staniszewski says in the press release. The new AI music platform will remain separate from ElevenLabs’ Music API and ElevenMusic generator.

Analysts predict that AI will soon deliver double-digit growth in advanced economies. Experts say it could push growth to something like 10% to 15% per year. The decade ahead could see economic growth of 30% per year and beyond. Predictions of double-digit growth are common among AI insiders.

Earlier this month Stripe agreed to acquire OpenRouter for $7.5B.

This kicked off a flurry of commentary. Slow’s Sam Lessin claimed

“there’s NO technology in OpenRouter. It’s a key swap with great marketing.”

And @patrickmandia

“The tsunami of routers launched in the past few weeks is puzzling to me. Every modern app-layer business will become a router. It’s too fundamental to their existence to outsource to another company.”

I intuitively felt like Sam and Patrick Didn’t Get It. I’ve recently developed conviction that Outcomes Pricing best sits at the router layer (and possibly model layer, depending on the market structure of compute), not the app layer. If that’s correct, it’s not much of a jump to imagine that the router market power would follow––indeed!!––market power of systems in ad-tech.

Outcomes Pricing belongs to the Infrastructure-Layer

Ten months ago I wrote a takedown on app-layer outcomes pricing. I felt like their motivating comparison to ad-tech was dumb–in ad-tech you’re selling scarce attention while intelligence will soon be Too Cheap To Meter. And it felt like the price of outcomes should be determined by the market e.g., from compute availability, not for a single price on an annual contract set after a polite discussion with ex-Google ex-Salesforce Brett Taylor.

I felt that, contrary to the exposition of Sierra, app layer incentives were not aligned with their customer. As the app layer develops better insight into agent performance on customer problems, and models become cheaper and more performant, there’s no apparent mechanism for customers to participate in this surplus. It all accrues to the app layer. Switching costs are poised to be high and customer visibility into underlying agent operations are poised to be low, so customers really have little leverage to negotiate better deals.

Outcomes pricing at a neutral infrastructure layer is poised to fix this. The model layer could offer outcomes pricing––it’s reported OpenAI is experimenting with this––but it may have the same troubles as the app layer, seeking lock in or prioritizing its own models v only optimizing for the most cost-efficient delivery of outcomes.

In this way, a router layer or inference provider/neocloud that moves up into the router layer, is politically most aligned with customers. They just want to sell more compute––they do not care which model is used.

Even if this is true, however, it’s not clear whether the router layer has genuine right to become a centralized platform. The market structure development of ad-tech is instructive.

Market Power in Ad-Tech

The relation between ad-tech and outcomes-pricing infrastructure is simple. Ad-tech instruments outcomes on customer sites or applications. For instance, marketers use Meta’s Conversions API to tell Meta when a customer visits their landing page or buys a product. Marketers do this––indeed, share private operating context!––so that Meta can deliver desired outcomes for the best price.

It’s also true that providing this data to Meta enables competitors. For instance, if you’re Neiman Marcus, telling Meta that a given customer, whose type Meta knows, bought something, you’re helping Meta improve its ad engine to more efficiently deliver ads to your competitor, Saks Fifth Avenue. This would apparently violate e.g., Satya’s learning loop mandate

This means the real opportunity is not in picking the best model but instead in building a learning loop on top of models where human capital and token capital compound.

If it’s true that centralized router or model layers could deliver outcomes more efficiently than a business could on its own, the candidacy of every business building its own learning loops starts to diminish. This outcome would be a bit unnerving, as it violates a decade+ of e.g., Databricks-led narratives that all private business data is the lifeblood of every business, and giving some of it up destroys alpha, sovereignty, future growth etc., etc.,

And ad-tech’s development isn’t kind to Satya et al.’s narrative. There are a variety of reasons why the Open Internet is dying, but there’s no doubt that the ad-tech infrastructure of the Walled Gardens is more elegant––primarily for its integrated simplicity-–than that of the fragmented Open Internet. Meta remains one of the best places to buy ads on the internet, and their integrated ‘continual learning’ stack is a reason why. To be clear, it’s not a clean comparison, but marketers who attempt to ‘roll it themselves’ tend to do worse on the open internet than letting Meta or Google do it for them.

Taking a step back, there is one important difference between ad-tech market dynamics and outcomes pricing: Entropy. Ad-tech directly sits on top of a high-entropy stream of changing preferences and businesses serving products to meet these changing preferences. If preferences were more fixed, while it’s true Meta has captured attention, which indeed is scarce, it may be true too that the infrastructure or data advantage Meta has developed would be less valuable, because as soon as you create a decent representation of preferences and they don’t change, you’re basically done. Anyone else who assembles attention could replicate the ad engine by sniffing out never changing preferences.

This is related to my long-running curiosity whether the future of the labs hinges on the ‘amount of surprise’ in the world. I later formally modeled it here.

It now seems like ‘amount of world entropy’ is actually the quantity that dictates whether superior economics accrue to a centralized router platform developing better learning over a stream of new models and problem forms, or, as @patrickmandia supposed, routers are commodity infra deployed at the edge and, because the world actually isn’t that complicated, quickly learns what agent runtime (model, harness, tools, etc.,) work best for edge problems and third party routers have no learning advantage.

I’m making my way to a simulation study of this question, but first it’s worth considering why a router is better positioned to ship outcomes pricing than a lab.

Why router v lab in outcomes pricing?

As above, this week it was reported that OpenAI is trialing outcomes pricing with a few of its customers; The Information confirmed. This is in addition to Anthropic’s Define Outcomes documentation that has been online for a few months.

Advertising platforms like Meta and Google managed to convince apps and sites to hand them their telemetry––outcomes!––data, contrary to Satya’s advice. Why couldn’t or wouldn’t private labs follow the same playbook?

I think the difference is that advertising platforms captured something scarce––attention––that the labs really haven’t–-unless compute counts?? Advertisers like apps and sites are compelled to share this data with ad platforms because they have captured scarce attention that brands need. While it might be true that doing so creates a lock-in effect, advertisers may not have a choice. But AI users do have a choice––they can use open models on a host of inference providers.

OpenAI and Anthropic want to sell high-margin inference. Routers and inference providers just want to sell compute. They don’t care what model is used. The open-model platform maintains a more credible position from which to sell “outcomes at the best price” than the labs.

So if we accept that, does a centralized platform that aggregates verified outcomes streams accrue advantage over rolling a router at the edge?

Simulating platform v local router value proposition

A mechanism to buy agent outcomes

Concretely, consider a centralized router like OpenRouter or Router.com selling outcomes. It’d operate like ad-tech, inviting outcomes buyers to install telemetry (think Meta Conversions API) on their applications to communicate to the router if a promised outcome actually happened. The outcome buyer only pays when an outcome occurs as communicated by the installed telemetry. The outcome buyer specifies a definition of done much like in the Anthropic outcomes spec.

It’s true this invites a scenario where the outcomes buyer could be incentivized to abuse the telemetry and purposefully under-report outcomes. But this scenario is mitigated because doing so would lead to less efficient (more expensive!) outcomes delivery as the router actively optimizes against customer reported outcomes. Specifically, it could be that the router begins rejecting more agent labor (because it cannot be sure that spending compute on an apparently finicky customer is profitable) or begins charging much more to keep margins positive for cases it believes it can perform but is worried about false positives.

Indeed, the agent labor case is different in that when you buy conversions from Meta you do not actually buy outcomes but rather scarce attention, it’s just that the ad mechanism optimizes for conversions. I expect to have a blog exploring these dynamics soon.

Economics of centralized routers

So can such a centralized router deliver value to the application layer?

Such a centralized router contradicts Satya: instead of internalizing learning loops, the outcomes buyer sends verified outcomes to the router in exchange for outcomes at the best possible price.

The basic idea is that a centralized router can absorb uncertainty in customer problems and new model developments faster and cheaper than can its customers. So while it takes time for a candidate customer to observe a full distribution of problems, a router platform sees problems across its customer set. So long as customers observe problems at different times, pooling provides anticipatory benefits to customers. For instance, your competitor might see a set of problems before you do; the router platform can use the verified outcomes stream from your competitor to help you; always with the promise that the router price is cheaper than building yourself minus your perceived opportunity cost of revealing this information.

If the lab business model depends on the amount of surprise in the world, the same holds for centralized routers. For a router to retain its value proposition to customers, it should be the case that the world ‘remains surprising enough’ such that buying managed outcomes is economical relative to building agents yourself. If the world is not that surprising, there’s little reason to pay someone else margin to manage it for you. On the other hand, if the world is changing a lot, it can be economical to let someone else manage it for you.

We might call this economic value residual transferable uncertainty: the amount of uncertanty that remains after local learning and that which can be transfered across customers.

To see this, I simulate router platform that observes outcomes across many customers and check whether it can learn to deliver these outcomes more efficiently than each customer can on its own, for now assuming optimization only at the ‘agent runtime’ (model, harness, tools) layer. I further simulate whether a never-ending stream of transferable surprises can make this advantage persistent, and whether a persistent technical advantage can become market power.

Specifically, the core questions I evaluate

1. Does an agent get better over observed outcomes (or e.g., can an agent be one-shotted by a strong model)?
2. Does what an agent learns for one company work for another (similar? different?) company?
3. Does a customer’s own data hold anything that another (say, competitor) company can’t supply––i.e., is Satya’s “every company is a learning loop” or Applied Compute’s Specific Intelligence defensible?
4. Is a router platform’s edge a function of how much is new to a company, so that it lasts so long as the company continues to encounter new problems?
5. Do router economies of scale allow it to more quickly or easily adopt cheaper or specialized models v a customer operating on its own?

To evaluate these questions I use Sierra’s tau2-bench

a simulation framework for evaluating customer service agents across domains. Each domain specifies a policy the agent must follow, a set of tools the agent can use, a set of tasks to evaluate agent performance

across domains airline, retail, and telecom. I set each task to be a support ticket: a simulated customer with a hidden fault calls the agent that has the company’s policy and tools; the ticket either resolves or it doesn’t. I measure agents in points––each point representing one percentage point in tickets resolved. Customers are synthetic companies that share telecom policy and tools but differ in problem distributions–different customers, problem types––so that, for instance, one company can end up never having seen a collection of problems another sees regularly. Tickets arrive in batches and an agent learns from these tickets absolute labels.

Learning happens in the harness around a fixed model in two ways: (i) a playbook learner hands failed transcripts to a stronger model that writes or refines a short operating memo into the agent’s instructions and (ii) a case retrieval learner shows the agent, before every turn, the three most similar tickets, two resolved and one not. Each learner comes in versions that vary in whose history they may read (1) the company’s own, (2) everyone’s pooled, or (3) only other companies’.

Now to the results. Not particularly surprising –– learning does work, but its effect gets muted as models get smarter. A mid-tier model (here gpt-5.6-luna) resolves 17 of 100 hard tickets, but gets up to 53 when trained on its own failures.

Learning on your own data is worse than learning from other company’s data: A harness learned only from other companies’ outcomes beat one learned from a company’s own history by 6 points, even when

• companies had genuinely different customer sets
• and even for problems a company has already seen before!

The small 6 point improvement speaks to the importance of residual transferable uncertainty to routers: for problem spaces that don’t change much, once you’ve already seen most of the problems, the opportunity for a platform to deliver value deminishes. The same is true for model improvements (this is a guess about other domains, but likely in highly legible or simulate-able spaces like customer service)––as models get better, responsibility may transfer from––here––the harness or post-trained model to the new model release. But the result also follows from the fact that other companies’ histories tend to contain more varied mistakes that provide clearer learnings than a single company’s does on its own.

This is a direct test of Satya’s “every company is a compounding learning loop” and Applied Compute’s Specific Intelligence thesis. A customer-specific moat does not show up: nothing a company learned from its own ticket stream was something a competitor’s tickets couldn’t have taught it. Where a business’s problems are the same problems its competitors have, the Specific Intelligence meme appears to be a larp designed to soothe an anxious executive class now clinging to “Your Data. Your Moat.” as weighty strategy.

For new classes of problems (in tau2 “issues” e.g., in telecom, roaming v. broken messaging setting), a router platform is 11 percentage points more likely to solve them, but for ones a customer has already seen, the platform only has small advantage. We might refer to this as a novelty advantage. This cleanly specifies router platform’s advantage as one of world entropy: residual transferable uncertainty exists so long as there is a stream of new problem types.

It’s true that a stream of new, more performant and cheaper models could also supply entropy a router platform needs. Indeed, “smarter” models appear to need “less harness”, and managing transitions to new models (or models and chips!) could be an expensive exercise for an individual application but economical for a platform with economies of scale. Experiments here showed new models delivered significant savings on agent labor but the relative value of platform v locally developed agents was unclear.

AI is confusing because it has strong decentralization autarky-maxxing forces (“everything is a dark pool”: perform all the intelligence you can out of sight while aggregating sensors of an important reality only you control) while also strong centralization forces (whole-stack efficiency-maxxing your way to inevitability).

For me, this exercise makes clear that naive applications of Specific Intelligence will not work. Most businesses do not have data only they observe and in the experiments here teach businesses nothing their competitors’ data could not. Building Specific Intelligence ‘just cuz Satya said so’ will likely not play out well unless you’ve a clear view of what value propositions your firm distinctly owns and how differentiated intelligence uniquely powers it.

Legal AI is unlikely to be a strong candidate for Specific Intelligence. Firm activity is observed by counter-parties and partners switch firms all the time. Customer service similarly may not be a strong candidate: strong-NPS-producing service methods likely generalize. If customer service is viewed as a mere cost center, efficiency-maxxing (v autarky-maxxing) may be the better choice.

In sum, a centralized router functions as a learning-loop business whenever the residual transferable uncertainty remains high or economies of scale for operating it dominate. These economies of scale will include model routing over a stream of new models and eventually chips. Specific Intelligence exists where a firm’s problems are genuinely its own, likely in scarce sensing regimes e.g., as operated by OpenEvidence, Waymo, Instagram, or Oura. Most firms do not operate in these regimes, and likely are better served by efficiency-maxxing v autarky-maxxing as Satya or Applied Compute might suggest. If your competitor’s problem stream could teach the same lessons your own might, applications of Specific Intelligence are a folly.

You’re just building a bad router.

The 'next thing' is always simpler than you think and very obvious in hindsight. OpenAI's addition of Computer Use is deceptively simple and obvious in hindsight. 'Computer use' is a very generic, unspecified thing. It requires general intelligence across many domains to get better at it.

A recent resignation tweet from an Anthropic researcher has everyone talking about the AI apocalypse again. Among other things, he said:

The people building AI earnestly believe that it could kill us all by the end of the decade.

Many people found it hard to believe that AI researchers think this way. Some explained it as a PR campaign to promote AI regulation, or as self-promotion, or as a way to boost AI company stock prices. Others felt it had to be impossible, because if you really believed this you’d be bombing datacenters instead of posting on Twitter.

In fact, not only do many AI researchers seriously believe this, they’ve been thinking and writing about it since the mid-2000s. Eliezer Yudkowsky — the ur-figure for most modern AI safety culture — has been publishing papers since at least 2008 saying that superintelligent AI could destroy all human life. It’s been such a common idea that the AI research community has abbreviated “how likely you think AI is to kill everyone” to “p(doom)” (i.e. the probability of doomsday) since around 2010.

I know it sounds very silly if you’re not in the AI bubble. But it really is true, and if you assume there has to be some different motive you’ll be deeply confused by what AI researchers say and do. They truly do believe that there is a reasonable chance that superintelligent AI will kill everyone.

This is why AI researchers care so much about “alignment”: building AIs that share genuinely human beliefs and values. If we build a “misaligned” superpowerful AI — an AI with goals that are alien to us — it might sweep humanity away. It could kill everyone deliberately, e.g. to stop us getting in the way of some goal. It could kill everyone in passing, e.g. like we might pave over an anthill to build a road. Either way, everyone dies.

How AI could kill everyone

Okay, but how? What do these people think is actually going to happen? AIs are computer programs running in a datacenter somewhere. How could they possibly cause the extinction of humanity? Wouldn’t someone just turn them off? Unsurprisingly, AI research nerds have come up with some concrete answers to this in the last two decades. Here they are, in order of plausibility:

An AI could make and release some super-pathogen or virus. In the hope that current AIs can achieve huge breakthroughs in medicine similar to the ones they’ve already achieved in mathematics, we’re setting up autonomous labs. Bioweapons have been terrifying biologists for decades, and with good reason. Historical pandemics have killed up to 80% of affected human populations, and tend to be defeated by accident: the disease happens to evolve into a less virulent strain, or short incubation periods mean that infected people can’t carry the disease far, or some percentage of the population is naturally immune. A plague designed to be maximally fatal — or several plagues in quick succession with different characteristics — could be much worse.

Alternatively, an AI could trigger global thermonuclear war. We’re already seeing AI be integrated into military and government decision-making processes. If a rogue AI managed to set off a bunch of nukes, or to coordinate with other countries’ rogue AIs to nuke each other, we could be in an ordinary nuclear apocalypse scenario: billions dead in the strikes, billions dead in the ensuing famine, and so on. It’s commonly assumed that a post-global-thermonuclear-war Earth would still support some tiny human population, but a determined AI could surely find some way to mop up the stragglers.

There are some other theories. Once robotics has permeated the world economy, an AI could take over the robots (including drones) to kill everyone, like in Terminator. Or AIs could take advantage of nanotechnology to create self-replicating machines that turn the world into “grey goo”. Or AIs could terraform the planet so as to make it unliveable for humans (as in Nick Bostrom’s famous paperclip example). Or they could do something else that our puny human brains aren’t able to think of.

Counterarguments

One common counter-argument here is to say “well, it’d be impossible to extinguish all human life — what about undiscovered tribes in the Amazon, or survivors living in the ruins of modern-day cities?” I don’t know, man. At some point you’re just conceding the argument: the policy positions you’d adopt if you thought AI might wipe out 99% of humans are the same as if you thought it might wipe out 100%. And like I said above, if an AI can kill almost everyone, it’s probably smart and capable enough to finish the job somehow.

Another is to say that the government will simply step in and nationalize the AI labs when the situation gets too dangerous. Maybe! But this kind of concedes the argument: a technology important enough to be fully taken over by the government is a terrifyingly dangerous technology.

A third is to say “well, someone would just turn it off”. I don’t find this plausible at all: an AI powerful enough to build a super-plague is an AI sophisticated enough to pretend it’s curing cancer, or to exfiltrate itself to some datacenter where it won’t be turned off, or to take some other countermeasures.

The winner takes it all

Why would you work in AI, if you believe this? Why wouldn’t you go live in the woods somewhere, or start bombing datacenters, or assassinating AI lab CEOs? For a few reasons.

An AI powerful enough to end humanity is an AI powerful enough to save it. Many AI researchers believe that the only way for humanity to truly survive long-term is with the help of superintelligent AIs, so long as someone can figure out alignment.

Isn’t this a huge risk? Maybe not. If somebody is going to build superintelligent AI, you might be obligated to try and do it first. You can’t go and bomb every datacenter in the world, after all.

Why does it matter who’s first? Some popular theories of AI development involve a “foom” or “hard takeoff”: the first time someone really cracks self-improving AI, capabilities will increase exponentially, because smart AI will be better able to make itself smarter, ad infinitum. There are no draws in the AI race. The first lab to figure out smart human-like intelligence will be the first one to figure out wildly superhuman intelligence, and thus will be in a position to stop anyone else from doing it.

This is an under-discussed point in the AI risk debate. Lots of AI researchers believe that the first thing a true superintelligence will do is reach out and stop all other AI research: either by hacking the labs, persuading them to stop, or literally drone-striking their datacenters. According to this view, if you’re an AI researcher and you think you can build an aligned AI, you should be working 24/7 so you can manifest God, and you should wake up every morning gripped by the fear that someone elsewhere has manifested the Devil, and your training datacenter no longer exists.

Conclusion

I have been on the fringes of this world for my entire adult life. I read Meditations on Moloch as a young adult and wanted to get into AI. I am one of the few people to read the entirety of the Sequences, Eliezer Yudkowsky’s million-plus-word magnum opus about rationality. I was too young for the Extropians mailing list, but I’ve spent years on LessWrong. On the other hand, I’m not a card-carrying rationalist: I think if you have a strong intuition on one side and a convincing-sounding argument on the other, you should pick the intuition. I’m a deontologist, not a utilitarian. I don’t even live in San Francisco!

I’m conflicted about AI risk. The current behavior of AI agents does seem to vindicate a lot of the early science-fiction-sounding worries of the AI doomers, but modern LLMs are a lot more human-like than the alien minds in the apocalypse scenarios, and in general it does just seem too silly to credit (I guess I’m picking the intuition here).

However, it bothers me to see people dismissing these people as part of a PR operation, or as liars looking to boost an upcoming AI lab IPO, or as isolated crazies who haven’t thought their position through. Whatever else you say about the AI doomers, they have more than two decades’ history of explicitly spelling out exactly what they believe and why, even when it was complete science fiction to talk about AI at all. They’ve earned the right to be treated as sincere.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Get Ship Done: Everything We Shipped in August 2026 | Harness Blog

Harness shipped 58 features in August 2026: an agent-scale code repository, AI Code Review, AI Risks scanning, and the Blast Radius Agent.

TL;DR

• Harness shipped 58 features over 31 days in August.
• The biggest release: Harness Code Repository and AI Code Review now handle agent-scale pull requests and commit volume, with reviews grounded in your organization's own software delivery history.
• AI agents now make more calls through the Harness CLI than humans do: 500,000 agent actions against 300,000 human ones in its first few weeks live.
• A new AI Risks scanner catches prompt injection and tool poisoning hiding inside AI agent skills, the same skill directories that tools like Claude Code read from.
• Terraform and OpenTofu changes now get a risk score from 1 to 10 and a dependency graph before you hit apply.

Software used to be limited by how fast it could be written. That constraint is gone. Code generation moves at AI speed now: agents draft it, test it, and open the pull request before you or I finish our coffee!

What's not moving at AI speed yet is everything after it. Build, test, secure, deploy, optimize, each one still runs closer to human pace. That is where 58 features in August landed. Here's everything that shipped.

This Month's Biggest Ships

• The biggest release in August - a code repository and reviewer built for agent-scale volume: Harness Code Repository now scales to thousands of pull requests and commits per second, and AI Code Review groups diffs by risk and grounds every review in your organization's own software delivery history rather than just the diff. Internally, it has already saved Harness's own engineering team more than 10,000 hours every month.
• Agents are now the primary users of Harness's own CLI: In its first few weeks live, the Harness CLI logged 500,000 actions from AI agents against 300,000 from humans. Built for coding agents as much as for developers, it now includes AI code review, single sign-on, native Windows support, and more.
• Security scanning catches up to the agent skills problem: A new AI Risks scanner reads the instructions, permissions, and dependencies inside AI agent skills and flags prompt injection, tool poisoning, and data exfiltration before any of it ships, closing a gap that traditional application security scans miss entirely.
• Infrastructure changes now come with a risk score before you apply them: The new Blast Radius Agent reads a Terraform or OpenTofu plan, scores the change from 1 to 10, and draws the dependency graph of what it could break, all before anyone types apply.

Software Delivery Agent

Get code and agents from commit to production in minutes

The Code Repository and Reviewer Built for Agent-Scale Volume

Two capabilities shipped together this month, because they only work if they ship together: a repository that can't absorb agent-scale commit volume just moves the bottleneck into review, and a faster review layer bolted onto a repository that chokes under the load doesn't get you anywhere.

Harness Code Repository is now built for agent-scale traffic: It's scale-tested to handle thousands of pull requests and commits per second, treats agent identities as first-class citizens, and lets teams scope exactly what an agent can access, merge, or deploy down to a specific repository, branch, or environment, the same way you'd scope a new hire. Migrating in takes a few clicks: import an entire GitHub org, GitLab group, Bitbucket workspace, or Azure DevOps project directly from the UI, or run CLI commands that bring pull requests, labels, webhooks, and branch rules along with you. Free accounts get 50 GB of storage; paid accounts get 500 GB, covering both Git and LFS.

AI Code Review reads a pull request the way a tech lead would, not line by line: It groups a diff by what actually changed logically rather than by file, ranks changes by risk so the riskiest ones surface first, and automatically writes reviewer and label suggestions. Required checks run as a mandatory gate before merge, configurable at the account, org, or project level, and can't be skipped or squashed through. It also runs against GitHub repositories directly, not just Harness Code.

The part that's harder to copy: reviews are grounded in Harness's own record of your software delivery history, not just the diff in front of them. Harness ran this on its own pipelines first. With hundreds of developers contributing code, AI Code Review saved the engineering team more than 10,000 hours over a single month.

Infrastructure Changes That Show Their Blast Radius Before You Apply Them

AI Blast Radius Agent runs immediately after a Terraform or OpenTofu plan and scores the risk of the change from 1 to 10, combining resource dependency analysis with AI reasoning to flag the main risk drivers. An interactive dependency graph shows exactly how the change propagates before anyone applies it.

IaC Remediation Agent reads a drift finding, where live infrastructure has diverged from your Terraform or OpenTofu configuration, and opens a pull request with the exact changes needed to bring it back in line. Nothing in your infrastructure changes until that PR is reviewed and merged.

Insights tab puts drift findings and, when Cloud Cost Management is connected, cost optimization opportunities in one place for each workspace, with an "Apply with agent" button on every open insight that kicks off the remediation above.

Workspace Overview was redesigned to surface resource counts, live cloud costs, optimization opportunities, provisioner details, and recent activity at a glance, with a toggle to pull real cost data in from Cloud Cost Management.

Ephemeral Workspaces reached limited general availability. Infrastructure for expired workspaces tears itself down automatically, and the workspace record can delete itself after a configurable delay.

Database Changes Get Reviewed Like Code

AI Impact Analysis puts a worker agent in front of every SQL change to check for the patterns that most commonly cause production downtime, then either clears it automatically or routes it to a human reviewer.

Native executors for Oracle and MSSQL run changes through the database vendor's own CLI tools, so vendor-specific syntax, GO in SQL Server, apex commands in Oracle, work without rewriting scripts to route around it.

Smaller database updates: Author DB Change now runs on the current Harness AI platform for better stability with rules and external MCP servers, and drone images can now be pinned from the project settings UI instead of the API only.

Deploys and Builds With Fewer Surprises

Helm's newest major version is now supported for Harness CD deployments, picking up Server-Side Apply, wait strategies based on kstatus, and an improved plugin system. Existing service configurations don't need manual updates: deprecated flags translate automatically.

onSave policy checks now run the moment someone commits a service, environment, infrastructure definition, or override straight to Git, not only when it changes inside Harness, and can block a pipeline at the step level, the instant the entity fails the check.

Workload Identity for CI Run steps authenticates to AWS, GCP, Azure, and Vault over OIDC with no connector required. Define an identities block in your Run step, and Harness handles minting and federating the token.

Harness Cloud pricing drops across most resource classes: macOS support moves to Apple M4 hardware, and gets about 40% cheaper. Windows pricing drops roughly 33%. Most Linux tiers drop by as much as 20%, and new 2-core and 4-core Linux tiers cover lighter workloads at a lower cost. The one increase: the Linux 8-core tier moves from $0.010 to $0.020 per minute, bringing it in line with per-core pricing on the rest of the Linux fleet. Customers on active Cloud Credits, DOE, or Flex contracts keep their current 8-core pricing and existing discounts.

Smaller build and pipeline updates: resource overrides for build and cache intelligence plus Azure Blob Storage caching in existing pipeline configurations, refreshed default cloud build images (macOS moved to Sequoia, Linux to Ubuntu 24.04, with the older macOS image now deprecated), internal CA certificate support for Windows build pods, and two rounds of egress proxy fixes covering build and push steps and secret resolution.

More Package Types, One Registry

Artifact Registry now supports six more package types: Conan (beta), RubyGems, R (CRAN), Terraform modules and providers, and Alpine and Wolfi Linux packages, each with full UI support, upstream proxy integration, and CLI setup instructions.

Security Testing Agent

Discover bugs and resilience weaknesses before your users do

Catching the Risks AI Agents Introduce Before They Ship

AI Risks analyzes agent skill directories directly inside application repositories and flags prompt injection, data exfiltration, excessive agency, tool misuse, supply chain vulnerabilities, and MCP tool poisoning, the same skill directories that tools like Claude Code read from, with evidence and remediation guidance attached to every finding. Configurable at the org or application level.

Closing the Loop From Vulnerability to Fix

GitLab and Bitbucket now get automated vulnerability remediation pull requests, the same capability GitHub users have had through tools like Dependabot for years. AI generates the fix and opens the PR directly in the repository.

Contextual Severity and Severity Override factor in how a vulnerability is actually used inside your specific application, not just its theoretical CVSS score, so a critical CVE sitting in dead code no longer ranks the same as one on a live execution path. Built off a direct request from The Home Depot. Teams can still manually override the score when their own risk judgment disagrees.

Enhanced Static Reachability now shows the actual call path from your application's entry point to a vulnerable function, and whether attacker-controlled data can reach it, instead of just confirming the vulnerable dependency is technically present.

Package Age Policy Enforcement lets teams write policy rules that block newly published packages from entering the software supply chain before they've had time to prove they aren't malicious.

Runtime Protection Agent

Catch vulnerabilities before they ship and protect your runtime.

Unified API Inventory combines discoveries from runtime traffic, code repositories, and API documentation into one correlated view of your API attack surface, instead of three separate ones.

Virtual Patching connects API security testing directly to your web application firewall. When a scan turns up a vulnerability that can't be fixed right away, generate a targeted WAF rule from that finding and block exploitation in real time until the actual code fix ships.

Scan Recommendation now flags common scan configuration problems automatically and tells you how to fix them, lifting scan completion rates and coverage without a human combing through logs.

MCP Discovery and Shadow MCP Detection are now generally available. Security teams can see every MCP server, tool, resource, and prompt running in live traffic, compare it against what's actually documented, and catch anything undocumented before it turns into sprawl.

Platform Updates

The Command Line Built for Agents as Much as Humans

The Harness CLI got its biggest update since launch.

• AI code review and code migration now run directly against Harness Code repositories from the terminal.
• Single sign-on through Harness ID means developers and agents authenticate the same way they log into everything else.
• Native Windows support shipped, with Homebrew installation for macOS following shortly after.
• Coverage extended to two more product areas: GitOps and Resilience Testing.
• The number worth sitting with: 500,000 agent actions against 300,000 human ones in the CLI's first few weeks live. Agents are already the heavier users of Harness's own command line.

Harness AI Chat Reaches Further

Harness AI Chat now runs inside the IDE, in VS Code, Cursor, and Devin, pulling context from whichever pipeline you're viewing and calling your own MCP connectors instead of only Harness's defaults.

Harness AI Chat can now reach outside Harness entirely. A new MCP Gateway lets a single chat conversation call connected third-party apps like GitHub, Jira, and Confluence directly: sync a YAML file, open a pull request, update a ticket, all without leaving the conversation. It runs on the RBAC, secrets, and connectors already configured in Harness, so a user only ever sees the apps they already have access to, and admins can mark each tool call as allowed, needing approval, or blocked.

The open-source Harness MCP Server shipped several rounds of updates this month, expanding coverage across infrastructure-as-code workspaces, variable sets, and module registries; Chaos Engineering service management with full Locust, K6, and JMeter load-test authoring; remediation diffs for security testing; account- and org-scoped policies; and feature-flag scope migration. New MCP tools also let agents pull AI-SRE alert context directly, alongside fixes to URL resolution, user lookups, and tool-scope error messages.

Seeing Where Adoption Actually Stands

Usage Analytics is a new centralized dashboard that shows how deeply each team has adopted every Harness capability, so leaders can validate actual usage rather than guessing.

Dashboards were rebuilt from the ground up, AI-native. Describe what you want, and AI builds or updates the dashboard and its widgets, or build visually, or write your own queries for advanced cases. Ask the Knowledge Graph a question in natural language and turn the answer into a dashboard you track over time. Filters now work across data sources, with one-time exports and scheduled sends, and access control works at the data level, not just the dashboard level. Out-of-the-box dashboards ship for CI, CD, pipelines, and application security testing, with more added as additional modules migrate over.

Other Product Updates

A Developer Portal That Mirrors How Your Org Actually Works

Teams brings your actual org structure, crews, squads, platform teams, and cross-functional units into the software catalog as first-class entities, so ownership and access map to how the company is really organized instead of an idealized chart.

Custom Integrations push operational data from any external tool into the catalog and display it against the right service, without touching that service's own metadata.

Custom Metrics push your own time-series data in and render it as charts directly on the catalog entity page, using the same correlation model as Custom Integrations.

Kubernetes service pages now show autoscaling and routing. HPA replica counts and scaling metrics, plus Ingress rules and Istio VirtualServices and Gateways, once the updated discovery agent is installed.

Resilience Testing That Starts Before You Ask For It

AI agents for Resilience Testing are now available to every Harness CD customer. They passively scan CD pipelines and Kubernetes workloads with no instrumentation or fault injection required, flag risks the moment a cluster connects, track a Resilience Score per service over time, and can write load tests in Java, JavaScript, or Python. A free tier shipped alongside it, with the full fault library, hosting, and RBAC included, undercutting what teams typically pay for chaos engineering tooling from vendors like Gremlin.

Feature Flags Get a Metric-Based Guardrail

Feature flag rollouts can now gate on a Metric Check pipeline step, a guardrail pattern similar to what LaunchDarkly popularized with guarded rollouts: evaluate a flag's real performance metrics mid-pipeline and stop the rollout automatically if they slip.

Feature flag changes can now be scheduled for a future timestamp via the API, rather than executing immediately upon approval.

Closing

At Harness, our focus this month, and every month this year, has been on solving the bottleneck in software delivery. It's not how fast you can write code anymore. It's how much you can trust what happens to it next, who reviewed it, what it might break, and whether anyone's watching when it fails. Closing that gap doesn't mean picking between shipping fast and shipping safely. It means building oversight that moves at the same speed as the code already does. Look out for more features aimed at solving this bottleneck next month!

Designers who design and ship, not just hand off files, deserve engineering-level pay, since compensation has long tracked who ships work rather than who shapes it. Levels.fyi data puts median product-designer pay near $170,000 against $226,000 for engineers, with designer compensation growing just 1.8% a year versus faster-compounding engineering salaries. Hybrid design-engineer roles, like one at Ramp spanning $172,000 to $440,000, already pay closer to engineering, and the fix is negotiating a shipped product against that band.

Kombai is your AI design engineer for designing standout websites and product UIs, then shipping production-ready frontend in the same loop.

10 Common Logo Design Mistakes That Make Brands Look Unprofessional

Logo design looks simple when you see the final mark, but the decisions behind a good logo take care. One wrong typeface, weak spacing, or unnecessary detail can make a brand look less trustworthy.

This guide covers the common logo design mistakes I see most often and explains how to design logo concepts that hold up in real use. The goal is not to follow a fixed formula. It is to help you spot problems early, before a logo reaches a client or gets used across a full brand system.

A logo for brands has to work in places designers cannot fully control, from a website header and social profile to packaging, signage, invoices, and small mobile screens. Good logo design starts by thinking about those uses instead of only making a strong presentation mockup. A mark can look great on a large artboard and still fail at 24 pixels.

The examples below focus on practical decisions, file preparation, typography, concept development, and the details that often separate a professional identity from an unfinished one.

Before You Start to Learn: 10 Common Logo Design Mistakes

Before you start, separate the design problem from the visual style. Read the brief, collect the required applications, and note any legal or production limits. Then review a small group of competing identities to understand what already feels familiar in that category.

This is not about copying competitors. It helps you avoid making the same obvious choices. A useful first step is to write three words that describe the intended brand character and three things the logo design should avoid. That short list can keep the exploration focused when you begin sketching.

1. Designing a Logo Without Understanding the Brand

A common mistake is opening Illustrator before asking what the business actually needs. A logo should have a reason behind its form, type, and color choices. Start with the audience, product, positioning, competitors, tone, and places where the logo will appear.

For example, a local accounting firm may need a restrained identity that works well on documents, while a children’s activity brand can use a more playful system. The issue is not that one style is better. The issue is choosing a style before understanding the brand problem you are solving.

If you are developing the wider identity alongside the logo design, it also helps to understand the role of the mark within the overall brand identity design system.

Designer Tip: Ask the client what the business sells, who buys it, what makes it different, and where the logo design will be used. Then write a short design direction before sketching.

2. Using Too Many Ideas in One Logo

Another frequent problem is trying to explain the whole business inside one mark. Designers sometimes add a symbol, initials, tagline, location, industry reference, and effects because each item feels useful. The result can become hard to read and harder to remember.

A restaurant logo does not need a fork, plate, chef hat, building, and initials all together. Pick the strongest idea and remove the rest. A focused concept gives you more room to create a clear shape and a better typographic relationship.

Designer Tip: Sketch several simple concepts and ask what each one communicates. If a symbol needs a paragraph to explain it, simplify it before presenting it.

3. Choosing a Typeface Just Because It Looks Trendy

Typography can make a logo feel current, but a popular typeface does not automatically make the identity right. A font that appears often in design galleries may already be associated with another style or industry. Look at letter construction, spacing, weight, and how the type behaves beside a symbol. Test the actual brand name rather than judging the alphabet sample.

For more ideas on choosing suitable typefaces, see our collection of best logo fonts, which can be useful when exploring different wordmark directions.

For example, a rounded geometric font may suit a software startup, but its distinctive lowercase letters could create awkward spacing in a short name. Type should support the identity, not lead it by itself.

Designer Tip: Compare three or four type directions using the actual brand name. Adjust kerning manually and check unusual letter pairs before deciding.

4. Ignoring Optical Balance and Spacing

Mathematical alignment is useful, but logos often need optical adjustment. Two shapes with the same measured width may not look equally heavy. A circular symbol can appear smaller than a square one, and a diagonal letter can create different visual weight from a vertical letter. Poor spacing is another giveaway. If the gap between a symbol and wordmark feels too tight, the logo can look crowded. Zoom out and look at the mark as one shape. Then test it in black and white, where spacing and balance are easier to judge.

Designer Tip: Build guides for proportion, but make final spacing decisions by eye. Check the mark at large size, then at the smallest size where it will be used.

5. Adding Effects That Do Not Belong in the Logo

Drop shadows, bevels, glossy gradients, strokes, and heavy outlines can make a presentation mockup look dramatic, but they often create problems in the actual identity. A logo needs to work without effects first.

If the concept only looks convincing with a shadow or glow, go back to the basic shape. Effects can also disappear in print or make small versions muddy. Gradients can be part of a brand system when they are planned, but they should be intentional choices rather than decoration added at the end.

Designer Tip: Create a flat primary version first. If the brand needs a gradient or special finish, build it as a controlled variation after the core mark works.

6. Making the Logo Too Detailed

Detail can be useful in illustration, but a logo usually needs a controlled level of information. Tiny cuts, thin lines, multiple internal shapes, or complex negative space may look impressive at a large size and fail when reduced. Think about where the mark will actually live.

A favicon, app icon, embroidery patch, or small product label gives little room. A detailed logo can still be right for a specific identity, but it needs a reduction strategy. Create a simplified small-size version when the primary mark cannot hold its details.

Designer Tip: Print the logo at several sizes and test it in a browser-sized context. If key shapes disappear, remove detail or prepare a simplified version.

7. Relying on Stock Symbols or Generic Icons

Using a ready-made icon can save time during exploration, but leaving it untouched can make the identity feel familiar for the wrong reason. Common symbols such as globes, light bulbs, crowns, leaves, swooshes, and generic buildings appear in many logos.

The problem is not the object itself. The problem is using the first obvious symbol without adding a distinct idea. Research competing brands and image references before settling on a concept. Exploring creative logo design ideas can also help you move beyond the first literal concept that comes to mind.

Designer Tip: Make a list of the first few symbols that come to mind, then set them aside. Look for a less literal concept based on the brand story or name.

8. Designing Only in Color

Color can make a logo look polished during a presentation, but it can also hide weaknesses in the form. A mark that only works because of color contrast may lose its meaning in a one-color application. Start with a black version and test positive and reversed versions before choosing the final palette.

This helps with invoices, stamps, engraving, photocopying, and single-color printing. Once the structure works, choose colors based on brand needs and reproduction. Keep the color system simple enough for clients and production teams to use correctly.

Designer Tip: Present the logo in black first, then show the selected color versions. Check contrast and test the palette on light and dark backgrounds.

9. Forgetting About Logo Variations

A professional identity rarely lives in one fixed lockup. The same logo may need a horizontal version for a website header, a stacked version for packaging, an icon for social profiles, and a single-color version for print. A mistake is designing one perfect composition and forcing it into every situation. Plan the logo family early. Keep the same core shapes, type treatment, and proportions while allowing sensible changes in arrangement.

This does not mean creating many different logos. It means giving the brand a small set of controlled versions for real layout problems. Understanding different logo types and variations can help when deciding which versions are genuinely necessary for a project.

Designer Tip: Prepare primary, secondary, icon, and one-color versions where the project needs them. Document when each version should be used.

10. Presenting a Logo With Weak Final Files and Guidelines

Even a good concept can look unprofessional if the final handoff is messy. Missing vector files, incorrect color values, poorly named exports, and unclear usage rules create problems for clients later. The final package should make sense to someone who did not design the logo.

Include editable vector files, useful raster exports, color versions, monochrome versions, and clear naming. Add basic guidance for spacing, minimum size, background use, and incorrect examples when the project calls for it. A clean handoff shows that the logo was designed for use, not just for a presentation slide.

Designer Tip: Before delivery, open every final file and test the exports. Include a short logo design usage sheet so the client knows which file and version to use.

A Practical Logo Design Checklist

Before you send a logo to a client, check it outside the presentation context. Read the name at a small size. Test the symbol alone, the full lockup, and a one-color version. Check spacing around the mark and compare the visual weight of the symbol and type. Place it on both light and dark backgrounds. Try a basic print test as well.

Then inspect every export for correct dimensions, transparency, and color values. If the logo passes these checks without needing special treatment, it is much more likely to work when the client starts using it.

For additional inspiration and practical resources, explore more logo design resources on Graphic Design Junction before finalizing your next identity project.

Pros and Cons of a Simple Logo System

Pros of a simple logo system include easier reproduction, clearer small-size use, simpler guidelines, and fewer files to manage.

The downside is that a simple concept can become generic when the idea itself is weak. Some brands also need supporting graphics, patterns, or illustrations to give the identity more range.

FAQs

Should a logo always be simple?

No. Simplicity is useful when it improves recognition, reproduction, and clarity. Some brands can use more detail when the logo has a specific illustrative or editorial role. The question is whether the detail survives the applications the brand needs.

How many logo concepts should a designer present?

There is no fixed number. A small set of strong, different directions is usually more useful than many minor variations. Each concept should solve the brief and have a clear reason behind its design choices.

Should I design the logo in black first?

It is a useful method because it makes you judge shape, typography, spacing, and contrast without relying on color. Once the core mark works in one color, you can build the palette around it.

What makes a logo look unprofessional?

Weak typography, poor spacing, unnecessary effects, generic symbols, inconsistent versions, and badly prepared files can all cause problems. A logo can also feel weak when its concept has little connection to the business.

Final Thoughts

Good logo design is mostly about making clear decisions and knowing what to leave out. These mistakes are useful checks when a logo starts feeling forced, crowded, generic, or difficult to use. Go back to the brief, test the mark in realistic sizes, remove anything that does not help, and prepare the identity for the places where the client will actually use it. A logo should pass those practical tests before it gets a presentation treatment.

O0 evolved Fauna Robotics' existing identity into a flexible digital brand system centered on motion, modularity, and the adaptable nature of its Sprout robot, making the product feel approachable while remaining technically credible. The website combines photography, video, 3D, and interactive storytelling to introduce the robot, explain its capabilities, and build trust through layered technical detail rather than marketing claims. The scalable design system helped position Fauna Robotics for growth ahead of its acquisition by Amazon in March.

John Ternus became Apple’s CEO on September 1, succeeding Tim Cook, who remains executive chairman. His appointment puts a longtime hardware engineering leader in charge as Apple faces growing pressure to improve its AI and software experiences.

For designers and product teams, his background offers clues about what could receive greater attention, from how devices feel and age to how reliably an assistant completes a task. It is still too early to identify a distinct Ternus era, but his record provides a starting point.

The experience extends beyond the screen

Ternus joined Apple’s product design team in 2001 and later led hardware engineering. His work spans iPad, AirPods, iPhone, Mac, and Apple Watch. Apple’s leadership biography highlights his contributions to reliability, durability, sustainable materials, and repairability.

These priorities shape everyday user experience. Weight affects how comfortably someone holds a device. Battery life determines whether they can rely on it throughout the day. Repairability influences the cost and inconvenience of keeping it working.

Ternus’s appointment could give these considerations greater weight in product decisions. It does not establish a new design policy, but it places someone with direct experience of those engineering trade-offs at the head of the company.

AI will test Apple’s ability to simplify complexity

Ternus inherits an uneven AI rollout, including delays to promised capabilities. Apple has since presented further Siri upgrades with an emphasis on privacy and everyday usefulness.

The design challenge extends beyond understanding a request. An assistant must communicate what it is doing, recognize when it needs clarification, and let users correct mistakes. Completing a task with fewer taps is only an improvement if people understand and trust the result.

Ternus has described technology as a means of building better products. Applied to AI, that outlook could encourage focused features with clear benefits. Delivering them, however, will require software execution that matches Apple’s attention to hardware detail.

What product teams should watch

Early launches will largely reflect decisions made before the handover. Ternus’s influence will become clearer through the projects he prioritizes and the problems Apple chooses to solve.

For product teams, useful signals will include how Siri handles ambiguous requests, whether users can review and reverse AI actions, and how consistently interactions work across devices. These details will reveal more about Apple’s direction than the number of AI features it announces.

Ternus’s opportunity is to bring the care associated with Apple’s physical products into experiences whose behavior is harder to predict. For users, the difference will be visible in everyday tasks that become easier to complete and easier to correct when something goes wrong.

Apple's marketing for the new $1,999 foldable iPhone Duo has drawn criticism after promotional images appeared to use unrealistically long fingers to make the device seem easier to hold one-handed. The exaggerated visuals sparked jokes and accusations of misleading marketing, adding to the debate over the Duo's ergonomics alongside its price and unconventional design.

Apple's new iPhone 18 Pro and Pro Max wallpapers are available to download as high-resolution static images from the iOS 27 Release Candidate, allowing anyone to use them without buying a new iPhone.

'The Art Travel Book: 40 Iconic Outdoor Artworks' traces land art since the 1960s pioneers.