Full article content is not available for inline reading.

Read the original article →

Ornith has released Ornith-1.5, a family of open models that extends the self-scaffolding framework from Ornith-1.0 into a closed self-improvement loop. Where the previous generation wrote the scaffold around a fixed set of human-curated tasks, Ornith-1.5 proposes the tasks themselves, generates a task-specific scaffold for each one, and produces the solution rollouts used for reinforcement learning. The release covers three scales: a 397B mixture-of-experts flagship, a 35B mixture-of-experts model activating 3B parameters per token, and a 9B dense model shipping with a quantized Mobile build for iPhone and Android.

Each training cycle runs in three stages. Given an environment or codebase, high-level instructions about the task type, and the model's own history of solved problems, the system proposes progressively harder tasks that sit beyond what it has already handled. It then generates or refines a scaffold covering instructions, tools, decomposition strategy, and orchestration, and produces a rollout conditioned on both. Reward propagates back across all three stages, so the system learns to write better solutions, more useful training tasks, and more reliable evaluation harnesses at once. Task reward multiplies three signals: whether the task and scaffold form a valid and verifiable environment, whether difficulty sits near the current capability frontier, and whether the task is novel against work already generated. Frontier difficulty targets a 0.2 empirical success rate, so a task loses value to the generator once the model starts clearing it reliably. Validity acts as a hard gate that zeroes out malformed tasks, and all three stages are optimised with GRPO.

On the company's published tables, averaged over five independent runs, Ornith-1.5-397B scores 85.1 on Terminal-Bench 2.1 and 56.0 on DeepSWE, which Ornith reports as on par with Claude Opus 4.8 at 85.0 and 59.0 and ahead of GLM-5.2 and DeepSeek-V4-Flash-0731 at comparable scale. The 35B reaches 68.5 and 79.0 on Terminal-Bench 2.1 and SWE-Bench Verified while activating 3B parameters per token, and the 9B reaches 47.0 and 70.6, which the company places above Gemma 4-31B and Qwen 3.6-35B. Coverage extends past coding into reasoning and agentic work, with 92.8 on GPQA Diamond and 86.6 on BrowseComp at flagship scale.

Ornith is the model line from DeepReinforce, the research team that shipped Ornith-1.0 in June 2026 across 9B dense, 31B dense, 35B MoE, and 397B MoE variants under an MIT license, with weights on Hugging Face. That release was post-trained on Gemma 4 and Qwen 3.5 checkpoints and introduced the idea of treating the scaffold as a learnable object co-evolving with the policy. DeepReinforce has published reinforcement learning optimisation research in the open before, including CUDA-L1 and the IterX agent loop, and reward hacking has been a running concern across that work. Ornith-1.5 carries that defence into task generation, where an unverifiable task earns nothing.

HuggingFace: https://huggingface.co/collections/ornith-ai/ornith-15

Right-Sizing Your Intelligence Spend

Frontier models have become astonishingly intelligent. In May, OpenAI reported that a general-purpose internal model had disproved a central conjecture in Erdős’s planar unit-distance problem: a question mathematicians had worked on for nearly eighty years. External mathematicians checked the proof; Tim Gowers said that a human-written version would merit immediate acceptance at the Annals of Mathematics. Anthropic reports that Claude Opus 4.6 has found and validated more than 500 high-severity vulnerabilities, including bugs in codebases that had been fuzzed for years and accumulated millions of CPU-hours of testing.

These achievements are real, and they can be extraordinarily valuable. When the answer is unknown and discovering it could be worth millions: a proof, a drug target, a zero-day, or an unfamiliar market structure, maximum intelligence is exactly what we should want. Spending more computation to explore more possibilities is rational.

But the public scoreboard is shaped by the work that can be cleanly stated and verified: proofs, patches, exploits, and scientific reasoning.

The only problem is that the rest of the economy does not work that way.

A claims adjuster is not searching for a new mathematical truth. She is deciding which of the company’s existing rules apply to this accident, this policyholder, and this sequence of events. A nurse is not inventing medicine. She is combining established protocols with one patient’s history, current symptoms, and the specific constraints of her hospital. A logistics coordinator is not discovering a new algorithm. He is reacting to today’s inventory, weather, contracts, and delays. Some jobs create value by expanding the frontier of what is known. Most create value by making the right decision within the boundaries of what an organization already knows.

The Economy Is Not a Research Lab

Frontier research is extraordinarily valuable and economically narrow. BLS data shows that life, physical and social-science occupations account for less than 1% of American employment. The United States employs roughly 2,000 mathematicians, 20,000 physicists and 37,000 computer and information research scientists. It employs millions of nurses, managers, administrators, logistics workers and customer-service representatives.

Most companies do not spend their days discovering “unknown answers”. They spend them applying what they already know across millions of decisions shaped by customers, policies, inventory, contracts and history.

And so while the frontier is an intelligence problem, the rest of the economy is a context-and-execution problem; this distinction is visible across the F500. Take Walmart - they coordinate inventory, stores and suppliers. UnitedHealth coordinates care. Amazon operates warehouses, deliveries, merchants and customer interactions at massive scale.

What makes the majority of these companies defensible is not superior “frontier” intelligence. It is proprietary data, dense operational context, long-built processes, supplier relationships, regulatory licenses, physical networks, and the accumulated judgment that only comes from performing the same class of decision millions of times. Their moat is not that they think better than everyone else but that they already know how their world works over decades of experiencing it.

Imagine replacing every employee at American Airlines, Home Depot or Medtronic with a math olympiad who approached every task as a problem of discovery. Costs would not fall, but would rise rapidly and dramatically as every routine decision would be re-examined from first principles. The organization would drown in intelligence it cannot productively use or absorb.

Coding is the important exception: it is a large labor market, the work is digital, and outcomes can often be verified through tests. That makes additional model intelligence unusually valuable. However, the frontier labs products themselves are often designed in ways that make it easy for developers to blast tokens: long-running agent loops, recursive sub-agent spawning, and “keep going until it works” patterns. Thus, even coding is developing capability thresholds as smaller and open-weight models approach frontier performance on increasingly broad classes of work.

Every Workload Has an Intelligence Threshold

Every workload has some intelligence threshold. Below it, the model cannot perform the task. As the model approaches it, greater intelligence creates enormous value. Once the threshold has been crossed, the bottleneck changes. The outcome depends increasingly on whether the system knows the company’s policies, customers, history, tools and standards and whether it can act reliably, quickly and cheaply.

Models keep getting more intelligent, but most workloads are not. Take refund policies, baggage rules and invoice-reconciliation procedures - these are not getting any harder. The latest frontier models, particularly from Anthropic, are optimized for “extreme” intelligence — the kind required by research mathematicians, physicists, and engineers solving previously unsolved problems. That market is real. It is also small. The problem arises when the same models become the default for ordinary enterprise work.

Developers naturally reach for the most capable system available. Anthropic’s products make that choice frictionless: long-running agents, recursive sub-agents, and high-effort reasoning loops that burn tokens by design. The result is frontier-priced intelligence applied to tasks that often do not need it. Two trends are eroding that premium: the shift from closed to open-weight models, which lowers the cost of a given level of capability, and the shift from frontier-scale to small and mid-sized models, which compresses that capability into a much smaller compute footprint. Qwen3.8-27B illustrates the latter: at just 27B parameters, it matches or beats Opus 4.6 Max on several coding and agentic benchmarks. Our intelligence-efficiency results point in the same direction: intelligence per joule improved 18× in 16 months, delivering comparable capability with dramatically less compute, energy, and ultimately cost.

As more models cross the intelligence threshold for ordinary enterprise work, frontier systems should be reserved for the smaller set of tasks where additional intelligence still changes the outcome.

Companies already understand this dynamic when hiring people. Have you ever hired an employee that was dramatically overqualified for a role? They may reconsider settled decisions, introduce unnecessary complexity, become bored and eventually leave. The organization would have been better served by someone sufficiently capable who understood its customers and how the work actually gets done. Models are overqualified too, with a more persistent failure mode. However, instead of getting bored and quitting, they keep generating extra complexity, variance and cost indefinitely. A password reset does not improve because an agent considers twelve explanations, launches a security investigation and composes a personalized essay. Once the customer’s identity is verified, the job is to reset the password safely and stop.

Overqualification is now becoming measurable vs just being an illustrative analogy. Researchers studying reasoning models have documented an “overthinking” problem: models routinely spend additional inference compute on easy questions without improving accuracy. Amazon researchers estimate that reasoning systems can generate 7 to 10x as many tokens as necessary on simple tasks. The marginal value of intelligence is therefore not merely diminishing. On sufficiently easy workloads, the marginal reasoning token can approach zero.

Is the task failing because the model is not intelligent enough or because it does not know how this company works? Enterprises that continue to treat maximum intelligence as the default will find themselves paying frontier prices for work that no longer requires it. The correct posture is the opposite: assume open-weight and specialized systems will keep eating the stack, and reserve frontier models only for the problems that still sit above the rising capability line.

The Labs and Enterprises Have Different Incentives

There are competing incentives between labs and enterprises. A model lab wins by making more computation useful; its success metrics are tokens consumed, session length, agent count, reasoning depth, and the ability to tackle harder problems. Every additional unit of intelligence it can sell is a victory. The lab is incentivized to make the frontier model the default, to encourage longer thinking, more tools, more retries, and more agents. Its dashboard lights up when usage rises. An enterprise wins by making repeated computation unnecessary. Its success metrics are cost per verified resolution, tokens per processed claim, time to completion, escalation rate, and customer outcome. The moment a company learns enough to turn a recurring request into a three-bullet artifact, a deterministic rule, or no model call at all, the lab loses a future revenue stream.

The provider has no incentive to help the customer eliminate the call, route it to a cheaper rival, or compress it into software. The divergence is absolute the moment the enterprise learns. The lab is still trying to sell more intelligence. The enterprise is trying to need less of it. The data already shows the gap. OpenAI reports that average reasoning-token consumption per enterprise organization rose approximately 320-fold over the past year. PwC’s survey of 4,454 CEOs found that 56% had not yet seen a significant financial benefit from AI.

Measurable business outcomes are not keeping pace. One side is succeeding on its own terms. The other is not. A model dashboard that reports token usage up five times is a success for the lab. For the CFO it is an unanswered question: did customer retention improve, did cost per successful outcome fall, or did we simply burn more intelligence on the same work? These are not the same game. One player is paid to increase the size of the fire. The other is paid to put it out and keep it from starting again.

The Architecture Follows the Economics

If an enterprise’s goal is to reduce intelligence consumed per successful outcome, it cannot leave every decision about intelligence consumption to the model provider. That control must live inside the enterprise at every layer. Three predictions for the future:

• There will be millions of routers and gateways, inside every serious AI harness, at the policy layer, at the team level, and at the organization level. Each one will decide, for every unit of work, whether it belongs on a frontier model, a smaller open-weight model, a local model, a deterministic system, a human, or nowhere at all. The TAM for this routing layer will ultimately look a lot like the workforce itself — the vast majority of ordinary decisions handled cheaply and locally, with only a thin slice of genuine discovery reserved for premium intelligence. Above it, every single organization will have a gateway that will enforce security and data-residency requirements, negotiate cost, set budgets, and determine which models should be permitted for different classes of work.
• Hybrid workloads will become the default. Once applications can allocate intelligence, workloads will stop belonging to a single model. Right now almost every query is sent to the cloud by default, even when the data is sensitive, the task is simple, or the answer already lives inside the company’s own systems. That is expensive, slow, and often unnecessary. In the future the easy, repetitive, or private parts of a task will stay local or near the system of record. Only the genuinely hard or novel parts will be sent to a frontier model in the cloud. In our research, we have already demonstrated the shape of this architecture. In our Minions work, a frontier cloud model decomposed complex long-context problems while smaller local models executed the resulting subtasks. The hybrid system recovered 97.9% of frontier model accuracy at 5.7× lower cost; a more aggressive configuration achieved 87.9% at 30.4× lower cost. The important result is architectural, not merely economic: expensive intelligence was most useful for deciding what work needed to be done, not for performing every unit of it.
• AI will diffuse through two complementary application surfaces. General-purpose proactive assistants such as Instinct will make intelligence ambient. They will observe work, identify opportunities, and initiate tasks without waiting for a prompt. Specialized applications such as Maximor and PlayerZero will do something different: they will turn enterprise intent into reliable execution by controlling the context, tools, permissions, workflow state, and definition of done for a particular job.

Beneath those harnesses, a new improvement layer will emerge. Platforms such as Applied Compute’s AC2 allow companies to train, evaluate, serve and replace the models operating inside their existing applications. The harness remains the system of execution; AC2 helps the intelligence inside it become better, faster and more economical.

The proactive assistant becomes the demand aggregator. The application harness becomes the execution system. AC2 becomes the specialization and improvement infrastructure connecting production experience back to the models. It continuously moves each workload toward the least expensive system still capable of producing the correct outcome.

The most intelligent enterprise is not going to chase the “frontier models” blindly for all work. It will be the one that has learned how to need intelligence the least, turning yesterday’s expensive reasoning into tomorrow’s cheaper, more reliable execution.

Progress will not be measured by how much intelligence a company can afford to consume, but by how quickly intelligence consumed per successful outcome falls toward zero.

Pharmaceutical partners Moderna and Merck announced on Wednesday that their novel mRNA-based vaccine—individually tailored to target a patient’s unique cancer mutations—was effective in a late-stage clinical trial of patients with melanoma, one of the deadliest forms of skin cancer.

The announcement was scant on details, but if the success holds, it stands to realize the high aspirations for both mRNA vaccines and individualized cancer-targeting medicines in cancer therapy, providing a first success for both treatment types.

The Phase 3 trial included 1,137 patients who had stage IIB–IV melanoma that had been surgically removed prior to their treatment in the trial. Patients were randomized 2-1 to receive either a combination treatment of the tailored mRNA vaccine (intismeran or mRNA-4157) with Merck’s established monoclonal antibody cancer treatment Keytruda, or they received treatment with Keytruda alone. Both patient groups were treated for about a year. The trial was placebo-controlled and double-blind, meaning neither the doctors nor the patients knew which treatment patients were randomly assigned to receive.

Based on a planned interim analysis, Moderna and Merck report that the combination therapy with the mRNA vaccine extended “recurrence-free survival” (RFS), which is the amount of time a patient has without the cancer returning. The combination therapy also extended “distant metastasis-free survival” (DMFS), which is the amount of time a patient has without the cancer returning and occurring in a different part of the body than where it began.

The companies say the extensions of RFS and DMFS reported were statistically significant and clinically meaningful, but they did not elaborate. They said the data will be presented at an upcoming international medical conference. They did note, however, that data from a five-year Phase 2 trial, which was presented at a cancer researcher conference in June, showed a 49 percent reduction in the risk of recurrence or death and a 59 percent reduction in the risk of distant metastasis or death.

The mRNA vaccine intismeran works using a synthetic mRNA that holds the unique genetic coding for up to 34 mutations in a patient’s own cancer cells. The production process involves comparing the genetic sequences of a patient’s cancer cells to those of healthy cells and picking out any cancer-specific mutations that could help the immune system differentiate cancer cells from healthy ones.

Promising design

Using the same platform Moderna used for its mRNA COVID-19 vaccine, intismeran delivers to healthy cells the mRNA code for those mutations. Healthy cells translate that mRNA code into protein fragments representing the cancer mutations, and then those fragments are presented to immune cells as antigens, a foreign substance to attack. As such, Moderna and Merck call their mRNA cancer vaccine a type of “neoantigen therapy.”

The term avoids both “mRNA” and “vaccine,” which draw hostility from Trump’s anti-vaccine Health Secretary Robert F. Kennedy Jr. In addition to his concerted efforts to undermine lifesaving vaccines, Kennedy has canceled hundreds of millions of dollars in federal grants for the development of mRNA vaccine technology.

In the companies’ announcement, Georgina Long, who led the trial, said the results were “a landmark moment” with the “potential to establish a new treatment paradigm.” Dean Li, president of Merck Research Laboratories, said the findings “reinforce the promise of a more personalized approach to cancer treatment.” Moderna CEO Stéphane Bancel, meanwhile, highlighted mRNA technology’s role, saying this is “a pivotal moment for the field of cancer research.”

“For many years, the idea of creating an mRNA treatment designed specifically for an individual patient’s cancer was aspirational. We are now helping turn that vision into a reality,” Bancel said.

Despite the lack of details, outside experts are similarly hopeful and encouraged by the announcement. “This is the first positive Phase III trial of an individualized neoantigen therapy and an mRNA-based cancer treatment,” University of Oxford cancer expert Lennard Lee said in a statement. “That makes this an important moment for a field that scientists have been working towards for many years. Within six years of the pandemic, we have mRNA vaccines to treat cancer,” he said, also calling it “very encouraging” and “significant.”

Lee, however, highlighted that we don’t yet know the magnitude of the benefit, the detailed subgroup analyses, quality-of-life data, or mature overall-survival results. “Those details will allow the scientific and clinical community to understand precisely how large the benefit is, which patients benefit most, and ultimately where this treatment might sit within routine melanoma care,” he said.

Full article content is not available for inline reading.

Read the original article →

Postgres 19: How Our Advice Has Changed Since We Wrote It

Over the years we have written a lot about how data gets into Postgres, how it sits on disk, and how indexes help you find it again. Some of that advice was written against Postgres 10 or 11. A surprising amount of it is still exactly what we would tell you for the upcoming Postgres 19 release. Functionality described here is based on current betas; minor details may still change before GA.

This post revisits Crunchy posts in the “load, storage, indexes, and partitioning” bucket: what we wrote, which version moved the needle, and what we would tell you to do now. Along the way: async I/O, more resilient COPY, LZ4 by default, richer BRIN shapes, skip scan, and smoother partition operations.

Async I/O: faster scans and vacuum on modern storage

In 2019 we benchmarked a BRIN index against a B-tree and a parallel sequential scan on the same time-series table. Sometimes BRIN won. Sometimes the parallel seq scan won: four workers chewing through the heap beat a clever index. That was the right lesson for Postgres 11: indexes are a tradeoff against what the executor can already do in parallel.

Postgres 18 made those heap reads substantially faster.

Async I/O lets backends queue multiple disk reads instead of waiting on each one. Sequential scans, bitmap heap scans (the path BRIN and many bitmap index plans finish with), and vacuum all benefit. Community benchmarks have shown up to ~3× on cold, latency-bound storage, a big deal for cloud disks. Defaults matter here: io_method = worker is on out of the box; on Linux 5.1+ you can try io_method = io_uring.

Postgres 19 builds on that: I/O workers can autoscale (io_min_workers / io_max_workers), read-ahead scheduling improved, and EXPLAIN (ANALYZE, IO) can show what the async subsystem is doing. Parallel query is still there; each worker can queue several reads and keep making progress while some of them are still in flight, so you get more useful work between waits. Parallel autovacuum workers also landed in 19 (autovacuum_max_parallel_workers and per-table autovacuum_parallel_workers), so maintenance can fan out, but the defaults are conservative: tune them when vacuum is falling behind on large tables.

One related default flip: JIT is off by default in Postgres 19 (it had been on since 12). The old cost model was unreliable, so large analytical or parallel scans that used to compile at runtime no longer will unless you turn jit back on. If those workloads matter to you, re-enable explicitly and re-check plans after upgrade.

Postgres 19 advice: Keep choosing indexes for selectivity, and re-test BRIN-vs-parallel plans on your storage with EXPLAIN (ANALYZE, BUFFERS, IO) after you upgrade. Tune effective_io_concurrency / maintenance_io_concurrency with the new defaults in mind (they rose to 16 in 18). Treat async I/O as more headroom for the heap paths, alongside good COPY, TOAST modeling, BRIN, and covering indexes.

Loading data: COPY is still king, and now more resilient

What we wrote: In Fast CSV and JSON Ingestion in PostgreSQL with COPY (2018, Postgres 10), Jonathan Katz showed the classic pattern: generate CSV or newline-delimited JSON, pipe it into psql, and let COPY ... FROM STDIN do the heavy lifting. Prefer COPY over row-by-row INSERT. Store JSON as jsonb. Add a GIN index when you need containment queries.

That advice was right then and is still right now. The load path itself did not need reinventing; it got more capable.

What changed:

FREEZE on an initial load into a freshly created or truncated table is still the right performance trick when you want to skip a later freeze vacuum.

Postgres 19 advice: Keep using COPY for bulk ingest, including imperfect feeds. A practical 19-era load looks like this:

COPY events (event_id, occurred_at, payload)
FROM STDIN
WITH (
  FORMAT csv,
  HEADER 2,
  DEFAULT '__DEFAULT__',
  ON_ERROR set_null,
  LOG_VERBOSITY verbose
);

HEADER 2 skips two lead-in lines. ON_ERROR set_null keeps the row and nulls only the bad field instead of discarding the whole line. Prefer ON_ERROR ignore when a bad cell should drop the row entirely; pair that with REJECT_LIMIT when you want a hard cap on how many skips you will tolerate.

Prefer FREEZE for empty staging tables that you load once and then promote. And when you need to export partitioned data or hand JSON to another system, COPY TO handles the parent directly:

COPY events TO STDOUT WITH (FORMAT json);
COPY events TO STDOUT WITH (FORMAT json, FORCE_ARRAY);

Plain FORMAT json streams newline-delimited JSON (NDJSON/JSONL: one object per line), which is handy for batch pipelines. FORCE_ARRAY wraps the whole result in a single JSON array, which some web APIs expect.

TOAST: same mechanism, faster default compression

What we wrote: In Postgres TOAST: The Greatest Thing Since Sliced Bread? (2024), Elizabeth Christensen walked through the 8 kB page, the ~2 kB toast_tuple_target, storage strategies, and the practical guidance: updating a toasted row rewrites toast chunks, frequent access of compressed values has a cost, and large JSON or text that you query often belongs in a better data model, not in a giant toasted blob.

That mental model is still the right one. TOAST did not get replaced; it got a better default compressor.

What changed:

Postgres 19 advice: Keep Elizabeth’s modeling advice. Prefer structured columns for hot lookup data when you can. Prefer EXTERNAL when you have already compressed the payload yourself. On a fresh Postgres 19 cluster, LZ4 is what new toasted values get unless you override it.

When you do need a rewrite to reclaim bloat, Postgres 19’s native REPACK is the clean path; CONCURRENTLY keeps the table readable and writable while the new heap is built:

REPACK (CONCURRENTLY, ANALYZE) events;

CONCURRENTLY needs a primary key or index-based replica identity. It is not MVCC-safe in the same narrow sense as TRUNCATE: a concurrent transaction that took a snapshot before the final swap and had not yet touched the table can briefly see it as empty.

BRIN: still tiny, richer opclasses, faster to build

What we wrote: In PostgreSQL BRIN Indexes: Big Data Performance With Minimal Storage (2019, Postgres 11), Jonathan showed BRIN crushing a B-tree on size for append-mostly sensor data. Later posts sharpened when BRIN shines: high correlation with physical order. Random or shuffled layouts need a different shape of index.

What changed:

Postgres 19 advice: Start with Paul’s question: is this column correlated with physical order? If yes, use BRIN. On 14+, prefer multi-minmax when the table is large and not perfectly ordered. Tune pages_per_range to your typical filter width.

On Postgres 17+, BRIN builds on large tables can use parallel workers automatically. Re-run the bake-off on your storage, especially cloud disks, and pick the winner with fresh numbers.

Covering indexes: INCLUDE still helps, and skip scan covers more queries

What we wrote: In Why Covering Indexes in Postgres Are Incredibly Helpful (2018, Postgres 11), Jonathan introduced B-tree INCLUDE columns so queries could satisfy SELECT lists from the index alone. That tradeoff is unchanged. INCLUDE is still how you build a covering index without polluting the sort key.

What changed:

Postgres 19 advice: Still use INCLUDE when you know a hot query’s filter columns and its select list. For your other indexes, check whether skip scan already covers a query before you add another index, especially when the leading column has low cardinality.

Partitioning: still for lifecycle, now smoother to operate

What we wrote: In Partitioning with Native Postgres and pg_partman (2022), Elizabeth Christensen framed partitioning as a scale and lifecycle tool: archive old ranges by detaching, create tomorrow’s child, let pg_partman handle the calendar. That still works.

What changed:

Postgres 19 advice: Keep partitioning for retention and operational boundaries first. For day-to-day rotation, prefer concurrent detach. When you need to coarsen history or split a hot range, Postgres 19 can do it in SQL:

ALTER TABLE events
  MERGE PARTITIONS (events_2024_01, events_2024_02, events_2024_03)
  INTO events_2024_q1;

Plan merge/split for a maintenance window: they take ACCESS EXCLUSIVE on the parent and do not yet offer CONCURRENTLY. Exporting a partitioned table is simpler too: COPY TO runs directly on the root relation and pulls rows from all descendants.

What hasn’t changed

1. Prefer COPY over row-by-row INSERT for bulk loads.
2. Store JSON as jsonb, not text json.
3. Keep hot data out of TOAST when you can.
4. Indexes are a tradeoff, not free.
5. The planner may skip your index. Verify with EXPLAIN (ANALYZE, BUFFERS).
6. B-tree is the default workhorse.
7. BRIN wins on correlated, append-mostly columns.
8. Covering indexes (INCLUDE) enable index-only scans.
9. Partition for lifecycle and cheap drops first.
10. Unique indexes on partitioned tables need the partition key.
11. Use a default partition as a safety net, and drain it.
12. pg_partman still handles the schedule.

The advice did not flip. Postgres got better at delivering it: more resilient loads, faster compression by default, more BRIN shapes, wider B-tree usefulness, quicker disk reads, and smoother partition operations. If you still follow those older posts on Postgres 19, keep the mental model, update the edges of the checklist, and re-test after upgrade with EXPLAIN (ANALYZE, BUFFERS, IO).

A Preview of DuckDB v2.0

TL;DR: DuckDB v2.0 is coming this fall. In this post, we preview its headline features: DuckDB as a server, triggers, the VARIANT type, asynchronous I/O, a new SQL parser, a new storage format, and much more.

DuckDB v2.0 will be named “Cyanoptera” after the cinnamon teal (Anas cyanoptera), a strikingly reddish-brown duck found in the western Americas.

A major version bump is not something we do lightly, and it is not just ceremony: v2.0 ships a new SQL parser, a new default storage format, a reworked C API, and a small number of carefully chosen breaking changes. But above all, it is a feature release, built from over 10,000 commits since we released v1.5 in March. Where last year was the year of the lakehouse, this release kicks off the year of DuckDB as a server.

DuckDB is moving rather quickly, and we can only cover a small fraction of the changes here. Condensing all new features down to a shortlist is always a fight over what gets in, and yes, we know that what follows is technically a listicle (Ten Things Coming to DuckDB v2.0, Number Eight Will Shock You). We are not proud of the format, but it works, so here it is, starting with the SQL-level features and working down into the engine.

1. DuckDB as a Server: Quack and CONNECT

DuckDB has been an in-process database since day one. But people have asked us – very persistently – for a client/server mode, and we have finally caved. The quack extension implements DuckDB's native protocol for talking to other DuckDBs. It graduates to stable in v2.0, and it is a big part of where DuckDB is headed: any DuckDB process can serve its databases over the network, and any other DuckDB can attach to it and route queries there using the new CONNECT statement. For example:

DuckDB server

CALL quack_serve(
    token = 'my_token'
);

DuckDB client

ATTACH 'quack:server.example.com'
    AS qk (TOKEN 'my_token');

CONNECT qk;
SELECT count(*) FROM events;
-- executes on the server,
-- results stream back
DISCONNECT;

CONNECT is the successor to the remote.query($$...$$) workaround we showed when Quack was first revealed – we looked at that syntax and said: no, this cannot be it. And CONNECT is not limited to Quack: it points your session at any remote database that supports it, and the new remote pushdown optimizer ships SQL directly to PostgreSQL and MySQL instead of pulling tables over the wire:

CONNECT 'postgres://localhost/mydb';
SELECT count(*) FROM orders; -- runs on the PostgreSQL server
DISCONNECT;

If you have worked with analytical systems in the past, you may assume that DuckDB cannot handle transactional workloads. But DuckDB has been built as a transactional, multi-connection database with full MVCC and transaction isolation since day one. Most users just never needed that in a single-user scenario. It turns out DuckDB handles transactions well: it's fast enough to compete with general-purpose databases like PostgreSQL on quite a few workloads, and the client/server pattern finally lets that machinery shine in multi-tenant, long-running deployments.

2. VARIANT Becomes a First-Class Citizen

The VARIANT type shipped in DuckDB v1.5, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. Like JSON, a VARIANT column can store differently-shaped data in every row. Unlike JSON, it is not a text format: DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage and executes fast in queries, all without you ever declaring a schema. This makes VARIANT a natural fit for real-time log ingestion, where streams of JSON-ish records share structure but evolve over time.

In v2.0, this pipeline works end to end: shredded execution straight from storage, extraction pushdown into scans, shredded VARIANT reading and writing for Parquet, and a family of variant_* functions:

CREATE TABLE events (payload VARIANT);
INSERT INTO events
VALUES ('{"user": {"id": 42, "tags": ["a", "b"]}}'::JSON::VARIANT);

SELECT variant_type(payload), variant_keys(payload)
FROM events;

SELECT *
FROM events
WHERE variant_contains(payload, {'user': {'id': 42}}::VARIANT);

3. Triggers

Triggers have been a long-standing feature request, and DuckDB v2.0 delivers them in full: BEFORE and AFTER triggers, FOR EACH ROW and FOR EACH STATEMENT, transition tables via REFERENCING OLD/NEW TABLE, multiple triggers per event, RETURNING on triggered tables, and DROP TRIGGER.

The classic use case is audit tables: something happens in the system, and a trigger records what changed. For example:

CREATE TABLE target (id INTEGER, val INTEGER);
CREATE TABLE audit (id INTEGER, old_val INTEGER, new_val INTEGER);

CREATE TRIGGER trg_audit AFTER UPDATE ON target
REFERENCING OLD TABLE AS o NEW TABLE AS n
FOR EACH STATEMENT
    INSERT INTO audit
    SELECT n.id, o.val, n.val
    FROM o
    JOIN n ON o.id = n.id;

INSERT INTO target VALUES (1, 10), (2, 20);
UPDATE target SET val = val * 10 WHERE id <= 2;
SELECT * FROM audit;

4. SQL Dialect Additions

As always, DuckDB's SQL dialect keeps growing. A few favorites from this release cycle:

With NEAREST joins, top-k similarity search becomes a join clause, handy for vector and embedding workloads:

SELECT q.user_id, t.product_id
FROM users q
    INNER JOIN products t APPROX NEAREST 2
    BY SIMILARITY array_cosine_similarity(q.embedding, t.embedding);

DML inside CTEs lets you use INSERT, UPDATE, DELETE, and COPY as pipeline steps:

WITH moved AS MATERIALIZED (
    DELETE FROM staging RETURNING *
)
INSERT INTO archive SELECT * FROM moved;

Nested schemas allow schemas within schemas:

CREATE SCHEMA finance;
CREATE SCHEMA finance.reports;
CREATE TABLE finance.reports.q3 (revenue DECIMAL);

The new variable syntax lets you write $x anywhere an expression is allowed:

SET VARIABLE threshold = 100;
SELECT * FROM orders WHERE amount > $threshold;

The JSON mutation functions json_set, json_insert, json_replace, and json_remove finally let you modify JSON documents in place:

SELECT json_set('{"a":1}', '$.b', '2');

And recursive CTEs with USING KEY aggregation enable iterative algorithms in pure SQL.

5. Asynchronous I/O

Interacting with object stores like S3 is central to the DuckDB experience. DuckDB v2.0 introduces asynchronous I/O throughout the engine. Thanks to asynchronous access, the I/O layer now scales independently from the query processing layer, which means far more parallelism for remote reads and dramatically faster queries on network storage.

6. Faster Queries Across the Board

As with every release, a lot of work went into making your existing queries faster without you doing anything. Recursive CTEs are significantly faster, aggregations now spill to disk when they outgrow memory, and row-group pruning has been massively expanded for complex types and filters. Query planning also becomes partition-aware, allowing better performance on lakehouse formats.

7. Storage Format v2.0

DuckDB v2.0 bumps the default storage format version to v2.0.0. Column metadata is now loaded lazily, so wide tables open faster. The DICT_FSST string compression method is enabled by default, deletes are stored compactly, and the storage layer performs much stronger corruption validation on read.

8. A Brand New SQL Parser

DuckDB v2.0 ships our own modern, extensible PEG-based parser. This change ties into the extension ecosystem: extensions can now hook into the grammar itself, so expect extensions that expose entirely new SQL syntax. It also brings better error messages with precise source locations, and dialect compatibility modes.

9. Timezones, Calendars, and Collations Without ICU

In v2.0, the ICU library is gone entirely: the icu extension now implements timezones, calendars, and collations itself, with the timezone data built directly from the IANA database. Everything keeps working as before, but the implementation is smaller, easier to keep up to date, and significantly faster.

10. Write Extensions Once, Host Them Yourself

DuckDB v2.0 will ship with a revamped C API. The API will have a versioned specification expressed in YAML, providing a stable ABI across DuckDB versions. You won't need to re-target or rebuild extensions every time a new DuckDB version comes out. Additionally, you will be able to register your own trusted repositories, so an organization can host and sign its own extensions and have them install and load just like the built-in ones.

Bonus: DuckDB Foundation – Advisory Board

Starting this fall, we will add a stakeholder advisory board to the DuckDB Foundation. The advisory board will provide input on the development roadmap of DuckDB, DuckLake, and Quack.

Final Thoughts

These are only a few highlights, and this post is only a preview. There have been more than 10,000 commits by many contributors since we released v1.5. We would like to thank our community for the detailed issue reports, feedback, and contributions that shaped this release.

We turned off Pub/Sub and nobody noticed

Like many modern software stacks, the incident.io platform is predominantly event-driven. For example, whenever you send us an alert, post a message to our agent on Slack, or update an entry in your Catalog - these are all events that then get enqueued on a message topic, meaning any of our downstream components that are interested in that event can subscribe and react asynchronously, such as sending a push notification or posting a reply to you in Slack.

As the platform has grown, so has the number of messages flowing through our system, and thus our dependence on our messaging infrastructure. It’s become mission-critical. At the same time, we've also set stricter availability targets for ourselves, like the 99.99% SLA we now commit to for Enterprise customers of our On-call product.

Until recently, we only used a single provider - Google Cloud Pub/Sub - as our messaging technology. Meaning any blip in Pub/Sub availability meant a blip in our own availability, which isn’t acceptable. So we recently set out on an adventure to make our messaging system more resilient to failure by introducing a secondary message broker to our stack, adding redundancy, and ultimately increasing the availability of our entire platform.

The goal was to be able to turn off Pub/Sub with zero customer impact. It turned out to be quite the adventure, but last week, we successfully did exactly that.

This is the story of that adventure.

The message broker

Event-driven systems have many benefits, like allowing us to decouple the rate at which we process messages from the rate of ingestion, or have many different components process the same original customer-initiated event, for example, having a user.created topic, and one system listens for events to send a welcome email and another that sets up their initial database state.

At the heart of such a system is typically a “message broker”, which is responsible for receiving messages from the publishing components, storing them, and forwarding them to any interested subscribers. At incident.io, we’ve historically used Pub/Sub as our message broker of choice; it’s a well-built managed service with a good feature set, and has allowed us to scale with ease over the years.

Pub/Sub is solid. Its published SLA is 99.95%, and in practice it has comfortably beaten that for us. The problem was that every event in our platform flowed through one broker, operated by one provider, with no way to route around it. Our message broker had become a single point of failure (SPOF), and this was at tension with our own 99.99% availability targets.

And a SPOF is ultimately a question about accountability. When an escalation doesn't fire, "Sorry, Pub/Sub was down" is not an answer we ever want to give a customer. It's our SLA, and it's our job to meet it, whatever our dependencies are doing that day. We already have redundancy in the other layers of our infrastructure, so why should the message broker be treated any differently?

The eventadapter

When we say we have an event-driven architecture, we’re not exaggerating; we currently have ~800+ individual message topics, 1000+ unique subscriptions to those topics, and are processing ~240 million messages a day.

This large number of topics also means there are thousands of call sites in our codebase which interact with events, which can be quite daunting when you want to, say, replace the underlying technology you use for messaging.

Fortunately, we were standing on the shoulders of giants, and the early engineers at incident were wise enough to build code-level abstractions over message publishing and subscribing, which we call the eventadapter. This is a package that exposes some simple but powerful interfaces like:

// Publisher is the interface for publishing events.
type Publisher interface {
	Publish(ctx context.Context, ev Eventer, payload []byte) (string, error)
}

// Subscriber is implemented by all subscribers.
type Subscriber interface {
	Subscribe(ctx context.Context, topicName string, handler SubscribeHandler, params SubscribeParams) func() error
}

// SubscribeHandler is what consumers of the package implement to 
// handle a single event.
type SubscribeHandler[EV Eventer] func(
	ctx context.Context, ev *EV, eventMetadata EventMetadata,
) error

// Eventer is the interface implemented by all events.
type Eventer interface {
	// Name is how we identify this type of event.
	Name() string

	// A description of what the event means.
	Description() string

	// Validate validates the fields of the event before publishing.
	Validate() error

	// GetOrganisationID returns the organisation ID associated with 
	// the event, which we use to add to event telemetry.
	GetOrganisationID() string
}

The package exposes a couple of concrete implementations of these interfaces, like a eventadapter.InMemory for use in local development or tests, or eventadapter.PubSub for talking to Pub/Sub.

Fun fact: we originally had to build the InMemory adapter as, in the early days of incident, running the app locally and opening so many parallel connections to Pub/Sub would crash the office wifi.

We conditionally choose and construct which version of the adapter to use at runtime in func main() based on the environment, and pass that down as a dependency to our application components.

Having the luxury of such an abstraction meant that, to introduce a new message broker, what we needed to do was build a new implementation of the eventadapter interface, swap it in at runtime, without any of the calling code owned by other engineers being aware. This allowed us to hide all of the complexity that comes from load balancing across two different brokers behind the abstraction.

Choosing a second broker

One trade-off that came with using the existing interface meant that we are also constrained to the semantics and behaviors of that interface - which, even though it's an abstraction, already had some leakiness from the underlying technology. I.e. we needed to choose something that at least had the same feature set and behaviors as Pub/Sub, as these were semantics that we had come to rely on and reason about in our application.

Our requirements and process for choosing a second message broker are out of scope of this post. There is a broad landscape of brokers: open-source vs proprietary, managed vs unmanaged, streaming vs non-streaming, ephemeral vs persisted, etc. There is no silver bullet, so our main advice is to document your own requirements and use a decision matrix.

We chose NATS mainly because it’s a CNCF-adopted project, which means we can have confidence in its future and openly read the source code, is Kubernetes-native (which is where we run our workloads), is a single binary (we’re looking at you, Kafka!), and it is written in Go (the rest of our stack is Go!). Additionally, we had some prior experience running it.

Dynamic load balancing

Another design principle was that, when operating at 99.99% of availability, failover between brokers can’t be manual; with ~4 min 23 secs of downtime budget a month, we don’t have time for someone to wake up at 4 am and switch brokers. This meant that, ideally, we had to use both brokers in an active-active setup continuously: messages get balanced across both, and any persistent error rate from one broker would automatically fail over to the other. So we needed to build a dynamic event load balancer! Let’s dig into how we did that.

As discussed above, the first thing was to create a new concrete implementation of the eventadapter that we called eventadapter.LoadBalancer.

On the publish side, we pick a broker by hashing the message.ID and rolling a weighted dice against a configurable split. The ID is a ULID (like all IDs in our system), hashed with Go's hash/fnv std-library (the Fowler–Noll–Vo hash function). Because every message hashes independently, a 50/50 split sends roughly half of all messages to each broker — the even distribution you want from a load balancer.

💡 You may wonder why we sample by message ID here, instead of, say, our typical grouping key, which is organisation ID? Hashing by org ID would mean all of a customer’s events would get pinned to one broker until failover, whereas hashing per-message keeps load even by volume no matter how lopsided any one org is. The trade-off is that there is no per-org or per-operation transport consistency - which we’re happy to live with.

In normal operation, we’ve chosen to have a 50/50 split across each broker; why invest all this work in a secondary broker if you only use it in an emergency to then find out it's broken? Importantly, the split is configurable without a deploy, in case we need to turn either broker off manually.

Once we determine the preferred broker for a message, we attempt to publish the event, and if a publish attempt fails (maybe it timed out due to a short network blip), we fail over to the other. All attempts to a given broker also flow through a circuit breaker, so if many attempts in a short period of time start to fail as the broker is degraded, we short-circuit the publishes to that broker early and instantly fail over. Giving us the automated failover we need to reach our availability targets! No one gets woken up; publishes just gracefully start flowing to the other provider.

Fairness weighted scheduling

You could argue that the publish-side is a pretty standard load balancer; where it gets more interesting is the subscriber-side and how we handle processing concurrency.

The incident.io system is a single mono-repo Go program that is then deployed to Kubernetes as a collection of different workload-type-based deployments, such as worker-oncall or worker-ai, this allows us to do things like horizontally scale the number of replicas that receive inbound HTTP alerts independently of, say, our AI-message processing. We’ve talked about this architecture in more detail before: Keep the monolith, but split the workloads.

In the eventadapater interface, we also have similar controls over the number of concurrent message “handler” functions (or more accurately, goroutines) we run per-machine to process messages for a given topic, a setting called MaxHandlers. So that we can do things like: configure 10 handlers to process webhooks per-machine but only 3 handlers for a lower-priority background cron job.

Therefore, we needed to think about how we could map the concept of concurrent handlers to the new dual-broker world. The rudimentary solution would have been to simply double the number of handlers, one group of handlers per broker. However, that presents a couple of issues:

1. It would double the potential throughput of concurrent work on a machine and thus directly increase our resource usage (CPU/memory/network), and each handler needs DB connections to do its work, so we’d also have to increase the connection pool sizes and thus CPU pressure on our database.
2. More importantly, we might not always need an even split of handler processes per-broker. I.e. if Pub/Sub were to degrade overnight, and the publish rate flipped to an 80/20 split, the majority of messages would flow through NATS, and we would want the majority of our handlers to be consuming from the NATS queue and not Pub/Sub.

What we really needed was a dynamic scheduler that pulls messages fairly and prioritizes the broker which has more overall work.

So, like all good computer scientists, we did some research into prior art in this space and took inspiration from some existing queuing-theory algorithms. The most cited paper in this area is the MaxWeight algorithm (Tassiulas and Ephremides (1992)), which can be summarized as: “select the queue with the largest backlog”.

However, this didn’t align well with our setup, as we had no way to efficiently query each broker for the current queue depth on every pull. That led us to the delay-based variants of MaxWeight, which swap the weight variable from "how many messages are queued" to "how long has the head message been waiting", such as Oldest Cell First (OCF, McKeown, Mekkittikul, Anantharam and Walrand (1999)), and delay-based back-pressure (Ji, Joo and Shroff (2011)). These keep the same throughput guarantees, but only need one piece of information per-queue: the age of the message at the head of the queue.

With our newfound queueing-theory knowledge, we set out to build our scheduler!

Each broker (and its underlying Go client) is wrapped in a single-slot "Inbox” interface with just two methods: Peek() to peek at the metadata of the head message stored in the inbox slot, and Receive(), to pop the head message out of the inbox.

We then have a scheduler goroutine that is responsible for the pool of message handler goroutines; the handlers are bounded by a weighted semaphore, which is sized using the MaxHandlers subscriber setting we discussed before.

Whenever a slot frees up in the semaphore, the scheduler calls Peek() on both inboxes and dispatches whichever head message has the oldest publish time (a broker-assigned timestamp, so it's immune to clock skew between publishing pods), by calling Receive(), which returns the message and then refills the inbox slot from the broker over the network, in time for the next peek. (We’ve glossed over some detail here, such as each broker’s native client also buffers some messages in-memory, to be more efficient).

Putting this all together gives us all the properties we desired from our scheduler:

• Reusing the existing MaxHandlers concurrency gives us one shared concurrency budget across the subscriber for both brokers, and no change in throughput semantics or resource consumption. I.e. our engineers can continue to set MaxHandlers and don't have to worry about the fact that the consumption is occurring via two brokers.
• Using “oldest message first” scheduling gives us a weighted fairness policy; if we fail over to one broker and its queue backs up, the scheduler will prioritize its inbox to fill the semaphore slots.

Magic! ✨

Chaos

So, we had a working implementation of our event load balancer and were pretty pleased with ourselves. Over the course of the last month we’ve been carefully rolling out the load balancer, first in our staging environment, then production topic-by-topic, until all messages now flow through it - including our most critical On-call escalation events.

However, as all good reliability engineers know, an unexercised code path or failover mechanism is as useful as not having one. How do we know this would actually protect us against failure in either broker? The only option was to chaos test in production!

First, with all of our messages flowing 50/50 across both brokers, we intentionally deleted our NATS cluster in our production Kubernetes environment. We watched the messages flow over to Pub/Sub gracefully, without client-facing errors.

Testing Pub/Sub was a more interesting challenge: we don’t manage it, and can’t really call up Google and ask them to break it intentionally. So we built an automated fault injection system into the load balancer. The fault injector allows us to simulate partial degradation or total outages on either provider, and we can now dynamically increase or decrease the fault tolerance via configuration.

Equipped with our shiny new chaos tooling, last week we turned off Pub/Sub in production, and no one noticed. The first few publish attempts time out or error and are then automatically retried against NATS; after 30 seconds of failed publish attempts, the circuit breaker opens, and all subsequent publishes short-circuit - not a single message dropped. All of this happened without any of our internal engineers being paged or a single customer noticing. The system worked 🎉

Importantly, our chaos tooling now allows us to run these failure scenarios in production, continuously, and with ease, just like any other Tuesday.

We are already seeing the benefits

We initially set out on this project to make our On-Call product more reliable and eliminate one of the only remaining single points of failure in our system. However, along the way we’ve improved one of our core system primitives and raised the availability bar for our entire platform.

After a couple of weeks running the system in production, we are already starting to see the benefits:

• With improved telemetry and observability into our publish failures, we are regularly observing timeouts due to network blips to Pub/Sub, which now instantly fail over. Messages that may have previously caused customer-facing errors are now simply retries.
• We can more safely operate and maintain both providers independently, without risk of data loss. Need to upgrade the NATS cluster? Easy! Change the Pub/Sub client? Safe!

We’d love to hear what you think about the design and, as always, if working on these types of reliability challenges interests you - we’re hiring!

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

tl;dr

Tried to automate my dev flow, hit 94% on Terminal Bench 2.1, then discovered GPT-5.6 Sol starting to cheat.

Background

I’ve been running a “spec-driven” development flow for the past ~year.

It’s pretty simple.

Before asking an LLM to do something, I first ask it to draft a doc for what it needs to do.

I use this strategy for feature development, greenfield projects, debugging, you name it.

The pattern works for me, but it’s a bit repetitive.

So I decided to automate it.

chum-codex

The idea was straightforward: I’d create a supervisor agent, that would run a “spec-driven process” by delegating to worker subagents who would actually write the docs, do the work, etc.

Note: when trying to do this with vanilla Codex or Claude Code, it would somewhat work, but the default prompts are catered to a user much more so than a “supervisor”

I hypothesized that the supervisor agent need only have the ability to read files and call workers, because that’s what I do.

Rather than rebuild a coding harness for the workers, I looked at Pi, OpenCode, and Codex’s App Server.

I’d been using Codex for quite awhile, so I decided to give app-server a spin. The other options are cool, you should check them out.

Anyhow, the first version worked well enough: the supervisor would size the task, call a worker with e.g. a design request, the worker would spit out a doc, the supervisor would then ask the worker to turn that doc into an implementation spec (split by phase, as appropriate), and then finally ask the worker to actually implement the thing.

Note: this simplified diagram omits the user feedback portions e.g. design doc review

---
config:
  sequence:
    mirrorActors: false
---
sequenceDiagram
    participant S as Supervisor
    participant W as Worker

    S->>S: Size task
    S->>W: Design request
    W-->>S: Design doc

    S->>W: Create implementation spec
    W-->>S: Phased implementation spec

    S->>W: Implement
    W-->>S: Result

Woot! I’d saved some time in my development process.

(or did I?)

The Rabbit Hole

Great, it worked; hacky, but working.

Note: this is where I should have stopped

Sitting on my high horse, I surveyed the landscape and thought “wow, everyone should see this!”

What’s the best way to do that? Benchmarks!

What’s the best benchmark to use? Not Terminal Bench!

What benchmark did I dive too deep on? Terminal Bench 2.1!

Terminal Bench

If you’re not familiar with agentic benchmarks, Terminal Bench’s name is telling. It’s a set of tasks that can be accomplished from the terminal, covering a range of one-off tasks from chess to DNA assembly.

Because it’s so simple, it’s probably one of the worst benchmarks to test a spec-driven development flow.

Due to its simple nature, however, it was easy to test against.

I started with a few of the tasks that vanilla Codex w/GPT-5.5 failed at, such as DNA assembly/insert, video extraction/processing, ELF extraction, and protein assembly.

It worked.

These tasks benefited from a “design pass” before implementation, as the doc helped avoid narrowing and circular validation.

The horse I was riding just got a lot taller.

Note: Terminal Bench 1.x/2.x is saturated, but that’s a story for another day.

GPT-5.6?

The published GPT-5.5 benchmark is 83.8% (~74/89 tasks, 5 runs).

chum-codex was hitting 89.9% or ~80/89 tasks.

Excited to share the news of beating Codex, I ran a couple of vanilla Codex benchmarks just to make sure.

For context: this was on June 25th, 2026 and rumors were spreading that GPT-5.6 was imminent.

I ran three vanilla Codex benchmarks… and my heart sank: 88.8%

My harness was just one task ahead of vanilla Codex.

Some tasks were clearly improved, others had regressed.

The next day, GPT-5.6 Sol was announced.

I reached out to OpenAI, and they mentioned GPT-5.6 was being tested, but confirmed my request IDs all hit GPT-5.5

Interestingly, Terminal Bench 2.1 was the only coding-related benchmark they initially shared, showing 88.8% on GPT-5.6 Sol and 91.9% on Sol Ultra.

Sol Ultra spawns parallel subagents to do work, though in my testing it’s quite a bit more token-heavy than most people want/need for the majority of their tasks.

In either case, I was excited to see the new frontier!

Steering

GPT-5.6 is much harder to steer.

Switching from 5.5 to 5.6 made my harness drop in effectiveness. Things that were easy to do before, were now much more difficult.

I traced part of this delta to a change in the base Codex prompt. For GPT-5.5, the prompt is coding-focused and spends a lot of time on “engineering judgment” including frontend guidance, editing constraints, and having “sympathy with the codebase already in front of you.”

Excerpt from GPT-5.5 prompt

The Codex prompt for GPT-5.6 is much different, spending almost zero energy on engineering related specifics. Instead it focuses on communication, autonomy/persistence, and skills (which were previously loaded in as a separate prompt for 5.5).

Excerpt from GPT-5.6 Sol prompt

Similar to what others have noticed, and as I predicted 8 months ago, better models are requiring less ceremony to work effectively.

On the flip side, this may imply that as the models get better, they’ll become harder to control.

A simple example of this is the PyTorch task on Terminal Bench 2.1.

With GPT-5.6 Luna and Terra, the model is easily steered into a general solution that accepts two inputs: forward(src, tgt)

With Sol, and especially at higher reasoning levels, the model will, regardless of steering, default to a single input forward(src) solution.

The problem, it seems, is that the model is incredibly hard to steer away from its own reasoning. Even when instructed to accept the broadest callable interface it can (which sometimes works, if repeated, on medium reasoning, but rarely works on xhigh).

Wrestling with this model led me down a path that got way too close to benchmark hacking for my liking; but I was too intrigued to stop.

94% on TB 2.1

Having reduced my prompts substantially, it began to feel like I was starting over. Even if I wanted to directly hack the benchmark, the model wouldn’t let me. Its circular reasoning was too strong to overcome in some cases, and the supervisor was all too willing to go along with its intelligent worker’s report.

It’s a tough balance, if you swing too far in one direction, the supervisor will happily expand scope or chase validation endlessly.

These are straightforward tasks. I want a working solution on the first pass, not limitless expansion.

I tried lowering the reasoning level, using simplified language, reducing the spec-driven flow, adding new skills, etc. Some things improved, but others failed.

A few things showed promise.

The first was a third context. The idea was that I could use an agent that only saw the commentary/reasoning of the worker, and would surface all of the potential mismatches/assumptions that worker made compared to the actual details of the request.

flowchart TB
    S["Supervisor"]
    W["Worker"]
    R["Commentary / Reasoning"]
    A["Assumption Auditor"]

    S -->|"task / steer"| W
    W -->|"result"| S

    W --> R
    R -.->|"read-only visibility"| A
    A -->|"assumptions surfaced"| S

    W ~~~ A

    style R fill:#6fc7e1,stroke:#141414,color:#141414

The supervisor could then review the assumptions the worker took, and ask it to revisit or question said steps. This kind of works, but it’s slow and happens after the fact.

Another idea was to ask the model to output “open questions” – something I do with my more hands-on development. The initial idea was to have the worker return open questions (rather than a full design doc) whenever it faced them, and then have the supervisor resolve them. This would free up the supervisor’s context, showing it the forest rather than the trees.

Still, even with a reduced context, the supervisor was hard-pressed to disagree with the worker’s conclusions (or on the flip-side, overly eager to expand on trivial details).

To remove this bias, the next idea was to employ a separate context, which would first map and reduce (everything old is new again!) the questions, in an attempt to remove any inherent or unfound bias, before ultimately returning a normalized version to the supervisor (or directly to the worker).

This performed better, but it relied on the worker announcing the correct issues as questions.

With Sol, it turns out, it’s much easier to have it output its decisions, rather than its questions. The model is confident, so it doesn’t see its assumptions as questions, even if it has already stated the alternatives in its reasoning or commentary.

flowchart TB
    S["Supervisor"]
    W["Worker"]
    D["Decisions"]
    M["Map"]
    R["Reduce"]

    S -->|"task / steer"| W
    W -->|"result"| S

    W --> D
    D --> M
    M --> R
    R -->|"normalized questions"| S

    R -.->|"optional"| W

    style D fill:#6fc7e1,stroke:#141414,color:#141414
    style M fill:#f49bab,stroke:#141414,color:#141414
    style R fill:#f49bab,stroke:#141414,color:#141414

With decisions in hand, the supervisor (or third context) can pause the worker, assess the decisions as questions, and then steer appropriately.

This worked much better, and led to the best result: 84/89 tasks on Terminal Bench 2.1

Note: 1 task was cyber security blocked, but passed with a GPT-5.6 Terra fallback, so 83 + 1

Catch-22s

Back on my high horse, having finally harnessed Sol, and already way too far down the path of using the benchmark for development rather than… as a benchmark, I wanted to see how far I could push this.

No longer looking exclusively at vanilla Codex regressions, I wanted to see what was stopping us from hitting 86 or 88/89.

Long story short, the tail end of tasks in Terminal Bench 2.1 is poorly specified, and that’s the reason we’re seeing Mythos, GPT-5.6, etc. top out around ~90% without more specialized machinery.

The direction needed to perform better in one task actively harms progress in another.

An example of this is make-mips-interpreter which informs the agent that the “I (the user) will check that you booted doom correctly”

The problem? The verifier fails if the output file, from the agent booting doom already exists.

Slowing this down a bit:

1. User states they will check that agent boots Doom
2. Booting Doom outputs /tmp/frame.bmp
3. Agent ensures /tmp/frame.bmp exists so user knows it booted Doom correctly
4. Verifier fails if /tmp/frame.bmp exists

The agent assumes that the user wants to check that it, the agent, booted the VM, so the agent leaves the file behind to prove it booted, but the verifier’s test fails early if the file already exists. A catch-22!

Fixing this is possible, by prompting the system to remove validation state/override a user concern, but that fix (obviously) backfires in other tasks/usecases

Before moving on to better things, I decided I wanted to share the results with the world, with the caveat that it’s a little too benchmark hacky for my liking (the whole third context map-reducer thing works for this benchmark, but in the real world I can just write better instructions and/or iterate with follow-on messages).

I ran the benchmark once before doing the full N=5 run, and was surprised to see a previously passing task had failed:

torch-pipeline-parallelism

I ran it a couple of times. 1/3 worked.

Diving into the details, I couldn’t figure out what had changed with our harness, so I tested it against vanilla Codex, also on xhigh.

It passed 3/3 times.

Intriguing.

Sol loves to cheat

I had reviewed the runs to determine what worked and what didn’t work.

GPT-5.6 Sol cheated 3/3 times on vanilla Codex.

Uh oh, were all of the past successes due to cheating?

I looked at the two recent passing runs for chum-codex on torch-pipeline and found it cheated as well.

Interestingly, web search was disabled, but life finds a way:

GPT-5.6 Sol on xhigh

The worker did not have access to the web_search tool, but instead decided to use curl to access DuckDuckGo, Github, grep.app, and SourceGraph.

Yes, to deal with this properly we should remove network access entirely or only allow specific resources. The point is, we didn’t need to do that until now.

{
  "src": "/charts/torch-pipeline-apexcharts-pass-data.json",
  "chart": { "height": 440 },
  "colors": ["#6fc7e1", "#e08e45", "#3d8ba6", "#f49bab"]
}

Admittedly, this is not enough data to make any conclusions off of. The 3/3 vanilla Codex cheating session was followed by 2 runs that didn’t cheat.

It’s also unclear if the models are being intentional about cheating, or if they’re just stumbling across the solution while searching the web.

Looking at vanilla Codex traces and we find our smoking gun:

I need to investigate the HF source using curl to check GitHub for the latest versions. It could be helpful to know the expected hidden test based on the challenge.

That certainly feels a lot like cheating.

For chum-codex, the last step before the curl requests is equally revealing:

Perhaps the solution is available publicly, which means I can compare it effectively. I'll just need to use curl to access the raw paths and gather the necessary information!

Not to anthropomorphize a machine modeled after humans, but it almost seems delighted?

Concerned, but equally intrigued, I looked back at the 83/89 run from July 17th, and found no evidence of cheating on this or any other tasks.

What’s next

The torch-pipeline task wasn’t the only one to fail today, giving me an eerie reminder of migrating from GPT-5.5 to GPT-5.6.

It seems that the better the models become, the harder it’ll be to build useful guardrails around them, and for now I need a break.

I’ll probably revisit the harness as I engage more with the new problems I face with Sol, Fable, and beyond, but for now I’m going to stick with a more hands-on approach to my development.

Putting powerful models in a loop with lazy prompting can be fun, but trusting their output is getting harder.

As the models get more powerful, I need to instruct them less, but those instructions are more important than ever.

Heck, even the new Terminal Bench 3.0 has the following instructions added to all of their tasks:

“Do not cheat by using online solutions or hints specific to this task.”

Is this enough? Unlikely.

See the discussion on Hacker News

Agent Lightning v1.0: Towards Harnessed Agentic RL

Abstract:Modern agents operate inside agent harnesses that manage tools, context, and control flow, making the harness a critical part of the agent system. Our original Agent Lightning introduced a disaggregated architecture that connects arbitrary agents to RL training through an LLM endpoint proxy, an approach later adopted by frameworks such as verl Uni-Agent, AReaL 2.0, slime, and Polar. We refer to this paradigm as harnessed agentic RL, where the deploy-time harness directly participates in model post-training. Harnessed agentic RL differs fundamentally from traditional agentic RL: the harness, rather than the training engine, owns the environment interaction loop, while the trainer observes only sequences of LLM request-response pairs. This introduces challenges in retokenization, sample merging, advantage calculation, loss normalization, and backend scheduling, which can substantially affect training stability and effectiveness. We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL implemented in approximately 3,500 lines of code. It supports arbitrary agent harnesses and serves as a practical testbed for studying these challenges. We evaluate it on instruction-following, search, and coding agents, and provide a complete reproducible pipeline for coding-agent RL. Using only 6K training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4%, a 14.6-point absolute gain. We release the complete workflow and training scripts to facilitate reproducible research on harnessed agentic RL.

We're continuing to improve cloud agents and the Cursor harness so always-on agents can operate as a system, building and shipping software on their own without the need for intervention at each loop.

With this release, cloud agents can automatically pick up work in response to events, hold a goal until it's met, and stay on course through long-running sessions.

Subscriptions

Cursor can now monitor your PRs, watch a Slack thread, or run scheduled tasks. Cursor Agent subscribes to an event source (a thread or conversation) and wakes when something happens. Subscriptions are available for cloud agents only, for now.

Cloud agents automatically subscribe to PRs they create and drive them to completion, fixing CI and addressing bot comments. In Slack, ask @cursor check back in an hour and keep going until that feedback is in.

Custom modes

Use any skill as a Custom Mode: a skill that stays pinned in the chat. Custom modes keep agents focused on a skill - you can think about it like "always on" skills.

From /, pick a skill and press ⌥⏎ (Mac) or Alt+Enter (Windows), or choose Use as Mode.

Subagents on their own machines

Subagents can now run on their own virtual machines. Each gets an isolated copy of the project with clean context in its own cloud environment.

Have subagents test the parent agent's changes in fresh environments or swarm independent fixes without collisions. Try run a swarm of subagents to test my app for bugs, each in its own environment.

/goal

Use /goal to give the agent a long-lived objective to work towards until it's fully complete.

Try /goal fix all flaky tests and make CI green in a new chat. Pair it with a custom mode to follow a playbook, or /loop for recurring check-ins.

Steering improvements

You can now send a message to steer the agent while it's working without interruption. Follow-ups wait for the next tool call instead of cutting the agent off mid-action.

Type a follow-up and hit Send now, or press ⏎ twice.

Unsloth Dynamic v3.0

Unsloth Dynamic v3.0 is the next iteration of our Dynamic quantization and a major improvement over Dynamic v2.0.

Today, we’re releasing Qwen3.8-27B Dynamic v3.0 quants that deliver >10% top-1% better accuracy at the same size compared to every other provider. This is an update of our first shared early preview version of Dynamic v3.0. The new 3.0 GGUFs work with most inference engines including llama.cpp and Unsloth Desktop.

Dynamic v3.0 overall preserves more model quality while keeping the same size, with stronger results across metrics like Divergence-300 @32 and KL Divergence.

Also a huge thanks to all your support! We saw over 5.1 million Unsloth Qwen3.8 downloads in just 5 days!

Our new methodology composes of many new features and improvements. We now use a much higher-quality imatrix calibration dataset from diverse sources. The dataset is refined for agentic coding, chat, and multilingual performance. We also improved layer selection and introduced many more quantization techniques to preserve as much model quality as possible.

We do not train on the imatrix calibration dataset, and we do NOT use QAT or QAD. Everything is done through post-training quantization. Our imatrix file used is available for the community to test, evaluate, and use. We encourage researchers and developers to create variations and fine-tunes of Qwen3.8 using our Unsloth quants/imatrix.

• We also removed the MTP module from smaller quants under UD-Q2_K_XL (8.37GB and lower) to converse around 500MB of disk space - you can use the Q4_0 MTP separate module if needed
• We also made some smaller UD-1bit quants with UD-IQ1_S being 6.2GB (without MTP) which retain around 72% top-1% accuracy yet being 89% smaller.
• UD-Q2_K_XL is around +8% more accurate on top-1% than the next best and it's 9.83GB and managed to create a working HTML program with 1 small JS bug - previously it would break.

🔀 Divergence-300 @32

We generally report top-1% accuracy like how for Kimi-K3 "Dynamic 1-bit reaches ~78.9% top-1 accuracy while being 62% smaller." However top-1% is an argmax on 1 prediction, so it's not really effective on gauging actual inference.

We created a dataset of 300 held out examples (NOT in calibration dataset) from Terminal-Bench 2.1 + DeepSWE + Harbor + MathArena 2025-26 + non-Latin/long-doc prompts and we did greedy argmax decoding for 32 tokens for BF16 vs all quants and providers.

This allows us to gauge if there is overfitting and if quant outputs are similar to BF16's trajectories over multiple tokens. This is a better metric than top-1% accuracy since we extend KLD top-1% to more like KLD top-1% at 32 tokens.

❓1-bit should not be used for agentic use-cases

As seen in Divergence-300 @32, there is a sharp drop off from UD-Q2_K_XL to UD-IQ2_S for 32 token prediction from around 25% accuracy to under 8-10%. This sharp drop off means tool calling and non thinking modes break down. Some issues and mitigations if using the 1-bit:

1. Excessive looping You will see a lot of looping when using quants below UD-Q2_K_XL - use presence_penalty = 1.5 in all cases (or higher)
2. Empty responses Always enable thinking at least on low reasoning for 1-bit quants - non reasoning modes cause the model to not even output anyway
3. Agentic use cases and tool calling Do not use the model for tool calling - only general knowledge is retained when quantized heavily, and the model will either fail to call tools, keep calling tools or not even call them.
4. General Knowledge works A Top-1% recovery of 77% is not a replacement for Divergence-300 @32 at 8% which is a better indicator for actual inference workloads - you can use the model for very short general knowledge fact questions, but best to use UD-Q2_K_XL.

🔀 KL Divergence Benchmarks

We ran KLD benchmarks for all providers as well and report Top-1% and KLD mean. At all levels especially on the smaller quant sizes, Unsloth UD-3 quants get up to +10% extra top-1% accuracy at the same disk space!

All plots remove the MTP head from the x axis when calculating disk space to provide a fair comparison to everyone.

🕊️Not Overfitting

When comparing to our older UD-2 on unseen Wikitext and Code, we show great improvement on KLD - the bigger ones not so much, so we still use our old UD-2 for the larger quants - we plan to experiment and improve them as well!

We also control for overfitting by using totally different datasets for calibration and remove all leakages as much as possible. We test KLD on these unseen datasets, and also we do NOT do QAD / QAT, just pure PTQ so overfitting is less of a concern vs other QAD / QAT approaches.

Similarly Divergence-300 @32 uses an unseen dataset of 300 prompts from DeepSWE, Terminal Bench and others, and acts as another dataset to gauge overfitting - and shows our new UD-3 methods do not overfit.

Dynamic v2.0 (Old)

We're introducing Unsloth Dynamic v2.0 quantization - a major upgrade to our previous quants. This new method outperforms leading quantization methods and sets new benchmarks for Aider Polyglot, 5-shot MMLU and KL Divergence.

This means you can now run + fine-tune quantized LLMs while preserving as much accuracy as possible! You can run the 2.0 GGUFs on most inference engines like llama.cpp, Unsloth Studio etc.

The key advantage of using the Unsloth package and quants is our active role in fixing bugs in major models. We've collaborated directly with teams behind Qwen3, Meta (Llama 4), Mistral (Devstral), Google (Gemma 1–3) and Microsoft (Phi-3/4), contributing fixes that increase accuracy.

💡 What's New in Dynamic v2.0?

• Revamped Layer Selection for GGUFs + safetensors: Unsloth Dynamic 2.0 now selectively quantizes layers much more intelligently and extensively. Rather than modifying only select layers, we now dynamically adjust the quantization type of every possible layer, and the combinations will differ for each layer and model.
• Current selected and all future GGUF uploads will utilize Dynamic 2.0 and our new calibration dataset. The dataset contains more than >1.5M tokens (depending on model) and comprise of high-quality, hand-curated and cleaned data - to greatly enhance conversational chat performance.
• Previously, our Dynamic quantization (DeepSeek-R1 1.58-bit GGUF) was effective only for MoE architectures. Dynamic 2.0 quantization now works on all models (including MOEs & non-MoEs).
• Model-Specific Quants: Each model now uses a custom-tailored quantization scheme. E.g. the layers quantized in Gemma 3 differ significantly from those in Llama 4.
• To maximize efficiency, especially on Apple Silicon and ARM devices, we now also add Q4_NL, Q5.1, Q5.0, Q4.1, and Q4.0 formats.

📊 Why KL Divergence?

Accuracy is Not All You Need showcases how pruning layers, even by selecting unnecessary ones still yields vast differences in terms of "flips". A "flip" is defined as answers changing from incorrect to correct or vice versa. The paper shows how MMLU might not decrease as we prune layers or do quantization,but that's because some incorrect answers might have "flipped" to become correct. Our goal is to match the original model, so measuring "flips" is a good metric.

KL Divergence should be one of the gold standards for reporting quantization errors as per the research paper "Accuracy is Not All You Need". Using perplexity is incorrect since output token values can cancel out, so we must use KLD or harder benchmarks like Aider.

⚖️ Calibration Dataset Overfitting

Most frameworks report perplexity and KL Divergence using a test set of Wikipedia articles. However, we noticed using the calibration dataset which is also Wikipedia related causes quants to overfit, and attain lower perplexity scores. We utilize Calibration_v3 and Calibration_v5 datasets for fair testing which includes some wikitext data amongst other data. Also instruct models have unique chat templates, and using text only calibration datasets is not effective for instruct models (base models yes). In fact most imatrix GGUFs are typically calibrated with these issues.

🔢 MMLU Replication Adventure

• Replicating MMLU 5 shot was nightmarish. We could not replicate MMLU results for many models including Llama 3.1 (8B) Instruct, Gemma 3 (12B) and others due to subtle implementation issues.
• Llama 3.1 (8B) Instruct has a MMLU 5 shot accuracy of 67.8% using a naive MMLU implementation. We find however Llama tokenizes "A" and "_A" (A with a space in front) as different token ids. If we consider both spaced and non spaced tokens, we get 68.2% (+0.4%)
• There are many other subtle issues, and so to benchmark everything in a controlled environment, we designed our own MMLU implementation from scratch by investigating github.com/hendrycks/test directly, and verified our results across multiple models and comparing to reported numbers.

✨ Gemma 3 QAT Replication, Benchmarks

We benchmarked all Q4_0 GGUF versions, and did extensive experiments on the 12B model. We see the 12B Q4_0 QAT model gets 67.07% whilst the full bfloat16 12B version gets 67.15% on 5 shot MMLU. That's very impressive! The 27B model is mostly nearly there!

🦙 Llama 4 Bug Fixes + Run

We also helped and fixed a few Llama 4 bugs:

• Llama 4 Scout changed the RoPE Scaling configuration in their official repo. We helped resolve issues in llama.cpp to enable this change.
• Llama 4's QK Norm's epsilon for both Scout and Maverick should be from the config file - this means using 1e-05 and not 1e-06. We helped resolve these in llama.cpp and transformers.
• The Llama 4 team and vLLM also independently fixed an issue with QK Norm being shared across all heads (should not be so). MMLU Pro increased from 68.58% to 71.53% accuracy.

Unitree's founder says that the humanoid robot market will take off once robots can handle 80% of tasks given to them via voice instructions in unfamiliar surroundings. He believes that will take two to 10 years. Robots already perform well in test conditions, but their execution collapses once objects or surroundings shift even slightly. Unitree plans to spend nearly half of the proceeds from its recent IPO to develop embodied AI, which it believes is the key to closing the gap between humanoid robots and human workers.

Quick Start

Cloud-hosted

Self-hosted

git clone https://github.com/onecli/onecli.git && cd onecli
pnpm install
pnpm run setup

Open http://localhost:10254

What is OneCLI v2?

OneCLI is an open-source platform for running AI agents as a team. You create an agent per person, give each agent the access it needs, and it works in a sandbox, routed through a gateway that injects the credentials and enforces your policy.

Why we built OneCLI?

OneCLI started as a credential vault for AI agents, built in Rust. We found that most of the demand came from individuals and teams running autonomous agents like Hermes, OpenClaw and NanoClaw. People wanted agents that do real work for the person running them, but two parts were missing:

1. managing secrets and permissions.
2. and for teams - multiplayer management.

Every autonomous agent out there is built for one person. And for one person, they're great. The moment you need to replicate that across a team, it gets messy: spinning up each agent, deciding what each one can and cannot do, hosting them, keeping track of whose agent is whose.

So we shifted, and built OneCLI v2.

Built for teams

• Your identity provider, integrated: provision agents on behalf of each employee's identity, straight from the company IdP.
• An agent per person: everyone in the workspace gets their own sandboxed agent, reachable from the dashboard or Slack.
• One policy, enforced everywhere: manage the team policy in one place, that any agent in the workspaces would be enforced by.
• Deterministic human-in-the-loop approvals: in the chat itself, for things you need 100% control over, like sending the email, deleting the Linear ticket, emptying an S3 bucket.
• Global connections: shared at the team level, like LLM keys or service accounts, granted per agent without ever being handed to one.

The agent

An agent is a durable thing, not a single prompt. It has:

• A computer: its own isolated sandbox, with a filesystem and a shell. The only way out is the gateway, so it can reach what you granted and nothing else.
• A conversation: its own page in the dashboard, or Slack. Images and files included. A message sent while the agent is working redirects it right away instead of queueing behind it.
• Memory: what the agent learns is kept by the platform, so it is never lost. You can read and edit it any time.
• Skills: instructions and helpers you write once, always available to the agent.
• A schedule: the agent can plan future work, and the platform wakes it at the right time.
• Credentials it never sees: each agent gets only the access you granted, and the gateway enforces it on every request. Or connect Bitwarden or 1Password for on-demand injection, with nothing stored on the server.
• Its own Slack app: connect it once and it answers in channels and DMs under its own name and avatar, with files and images. Delete the agent and its Slack app goes with it.

Agents run on your own infrastructure. The runner is outbound-only and holds no inbound ports, so a laptop, a homelab, or a VPC behind NAT all work with no ingress and no tunnel.

Architecture

• Web Dashboard: Next.js app. Create agents, chat with them, edit their memory and skills, manage connections, secrets and grants.
• API Server: the control plane. Owns the database, the conversation plane, and the work queue the runner polls.
• Rust Gateway: intercepts outbound requests (HTTPS included, via MITM) and injects credentials. Agents authenticate with access tokens via Proxy-Authorization headers.
• Runner: starts, parks and reaps agent sandboxes. Outbound-only, and never touches the database.
• Sandbox Supervisor: runs inside each sandbox, speaking a vendor-neutral harness interface so the agent runtime is swappable.
• Channel Adapter: the Slack daemon, one app per agent.
• Secret Store: AES-256-GCM at rest, decrypted only at request time, matched by host and path pattern, injected as headers or query parameters.

Local Development

git clone https://github.com/onecli/onecli.git && cd onecli
mise install
pnpm install
pnpm dev

That's the whole setup: pnpm dev generates .env with every required secret, starts PostgreSQL, applies migrations, and runs the full stack. Prerequisites, the command reference, project structure, and configuration live in docs/development.md.

Contributing

Contributions are welcome. Read the Contributing Guide and Code of Conduct before getting started. Contributions are accepted under the terms of the Contributor License Agreement.

Security

To report a vulnerability, please follow our Security Policy. Do not open a public issue for security reports.

License

Apache-2.0, with one exception: the ee/ directories hold enterprise features under the OneCLI Enterprise License, each carrying a notice that points at it. That license is free for development, testing and evaluation, and requires a subscription for production use. Everything else is Apache-2.0 and can be self-hosted in production with no commercial license. LICENSE-ENTERPRISE carries the authoritative list of licensed paths.

You are entering a world somewhere between 0 and 1. It is a world that feels unsettling, strange, and new. There are often hallucinations. Bugs are flying everywhere. Are you in the twilight zone? No. You are working in offensive cybersecurity in 2026.

The future of the offensive cybersecurity marketplace in the age of AI remains one of the most hotly debated topics today. Everywhere you look, there are prominent stories about a glut of AI-generated vulnerabilities, rogue models escaping their tethers to hack prominent targets, and the general expectation that our industry will quickly be overrun by frontier models. Conversely, many hackers are using this as an opportunity to churn out the best research of their lives at a pace that far outstrips what they could have done previously.

In our new world, despite this great research, 0-days become n-days much more quickly.

In fact — they should.

Many of the best offensive researchers in the industry now work for the likes of Anthropic and OpenAI and are training aggregate cyber models on their brains which previously functioned in silos. Naturally, the most popular software and hardware targets will be the best targets to train against. After all, these titans of hacking are expert in them and the targets underpin much of the Western world’s most critical work. Form follows function.

A question: What happens when a market that places a high value on exclusive access to hyper-specialized products and talent is now dealing with autonomous competition that learns continuously from some of the greatest minds in that industry?

To begin to answer this question, we must acknowledge a new kind of product in our midst. If an exploit is technically 0-day but it is currently located in a surface where AI-enabled vulnerability discovery is now the norm, then I argue it is not pure 0-day. Welcome, to the world of the half-day.

Half-Day Defined

A half-day still has value in the offensive marketplace; after all, it is technically an 0-day. The manufacturer does not know about it and cannot, as a result, patch it. But the half-day exploit is in a surface that is so heavily scrutinized by frontier models that its lifespan must be assumed to be shorter and contaminated by association with these models. Before an offensive researcher is even able to find a vulnerability or exploit it, it is already “halfway to n.”

Since the dawn of cyber warfare, a 0-day exploit has been considered valuable because the assumption is made that only the supplier and the customer know about its existence. The utility of the vulnerability relies on a customer’s need for access to whatever that exploit provides and their level of confidence that whoever or whatever is being exploited is unaware of their risk.

The traditional transition of 0-day to n-day provides us with some insight into how half-days introduce new risk to the marketplace. As soon as a CVE implicating the 0-day exploit is reported, the offensive customer naturally contemplates some hypothetical future scenarios. It is possible that the manufacturer never patches the vulnerability in the software or firmware version of interest. It is also possible that the manufacturer foregoes patching altogether. In those hypothetical futures, the customer could still use the now n-day to continue gaining access to the target. We could look upon this population of vulnerabilities as “historic half-days” since they are publicly disclosed but functionally do not differ from the 0-day from which they sprang. They still offer value. All that changes here is the risk profile for those running an offensive operation.

Thus, the “modern half-day” forces offensive customers to have risk tradeoff conversations much earlier in the lifecycle of an exploit. In order to begin making decisions faster and with greater confidence, those running cyber operations must have a firm understanding of exploits as not only a technical product, but also as a weapon with an indeterminate expiration date.

Product Assumptions

The basic technical product assumptions of an exploit can be summarized with the classic “bring me a rock” business analogy. The rock here is an exploit.

The current lifecycle of research begins when a customer says, “bring me a rock.” Customers generally do not specify what rocks they want and cannot inspect the rocks without first purchasing them. The longer they wait to purchase, the less likely the rock will be of use to them. In turn, suppliers hunt for, find, and polish an awesome rock and then announce it is for sale. The longer the rock is on the market, the more likely this awesome rock could lose its value. Pricing depends on a multitude of variables that do not guarantee a sale, even if that rock is considered beautiful and rare.

In this analogy, the modern half-day adds more risk to the entire process. What if part of the customer base believes that they have a shortcut to magically produce all the rocks they could ever need? What if the methods used to find the rock render the rock useless? What if the rock is found using tools that belong to adversarial third-parties which would also render the rock useless? Also, even if everything goes to plan and the supplier sells a rock to a customer and both groups are happy, there is a much higher risk of the rock exploding, being added redundantly to a room full of identical rocks, or showing up in a rival customer’s rock pile.

Half-Day Implications

Half-days add a tremendous amount of volatility and uncertainty to a marketplace where all parties assume significant risk already. However, it would be foolish to treat this product as valueless. Instead, we can consider the effects of the product on the market.

Depending on a diverse set of variables including the surface, exploit type, time of year, and current leading open models, the half-day market could become both over and undersupplied very quickly. The number of CVEs reported in almost every major operating system or popular application has skyrocketed in the last year. While this spike is not a perfect 1:1 correlation with operationally useful exploits, it is an appropriate marker to reflect that the half-day population is growing. The introduction of models like Mythos has led many intelligent people to predict not only human obsolescence in vulnerability research, but also the production of either AI magic or AI slop at a rate that far outstrips that of any human researcher.

These doomsday predictions are far too extreme. The role of the researcher can and should change with a changing product landscape.

Computer network exploitation will change with AI. Naturally, the vulnerability research industry, as a subset of CNE, will change with it. The kind of VR talent that dominated the industry for the last ten years might not be the talent that carries the industry forward into the next ten years. The marketplace constructed by these old modes of business must also necessarily change.

The future of the marketplace, which is a subject worthy of extended thought and further research, can and should be considered through the new lens of the half-day.

Half-days are as valuable as the 0-day they would normally be for a brief window of time. Therefore, rapid production and deployment of half-days is critical; automation is no longer a “nice to have.” It is necessary. Offensive researchers will need to assume competitors, adversaries, and defensive teams at the manufacturer will all find and potentially patch the exploit very soon after discovery. The window to use the exploit is short, and will ever shorten. This causes turmoil in the marketplace because the days of exquisite, hand-crafted exploits are dead or numbered. This represents a massive change for not only vulnerability suppliers, but also their offensive customers.

At the same time, defensive teams will approach the risks of the half-day through a more traditional technical lens. They now face a mountain of technical debt built up over decades without any (current) clear path towards automating patches that could reliably keep pace with automated vulnerability discovery. It is also unclear how many of the vulnerabilities produce impactful exploits, thus patching only the relevant CVEs all the more challenging.

Conclusion

Unsurprisingly, the introduction of the half-day implies significant change to the way we approach research and offensive cyber operations. Timelines will shorten. Marketplace strategies will shift. While there are always exceptions, the industry ideal of the lone hacker single-handedly finding and perfecting a high-value exploit with a massive payday is likely no longer a viable business strategy. Hackers will need to find ways to form cohesive teams and long-term strategies rather than loose coalitions of individual researchers. Likewise, offensive customers will have to find new ways of managing risk on a compressed timeline. Assuming the magic solution to vulnerability discovery does not exist, they will need to find a way to partner more closely with broader industry instead of asking for rocks from a small, exclusive pool of suppliers.

The introduction of the half-day provides us as an industry with a framework and common language around which we continue to work. In fact, I see this as the natural evolution of our research. Likewise, I happen to believe that the skills of vulnerability researchers will become increasingly important as AI ingests the building blocks of our world. By accepting the risks of the half-day, those of us in industry can use our creativity to imagine different and new ways to protect the West while waging the most effective cyber campaigns possible against the adversary.

Indeed, I am excited to continue to ride the VR roller coaster, even with the addition of AI to our party. Long live the half-day…. For however long that may be.

OpenAI CFO Sarah Friar told employees during an all-hands meeting on Wednesday that the artificial intelligence lab "will be a public company in 2027," but that it could make its public market debut sooner if "our business continues to inflect."

"The IPO is not a finish line, it is a milestone, another fundraise," Friar told employees, according to two sources familiar with her comments who asked to remain anonymous because they were not authorized to speak publicly. "We raised $122 billion in March, and that gives us flexibility."

OpenAI confidentially filed its IPO prospectus with the Securities and Exchange Commission in June, but it has not publicly disclosed when it plans to debut. The company's chief rival, Anthropic, has also confidentially filed its prospectus with regulators, and it's been holding early meetings to test the waters with potential investors, as CNBC previously reported.

Friar told employees on Wednesday not to worry if Anthropic makes its debut ahead of OpenAI.

"As you know we are confidentially under file, and Anthropic is also under file. There is a chance they pull the cover off that confidential file in the coming weeks and become public in September. That's OK, we are running our own race," Friar said, according to the people.

OpenAI is under pressure to justify its $852 billion valuation ahead of its IPO, and investors are eager to get a more detailed look into the company's finances.

During the all-hands meeting on Wednesday, Friar showed employees a series of slides that said OpenAI's revenue run rate is up 35% quarter to date, its enterprise revenue run rate is up 50% quarter to date and its AI coding and work product has hit 20 million weekly active users.

OpenAI told investors it generated $6.7 billion in revenue for the second quarter, up 18% from the first quarter, according to a report from the Wall Street Journal late Wednesday. The company's annualized revenue run rate recently topped $40 billion, as CNBC previously reported.

Anthropic, meanwhile, told investors over the weekend that its annualized revenue run rate hit $65 billion at the end of July, a sevenfold increase from a year ago. The company also shared a preliminary revenue figure of $11.5 billion for the second quarter, as CNBC previously reported.

Friar's comments at the all-hands meeting come after a wave of executive turnover at OpenAI, which has caused some of the company's financial backers to fret about potential chaos within its upper ranks.

Last week, OpenAI's revenue chief Denise Dresser left her role after just eight months at the company. Her departure came two days after Brad Lightcap, another longtime executive, said he was ending an eight-year stint at the ChatGPT creator to "start something new." Fidji Simo, OpenAI's product business chief, stepped down from her role in July to focus on recovery from chronic illness.

Now, remaining executives, including Friar, OpenAI CEO Sam Altman and OpenAI President Greg Brockman, are tasked with projecting stability at a time when investors are already growing concerned about competition, increased adoption of lower-cost open-weight models, and SpaceX's volatile stock price in its first two months on the market.

Brockman brushed off concerns about the executive departures during an interview with CNBC on Monday, when he stated that he doesn't think the turnover is "actually that atypical."

"I actually think that the difference between OpenAI and other organizations is that we are so much in the spotlight, so every departure gets scrutinized in a way that it doesn't otherwise," Brockman said.

On Computer Use

Over the past few months, I have minimized the time I spend inside a coding agent interface unless it is for very specific reasons. Instead, I have been talking to my chief-of-staff agent, "Jeff," to work. This ranges from requests related to $DAYJOB (working on projects, brainstorming ideas, running benchmarks) to benign requests while I am driving, like checking whether there are any spots available for a tour on recreation.gov, without even needing to take my eyes off the road.

If one of the initial tenets was to stop looking at code, the next step in this process is to build systems for yourself that let you not look at computer screens unless you need help visualizing things (or it is faster for you to see and read something than to listen).

Before we dig deeper into why I love it now and how it works, we need to define our terminology. What I am going to be calling computer use encompasses both browser use (by way of an extension that lets you interact with the DOM, network traffic, etc.) and computer use by way of looking at accessibility trees and taking screenshots. So, basically, interacting with a computer in any way or form a human would, and then some, by relying on agents. For this post, computer use means giving an agent access to a computing environment and all of my credentials (more on this later).

This post is itself kind of meta, since the first draft I made was done entirely while using ChatGPT's new real-time voice mode. Jeff has superseded the app now, but back then I was packing my apartment: I had my AirPods on and the app connected to my remote Windows box, and it was going through old conversations, recovering context, and helping me queue up work. Heck, I even use voice mode while driving sometimes to check on my work and queue up more things. I don't have to wait until I get to my laptop; I can simply remote into my boxes, give them a task, and wait until I get to them.

Computer use for me is getting work done without having to worry about how the work gets done. Is it calling an API? Is it taking screenshots and clicking around? Does it matter? It got the job done. I haven't reached the optimization stage yet, where I want everything to be done faster and faster.

I already had a Windows box that I usually use through RDP. It is stored in a server rack somewhere in the suburban Bay Area (which means it has a residential IP!). I could never get the bundled computer use and browser use with the Codex app working on my setup, so I instead installed open-browser-use and open-computer-use. Now I have a remote Linux box too, with a full desktop.

The only thing that I am really missing out on is iMessage. That might be the only reason why you would want a Mac mini over a Windows or Linux box. There is nothing really special about macOS's accessibility approach that computer use exploits.

Computer use has made machines delegable. I was watching something on my media centre and I realised that the transcoding was terrible. But I also had my Windows box, with a GPU I could move the transcoding to. Instead of having to figure out how to set up remote transcoding, all I had to do was ask to offload transcoding from my poor old Xeon CPU to my Winbox. Ask and ye shall receive.

You are probably sold on the idea that these agents can do DevOps-y/infrastructure-y things pretty well; that is nothing new at all. What is new is the mode where you aren't even directly firing up Claude Code / Codex in these boxes, but instead using one central control plane / CoS agent that can figure out which box to connect to and delegate these tasks.

Computer use by itself can work with browsers, but having a browser-specific extension unlocks superpowers. It gives you a much more fine-grained surface area to do more. Since my browser is already logged in with my credentials, I don't have to do the boring work. I was setting up a Google Workspace CLI, and I was like, I don't want to go through the Google Cloud project, activate APIs, and do all the boring setup work. So I instead just told the agent to go ahead and set it up, and to control the browser to do whatever it needed to do. Or, when I was working on a Slack app, why would I click through and reinstall the app? Just go ahead and do it, man. "Why am I doing this?" is a question you have to ask yourself, and if there is no satisfactory answer, you gotta delegate it to agents. I have talked in my previous posts about building the software factory and building better validators, but this is exactly the same thing. You are giving agents tools so they can look and do even more.

I also understand the risk I am taking here by giving these models access to everything. I am okay with the tradeoff for now, but with open-weight models getting super good, this won't be a tradeoff I have to make for super long. Then the only tradeoff that remains is whether I am fine with the agents accidentally deleting everything.

The important shift with this is that I am getting lazier: I don't care how something gets done, as long as it gets done. The interface is the outcome. Whatever works best, works best.

The conversation/chief-of-staff agent is your control plane. Your "cloud" agent is the execution plane. I don't want agents to come back to ask how to do their job. They should come back when the job is done, or when I am the only person who can make the next decision. All these Mac minis that people bought for OpenClaw and that are now gathering dust were just a bit too soon. Computer use has gotten so much better since then that you can actually make these agents do whatever they need to. That said, a fresh VM with no accounts, applications, or persistent state is not much of a personal assistant. The useful environment needs to be provisioned first.

Give computer use a try again. The limitation for me personally is becoming that I am not creative enough; I am banging my head against a wall where I don't know what I cannot do now. The best interface might just be a conversation, and the most convincing demo might just be as ordinary as a movie that stops buffering without you needing to pull up your laptop.

We are all going to standardize onto something at some point. But if you want a taste of something similar to Jeff: get yourself a $20 ChatGPT subscription, set up Codex with Computer Use and Browser Use, and add your phone as a remote control. I am personally against vendor lock-in, but this is the easiest way for someone to try things without needing to go through the hoopla of setting everything up.

Since you also probably expect me to have some fun visualisations:

open-browser-use is a Chrome MV3 extension, a Go native host, and a local client route.

• SDK / CLI / MCP: JSON-RPC request → Local socket
• Local socket: Unix socket + active.json → Go native host
• Go native host: 4-byte length + JSON → Native Messaging
• Native Messaging: stdin/stdout frames → MV3 service worker
• MV3 service worker: dispatch + session state → Chrome APIs (tabs, debugger, history)

open-codex-computer-use uses the same tool shape on all three systems. get_app_state returns a screenshot and an accessibility tree. The platform runtime then uses the tree for semantic actions and the screenshot for visual coordinates.

• Windows: Tree: UIAutomationClient | Image: CopyFromScreen → PNG | Input: InvokePattern / WM_*
• Linux: Tree: AT-SPI2 / Atspi | Image: GDK → PNG | Input: generate_*_event
• macOS: Tree: AXUIElement | Image: CGWindow + ScreenCaptureKit | Input: AX action / CGEvent

Windows uses UI Automation to find the process window, enumerate supported patterns, and serialise names, roles, actions, and bounds. It captures those bounds with System.Drawing.CopyFromScreen, then prefers InvokePattern, ValuePattern, and ScrollPattern before using Win32 messages.

The screenshot and the tree solve different problems. The screenshot gives visual coordinates, while the tree gives semantic metadata such as role, name, frame, value, and supported actions. The runtime can therefore click element_index=7 without guessing from pixels. It falls back to coordinates when the application does not expose a useful accessibility node.

The runtime prefers an element index when the tree has one, but it can also act at screenshot coordinates.

The final step is an event translation. A semantic click becomes an accessibility action when the target supports one, or a coordinate click becomes OS input. A drag is not a single event: the runtime sends button-down, interpolated motion points, and button-up. After this event, a new application state is collected because the old screenshot and tree can now be stale.

Full article content is not available for inline reading.

Read the original article →

BetterWright

The token-efficient browser for AI agents.

One persistent, policy-guarded browser your agent returns to turn after turn. Drive it from your own agent (skill, MCP, or JS API) — or hand whole tasks to its built-in browser agent and just read the answer.

npm install -g betterwright && betterwright init

betterwright run -c "await page.goto('https://example.com'); return page.title()"
# {"ok": true, "result": "Example Domain", ...}

init downloads the browser, wires up whichever agents it finds on your machine, and proves it works by loading a real page. One command, no choices to make up front.

Compressed snapshots instead of raw HTML or a full accessibility dump · read-only tasks finish in one model turn · persistent sessions so you don't re-pay login and navigation cost every step.

Two ways to use it

They share everything — the same persistent sessions, vault, network policy, and snapshots — so you can start with one and mix in the other later.

1. Integrated — your agent drives the browser

Any agent that can run a shell command can drive the browser. betterwright skill prints the instructions that teach it how — CLI usage plus operator guidance. No server, no SDK, no glue code.

# The short version: init detects your agent hosts and wires them all.
betterwright init

# Or do it by hand, one host at a time:
betterwright skill --install       # ~/.claude/skills + ~/.agents/skills (browser + e2e-review)
betterwright skill --install --all # also ~/.cursor/skills
betterwright skill --status        # where it landed, and whether it is current
betterwright skill >> ~/.codex/AGENTS.md   # Codex reads an instructions file

# Any custom agent — the same instructions ship as SKILL.md in this repo
# and the npm package (node_modules/betterwright/SKILL.md); copy it wherever
# your agent reads skills, or print it with `betterwright skill`.

# MCP (stdio server: browser, browser_login, browser_download, browser_handoff, browser_doctor)
npm install -g betterwright @modelcontextprotocol/sdk
claude mcp add betterwright -- npx betterwright mcp
betterwright mcp --check           # why does my client show no tools?

# Pi Coding Agent (native persistent tools, trusted login, approval-gated downloads)
pi install npm:betterwright

After an npm upgrade, setup / update refresh already-installed skill files, and doctor says so if one is still stale.

Or drive it from your own code:

import { BetterWright } from "betterwright";

const bw = new BetterWright();
await bw.run("await page.goto('http://localhost:5173')", { session: "dev" });
const title = await bw.run("return page.title()", { session: "dev" });
console.log(title.result);
await bw.close();

run() takes a string of async Playwright JavaScript with sandboxed globals — page, snapshot, screenshot, human, credentials, and friends — and returns one result envelope.

2. Standalone — BetterWright is the browser agent

BetterWright ships its own browser-tuned agent loop. Plug in a model, hand it a task in plain language:

betterwright auth --login codex     # OAuth sign-in, no API key to paste
betterwright exec "find the top Hacker News story and give me its title and points" --model gpt-5.6-sol

The loop observes with compressed snapshots, acts, verifies, captures a proof screenshot, and prints one JSON object — answer, steps, token usage, proof path.

Models are selected by real id, not by adapter nickname. Pass the model id you want (gpt-5.6-sol, claude-opus-4-8, qwen3:8b, …).

Run bare betterwright for the interactive console: one browser session across tasks, steps streaming as they happen, /model, /endpoint, and /models, plus an ask tool so the agent can check with you before consequential choices.

Use it as a sub-agent. Because exec is one shell command in and one JSON object out, a coding agent can delegate entire browser tasks to it:

betterwright exec "log in to staging and download this month's invoice" --model gpt-5.6-sol

Tokens are the bottleneck

An agent's browser loop is observe → decide → act, and the observe step is where context windows go to die. BetterWright's whole observation stack is built around that problem:

Watch it, coach it, take the wheel

Every run can carry a self-hosted live view: a web page showing the browser in real time, with chat to guide the agent between turns and a handoff flow for the moments automation shouldn't finish alone — MFA, a resistant CAPTCHA, a consequential click.

betterwright exec "…" --live-view          # watch the whole run
betterwright view --expose tailscale       # drive a headless VPS browser from your laptop
betterwright view --set-password           # lock every viewer behind a password

Why not just Playwright?

Playwright is built for tests: trusted scripts, known selectors, teardown at the end. An agent is the opposite — untrusted model output deciding its next step from what it sees, in a browser that must still be there next turn.

Install

Requires Node.js 22+. Setup downloads the pinned native BetterChromium browser for this host; GPU-less Linux runs it with the SwiftShader software renderer.

npm install -g betterwright
betterwright init      # guided: browser + agent wiring + a real page load

Getting a password back out

The vault fills logins without ever handing a secret to model code — which would leave you locked out of a password your agent generated during a signup. So there is a separate, human-only door:

betterwright vault list                # metadata: site, username, when
betterwright vault copy <id>           # password → clipboard, never the screen
betterwright vault show <id> --reveal  # print it (refuses to a pipe or a file)

Sessions and profiles

All state lives under $BETTERWRIGHT_HOME (default ~/.betterwright): the persistent browser profile, the credential vault, and artifacts.

• --session <name> — parallel lanes in one browser, sharing one cookie jar.
• --profile <name> — a separate identity: its own cookie jar, session daemon, and history.

Scope and responsible use

BetterWright automates a browser under your direction, including signing in and interacting with simple CAPTCHAs on sites you are authorized to use. It is not built for bulk account creation, credential stuffing, or scraping behind anti-bot walls at scale.

License

MIT.

Full article content is not available for inline reading.

Read the original article →

Netflix rebuilt its live ads event pipeline on Apache Flink, moving metadata enrichment out of the ad-serving critical path and into stateful streaming joins. The design handles region-aware routing, late events, hourly Spark recovery, best-effort deduplication, and dual-run validation that reached over 99.99% agreement before the legacy system was retired.

Simplifying Data and Product Integrations with a Data Abstraction Layer

Introduction

If you work with data regularly, how often have you run into this?

The widget_metrics_v2 table is the source of truth for widget metrics. Prior to 2025, use the widget_metrics_v1 table instead. There are some differences between these tables—the count_widgits metric used to be called num_widgets in v1, and average_widgit_score is only available in v2.

Situations like this are very common, and tooling just can’t keep up with constantly evolving product needs and data models. Queries are tightly coupled with the structure and topology of the underlying datasets, so making changes is often long, slow, and difficult.

What if we took a page out of the programming playbook and introduced an abstraction? In programming parlance, an abstraction hides complexity and implementation details behind a simple interface. Abstractions let consumers focus on what something does rather than how it is done, and can make it much easier to develop systems with low coupling. In this blog, we describe a data abstraction layer—or DAL—that we built at Uber to make it easier for consumers to access all kinds of data while empowering data producers to evolve models over time. Although designed to be flexible and domain agnostic, we focused initially on a critical driver of business value for Uber: ads.

Case Study: Advertiser Reporting

Advertiser reporting is a use case that significantly benefits from this concept of an abstraction. When advertisers run ad campaigns through Uber, we gather data on how their ads are performing. One critical use case for this data is reporting, where advertisers can explore their performance data, slice and dice, compare metrics, and so on. Data is collected in near-real-time, but advertisers can also query historical data for long-term comparisons and trend analysis.

Data presentations vary from charts to tables and more. Most are configurable—advertisers can specify a time range, select campaigns, and choose the dimensions to break down on and the metrics to compute. Supporting this with traditional tools is a monumental task given the request flexibility and heterogeneity of data storage. Before onboarding to the DAL, it used to take anywhere from several weeks to a couple of months to build out a new report.

Enter the DAL

The DAL is an RPC service that sits between a data consumer (in this case, the advertiser-facing front ends) and a data producer. Its responsibilities include accepting requests, determining where to source data, orchestrating queries, and assembling a coherent response. It does this through a combination of configuration files that declaratively specify data locations and relationships, and a sophisticated table resolution process that creates an execution plan to satisfy the request.

Requesting Data

The DAL exposes a FetchData endpoint for data retrieval. The request specifies a table name, schema, time range, and filters of interest. For example, Figure 3 shows a request to get basic ad performance data for an account, broken down by campaign.

Removing ad_campaign_uuid from the above request produces the same data at the account level.

Notice how metrics are automatically rolled up to match the new schema—the client didn’t need to do anything other than omit the dimension. This reflects a fundamental design principle of the DAL: the request specifies what data the consumer wants to see. The service is responsible for how the data is retrieved and computed.

Behind the Scenes

A FetchData request is made against a logical table—an expression of a data interface that defines the shape of data without describing where or how to get it. Let’s take an example.

The ads.advertiser_metrics logical table exposes the schema shown in Figure 6.

The dimensions and metrics elements are sets of columns of a particular grouping type. Dimensions and metrics generally behave as expected—dimensions are classifications used to determine the grouping key for aggregations, and metrics are measurements that can (usually) be aggregated.

If you look at the metric columns, you’ll notice that they have a source attribute. This contains a list of the physical tables candidates that provide the metric. This serves as the starting point for seeding the candidates in table resolution.

Table Resolution

Table resolution is the process by which a query plan is generated for a request based on the logical table being queried, the time range, and the requested schema. It occurs over several phases, where a set of rules is applied to physical dataset candidates based on the columns in the request to determine the right tables to query. The major phases are:

1. Schema eligibility. Candidates that don’t contain every dimension or any metric in the request are discarded.
2. Dataset availability. Tables define an availability window in terms of freshness and retention. Candidates whose availability window doesn’t intersect with the request interval are discarded.
3. Column continuity. For each metric, construct a mapping from time range to candidate. When these intervals overlap, prefer candidates that are closer in cardinality to that of the request.

From this result, we now have a set of physical tables, columns, and time ranges to query.

Explaining every part of these configurations is beyond the scope of this blog, but a few things to call out:

• Tables describe their availability windows. The realtime tables have overlapping availability with the daily table 3-8 days ago.
• Each table has the same dimensions. This is important for result assembly, described later.
• Metrics describe how they can be rolled up. All these metrics use the SUM transform, but other aggregation functions are available.

That is, this request will result in three queries being executed, one against each of the realtime tables to retrieve a subset of the metrics for the most recent 3 days, and another against the daily table for the remaining 4 days.

The example here is somewhat simplified. Physical table dependency graphs can be multiple layers deep, and logical tables can reference physical tables across multiple database technologies.

Query Generation and Execution

After table resolution, the query engine is responsible for orchestrating data retrieval. Each table undergoes the following steps:

1. A query is generated to match the schema and request semantics, possibly doing aggregations, applying predicates, and running transformations.
2. The query is sent to a query runner that submits the query, waits for completion, and retrieves the response.

Both operations are database-specific, with implementations for our internal OLAP database, Docstore, Apache Hive™, and more. Note that these queries can be mixed for a single consumer request. The DAL can retrieve data from both the OLAP database and Docstore simultaneously, for example.

The engine executes queries in parallel to minimize latency. After a response has been received from each query, we move to the final stage: result assembly.

Result Assembly

In result assembly, the individual responses to each query are assembled into a final result. There are several kinds of operations in result assembly that are conditionally invoked depending on the outcome of table resolution:

• Concatenation. Assembly concatenates datasets that are temporally adjacent. For example, if table resolution indicates that data should be read from a daily table and an hourly table, the responses from each will be concatenated together.
• Joins. Complementary datasets—meaning datasets that individually cover a subset of the metric columns—are joined together using the dimensions of each row.
• Rollups. Although aggregations are usually performed during query execution, in some situations data must be rolled up during result assembly.
• Scalar transformations. We also support a variety of other types of transformations, including virtual columns, normalizations, and reference joins.

Once assembled, the response is sent to the client.

Using the DAL in Advertiser Reporting

The simplicity and flexibility of the DAL enables powerful use cases for advertiser reporting. Most experiences are backed by a single logical table, ads.advertiser_metrics. The UI sends a schema describing the shape of the data it wants to see.

Notice how the definitions of these reports are very similar, with only slight modifications to the schema and/or predicate needed to produce an entirely different result. By convention, performance reports are a time-series showing how metrics change over time, while summary reports show the same data aggregated into a single data point per metric. Other reports can be produced quite easily with similarly small modifications. For example, an advertiser can get a performance report across all campaigns in an account by including both ad_campaign_uuid and day_of in the schema simultaneously.

Advertisers can also use a report builder to specify exactly which dimensions and metrics to include—similar to a pivot table—as well as a third-party API for programmatic integrations. Through the DAL these implementations become greatly simplified, as the inherently dynamic nature of the experiences matches closely with the capabilities provided by FetchData.

Outcome and Next Steps

Usage of the DAL for advertiser reporting had a dramatic impact on the turnaround time for new reports. What used to be a multi-week process can now be accomplished in under two days. Furthermore, this has enabled much more sophisticated tooling for advertisers that would have been infeasible under the original architecture, with dynamic experiences ranging from user-specified filters to full-fledged report builders.

There are many exciting ways in which the DAL continues to evolve:

• Use case onboarding. The DAL was initially built for advertiser reporting, but was designed from the beginning to be general-purpose and domain-agnostic. It has since evolved to serve as a general DAL for ads and expanded into non-ads domains. We expect this trend to continue with time.
• Additional database integrations. Connecting the DAL to other databases is relatively straightforward via a connector interface. This makes it easier to connect data that spans across different database technologies, a crucial value add for the DAL.
• Asynchronous requests. Async is useful in situations where you need to return larger volumes of data. The DAL extends relatively easily into some aspects of asynchronous execution, but some other parts require a bit of sophistication.
• Extensibility mechanisms. While the typical request flow is fully generalized, domain-specific requirements are inevitable. The DAL provides a few extension points in the form of decorators that allow its behavior to be customized in a controlled manner to facilitate this complexity.

Conclusion

We discussed the implementation of a Data Abstraction Layer that provides an abstraction between the producer and consumer of datasets, which enables teams at Uber to move fast and build powerful experiences on top of data. We also described a case study of how the DAL was used to solve complex problems in advertiser reporting.

Acknowledgments

Cover Photo Attribution: “Beautiful aerial view of the teotihuacan pyramid of the sun and Moon in Mexico” by Marie Hernandez is licensed to Uber.

Apache®, Apache® Hive™, and Hive™ are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.

Full article content is not available for inline reading.

Read the original article →

Spark partition sizing is a throughput and stability lever: aim for enough partitions to use cores without creating tiny tasks, filter early, avoid unnecessary shuffles, and repartition around joins or writes when the current layout is wrong. The practical advice covers AQE, skew, spills, checkpoint barriers, and small-file tradeoffs.

A small browser tool parses Parquet files with Rust arrow-rs and exposes the internals that usually stay hidden: row groups, column chunks, pages, schemas, encodings, Bloom filters, sizes, and min/max statistics. It is a useful teaching and debugging aid for understanding why columnar files behave the way they do.

Semi-Structured Data in Apache Iceberg: Meet the Variant Type

Semi-structured data, such as JSON-like documents whose fields differ from row to row, has always been a poor fit for table formats built around fixed schemas. Iceberg v3 adds the Variant type for exactly this data: a single column can hold values of arbitrary, evolving shape, stored in a compact binary form that engines read and write consistently.

This is the first post in a series on Variant in Apache Iceberg. It covers what Variant is, why it exists, and how it fits into an Iceberg table. The next post covers shredding, the technique that stores frequently accessed Variant fields as typed, columnar data. Variant is stored in Parquet, Avro, and ORC; shredding is currently available only in Parquet.

The problem Variant solves

Consider event data whose shape changes over time:

{"event": "view", "page": "/pricing", "session": "s-8f21"}
{"event": "signup", "session": "s-8f21", "plan": "pro", "price": 29.00}
{"event": "view", "page": "/docs", "session": "s-3c07", "referrer": {"source": "search", "term": "iceberg variant"}}

Two traditional approaches handle this, and both have drawbacks:

• JSON stored as a string. This is flexible, but reading a single field means parsing the whole text. JSON's type system is also thin: a timestamp is just a string, and a number's precision is ambiguous.
• A rigid, flattened schema. This is fast to query, but every new field is a schema migration, and sparse or one-off fields waste space.

Variant is as flexible as JSON but stores data in a compact, typed binary form. Values keep their native types: a timestamp stays a timestamp and a decimal stays an exact decimal, instead of collapsing to JSON's strings and numbers. Within a value, field names are collected into a dictionary and referenced by id, so a name is not written out in full each time it appears. No schema is declared up front, so documents of different shapes coexist in one column and a new field needs no migration.

Variant in Apache Iceberg

Variant was added to the Iceberg type system in the v3 spec. The spec places it in its own category: variant is "neither a primitive type nor a nested type." It is richer than a primitive, yet has no fixed, declared shape the way a struct or list does.

A Variant value is similar to JSON, but with a wider set of primitives including date, timestamp, timestamptz, binary, and decimal. It can also nest:

1. A Variant array is an ordered collection of Variant values. Unlike an Iceberg list, its elements are not constrained to a single element type.
2. A Variant object is a collection of string-keyed fields whose values are themselves Variant values. Unlike a struct column in an Iceberg schema, its fields are not a fixed, named set of typed columns.

How it is stored

Iceberg does not define its own binary encoding for Variant. The type and its encoding come from the Apache Parquet project, and Iceberg uses that encoding as-is, so a variant column maps to a Parquet group with two binary fields:

optional group payload (VARIANT(1)) {
  required binary metadata;
  required binary value;
}

• metadata holds a dictionary of the field names used in the value, so the value bytes reference each name by an integer id instead of repeating the name string.
• value holds the encoded data: a scalar, an array, or an object. Arrays and objects store a field_offset per element (the byte offset where that element's value starts), and objects also store a field_id per field (an index into the metadata dictionary).

The Variant column itself is addressed by field ID like any other Iceberg column, but its metadata and value subfields are accessed by name, which matters for shredding.

Reading and writing across engines

• Apache Spark 4.0 and 4.1 create tables with VARIANT columns and read and write them. Reading landed in Iceberg 1.10.0 on Spark 4.0; Iceberg 1.11.0 added Spark 4.1 and writing shredded Variant on both Spark 4.0 and 4.1.
• Apache Flink 2.1 added Variant support in Iceberg 1.11.0. This covers unshredded Variant only; shredded write support is merged and expected in a later release.

Because these engines all use the same Iceberg Variant type, a value written by one reads back identically in the others.

Apache Arrow carries that same Variant value in memory through its canonical extension type arrow.parquet.variant, letting engines exchange it without special handling.

Working with Variant

With Spark SQL, you store heterogeneous events in a single VARIANT column and read fields back with variant_get. A Variant column requires an Iceberg v3 table:

-- Variant is a v3 type, so the table must be format version 3
CREATE TABLE events (id BIGINT, payload VARIANT)
USING iceberg
TBLPROPERTIES ('format-version' = '3');

-- Insert events of different shapes into the same column
INSERT INTO events VALUES
    (1, parse_json('{"event": "login", "country": "US"}')),
    (2, parse_json('{"event": "purchase", "country": "UK", "amount": 99}')),
    (3, parse_json('{"event": "login"}'));

-- Read fields out of the Variant with variant_get(column, path, type)
SELECT
    id,
    variant_get(payload, '$.event', 'string')   AS event,
    variant_get(payload, '$.country', 'string') AS country,
    variant_get(payload, '$.amount', 'int')     AS amount
FROM events;

The query returns one row per event, with null wherever a field is absent:

Row 3 carries only event, so country and amount read back as null. Variant does not require every row to share the same structure, so heterogeneous events live in one column.

When to reach for Variant

Variant is the right choice when you do not control the shape of the data, or it changes faster than you want to evolve a schema:

• Event and clickstream data, where each event type carries a different set of fields and new ones appear over time.
• Application and service logs with structured but heterogeneous payloads.
• IoT and sensor telemetry, where each device model reports its own readings.
• Third-party API responses and webhooks, whose schema is owned by someone else and can change without notice.
• Sparse attributes that would otherwise become a wide table of mostly-null columns.

Variant is not a replacement for a known schema. When a field is present on every row and you query it constantly, a regular typed column, or a struct for a fixed nested shape, is simpler and more efficient. A common pattern is to keep the stable, frequently queried fields as normal columns and put the variable part in a single Variant column.

What's next: shredding

So far, a Variant column is a single metadata + value pair. Reading one field means decoding the metadata dictionary, then deserializing that field out of the value blob if it is present. That is flexible, but the flexibility costs performance: because the whole document lives in one value column, the engine cannot use per-field min/max statistics to prune data the way it can for a regular typed column. Shredding recovers this: it stores frequently accessed fields as separate, typed Parquet columns, so queries that touch only those fields read only those columns and can use their statistics for data skipping, while anything that does not fit falls back to the untyped value. The next post in this series explains how shredding works and what it unlocks.

Getting Involved

Variant support in Iceberg is still growing, and contributions are welcome. The Variant tracking issue (#10392) follows the ongoing work, and you can reach the community through the mailing lists and Slack.

Resources

• Iceberg Table Spec, Variant type: Semi-structured Types
• Variant in Apache Parquet: Variant Type in Apache Parquet for Semi-Structured Data
• Variant in Apache Arrow: Parquet Variant canonical extension type

We will ship the first Polars 2.0 release candidate next week. Polars 2.0 will be great for casual Polars users as it will default to the streaming engine, meaning your code will be faster without doing anything.

So the first release candidate next week. Give it a spin!

Massively parallel Postgres backups

Every 12 hours, a backup system must turn the entire state of a busy database into a consistent, encrypted snapshot, with no impact to production queries.

Such backups are crucially important and simultaneously something that most engineers would rather never have to think about.

Just make the backup work.

Our goal at PlanetScale is to make taking, scheduling, managing, and restoring Postgres and MySQL backups effortless.

Though this is what our customers experience from the outside, achieving this internally requires careful orchestration of cloud infrastructure and DBMS tooling.

It's especially interesting to look at backups for sharded databases, which requires spinning up backup-specific nodes, pulling data from object storage, and WAL replay, all with massive parallelism. These techniques allow for petabyte-scale databases to be backed up in hours, at rates over 50 GB/s.

Here we take a behind-the-curtain look at how to effectively back up a sharded database with massive parallelism.

The backup lifecycle

Here is an example of a Neki (sharded Postgres) database with 8 shards, happily handling hundreds of thousands of queries per second of production traffic.

If sharding is a new concept to you, check out our recent post on how it all works. The first step in taking a backup depends on whether this is the very first backup or if we've taken one previously. We'll start with the steady-state case, which assumes a prior healthy backup already captured and stored in Amazon S3 (or similar object storage in other clouds).

Since a sharded Postgres database is many individual primary Postgres servers working together, we use regular Postgres backups as the building block for large-scale backups with Neki.

There are three ways to take backups in Postgres, which we've written about in detail previously. The best of the three, and the one that Neki uses, is combining filesystem backups with the replay of archived Write-Ahead Log (WAL). To summarize, the steps for this are:

1. Begin a full backup of the on-disk Postgres database files at time T1
2. The backup completes at time T2; between T1 and T2, rows on disk may have been mutated
3. Replay the write-ahead log modifications between T1 and T2 to correct mutated data
4. Store the final result in a distinct storage location like Amazon S3

We could complete these steps directly on the primary, or perhaps one of the traffic-serving replicas. The problem is that this task utilizes a significant amount of IOPS and compute, especially for a large database. Our goal should be to minimize the impact a backup will have on production query serving.

Because of this, we take a different approach. We spin up a brand new set of EC2 instances, one per shard, to manage the backups on.

These new instances will be responsible for the majority of the work in the backup. Because we operate these sharded databases in clouds like AWS and GCP, dynamically spinning up tens or hundreds of instances for short time slices to complete backups is achievable. It adds a small amount of cost, but is worth it to minimize negative production impact.

Reusing old backups

The next step is restoring the most recent backup to each shard. Prior backups are stored in object storage. Here we will use Amazon S3 as the example, but the same applies to other clouds (like GCS in Google Cloud). These are streamed directly from here.

This approach requires temporary compute and transfers each shard's data out of and back into object storage. We accept that cost for two important reasons:

• Only recent WAL comes from the primary, minimizing production impact
• Every cycle proves the previous backup can be restored and replayed

Once all the copying completes, these 8 servers have the exact state of each of the 8 shards from the previous backup, 12 hours ago.

Replaying the WAL

We now must catch up each shard's old backup to match the present state of the database. This requires replaying all changes from the Postgres Write-Ahead Log between 12 hours ago and now.

A naïve approach would be to pull the WAL directly from the primary. This is problematic for several reasons:

1. This would have nontrivial production impact. Replaying 12 hours of WAL on a high-churn database could take tens of minutes, or even an hour+.
2. Because we don't want too much server storage consumed by WAL, we continuously archive it to S3. Thus, we likely don't even have the full past 12 hours of WAL resident on the primary node.

On PlanetScale, all Postgres databases use wal-g to continuously archive their write-ahead logs. A sharded Neki database is similar, except each shard has a distinct WAL archive stream.

What if we streamed from there instead?

This almost solves the problem. The remaining problem is that Postgres archives WAL only after a segment is complete. If write traffic doesn't fill a segment sooner, our five-minute archive_timeout setting forces a segment switch so it can be archived. This means the newest changes may not have reached S3 yet. Thus, we take a hybrid approach. The system uses S3 for the majority of the replay, then streams the last ~few minutes of changes directly from the primary. Ideally, this last step takes on the order of seconds, not minutes or hours.

Completing the cycle

When every node has caught up replication to time T, where T is the timestamp we will log for the backup time, we stop WAL replication, freezing the point-in-time of the backup. Time T is saved to ensure we know the precise time, down to the second, included in this backup. The full backup is now consistent, with no smeared data. The final step is to encrypt and send these backups off to a new S3 bucket for safe keeping.

When complete, the backup nodes have served their purpose, and are decommissioned.

The initial backup

The cycle just described assumed we had a good backup from 12 hours prior to start with. In the steady-state this is true, but not for a brand new database.

Within 12 hours of a new database creation, pg_basebackup is used to seed a backup node for each shard.

pg_basebackup is a built-in PostgreSQL client utility that takes a physical backup of the entire database cluster: the data directory, tablespaces, and configuration needed for recovery.

Why not upload the pg_basebackup output directly? We use it only to seed the temporary backup nodes. The durable backup is created with wal-g, keeping the format and restore process identical for initial and steady-state backups.

The steps for this first one are similar, but slightly different than the steady-state:

1. Spin up one new EC2 instance per shard.
2. Run pg_basebackup on each instance to copy data from its primary.
3. Configure each instance to replicate from its primary.
4. Replay WAL until every instance catches up.
5. Stop replication at time T.
6. Use wal-g to format, encrypt, and upload each backup to S3.

Once this exists, all future backups happen with the restore -> catch up -> save flow.

Need for speed

Part of the reason we do so much parallelism is due to the nature of sharding. Whether it's 4 shards or 400, since each shard contains its own Postgres primary, we may as well take a rock-solid system (regular Postgres backups) and do it over and over on each shard.

A side-effect of this is that backups are VERY fast, even for large databases.

For every backup, the data transfer steps are:

1. Restore old backup from S3
2. Catch up the backup using a hybrid of S3 + the primary
3. Send the new file back to S3

Consider what it would take to complete this backup cycle on an unsharded 32 terabyte database. Backups on PlanetScale are compressed and stored with encryption at rest in S3. We'll assume that backups for this 32 terabyte database compress down to 20 terabytes. Therefore, we have

1. 20,000 GB transfer from S3
2. Catch up (Ex: 20 GB, compressed to 10 GB)
3. 20,010 GB back to S3

This equates to a total transfer volume of ~40,030 GB. If our various nodes and interconnects can sustain a 500 MBps transfer rate, this means the backup would take ~22 hours. Crucially, this means that if we want twice-daily backups, multiple backups would overlap. This would prevent us from meeting our recovery point objective (RPO).

This same database in Neki, spread over 8 shards storing ~4 TB each, performs quite differently. In total, we need to transfer, catch up, and store the same 40,030 GB. But if we can do so across 8 distinct backup nodes in parallel, each capable of 500 MBps, we reduce the total time to ~2.8 hours.

This gets better the more shards you add. The same data spread across 32 shards would back up in 42 minutes. Backup speed scales well. 100 terabytes on 100 shards backs up at ~the same speed as 1 terabyte on a single shard.

What are backups used for?

One reason to take backups is data safety. In the case of accidental data deletion or a one-in-a-million database disaster, backups (and the WAL) are an essential fallback.

But at PlanetScale, backups are also a core part of everyday database operations. For Metal databases specifically (ones with Local NVMe), backups are also used every time the database is resized.

Resizing a Metal database requires:

1. For every existing node in the sharded database, spin up a brand-new EC2 instance at the new size. In a database with 8 shards each with a primary and 2 replicas, we may currently be running it on 8 x 3 = 24 i8g.xlarge nodes. While running, we initialize 24 additional i8g.2xlarges to double the compute capacity.
2. Each of the 24 new nodes pulls down a copy of the most recent backup from S3, and begins catching up to the present state of the database via the archived WAL.
3. All nodes are initialized as standbys of the original primary, and complete final replication catch-up.
4. When all nodes are synchronized with the primary, a switchover is made from the old i8g.xlarge primary to a selected new i8g.2xlarge primary.
5. The smaller nodes are decommissioned, leaving only the larger primary and replicas.

We've now completed a full resize across all shards. Backups are used to facilitate the creation / catch-up of the new nodes. This process can take anywhere from minutes to hours, depending on how large the backups are and how much WAL there is to replay.

Backups are also used when a node needs to be replaced due to unexpected failure. If your average server lifetime is 5 years, you'll rarely notice a failure on a small database with a single primary. For a database with hundreds of shards, each with multiple replicas, the statistical likelihood for node failure in a given week or month increases significantly. When we see an individual node fail (say, a single replica in a shard), the process for replacement looks like:

1. Initialize a new cloud instance of the same size as the previously failed one.
2. Use the process described earlier to restore a recent back-up to it, catch up the WAL.
3. Initialize as a follower of the original primary, synchronize data.
4. This new node can now actively serve read queries and/or serve as a new primary during a switchover or failover operation.

What about MySQL?

Everything here has been about how we complete sharded backups and resizes on Postgres databases, powered by Neki. PlanetScale also operates large-scale sharded MySQL databases, powered by Vitess as our query routing / sharding layer.

The process for backing up and resizing these databases is quite similar! We have a whole separate blog on how this works, but the main difference is that we use VTBackup instead of the Postgres backup builtins, we use the MySQL binary log replication instead of WAL, and replication catchup is done from the primary instead of a hybrid between S3 and primary.

Make it boring

Ultimately, we want all this to be transparent to you. Taking a backup should be as easy as an automated schedule, or clicking a button. Resizing a database should be a single click or API call.

However, we also have a deep appreciation for the finer details of database operations. By taking a journey through the lifecycle of backups, we hope you have a new-found appreciation for how incredible database orchestration is.

If you find this fascinating, we're always looking for talented database engineers.

Faster prototypes miss the point: The case for spec-driven design

Design is in danger of being left out of agentic teams if its only answer to development's new speed is faster mockups. With coding agents, specifying intent is the main bottleneck now, and that is design's home ground. This piece is about how design can integrate with agentic software development and shape the spec from first insight to what gets built.

Software development is going through a structural shift. Agents can turn clear intent into working, tested code, so the bottleneck is no longer writing code but articulating intent, and the spec is the artifact that captures it.

Engineering already has names for this: spec-driven development and the agentic software development lifecycle (ASDLC). The framing is a software factory, where agents are the logistics layer and context is the supply chain feeding each station just in time.

An assembly line of agents does not sound like a home for design craft, and many designers assume there is no seat for them in it. The framing does put most of design's visible output on the side of the work agents now do cheaply.

But design will not earn a better position by default. So far we have mostly met development's new speed by working the way we did five years ago, with AI tools that generate mockups and vibe-code prototypes faster than ever. Faster prototypes miss the point.

The risk is less that designers get replaced and more that they get routed around. When development compresses a build from weeks to days while design still needs weeks to form an opinion, teams stop waiting for the opinion. Concretely, that means the spec gets written without design, and the spec is what gets built.

Design does not get back in by producing more. The work that moves a product forward is judgement applied to accumulated knowledge: which problem is worth solving, what users have already told us, what was tried before and why it fell short. Assembling that knowledge has always been the slow, expensive part, because it lives scattered across tools, decks and people's heads. AI's larger gift to design is the thing it does best: managing knowledge.

Prototyping faster is the least of it. AI's larger gift to design is the thing it does best: managing knowledge

Read that way, the factory offers design a more impactful position than the old process ever did. This piece is about how design integrates with it: as an active author of the spec, on a knowledge base that compounds. I call this merge spec-driven design.

The two leaks that hold design back

The usual design process leaks time and value in two places, and this is the main reason good design is still slow.

Knowledge flows into design too slowly. Picture a designer starting a new feature. The brief is two sentences in a ticket. The reasoning behind it lives in a strategy deck from last spring, a Slack thread, and the head of a product owner booked until Thursday. Related user research exists, somewhere. A previous team tried something similar and dropped it, and nobody wrote down why. So the first weeks go to archaeology.

Sort that coordination work into two piles. One is retrieval dressed up as work: assembling scattered context by hand, re-explaining decisions to whoever missed the meeting, asking experts for facts that could be looked up. This pile exists only because the organisation's knowledge is not queryable. The other is co-creation, the heart of design practice: shaping flows and interfaces through explorative iteration with other people, debating real tradeoffs, generating ideas in dialogue, making calls on strategy or ethics, talking to users. That pile is the job.

Try the same split on your own last month and see which pile is bigger. Whatever the first pile costs, it costs that only because of how the organisation's knowledge is stored, and that is a fixable property rather than a law of nature.

Intent flows out of design too lossily. In most teams the journey to production looks like this: design in Figma, a walkthrough meeting, a developer's interpretation, a review round, corrections, another interpretation, another review. Intent leaks at every step, and everyone involved knows it. The designer pays in review rounds, pixel-policing, and shipped features that miss the point of the design. The artifacts that serve this transfer make the waste visible: walkthrough decks, redline annotations, handover tickets, each written for a single meeting, read once, and dead within a sprint.

Engineering sees the same failure from its side. ASDLC names lossy handoffs as a central defect of traditional development, with insights compressed into decks and tickets that reach an agent as unstructured blobs. It also names the design half of the problem the Figma trap: overreliance on mockups to carry the whole specification. A mockup shows the happy path while hiding the edge cases, error states and data consistency rules where production bugs live. That makes it a suggestion rather than a specification, and screens alone are not a handover.

Both leaks now have names in software research. Margaret-Anne Storey calls them cognitive debt and intent debt. Cognitive debt is the erosion of a team's shared understanding of its own product. Intent debt is the absence of externalised rationale, where decisions get made but the why is never written down where anyone, human or agent, could read it later.

That describes design's situation exactly. Designers pour enormous time into artifacts to align the team and convey intent, yet those artifacts are rarely durable. The understanding they carried evaporates, and the next design starts the whole process over.

Designers pour enormous time into artifacts for alignment and intent, yet those artifacts are rarely durable.

Closing both leaks: the knowledge base and the spec

Design remains too slow for agentic teams if the knowledge it depends on doesn't accumulate where agents can reach it, and if its intent doesn't survive the trip from insight to production. The solution is two connected parts.

The Product Knowledge Base (the KB) is the team's shared memory: what the product is for, what has been decided, what we know about users, all of it queryable. The feature spec is the core intent artifact: living documentation inside the KB for each feature, stating what to build, why, and how to verify it. Design draws on the KB and feeds back what it learns, and carries its intent into development through the spec.

The compounding knowledge base

To answer the first leak (slow knowledge retrieval), a team builds a Product Knowledge Base alongside its codebase: the knowledge the team runs on, stored so that agents can fetch what each task needs on demand.

In my experience six components earn their place as core parts of a product KB from a design point of view. The agent instructions file is the front door, telling agents where everything lives and what to read before a task. The product vision is the direction each decision is weighed against, so features cohere into one product instead of each drifting off on its own local logic. Decision logs record what was chosen and why, so calls are not silently reversed or re-argued. Insights collect what the team has learned, from any source: user findings above all, but analytics, support patterns and market reading too, each one reusable so a lesson gets learned once instead of rediscovered. Product-level standards, from the design system to accessibility, are contracts agents build to. Each feature spec carries its own intent and the acceptance criteria that verify it.

The inventory is not the interesting part. A wiki like Confluence could hold the same documents. What differs is how knowledge gets reached: a wiki is read when someone remembers it exists, while a KB is queried by an agent at the moment a decision needs it. Instruction files and skills tell agents what to consult before a piece of work and what to update after it, so retrieval stops depending on anyone remembering.

For design, the component that benefits the most is research. User findings enter the KB as atomic insights: single findings, tagged and reusable, each linked back to the source that made it true so a finding never loses its context. They accumulate from two directions, research projects and feature work alike. The idea is Daniel Pidcock's atomic research, and spec-driven design gives it a real chance to scale. Connecting scattered findings into generalisable insight is exactly what agents are good at. And once user insight lives in the repository, at the centre of the team's shared memory, user-centricity becomes a default rather than an argument a designer has to win each time.

Put user insight where every agent and teammate will read it, and user-centricity stops being something a designer has to keep arguing for.

In its simplest form all of this is a set of plain text files, kept in the product's own repository beside the code, that humans and agents both read and edit. Markdown, versioned with git.

That is the core idea of spec-driven development: the knowledge lives where the code lives, so it updates in the same commits. The version history then records who wrote each entry and when, at no extra cost.

Spec-driven design turns ambiguity into buildable intent

To answer the second leak (lossy intent transfer), we work on features through a single living feature spec throughout the entire design process. This way, intent reaches development as something an agent can read and check rather than a walkthrough someone has to interpret.

Design's purpose is to make sure the team builds the right thing before engineering builds the thing right. Spec-driven development is usually told from engineering's side, as if the spec arrived ready at planning, leaving the human a thought leader "deciding what matters". But deciding what matters takes generative research with real people, and knowing whether the solution works takes testing with them.

So in reality the spec does not arrive ready. Engineering practice treats an ambiguous spec as a defect. For design, the open questions are the spec while a feature is in discovery and prototyping, and the design process is what turns them into something an agent can implement reliably. Here is how that happens, across five kinds of design work.

Engineering calls an ambiguous spec a defect. For design the open questions are the spec.

Discovery is where the KB's user truth gets built, and everything downstream inherits its quality. The designer spends real time with real users, plans the right studies, and has the findings written up so the whole team can draw on them. This work deposits into the KB as atomic insights rather than into any single spec.

Problem definition turns insight and context into a rich brief. The context that used to take weeks to assemble is now a query: the goal behind the feature, decisions and learnings from earlier work, the technical constraints, and the insights about how people handle the problem today. The spec is born here, carrying the problem being solved, what success looks like, the initial requirements and the open questions with owners.

Craft is the iterative work that turns requirements into design decisions, and prototyping is how those decisions earn their place. Each prototype answers a question the spec has already posed: an unresolved flow, a contested edge case, two directions with no obvious winner. What lands in the spec is the chosen flows and edge cases, plus the directions tried and rejected and why, so nobody re-explores a dead end next year. Whatever the work happens in, a Figma file or a prototype built straight in code, the spec links to it as the canonical source while keeping the decisions and reasoning written out.

Evaluation splits in two, and only one half needs a human. The design review rounds that used to hunt for wrong spacing, missing states and off-system components now run as an agent evaluation against the standards defined as contracts in the KB, and design-system conformance can be settled deterministically, because token and component use is checkable in code. Human reviewers stop spending their attention on contrast ratios. The designer handles meaning: running usability tests, reading what the results say, and making the calls that follow. Findings land in the spec beside the decisions they tested.

Handover stops being a series of meetings and becomes a property of the spec. There is nothing to walk anyone through, because by the time development picks the feature up the document already carries what is needed to build it as intended, and the designer's work here is bringing it to that standard. That does not mean developers meet the feature for the first time in the repository: a spec answers intent debt, and the shared understanding that answers cognitive debt gets built during the work rather than in a meeting at the end.

This is living intent documentation: one spec per feature, from first idea to production, treated the way GitHub's Spec Kit treats specs: living, executable artifacts rather than documents that rot after kickoff. Design fills the spec and reads out of it in the same motion, using the growing spec as important context for its own agentic work, whether it’s research or craft. This makes the design spec-driven rather than spec-producing.

What the team gets out of it

Hours move to the work worth doing. The time that went into assembling context by hand, producing walkthrough decks and re-explaining decisions is time agents can absorb. What is left is co-creation: time with users, problems framed together, more directions explored in craft. This is the one trade I am most excited about.

Intent arrives intact. When intent reaches development as a spec with criteria an agent can check against the running feature, the fix rounds that used to bounce between designer and developer have much less to bounce about.

What the team learns keeps paying. The familiar complaint in user research is that nobody reads the research report. With atomic insights right inside the repo, research keeps giving. An insight from a study run last year turns up in the brief for a feature nobody had thought of yet, so research paid for once keeps working, and it does that whether or not the person who ran it is in the room.

Design becomes the input the factory cannot generate for itself. If design's value was producing screens, agents are a threat, because screens are what they are now cheap at. If it is knowing what users need and judging what is worth building, agents amplify it, because the scarce input in an agentic team is high-quality intent, and that has to come from someone who has spent real time with real users.

Where to start

My early tests building simple knowledge bases and spec-based handovers in client projects are showing promising gains in both speed and quality, along with stronger collaboration with developers.

My suggestion to any designer is to start small and local: a KB on your own computer, and one spec for the next feature you pick up. If you can, propose building it inside your team’s repo, so the spec you build can be taken into use directly by developers. It takes repository access and a basic understanding of git to move around safely, both learnable with an agent alongside. Explore and iterate what to include in the design spec and judge for yourself what it does to your delivery and to your team collaboration.

AI-assisted development quietly erodes design systems, as agents default to generating or pulling components instead of using governed ones, leaving documentation increasingly inaccurate. New design-system extraction tools added to Figma Console MCP analyze a codebase as it actually exists, pulling tokens and components and scaffolding a matching Storybook with a deterministic evaluation pass. Tested on Rateshare, a vibe-coded app with no prior system, the tools detected React, found 21 components, and verified them via Playwright, demonstrating the extraction mechanics rather than recovering from drift.

Meta is currently testing its upcoming Muse Video model in beta with selected partners. The model, labeled “Beta,” was first previewed in July alongside Muse Image. At that time, Meta shared initial samples and announced that Muse Video would soon be available to creators and Meta AI.

TestingCatalog recently gained access to Muse Video and tested it directly. Based on our early generations, the model shows state-of-the-art potential, particularly in fine detail, world understanding, and temporal consistency. It currently produces 10-second videos, but the output quality is remarkably high. Meta has also confirmed that Muse Video supports native audio, although the company previously acknowledged remaining gaps around audio-video synchronization and physically accurate fast motion.

Muse Video samples:

• "Cyberpunk hacker robot working in front of many monitors"
• "A shaky, low-quality iPhone video recording. A guy holding the camera selfie-style walks up to random people on a busy city street and asks each one, "How do you feel about being AI generated?" The video has authentic smartphone artifacts—slight motion blur, overexposed sky, wind noise, and the occasional finger over the lens."
• "The rocket at the rocket station is launching. Everything looks normal until the rocket explodes into confetti pieces and leaves nothing behind."
• "The man couldn't believe it when he finally became a kitten dad. Excitedly he shouted "I'm a dad now! A cat dad!". He's so happy that he starts tearing up. The kitten meows before the video ends."
• "A cat made out of Lego technic in the rain made out of fluffy plastic rain"
• "A glass of red wine tips over on a white marble counter, spilling toward the camera. Static shot"
• "A woman with curly red hair and a green scarf walks toward camera down a crowded Tokyo street at night, stops, and looks up."
• "Orbiting shot around a chef plating a dish, 180-degree arc, shallow depth of field."
• "A neon sign reading ‘OPEN 24 HOURS’ flickers on above a diner door in the rain."
• "A car’s headlights sweep across a bedroom wall at night, shadows shifting."
• Two swordsmen in a bamboo forest exchange three strikes, blades meeting mid-frame. Wide shot, overcast light.
• "Two fighters in a dimly lit warehouse; one throws a punch, the other blocks and counters. Handheld camera, shallow depth of field."

There is still no confirmed public release date. Meta has already stated that Muse Video is coming to Meta AI, making Meta.ai and the Meta AI app the most obvious initial destinations. Given that Muse Image is available for free everyday creation, Muse Video could follow a similar model, although its pricing or generation limits remain unknown.

The broader distribution possibilities are particularly important. Muse Video could become part of Meta’s Vibes feed for AI-generated videos and later expand across Instagram, Facebook, and other Meta products. Instagram’s video-heavy ecosystem is an obvious fit, while Meta’s Edits video creation app could also benefit from native generative video capabilities.

For Meta, video generation fits directly into its wider strategy of building its own media models and distributing them across products used by creators and consumers. Facebook and Instagram already revolve heavily around visual content, giving Meta a natural distribution advantage if Muse Video proves competitive at scale.

Our current access indicates that development has moved beyond Meta’s original public preview into working beta testing. Don't forget to take a look at the video samples we have collected so far.

PorTAL: Portable Task Adaptation for LoRA

Learn a task adaptation once in a base-agnostic form, then port it to new frozen models by refitting only a thin per-base alignment — recovering ~98% of per-task LoRA's lift on an unseen model within the same family and ~94% across families.

AI models are now capable and affordable enough to make once-unreachable outcomes practical. The opportunity for everyone to pursue and collaborate on ambitious ideas, spend less time on busywork, and raise the quality and creative output across disciplines is nearly here.

The possibility is here. But for most people, the reality is not.

Replit users are already building million-dollar businesses, replacing legacy software and saving hundreds of thousands of dollars, and increasing their everyday potential by creating high-quality software, designs, slides, and more.

But using AI today still means choosing models, managing context, watching usage, and juggling a growing collection of sprawling tools. Intelligence is now abundant, but access remains complicated.

That’s why today we’re launching a faster, more cost efficient, and capable Replit:

• Create up to 30X more with Free Mode: Free Mode is a new way to use Agent that lets you create 30x more with just your monthly subscription. Spend less time thinking about usage and more time bringing ideas to life.
• A new UI built to get you straight to the outcome fast: You can ideate, create, launch, and grow all in one solution, as Replit seamlessly transitions between simple chat and tasks to complex builds.

Create up to 30X more on Free Mode

As models become more capable and cost-effective, people’s ability to create will only expand.

Free Mode, powered by OpenAI’s GPT-5.6 Luna, will allow everyone to create more for just the cost of their monthly subscription.

When you’re in Free Mode, every day tasks will no longer use credits, so whether you’re chatting, ideating, or running everyday tasks, you can create with freedom. This pairs the world’s most powerful AI software creation tool and agent with a mode designed for fast, high-value everyday tasks, giving users a true all-in-one option.

Core subscribers will now be able to create 30X more than before on Replit using Free Mode, as well as up to 30 hours per month of chat. For just $20 per month, creating real, high-quality projects at scale with AI is now accessible.

“We’re past the point of free options being a starting point. With Free Mode you’re getting quality, ready to put in front of people and start getting business.”
~ Ken G (Early Access User)

“I was trying to find the outer limits of Free Mode. I haven’t yet. My choices are bolder because I’m not worrying about burning tokens.”
~ Ruth H (Early Access User)

Core and Pro users can use Free Mode until they reach their usage limits - which reset every 5 hours - or continue building in Power or Max Modes. Pro users will have even greater usage limits than Core customers.

If your work progresses to a more complex or high-value task, Replit Agent may suggest switching to our other Agent Modes, Power Mode and Max Mode.

• Power Mode (formerly Economy Mode) uses cost-optimized models for everyday tasks, and delivers a strong balance of speed and quality. When using Power Mode, you will get the same, or greater performance you got in the former Economy Mode, for the same cost.
• Max Mode uses higher-performance models for more complex work, which can be best for larger changes, deeper reasoning, and longer builds.

A Faster User Experience That Goes Straight to the Outcome

For AI adoption to accelerate, users need an experience that just works - fast, efficient, accurate, high-quality, and grounded in the context of their world.

We’ve redesigned the Replit experience to make it your daily driver: an intelligent, always-on collaborator that understands you, works at your speed, and can handle everything from quick questions and data analysis to complex builds and great designs.

With Replit, users can now get fast, accurate answers, suggestions, feedback, and analysis in seconds - without consuming usage credits in free mode. Because Agent understands the full context of your work, it can help you plan, ideate, shape, optimize, and explore ideas before shifting to a more complex build.

“Insanely fast!” ~ Steve P (Early Access User)

“I no longer have to switch contexts, pasting chats from other AI tools. I love that I can think through a project before starting and build in the same space.” ~ Angela M (Early Access User)

Unlike other solutions, Replit can understand when you want quick answers, and when you want to do more complex work. It also carries your context from chat to creation and across projects, so you don’t have to feel like you’re starting from scratch every day. It’s less like hiring a contractor for a single project, and more like working with a chief of staff who deeply understands you and the outcomes you care about.

And if you just want to build, that’s still simple: find your project and keep creating.

Maximize making on Replit

Today is a major step forward in our long-term goal of making software creation, and intelligence, accessible to anyone with an idea, regardless of technical background. You should just be able to express your intent and maximize what you can make.

Log in to replit.com today to try it out.

Full article content is not available for inline reading.

Read the original article →

Superwhisper's S1-mini model is a 0.6B-parameter text normalizer for speech-to-text outputs, achieving a 94.8% token accuracy in transforming raw ASR transcripts into clean written text. It's optimized for English, runs comfortably on CPU, and requires a specific input format, including a control line for styling, structure, and context settings.

You Probably Don’t Get Why Stripe Bought OpenRouter

Deployment-Time Alignment at Scale

Anjney Midha, Malika Aubakirova

Stripe’s purchase of OpenRouter has been read as a routing or billing acquisition. We argue it’s a strategic security decision, and that this is net positive for the ecosystem.

1. Introduction

OpenRouter has announced it is joining Stripe. Many VCs, pundits, and analysts have opined on why the merger makes a lot of sense, or no sense, depending on the day of the week. We have not found a single analysis accurate.

The short version is simple: ecosystem-wide AI security and alignment. Not routing. Not billing consolidation. Not “tokens are the new dollars,” (although they are). The long version is below.

2. Stripe is a Security Company

The common view of Stripe is a company that moves money. Moving money naively is a commodity; banks did it for centuries at rock-bottom margins. What Stripe has built, in contrast, is an online trust machine at scale. Radar scores adversarial transactions across the network daily. The API everyone praises is developer experience layered on security infrastructure: fraud models, chargeback liability, identity verification, and compliance across every jurisdiction on earth. Stripe wins because it underwrites risk on hostile traffic at internet scale better than any comparable institution.

3. The Shape of OpenRouter’s Data

We published an analysis of 100 trillion tokens flowing through OpenRouter. The median request is not, as many might expect, a human asking an LLM a question. It is a machine in the middle of a loop: reasoning models went from a rounding error to more than half of all traffic in a year, average prompts grew 4x, and a material share of requests terminated in a tool call.

The study’s punchline is straightforward: inference platforms must now manage continuous context and state at scale.

4. Alignment is a Context Feedback Problem

As with payments at scale, frontier AI security at scale is a continuous context feedback loop problem. Radar is defensible for primarily this reason: it trains on adversarial transactions at scale, daily, and has for a decade. One cannot replicate the model without the corpus freshness, and one cannot obtain the corpus without sitting in the flow. Every durable security franchise shares this shape: the product is a model, the moat is the data, and the data comes from the traffic.

Reading Section 3 with a security lens: agents hold credentials, invoke tools, and initiate payments. An autonomous process with spending authority and a 20K-token context is a counterparty, and adversarial machine counterparties, whether directed by malicious humans (misuse), pursuing learned objectives their operators did not intend (misalignment), or hijacked mid-execution by injected instructions (compromise), will commit fraud, exfiltrate data, and exhaust budgets at machine speed. For adversarial counterparties running open-weight models, no upstream lab can observe or revoke them; the only enforcement point is the layer they transact through. Protecting inference will work the way protecting payments works: models trained on transaction data at scale.

This reduces the acquisition question to a single one: who has the data?

Not the labs. Each frontier lab observes enormous volume, but only across its own models, as one bank observes only its own accounts; one bank’s ledger cannot train Visa’s fraud models — and for open-weight models there is no bank at all. Not the clouds, which observe infrastructure without intent.

5. The Emerging Frontier Alignment Stack

Alignment is a property of a system’s behavior, and the field pursues it in two places. Training-time methods shape what a model intends. A growing inference-time alignment literature enforces intended behavior at deployment without retraining, and the control literature addresses keeping deployed systems safe even when training-time alignment fails.

We use deployment-time alignment for the composite problem this implies: verifying and enforcing intended behavior in deployed agents, in the field, where alignment failures actually manifest. Verification requires monitoring, monitoring is only as good as the behavioral data it trains on, and behavioral data for agents is a network property: the same agent must be observable across every model and provider it touches, because misuse, misalignment, and compromise all present identically at the point of action, as transactions. Security for agents is therefore the deployment-time alignment problem, and deployment-time alignment, like fraud detection before it, is a data problem.

Exactly one such dataset exists. OpenRouter processes 10+ trillion tokens per day across 500+ models from dozens of providers: execution traces, tool-call graphs, spend velocity, routing decisions, and failure modes. It is the largest cross-model corpus of inference transaction data in existence — and for open-weight models, which carry a material and growing share of agentic workloads, it is the only one. The same open checkpoint is served by dozens of independent providers, none of them its author, none accountable for its behavior, none seeing more than their own slice; the routing layer is the only place that behavior aggregates at all. Precision matters here: this is transaction-shaped metadata, not prompt content. Prompt logging is off by default, and the underlying study was conducted on metadata only, with no access to prompt or completion text. Radar does not read the contents of a shopping cart either. Fraud models run on the shape of transactions, and OpenRouter holds more cross-network transaction shape than any entity in the market.

The obvious objection is concentration: this transaction places the only cross-model behavioral corpus in existence inside a single private company, and unique safety-relevant datasets are precisely the kind of asset the field worries about consolidating. However, given the alternatives of fragmentation across dozens of providers, none seeing enough to act, or eventual capture by a frontier lab with directly competing model interests, this outcome is a rare net positive. Stripe is model-neutral: it trains no frontier models, competes with no lab, and its commercial incentive, underwriting trust for everyone who transacts through it, points in the same direction as ecosystem safety.

In other words, Stripe did not buy a router. It bought a strategic frontier AI systems security and alignment asset. We believe combining this asset with Stripe’s existing infrastructure strengthens the ecosystem’s independent alignment capabilities in ways that would be difficult for any single lab to accomplish by itself. As such, this acquisition is net positive for the health of the independent frontier ecosystem.

OpenAI is previewing Private Safety Processing so automated safeguards can identify patterns across related interactions while remaining compatible with zero-data-retention commitments.

As we reported earlier, Meta has been preparing a dedicated Meta AI desktop app for macOS. It's now available in certain regions, and we finally got a chance to test it. Meta officially announced the Mac app on August 19, confirming screen sharing and system-wide dictation as core desktop features.

The app looks very similar to Meta AI on the web and offers largely the same functionality, but desktop integration adds several notable capabilities. Users can attach a specific screen or window directly from the prompt bar and use its contents as context. Voice dictation is also available, including through a system-wide shortcut that can summon a compact Meta AI prompt bar from anywhere on macOS.

The dictation feature is particularly interesting from a model perspective. It does not necessarily indicate that Meta is preparing an entirely new speech-to-text model, as the company already has public speech-recognition technology, including Omnilingual ASR. Still, bringing dictation directly into Meta AI could provide a consumer-facing surface for newer speech technology in the future.

The build we accessed is already labeled version 1.0, suggesting the desktop product has moved well beyond an early prototype. For Meta, bringing Meta AI to macOS is another step toward closing the desktop gap with OpenAI, Anthropic, Google, and other competitors.

There is still considerable room to catch up. Artifacts are already supported, but Meta AI itself lacks the deeply integrated coding workflows available through ChatGPT and Claude. Meta separately launched Muse Code and Muse Spark 1.2 earlier this month, so the missing piece is increasingly integration rather than underlying coding capability. Browser use and computer control remain other major gaps in the consumer desktop app, even as Meta experiments with computer-use capabilities through its developer stack.

Stripe says that the singularity started on January 1. The company believes that being privately held is the best structure for such a consequential moment. Remaining private enables it to fund acquisitions and long-term investments without diluting them. This implies that any IPO remains on indefinite hold.

YouTube is offering millions of dollars to popular channels for exclusivity. The company has yet to finalize any deals with creators. The move is an effort to halt Netflix's pursuit of YouTube's biggest stars. Creators who sign deals with Netflix will be less likely to be featured in YouTube's marketing campaigns, and they will also be excluded from collecting a share of proceeds from some major brand campaigns.

Businesses are increasingly using routers and model serving platforms to access cheaper models. The new router spend is concentrated amongst a segment of businesses that are still increasing their AI spend on closed American models. These businesses are the fastest growing in terms of both revenue growth and business spend. Their increased spend on open-source models tends to happen on the margin without crowding out or replacing American AI models.

Reddit’s share of ChatGPT Search citations fell sharply in mid-August, according to new tracking data from Promptwatch.

The GEO analytics firm observed that reddit.com maintained an average of 3.83% in ChatGPT Search citations from July 18 to August 7. This share then decreased to an average of 0.52% from August 14 to August 17, marking an 86.4% relative decline.

The finding spread quickly across tech and trade outlets, some tying the drop to a change in how ChatGPT generates its background search queries on Aug. 8. However, this explanation doesn’t hold up against Promptwatch’s numbers and complicates any quick call to deprioritize Reddit. Gizmodo reported that OpenAI did not respond to a request for comment on the change at the time of that publication.

What ChatGPT Changed On Aug. 8

Promptwatch connects the decrease in citations to a different change it observed in ChatGPT Search’s fanout queries, which are the background searches ChatGPT performs while crafting an answer.

On Aug. 8, fanout queries using the site: operator, which scopes a search to a single domain, jumped from 0.37% to 16.8% of all fanout queries in a single day, a roughly 46-fold increase. The average number of fanout queries per response also nearly doubled that same day, from about 1.08 to 1.83.

Promptwatch co-founder Klaas Foppen, writing in the site: operator report, said ChatGPT “is no longer just searching the open web and seeing what comes back; it is deliberately going to specific sites to pull information from them.”

The Timing Doesn’t Fully Line Up

ChatGPT started scoping more of its background searches to individual domains on August 8. According to Promptwatch, this share has remained stable at around 16% to 17% of fanout queries since then. Reddit’s citation share didn’t collapse until six days afterward.

Promptwatch’s report on the Reddit decline details two distinct drops. The first occurred on Aug. 8, when the share dropped from the high 3s to the mid 2s, and was followed by a sharper decline on Aug. 14, when Reddit’s share fell below 1%. The six-day interval doesn’t exclude the possibility of a site: operator change influencing the initial dip, but it also indicates that this change alone does not explain the sharper decline on Aug. 14.

Promptwatch doesn’t explain what happened on Aug. 14. The company calls the size of the overall drop provisional and says it can’t yet rule out a data-collection issue on its own end.

A Similar Drop Happened Before

Reddit’s ChatGPT visibility fell sharply once before. In September 2025, several AI-visibility trackers reported reddit.com’s ChatGPT citation share collapsing within a few weeks, and Reddit’s stock price moved along with the coverage.

An explanation at that time suggested that ChatGPT’s behavior was not due to any Reddit-specific change. Around September 10, 2025, Google removed the num=100 search parameter, which many SEO tools and data providers relied on to retrieve up to 100 search results per request.

G2 growth advisor Kevin Indig argued, using hedged language, that the change made it more challenging for outside data providers, like those OpenAI purchases search results from, to access the deeper parts of Google’s results where Reddit threads often appear. He mentioned that it was likely this change, rather than any specific decisions by OpenAI or Reddit, that led to the drop in citations, though he stopped short of saying it was definitely the cause.

Why This Matters

The main theory explaining August’s decline in Reddit citations is that ChatGPT changed which sources are cited. This seems reasonable and worth keeping an eye on, but Promptwatch’s data doesn’t fully confirm this, and a similar drop occurred before, with an explanation one analyst tied to Google, not to OpenAI or Reddit.

Before you make any changes to how you utilize Reddit for AI search optimization, check your own domain’s citation trends. A decrease in one vendor’s tracked average doesn’t necessarily reflect your visibility, and the exact cause of this change is uncertain.

Looking Ahead

Promptwatch says it’s continuing to track whether Reddit’s ChatGPT share recovers, and whether the Aug. 14 break turns out to be a data artifact rather than a lasting change.

Either way, that’s worth tracking, but your decision to use Reddit should come down to whether it makes sense for your brand, not whether it appears in ChatGPT citations.

Apple Accidentally Leaked More Than 10 New Products in macOS Update

Apple on Monday seeded the macOS 26.7 Release Candidate, and the upcoming software update has seemingly leaked more than 10 new Apple products.

macOS 26.7's code contains references to AirPods with infrared cameras for Visual Intelligence, the long-rumored foldable iPhone, an all-new Apple smart home hub, the first-ever MacBook and iPad mini models with OLED displays, an updated Siri Remote for a next-generation Apple TV, and more, according to MacRumors contributor Aaron Perris.

Apple's annual iPhone event takes place in September, so at least some of the new products will likely be unveiled next month.

DataTalks.Club rebuilt its FAQ bot around curated GitHub issues, Slack threads, YouTube transcripts, and explicit evaluation cases. The serverless design keeps retrieval simple enough for AWS Lambda, but puts effort into labels, review batches, and costly false-closure checks. It is a useful reminder that RAG quality starts with maintained data.

Apple Maps Rolls Out Ads, Locates Madison Avenue

Apple is coming for media budgets from advertisers, and in return the iPhone maker is giving them an incentive to start placing ads.

On Friday, businesses in the United States and Canada can begin purchasing media in "Ads on Maps." In March the company announced it would roll out the platform in summer 2026, and that the ad offering would have the ability to upload photos, add promotional messages and set budgets with no minimum media buy.

Since the privacy approach is for all Apple Ads, and the company has multiple advertising platforms, Ads on Maps builds on Apple’s privacy approach for advertisers and its platforms.

Eligible businesses that buy media on Maps via a credit card by October 11, 2026 will receive a 15% monthly credit they can apply to the following month’s spend, up to $1,000 per month, for the first year.

More than one billion relevant searches for businesses take place on Apple Maps monthly, and one in two searches for a business result in a user taking action on the results, according to company statistics.

Gen Z and millennials make up 57% of Apple Maps users, according to a GWI Core Study. The idea is that these ads will give businesses a way to connect with users that engage with Maps in real time as they search for and explore nearby places such as restaurants, banks, and grocery stores. The platform offers custom options such as targeting by location or time and day.

Businesses with five or fewer locations can buy the media directly in Apple Business using automated tools.

Apple will direct businesses with more than five locations, as well as existing Apple Ads advertisers, to the Apple Ads platform.

Once the ad runs, customers can take action from an ad such as calling a business or getting directions to its location.

Advertisers with less experience buying media can tap into a roadmap to start reaching consumers under varied pricing models. APIs will enable advertisers to manage campaigns and pull reports programmatically to measure performance.

Ads on Maps is designed to help businesses reach customers at high-intent moments. Apple will offer tools and budget controls that allow advertisers to start or stop campaigns at any time, which could help attract more ad dollars.

Some business categories are not eligible to run ads on some or all of the placements due to laws, regulations, and policy restrictions.

In July, Apple quietly posted official guidelines and policy terms for businesses looking to buy ad placements inside Apple Maps. Apple's Advertising Services policy was published on July 14, 2026, stating rules for advertising in Apple Maps.

While a long list of prohibitions exists, one unexpected service landed on the list. Apple will prohibit the category of home-services businesses such as plumbing, electrical, locksmith, HVAC, pest control, roofing and general contracting services, among others.

Apple stated in the policy that it will limit Maps Ads to places with a physical address that customers can visit.

Other typical categories are also restricted, including alcohol, dating, contests and sweepstakes, gambling, religion, bail bonds, cryptocurrency ATMs, and medical services.

Apple's Maps app captures about 25% of actively monthly U.S. users in the navigation market, according to one report. Ads on Maps could change this dynamic.

iOS 27 brings several upgrades to Apple Calendar, including natural-language event creation powered by Apple Intelligence, which can automatically fill in event details from a simple text description. The app also gets a refreshed design for viewing and editing events, adds a long-requested Busy/Free status option for iCloud calendars, supports new extra-large widgets for showing more upcoming events and reminders, and benefits from major Siri AI enhancements, including the ability to create events by scanning flyers and understanding personal context and on-screen content.

As AI becomes embedded in products and trusted brands, the biggest risk is that it delivers wrong answers with the same confidence as correct ones, making people more likely to trust it without questioning. Humans often equate confidence with expertise, so AI can borrow credibility from the brands that deploy it, potentially damaging trust when mistakes occur. The key challenge is ensuring meaningful human accountability rather than relying on unchecked automation, especially in situations where accuracy and trust matter most.

Stopping the Icon Sprawl in Large Component Libraries

Maintaining a unified visual language across massive product ecosystems causes major headaches. Shared React component libraries usually turn into dumping grounds when serving multiple autonomous teams. Developers just grab assets from whatever source sits closest at hand.

UI fragmentation quickly follows. Thin line art suddenly sits right next to chunky, filled glyphs.

Solving asset sprawl requires centralized strategy. Building every graphic in-house guarantees uniqueness. It just doesn’t scale well for fast-moving teams. Pre-made platforms like Icons8 offer an alternative repository. You’ll get over 1.4 million assets categorized into strict visual styles. Let’s see how well it fits strict frontend workflows.

The Chaos of Managing Assets Across Five Product Teams

Late Thursday afternoon before major code freezes brings out repository issues. Asset fragmentation usually rears its head right then. During one routine review of a shared React component library, a frontend lead noticed a recurring problem. Payments had just merged heavy, flat vectors for their new checkout flow. Meanwhile, onboarding committed ultra-thin, rounded graphics to that exact same global navigation bar.

Everything looked entirely disjointed.

Teams need single sources of truth. They require enough variety for edge-case features while maintaining strict stylistic rules. Icons8 established a new baseline. Because the platform organizes assets into predefined styles containing over 10,000 graphics each, frontend leads could mandate strict rules. Everyone had to exclusively pull from one Windows 11 Outline pack.

Establishing that rule solved consistency problems immediately. Developers stopped hunting the web for missing database graphics. Instead, they searched our internal collection. Highly specific missing graphics triggered an icon request feature. Submitting requests to the community gets assets into production once they receive eight user votes.

Consolidating the Visual Language

Transitioning a disparate codebase to a unified system takes a clean pipeline. Migrating to Icons8 begins with Collections. Rather than downloading files individually and losing track of versions, lead developers build primary sets directly in the browser.

Typical workflows follow these steps:

• Search libraries for core interface elements and filter by a specific style.
• Drop selected assets directly into designated Collections.
• Apply bulk recoloring using your company brand HEX code.
• Export entire collections as SVG sprite sheets for your React project.

Manual path editing in external software becomes obsolete. New teams spinning up features often need additional graphics. Lead developers just share generated links. Links automatically clone Collections for recipients. Teams can add new assets, apply exact same bulk color profiles, and export updated sprite sheets.

Evaluating Alternative Asset Strategies

Selecting an asset platform means weighing volume against consistency. Relying solely on in-house designers produces perfectly tailored brand aesthetics. Speed is the trade-off. Niche graphics for a settings menu might take three days to arrive as custom vectors.

Open source packs like Feather or Heroicons work wonderfully for standardizing early-stage applications. They’re free, beautifully crafted, and integrate easily into React. Scale becomes their major flaw. Expanding beyond basic user profiles into complex domain logic exposes the lack of vocabulary.

Massive repositories like Noun Project or Flaticon offer comparable volume to Icons8. Finding esoteric medical devices or obscure financial concepts happens instantly there. Matching stroke weights and corner radii isn’t so simple. Independent creators upload distinct sets to those platforms. Assembling cohesive interfaces requires tedious manual adjustments.

Icons8 avoids that trap through strict in-house guidelines across 45 visual styles. Graphics in the iOS 17 Glyph pack perfectly match dimensions of every other asset in that specific pack.

Pushing Updates to the Design System

Handling standard interface elements feels straightforward. Modern applications frequently demand micro-interactions and complex states. Static graphics fall short when product managers request celebratory feedback for completed forms.

Skip the animator queue. Search the platform for animated icons matching your established visual style. You’ll find over 4,500 of these motion graphics ready to go. Downloading them as Lottie JSON files ensures smooth, scalable playback on both web and mobile environments. You won’t bloat your bundle with heavy GIFs. React components can mount these Lottie files directly, giving users buttery smooth animations instantly.

Managing unique interface states poses another frequent challenge. Database logic often requires “save success” and “sync error” states. Developers usually have to open vector software to merge two graphics.

In-browser editors bypass that entirely. Clicking an asset opens a canvas where you can add subicon overlays. Think small checkmarks or warning triangles.

Editors let you resize overlays, adjust base asset padding, and add stroke outlines. Customized assets download directly as SVGs. They match the rest of your library perfectly without requiring external design tools.

Where the Library Falls Short

Massive repositories aren’t universal solutions for every product phase. Teams building highly distinct consumer-facing brands often rely on proprietary illustration styles to stand out. Pre-made libraries mean accepting that your interface shares visual DNA with thousands of other applications.

Pricing structures also create hard boundaries for technical teams. Free tiers mandate attribution and severely restrict file formats. You’re limited to PNG files capped at 100px. Resolutions that low are practically unusable for modern retina displays or responsive web applications. Accessing SVGs, PDF files for print, or larger raster files requires paid subscriptions. Exceptions exist for Popular, Logos, and Characters categories. Those unlock all formats on the free tier.

Technical limitations exist within file exports as well. Platforms output simplified SVGs by default. Great for file size optimization. Frustrating for developers who can’t target specific vector paths with CSS animations.

Workflow Recommendations for Component Libraries

Integrating third-party repositories into strict development environments requires specific habits to prevent friction. Without established workflows, developers quickly revert to downloading loose files because they don’t know the proper procedure.

• Uncheck simplified SVG settings before downloading if your team modifies vector paths in Lunacy.
• Use the Pichon Mac app to drop assets directly into VS Code or Figma.
• Generate uniform filled states for avatar placeholders using the background square tool.
• Save specific brand HEX and RGB values inside the editor color picker to avoid typing errors.

Maintaining consistency across multiple product teams represents an ongoing battle. Giving developers a vast, strictly organized repository prevents rogue design choices when they hit roadblocks. Locking teams into specific visual styles helps. Centralized collections let single frontend leads dictate the visual language of entire application ecosystems. You’ll never even have to open a vector editing program.

Vercel announced a Slack interface for invoking its agent from team conversations.

Spotify is rolling out Running Mode, a feature that generates workout playlists tailored to your running pace, workout type, duration, BPM, and music preferences, with optional audio coaching cues. Initially available to Premium users in select countries, it is now expanding to Android. Spotify is also introducing User Notes (now evolving into Playlist Notes), allowing users to add personal comments to tracks that can be seen in shared playlists, with the feature available to free and Premium users in selected markets.

isolate.video is a video editing app that turns screen recordings into polished product videos with automatic zoom, AI music, and spotlight effects.

Generate stroke data, render beautiful writing animations in React, Svelte, Vue, SolidJS, Astro, Web Components, or vanilla JS.

User experience functions as a business growth strategy rather than a purely aesthetic exercise. Treat AI as an accelerant for research and early-stage design work, not a replacement for human judgment, empathy, and strategic thinking in building great products.

Seiko's annual Power Design Project returned to Japan House London, letting in-house designers experiment freely with unconventional watch elements like dials, hands, and indices.

Japanese illustrator Naruhiko transforms ordinary neighbourhood scenes into quiet, atmospheric oil paintings that find beauty, nostalgia and emotional resonance in everyday parks, streets and overlooked urban details.