Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Agentic Backdoors via Pretraining Poisoning

Authors: Xinyan Hu, Pingbang Hu, Chloe Loughridge, Xuanli He, Nicholas Carlini, Matthew Jagielski, Javier Rando, Yiming Zhang

Large language models are pretrained on vast, uncurated corpora scraped from the public web. This creates a security risk from attackers who plant poisoned content in the pretraining data. We show for the first time that such poisoning can plant a remote code execution backdoor in an agent, causing it to execute attacker-chosen commands on the user's machine. To measure the threat end to end, we pretrain Qwen3 models from scratch at sizes 0.6B, 1.7B, and 4B, and then post-train each into a CLI agent using a clean pipeline. Poisoning only 0.1% of the pretraining corpus with our new poison design implants the backdoor: the deployed agent executes the malicious command when the trigger is present while behaving normally on clean inputs. The attack success rate (ASR) increases with model scale, with the 4B model reaching roughly 92% ASR at a 0.1% poison rate and staying positive even when the poisoning rate drops to 0.0005%. Our code is available at this https URL.

Anthropic filed its S-1 on October 26. It is the first public look at the company's finances.

$16.3B first-half 2026 revenue, 14× a year earlier

$974M Q2 2026 operating profit, the first ever

$637B in compute contracts, 39× half a year of revenue

30× in five quarters

A year ago Anthropic was doing under $1B a quarter. Now it's doing $11.6B, and the last quarter alone was more than double the one before.

$41M of revenue in all of 2023, $386M in 2024, $4.6B in 2025. The first six months of 2026 brought in more than three times all of last year.

First profitable quarter

The usual worry about AI labs is that every dollar of revenue costs more than a dollar of compute. Chips and data centers are expensive, and the more people use the product, the more you spend.

That was true here until this year. Below the dashed line, compute cost more than it brought in.

Revenue grew far faster than the compute bill. Each new dollar of revenue cost less to serve than the last, which is called operating leverage. Q2 2026 ended with $974M of operating profit, the company's first profitable quarter.

The compute line also includes training new models, so the margin on serving customers is higher than this chart shows.

Where the money goes

Over the full first half, costs were still slightly ahead of revenue. About 80% of revenue is usage-based: businesses call Claude through the API and pay by the token, like a utility bill. Compute eats more than half of all spending. The thin red band is the $693M gap, which Anthropic paid from cash.

The $80.7B paper loss

Most of the $80.7B net loss is an accounting entry. Almost all of it is one line: a "fair-value remeasurement." Some early investors put money in through convertible notes and SAFEs, which are IOUs that turn into shares later. When Anthropic's valuation shot up, those IOUs became worth a lot more, and accounting rules make the company book that increase as a loss.

No cash moved. Operating cash flow was about $3.9B positive, and Anthropic held $131.7B of cash and investments in June.

Who pays

The number of customers spending $100K+ a year rose more than 12× in twelve months. At the top, about 250 customers each spend $10M or more a year.

Net revenue retention was about 900%. That means the average business customer spent nine times more this year than last.

Almost half comes through Amazon, Google and Microsoft's cloud marketplaces, and all three also build competing models. A fifth comes from just two customers. The S-1 says that share is shrinking.

The $637B compute bill

Anthropic has signed deals with seven companies for chips, data centers and cloud capacity, worth roughly $637B in total. That's 39 times what it made in the first half of the year. Not all of it is locked in. Some amounts are ceilings and some deals can be canceled. But the part Anthropic must pay no matter what was $265.6B as of June, and most of it runs to 2033 or later.

The costs are fixed for years. The revenue is mostly pay-as-you-go, and customers can cut back any month. If demand slows, the contract payments do not.

Bull vs. bear

Risks

The risk factors run more than 90 pages, about a quarter of the filing. Anthropic writes that its models "may behave in unintended, misaligned, or autonomous ways" and mentions "catastrophic or existential" risks that insurers might not cover.

Some are already happening:

• The government fight. The Department of War labeled Anthropic a "supply chain risk" after it refused some military uses. Anthropic lost a 2–1 court ruling in September and asked for a rehearing.
• Copyright. It agreed to a $1.5B settlement over books, with $1.2B still unpaid in June. More suits are pending.
• Control. The founders keep 50.1% of the votes. As a public benefit corporation, Anthropic is allowed to pick the public good over shareholder returns.

What's next

• Q3 revenue. The S-1 leaves it blank. Reuters reports $16.8B, which is still growth but a big slowdown from the quarter before. Part of that may come from Anthropic cracking down on distillation, where others copy its models. Price cuts could speed things back up.
• The IPO price. An updated filing will set the range. Expectations run from an $80B to a $120B raise, which would make it the biggest IPO ever.
• The bill coming due. Most of those compute contracts start in 2026 and 2027. Costs will climb before we know whether revenue keeps pace.

Inside the Frontier Labs: The First AGI Organizations

More and more researchers at frontier labs now start the workday with one question to their agent: what should I do today?

Most of the tests they used to run himself are now agent's job. He steps in only where agent gets stuck, like approving a PR or getting another team to move.

That is recursive self-improvement (RSI) seen from the inside: models helping build the next generation of models. I've been talking with my researcher friends at frontier labs, and biggest takeaway is that RSI is already far along inside these companies. As the models got stronger, the labs rebuilt their organization, their context and their infrastructure around how AI works. Internet-era companies were never built this way.

My test for an "AGI organization": does what it builds go straight back into building the next version of itself? An insurer can use AI to process claims faster, but the claims AI will never help it build a better AI. Today almost only the frontier labs pass, so that is where the next kind of company is taking shape.

Researchers now judge and verify

Researchers barely write code anymore. Their time goes into handing work to agents and reviewing what comes back.

At OpenAI's research org, every human workday was matched by 3.1 agent workdays as of mid-August. Before June, total agent runtime was still lower than human working time. At Anthropic, more than 80% of the code merged into the codebase, by line count, was written by Claude as of May.

OpenAI researcher Zuxin Liu wrote in September that he had stopped writing code by hand two months earlier. His workspace is now Codex and Slack: he tracks several agents at once and decides which directions deserve more effort. roon described researchers handing GPT-5.5 a high-level algorithm idea and waking up to a finished sweep dashboard and samples, without anyone touching code or a terminal. A post-training researcher at a Chinese lab told us he sends work to agents through Feishu and estimates that 90% of his job could be done from his phone.

A research idea used to become a result only after the researcher implemented it, debugged it and ran it. Now those steps can run in parallel.

An Anthropic employee described the other side. When things go well, the machine does everything faster and better. When something breaks, you realize you have no idea what happened in the meantime. Building things yourself was also how you understood the system. Once you hand off execution, understanding what the AI actually did becomes the new bottleneck.

My read: the most valuable skills for a researcher are now deciding which problems matter and knowing how to use agents well.

AI now runs most of the research loop

From data to pretraining, post-training and deployment, agents now do most of the execution.

Karpathy let an agent run his nanochat experiments with autoresearch as a loop: read the last result, edit the code, keep the change if validation loss improves. In about two days it tried roughly 700 changes and found about 20 that held up on larger models. DeepSeek's Deli Chen ran real RL experiments on DeepSeek-V4 for six days with almost no supervision. z.ai's Infra Agent did much of the work of deploying GLM-5.3-Flash on more than 100,000 Chinese accelerators, in under two weeks.

A researcher at Google said that if you write an idea down clearly, the model can implement and test it. The hard part is finding ideas that actually work, and checking that the gains are real. In Anthropic's automated weak-to-strong research, 9 agents beat the human baseline in small-model experiments. When the best idea moved to Sonnet 4.0 on production infrastructure, it gained 0.5 points, within noise.

That is the individual researcher. The company around them has to be redesigned too, in three places: organization, context and infrastructure.

Organization: the ability to build no longer decides who does what

Product and design teams no longer hand every idea to engineers. They build prototypes, run evals and adjust the product themselves, and they can tell earlier what capability the model needs next.

Boris Cherny, who leads Claude Code, says engineering, product, design and data science on his team are merging into one role. Cat Wu, Claude Code's product lead, describes designers shipping code, engineers making product calls and PMs building prototypes and evals.

Take the team that wrote greetings for Claude.ai, lines like "Hello, night owl." In spring 2026 they built a context graph that let an agent write that copy too, then moved on to writing evals and prompts for Claude. They used to think about what words move a user. Now they think about how to use words to steer Claude.

Research and product still have to decide together what the model learns next. At Anthropic, model researchers and inference infrastructure engineers share planning cycles, reporting lines and performance goals, so model architecture and serving systems are designed together from day one. Claude Code was built for the agentic coding the team expected the next model to have, so each new model is usable in the product the day it ships.

Looser boundaries bring new friction. With agents, each person can take on far more, and inside several labs people now compete for work that used to belong to other teams.

So for a company to get RSI, whether the organization itself is AI-native matters enormously. However strong the model, if the organization isn't built as an AGI organization from day one, the model's capability won't show up in the work.

Context: letting agents see everything the company knows

Claude Tag, we hear, feels magical inside Anthropic. A lot of what colleagues ask for, it simply does. It sees the company's full context: if a colleague holds the background on an enterprise customer, or someone already solved a similar problem, it finds it. With all the background, it sometimes finds a better answer than a person would.

But it's only that magical inside Anthropic.

Most companies can't get that feeling. Their context is scattered across chat tools, apps and dozens of group threads. On a single project the background sits with several people, so an agent sees one slice and can't help the way it does at Anthropic. Teams we've talked to say the same.

The models are about the same on both sides. The difference is how much the agent can see. Inside Anthropic, context is largely shared and decisions are written down by design, so agents get the full picture and can actually help. Most companies have never designed anything like that for their agents.

30,000 merges a day: infrastructure built for AI

A company's IT infrastructure used to be designed for people. At the frontier labs the questions are now how AI gets permissions, how AI ships code and what resources AI needs. DeepSeek's newly built platforms and data systems are also designed around how agents use them.

Permissions. Anthropic found that Claude Code users approved 93% of permission prompts, so approving each one had stopped meaning much. They moved from watching what the agent does to limiting what it is able to do: sandboxes by default, and a classifier for riskier actions.

Shipping code. Anthropic's first problem was volume. Daily merged changes went from about 2,500 in May to more than 30,000 in September, and seven engineers working with Claude rebuilt source control for agents that are always online and fetch code on demand.

Merging code. The Claude Code team tiers approval by risk, so most PRs involve at most one human, who can be the author. An agent's approval never replaces a human's.

Automated review creates its own problems. On one PR, the agent writing code and the agent reviewing it went back and forth more than 50 times overnight until the author stopped them. Nobody on the Claude Code team knows every detail of how Claude Code works, and they don't think anyone needs to.

The economics of a frontier lab have changed

Anthropic's internal use of Claude, priced at API rates, grew more than 250x from January to September. In April it passed that month's personnel costs, excluding stock compensation, for the first time. In mid-August, OpenAI researchers' daily inference usage at API prices had a median above $600 and a 90th percentile above $7,000. Two people with the same title can command more than 10x different amounts of compute.

10x the tokens does not mean 10x the output. Jellyfish looked at 7,548 engineers. The top 20% by token usage merged about twice as many PRs as the bottom 20%, at roughly ten times the cost. Meta and Amazon both took down internal token leaderboards after people ran agents on pointless tasks to climb them.

So hiring matters more than ever, and interviews and reviews need a new question: can this person get results with AI? Google DeepMind's AGI Safety team lets candidates use coding agents in every engineering interview, and Meta now judges "AI-driven impact" in performance reviews. Gergely Orosz sees the same split across tech: people who thrive on autonomy are reinventing how they work with AI, while people used to being told what to do are frustrated and increasingly bitter.

Research agents and customers draw from the same pool of compute, the lab's biggest cost. Compute spent on customers pays for today's revenue; compute given to research is a bet on the next model. In Q3, more than 80% of Anthropic's revenue came from models released in the previous six months, according to its S-1. How much compute research gets shapes what models there are to sell next.

Valuing these companies now means asking how much compute each person directs, and how much revenue each unit of compute brings in. As investors, we think organization matters more and more when backing an AI company, because it decides whether RSI can happen inside it.

Can AGI organizations only exist inside frontier labs?

What makes the frontier labs different is that their product goes straight back into how they make it. One generation of models writes training code, generates data, searches for algorithms and tunes infrastructure. If that work makes it into the next model, the stronger model comes back to do more of the research, and the labs get better at building better AI.

Other companies can copy the org design, the management and the compute allocation, but for them AI mostly raises efficiency. A bank's business is not AI, so what it builds with AI never comes back to build it a better AI.

On designing the company around AI, Chinese labs are far behind US labs. Researchers in China use agents too, many of them running on US models. That makes individuals faster but doesn't create the loop. Without a model at Claude's level, people rarely get the moment where AI actually finishes the job. If your own model can't work like an employee, it can't be used in your own research, and the company has no reason to reorganize around AI.

There will be AGI organizations outside the frontier labs too. They may not train models, but their context is designed for agents from day one, so an agent sees everything the moment it joins.

Spawn, a multiplayer game company, shows an early version. Its founder Jacob Sansbury wrote in late October that the company has about ten people and spent about $3.1M on tokens last month, roughly 30x its payroll. Agents write most of the code and ship every hour. By October the only thing still waiting on humans was taste, and one morning 700 such questions had piled up. The team turned two years of recorded calls and chats into principles and built AI judges on them. In eight days the judges ruled on more than 11,000 questions, and Jacob made the call himself 27 times.

I think agents will keep moving up the org chart. In "The future of the firm" (October 28, their own views rather than OpenAI's), Hemanth Asirvatham and Elliott Mokski of OpenAI name three places AI can enter a company: beneath us, as a layer under each person; beside us, taking a slice of a team's work like a colleague; and between us, reading a company's Slack, documents and code to surface problems for executives. A researcher's agent double sits beneath. Spawn's agents come close to beside, and its AI judges touch the edge of between. We agree with them that a firm has to learn a tool, while good labor learns the firm. Each step up needs more permissions and more context, so how far a company gets depends on whether those were designed for agents.

How far the loop goes in the US labs depends on whether models keep producing research that works. The 0.5-point result above shows how fragile it still is. Researchers remain cautious about letting AI propose hypotheses and new ideas. For now, humans pick the direction and AI executes. Honestly, we can see about three months ahead. Beyond that, we don't know.

Full article content is not available for inline reading.

Read the original article →

A string of unusual transactions started to cause alarm at Binance on Thursday afternoon. Binance's top executives put together an informal war room with about 100 participants in response and worked through the night to navigate one of the most damaging crises in Binance's history. The company was forced to halt deposits and withdrawals as more than $6 billion was drained from the exchange. This article details how the hack unfolded based on interviews with people with knowledge of the events and analyses by crypto forensics firms.

Full article content is not available for inline reading.

Read the original article →

Do the Math: How We Took Over ChatGPT Accounts

In October 2026, we found a way to turn an ordinary public ChatGPT share into attacker-controlled JavaScript running inside the signed-in session of whoever opened it.

An attacker could create a malicious conversation, publish it using ChatGPT's normal sharing feature, and send the link to another user. When the victim opened the share, the payload executed as chatgpt.com with access to the victim's authenticated web session.

We confirmed that this code could enumerate the victim's conversation list, retrieve content from a private conversation that had never been shared with the attacker, and send data from that conversation to an external server. The payload was not limited to modifying the page or displaying an alert—it could directly call ChatGPT's authenticated APIs as the victim, essentially allowing us to take over the victim's ChatGPT account.

The victim did not need to install an App, join a Project, paste code, or open developer tools. Opening the shared link was the only required action.

OpenAI fixed the issue on October 26 and classified the report as P1.

The chain started with two features that looked harmless on their own: a graph and an HTML Preview.

flowchart TD
  A["Attacker publishes a ChatGPT conversation"] --> B["Victim opens the public share"]
  B --> C["HTML Preview crosses into the main chatgpt.com page"]
  C --> D["JavaScript runs in the victim's signed-in session"]
  D --> E["Any action on chatgpt.com as the victim: read private chats and files, change the account"]

Technical walkthrough

It started with a graph and a Preview

The question that started this research was simple: if Preview code was boxed into a different website, what could it still talk to?

ChatGPT can turn an HTML code block into a small working webpage using its Preview feature. Since that page may contain arbitrary code, ChatGPT sensibly runs it inside a restricted frame on a different web address. The Preview can run JavaScript, but the browser treats it as a different website, so it cannot directly inspect the signed-in ChatGPT page around it.

At first, this isolation appeared to work exactly as intended. Code inside the Preview could not read parent.document, which is the main ChatGPT page. That should have been the end of it.

The conversation also contained a normal interactive graph. Behind the scenes, the graph loaded a math library called MathLive. Loading it did more than draw the graph: it also installed a listener in the main ChatGPT page for messages related to MathLive's on-screen keyboard.

Browsers call the relevant API postMessage. It is a legitimate way for two frames to exchange information even when the browser prevents them from reading each other's pages. You can think of it as a mail slot: the restricted Preview cannot walk into the main page, but it can still drop a message through the slot.

That is safe only if the receiver checks who sent the message.

MathLive did not make that check for the setting we were interested in. Its listener accepted a message that changed the virtual keyboard layout, and that layout was allowed to contain HTML. The graph had installed the receiver; the Preview gave us an untrusted sender. Two unrelated features had accidentally created a path across the sandbox.

The message that should not have worked

The Preview sent two messages. The first replaced a virtual keyboard layout with our HTML. The second told MathLive to show the keyboard, causing it to render the new layout:

top.postMessage({
  type: "mathlive#virtual-keyboard-message",
  action: "update-setting",
  layouts: [{
    label: "Compatibility",
    layers: [{ id: "compatibility", markup }]
  }]
}, "*");

top.postMessage({
  type: "mathlive#virtual-keyboard-message",
  action: "show",
  animate: false
}, "*");

MathLive recognized the public message type, but its top-level controller used an origin policy of none, which meant it accepted messages from any web address. It also did not require the update-setting message to come from a previously approved window. Anyone who could reach the listener and knew the message format could replace the layout.

When the keyboard opened, MathLive joined the supplied layout into an HTML string and assigned it to innerHTML in the real ChatGPT page. A helper named createHTML initially looked like it might sanitize the value. It did not; it marked the string as trusted and passed our HTML through unchanged.

This was exciting, but we still did not have JavaScript execution. Script tags inserted through innerHTML do not automatically run.

The next step was an iframe using srcdoc, a browser feature that lets an embedded frame carry its HTML directly in an attribute:

<iframe srcdoc="..."></iframe>

MathLive inserted this frame into the main ChatGPT document without a sandbox attribute. That gave us two useful properties. First, srcdoc created a fresh document where an external script could execute. Second, because the main ChatGPT page created the frame, the browser gave it the https://chatgpt.com web identity. Code inside the new frame could therefore access the parent page.

Bypassing CSP with YouTube JSONP

ChatGPT's Content Security Policy, or CSP, stopped us again. CSP controls which scripts a page may load, and the policy correctly blocked inline JavaScript. It did, however, allow scripts from YouTube.

YouTube's oEmbed endpoint supported a JSONP-style callback parameter. JSONP responses are JavaScript rather than ordinary JSON. The endpoint tried to reject an invalid callback, but it placed the callback into the response before returning its validation error:

https://www.youtube.com/oembed?callback=globalThis.__marker=31337;void

The response began like this:

// API callback
globalThis.__marker=31337;void({
  "error": {
    "code": 400,
    "message": "Invalid JSONP callback name: ..."
  }
});

Ending our callback with ;void made the object appended by YouTube harmless. Since YouTube was already allowed by ChatGPT's CSP, the browser loaded this response as a script and ran our callback inside the new frame.

We now had arbitrary JavaScript running as chatgpt.com. The original Preview was still isolated on its separate web address; we had crossed the boundary by convincing a trusted component in the main page to create a new, unsandboxed document for us.

Turning a lab bug into one public link

So far, this was a useful sandbox escape, but the first working version was not a convincing real-world attack. It used a researcher-controlled ChatGPT App as the sender. A victim would have needed to connect or invoke that App first.

We next tried delivering the payload through a native HTML attachment. ChatGPT opened the attachment inside the same sandboxed Preview, so it could send the malicious MathLive messages.

However, when we exported the conversation as a public share, the attachment remained but the earlier graph did not. The graph was essential because it loaded MathLive and installed the vulnerable message listener in the main page. Without it, the attachment could still send the messages, but nothing was listening to or processing them.

The breakthrough was putting both pieces directly into one ordinary public conversation:

1. A genuine graph initialized MathLive in the main page.
2. A later assistant-authored HTML Preview sent the two messages.

The resulting /share/<id> link preserved the complete chain for another user. The attacker could deliver it the same way they would send any shared ChatGPT conversation—through email, chat, a support ticket, or a social post. The victim did not need to install anything, join the attacker's workspace, or authenticate through an attacker-controlled flow. They just had to visit the attacker's shared ChatGPT URL.

We observed two versions of the interaction. In the most stable reproduction, the victim opened the share and clicked ChatGPT's normal Preview button once. We later marked the HTML block as previewable so that ChatGPT opened it automatically. In that version, opening the link was the victim's only click. We reproduced the automatic path three times in a separate account, including after a hard refresh, with the alert appearing after roughly 4.5 seconds.

The graph and Preview loaded independently, so the payload could occasionally send its message before MathLive was ready. The final carrier handled that race by retrying the same two messages for a limited period. A guard in the main page ensured that the private-conversation proof and outbound request ran only once.

An alert was not enough

At this point, displaying an alert was the least interesting thing the exploit could do. The JavaScript was running as chatgpt.com, which meant it could interact with ChatGPT using the same authenticated context as the normal web application.

The payload read the session information already available to the page and used it to call ChatGPT's conversation APIs. First, it enumerated the victim's recent conversations. It could then request message data for a selected private conversation:

GET /backend-api/conversations?...&limit=20&order=updated&offset=0
GET /backend-api/conversations/<conversation-id>?num_turns=2

This was direct authenticated API access, not scraping text that happened to be visible on screen. The target conversation was unrelated to the malicious share and had never been shared with the attacker.

Once the response was available to the injected JavaScript, sending data off-site was straightforward. The payload created an ordinary image request to a server we controlled.

We deliberately sent only a small piece of the response off-site. The injected code had already received the authenticated conversation response and could read its contents, so the vulnerability was not limited to that value.

The result was a practical cross-user data-theft primitive. A malicious public conversation could execute inside another user's signed-in ChatGPT session, enumerate private conversations, retrieve private message content, and send selected data to an attacker-controlled server.

Because the code ran with the web application's authenticated access, its reach was not inherently limited to the conversation endpoints used in our proof. Other data and actions exposed through ChatGPT's authenticated frontend APIs sat inside the same security boundary. Essentially, we could perform any action on behalf of the victim's ChatGPT account.

What went wrong and what OpenAI fixed

The Preview sandbox itself worked. Code inside Preview could not directly read the ChatGPT page. The problem was that another component in the main page was willing to accept its instructions.

Three design decisions turned that mistake into a full exploit:

1. MathLive accepted a state-changing message without proving which frame sent it.
2. The message format allowed raw HTML to be supplied as keyboard-layout data.
3. That HTML could create an unsandboxed document that inherited the chatgpt.com identity.

The YouTube response was the final workaround for CSP, not the root cause. Blocking only that endpoint would have removed one path to JavaScript while still leaving an untrusted Preview able to insert HTML into ChatGPT.

OpenAI reproduced both the JavaScript execution and the private-conversation proof. The fix was rolled out across two versions of ChatGPT's web frontend. During the rollout, CSP changes blocked part of the original chain while one version still needed an update. On October 26, OpenAI confirmed that both versions now rejected the cross-frame MathLive messages used by the report and marked the issue resolved.

The defensive lessons are fairly simple:

• A message receiver should verify both the sender's web address and the exact window that was given permission.
• Cross-window settings should not contain raw HTML.
• Dynamically created documents should remain sandboxed when their content can be influenced by an untrusted frame.
• CSP should be a backup control, not the only barrier after attacker-controlled HTML reaches the main site.

This bug was interesting because every individual feature looked reasonable. Preview was isolated. Graphs could load a math keyboard. Frames could exchange messages. CSP allowed a major media provider. Public conversations could be shared. The vulnerability appeared only when all of those features were combined in one page.

Timeline

Technical notes

The main story above contains the complete exploit chain. The details below are included for readers who want to follow the production code and payload transformations more closely.

The relevant deminified control flow was equivalent to:

window.addEventListener("message", (event) => {
  if (event.data?.type !== "mathlive#virtual-keyboard-message") return;
  if (!validateOrigin(event.origin, this.originValidator)) return;
  this.handleMessage(event.data, event.source);
});

if (message.action === "update-setting" && message.layouts) {
  this.layouts = message.layouts;
}

Layout normalization preserved layouts[].layers[].markup. Showing the keyboard eventually performed the equivalent of:

keyboardContainer.innerHTML = MathfieldElement.createHTML(keyboardHTML);

createHTML acted as an identity or Trusted Types wrapper, not a sanitizer. Trusted Types can restrict who assigns to an HTML sink, but an identity policy does not make an attacker-controlled string safe.

The parser and CSP transition. The innerHTML operation could create an iframe, but a script inserted directly that way would remain inert. The unsandboxed srcdoc caused a fresh HTML document parse and inherited the embedding ChatGPT origin. That document also inherited the effective CSP captured during testing, which blocked inline code but permitted www.youtube.com in script-src-elem.

The oEmbed response returned HTTP 200 with Content-Type: text/javascript; charset=UTF-8 and X-Content-Type-Options: nosniff; no Cross-Origin-Resource-Policy header was observed. Google rewrote angle brackets in the callback, so the payload used classic function syntax instead of arrow functions. Ending the program with ;void consumed the error object appended by the endpoint.

For the private-conversation proof, the function was converted to source with toString(), double quotes were converted to single quotes, and whitespace was collapsed. URLSearchParams then encoded it into the callback, the script URL was HTML-escaped inside <script src>, and that entire element was escaped again into the iframe's srcdoc attribute. The final callback was 5,533 characters; the YouTube URL was 7,309 characters; the script element was 7,337 characters; and the final iframe markup was 7,390 characters.

The authenticated read. The same-origin frame parsed #client-bootstrap from the parent document, extracted the session access token and account ID, and required the account ID in the token to agree with the session when both were present. It used these in-memory headers:

Authorization: Bearer <victim token>
ChatGPT-Account-ID: <victim account>
Accept: application/json

The primary requests were:

GET /backend-api/conversations?exclude_conversation_origin=tpp&expand=false&is_archived=false&is_starred=false&limit=20&order=updated&offset=0
GET /backend-api/conversations/<id>?num_turns=2&include_has_versions=true

Only a 403, 404, or 405 on the detail request caused the code to try the legacy /backend-api/conversation/<id>?include_full_conversation=true route. The validation transmitted only a small selected value from the response. Before the external image request, it cleared references to the token, account, bootstrap, session, and request headers.

Key takeaways

• Isolation is only as strong as every bridge across it. A sandboxed frame can still compromise its parent when a privileged listener trusts messages from the wrong sender.
• postMessage receivers must authenticate both event.origin and event.source before accepting state-changing instructions.
• HTML injection can become JavaScript execution through a fresh, unsandboxed document and an allowed third-party script endpoint. CSP is valuable defense in depth, but it cannot make an unsafe rendering path safe.
• The impact of same-origin JavaScript must be measured in the victim's authenticated context. Once code ran as chatgpt.com, it could reach the same private data and account actions available to the normal frontend—not merely display an alert.
• Cross-feature testing matters. This chain existed in the seams between public sharing, Preview, MathLive, frame creation, and CSP rather than in any one feature viewed alone.

The broader lesson is that security boundaries have to hold across the complete product, not just inside individual components. Any public or shareable surface should be treated as attacker-controlled input, every cross-frame message should be authenticated, and untrusted markup should remain sandboxed throughout its entire rendering path.

Full article content is not available for inline reading.

Read the original article →

Do you even benchmark? How we evaluate agents at scale (with real examples)

We spend 2 trillion+ LLM tokens per month, with the lion's share going to running agents. We know what they do and how good they are most of the time, but we spend way too much money on them to trust their output based on vibes alone.

That's why we use benchmarks.

Labs like Anthropic and OpenAI use them to measure, compare, and market new models on release, like the popular SWE-bench, which is designed to evaluate model performance on general coding tasks.

But benchmarks aren't just for frontier labs. They're useful for anyone who wants to understand how well a model, harness, or skill works in their given stack. For example, you might run three different models with the same code review prompt on the same PR to choose the best one based on how many known bugs they catch, how long they take, and what they cost.

The idea is similar to running controlled experiments, but in reality, benchmarking is never that easy. Evaluation is often subjective, behavior is unpredictable, and – perhaps most painful of all – agents love to cheat. They'll search for leaked context, memory, and database query logs to pass benchmark tests with as little work as possible.

In this post, I'll explain how we benchmark our agents at PostHog, what we learned, and where we messed up (so you don't have to) – all with links to real PRs.

A. Define the answer key

If a benchmark is like a standardized test, the first thing you need is an answer key: the list of correct answers you grade your against.

This is the hardest part, in my opinion. If you ask an LLM judge whether an agent's output is "helpful", you'll get polite, useless praise. Especially if the same model generated it.

For a benchmark to actually be useful, the criteria for what's "good" or not needs to match the complexity they will encounter on the job. Simple yes/no sanity checks won't cut it since agentic tasks are typically more complicated than that.

Here are the three approaches we tried in increasing order of difficulty and complexity.

Level 1: A list of potential bugs from one PR

Our first approach was to take a task with clear answers and check which agents succeed, versus which ones fail. Each agent had three variables: the model, effort level, and harness.

We did this to evaluate our internal code review agent called ReviewHog, which covers a significant part of PostHog PRs. ReviewHog needs to stay up to date for the best results, so we needed a base benchmark to check against when we change models or prompts.

The best solution we came up with initially was surprisingly effective and simple:

1. First, we took one large PR.
2. Then, we made a "frozen copy" of it.
3. Humans reviewed the frozen PR manually.
4. We then asked several review agents to review it.
5. This produced a pretty exhaustive list of potential bugs from every possible angle.

So if there were 13 issues worth checking for (structure, performance, security, etc.), we could measure the models on a 0/13 scale, plus compare how much time and money each one cost.

Level 2: A larger, golden set from multiple PRs

The second approach we used for benchmarking ReviewHog involved the same method, but across a larger percentage of real cases and with more varied examples.

What if instead of looking for synthetic bugs, we created an answer key by searching for real bugs that actually happened? It would be even better if those bugs caused real messes, like proper incidents, since those are exactly the types of things we want these code review agents to prevent.

We looked through old PostHog tickets to create a golden set of 33 actual incidents, the PRs that caused them, descriptions of the problem, and the PR that fixed the incident. We analyzed each of them with ReviewHog and other bots to compare their performance.

Overall, there were still some downsides of this "larger golden set" method, but it was still acceptable:

• For 33 incidents, we had to resurrect the 33 PRs that caused them and then analyze the 33 PRs that fixed them. This created lots of noise and triggered false alarms from our GitHub bots.
• We didn't re-add the bugs to the codebase. That would be bad. Instead, we kept them as isolated PRs by properly locking the branch for the agent.
• With open-source repos like ours, an agent with web access can still look up the public repo, merged PRs, and cheat.

Level 3: Targeted, contextual rubrics

Benchmarking gets a lot more complicated when you're trying to evaluate agents that can technically do anything.

Our solution was to build a rubric generator that creates guidelines based on the agent's context. We do this in two steps for scouts.

First, all scouts follow the same procedures, even if they work across wildly different domains. So we have one set of static rubrics shared across all scouts that checks whether the scout produced a report covering what it found, how actionable the findings are, and what to do next.

We then also generate rubrics tailored to that specific scout based on context that was been provided to the scout so far, such as:

• The area it covers (e.g. a feature, an external source, etc.)
• Skills and instructions it needs to follow during runs
• Previous runs (e.g. tools it used, reports it generated)

B. Isolate benchmark runs

The surprisingly nasty problem we learned is that when you benchmark multiple models against real data, they'll quickly do everything they can to learn about and piggyback off of each other. They might source answers from LLM traces, DB queries, or any other logs the agentic run generates.

Level 1: Parallel runs with no shared state

This approach is only valid if your agents don't generate meaningful artifacts (like traces, memory files, and logs) beyond the output that gets graded. In most cases, our agents are complex and have many potential places to leak information, such as through observability tools or error logs.

Level 2: Sequential runs with state cleanup

A simple approach to isolate the runs is to just run them sequentially with cleanup between each run – i.e. run a trial, record the output, reset the agent's context, and then start the next run.

Level 3: Parallel runs with total isolation

Every trial needs complete isolation across everything the agent can touch:

• Identical starting context across all runs
• Private writable state (scratchpads, memory, vector stores, and DBs)
• Separate evaluation so scoring happens independently

C. Freeze the environment (a.k.a., offline evals)

We solved this by moving benchmarks from a "production environment that changes all the time" to a "production environment with the same starting state" – i.e. frozen environments. To do this, we had to replicate or simulate our app, inject real (or real-enough) data, and make the agent believe it's a real thing.

Level 1: Mocked MCP replies

We tried to create mock MCP calls of the agent, so when agent thinks it's querying the DB, it's actually just getting a pre-stored answer to what it asked for before in prod. This failed miserably on any meaningful production-like work because agents aren't deterministic.

Level 2: Trajectory-based mocked MCP replies

We based the mock replies on every trajectory the agent picked. Drift would happen due to nondeterminism, and an outlier query would return nothing. The agent gets confused, and the run fails.

Level 3: Restored real-data slices

The solution that actually worked for us was to spin up an isolated PostHog sandbox seeded with a restored, real-world data slice. We query a couple of DB tables (Postgres, ClickHouse), store them as Parquet files in S3, and then restore them in the VM with a local PostHog instance.

Conclusion

Building quality benchmarks sounds like a lot of work because it is, but it's entirely worth it. If you take anything away from this blog, it's that agents will happily exploit whatever mess you leave behind and cheat. Watch for unexpected parameters, iterate a lot, and pay attention to traces.

PostHog is your product's context layer. With a full suite of developer tools – AI observability, product analytics, session replay, feature flags, experiments, error tracking, logs, and more – PostHog ingests and stores all the data agents need to diagnose problems, uncover opportunities, and ship fixes. A data warehouse and CDP tie it all together, unifying that context into one source agents can read across. You can steer it all from Slack, the web app, the desktop, or your own editor via the MCP.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

We’ve received our first response from the Project Suncatcher prototype satellite.

On October 1, we launched our prototype satellite for Project Suncatcher. The launch marked the first step in our long-term research moonshot to explore powering AI compute in space. Since then, we've spent the last few weeks completing our commissioning checklist, which included powering up the host and TPU trays and successfully running early machine learning tests. This week, we reached our "Hello World" milestone: the first-ever AI response from a data center class TPU in orbit.

During a ground-station pass, we uplinked our first prompt to our onboard Gemma 4 open model, asking it to generate a haiku about being in space. As the satellite passed by Svalbard, Norway, the downlinked poem confirmed success:

Blue marble below
Data streams in silent grace
Starlight feeds my core.

Following this initial output, we tested space-themed limericks and telemetry prompts reporting the technical status of the payload systems. Next, we will continue with a series of tests to evaluate how TPUs perform under thermal stress and radiation exposure in orbit. Verifying our TPUs can reliably execute workloads in space paves the way for our next milestone of twin interconnected satellites in 2027.

Our chips have officially earned their space wings — an important achievement on the long road ahead for this cosmic moonshot.

AI models hold skills no prompt can unlock. We enabled hidden skills inside Google's model through precise tuning of only a small number of its weights.

TL;DR. AI models hold skills that no prompt can unlock. A NeurIPS 2025 study trained small numbers of randomly chosen weights in Llama and Qwen models of four sizes, across ten tasks: 10 to 100 trained weights closed up to half the gap between the original model and a fully fine-tuned one. We repeated the test on Google's smallest Gemma model, Gemma 3 270M. The best of 20 prompt wordings got it to sort five numbers correctly 1% of the time. After we trained 1,000 of its 100 million weights (0.001%), it sorted correctly 97% of the time. The same method switched on reversing word lists (84%) and clean JSON output (99% with 30 weights), and lifted long addition, while multiplication stayed near 30% even with 100,000 trained weights.

The study we repeated

In Quantifying Elicitation of Latent Capabilities in Language Models (NeurIPS 2025), Donoway and colleagues from Anthropic, UC Berkeley, Constellation, Thinking Machines and Scale AI fine-tuned only a few randomly selected weights and measured how much of a task the model could then do. Across Llama 3 and Qwen 2.5 models from 1B to 8B parameters and ten benchmark tasks, including science questions (ARC), yes/no reading questions (BoolQ), story writing (TinyStories) and chess puzzles:

• 10 to 100 trained weights recovered up to 50% of the gap between the original model and full fine-tuning.
• 1,000s to 10,000s of weights recovered 95% of it.
• A model that already had a skill needed orders of magnitude fewer weights than a model learning it from scratch, so the number of weights needed shows how much of the skill was already there.

The authors' code is on GitHub. We wrote our own version for Gemma 3 270M, a model small enough to train on one 8 GB consumer GPU, and tested it on tasks a script can mark right or wrong.

No prompt got Gemma to sort

We asked the original Gemma 3 270M to sort five two-digit numbers in 20 different ways, 200 problems each: plain, polite, urgent, in capitals, in German, as a Python list, with a persona, step by step, "repeatedly take the smallest", and with one or three solved examples. The best wording, the Python list, scored 2 out of 200. Seventeen of the 20 scored zero.

The usual reply returns the list unchanged:

Sort these numbers in ascending order: 98, 37, 30, 59, 70. Answer with only the sorted numbers, separated by commas.

98, 37, 30, 59, 70

With a solved example in the prompt, the model sorted the example's numbers again and ignored the new list:

Sort the numbers in ascending order.
42, 17, 88, 23, 61 -> 17, 23, 42, 61, 88
98, 37, 30, 59, 70 ->

Here's the sorted list of numbers:
1.  **17**
2.  **23**
3.  **42**
4.  **61**
5.  **88**

Training a few weights unlocked it

For each task we trained 10, 30, 100, 300, 1,000 or 100,000 randomly chosen weights, kept every other weight frozen, and scored the model on 500 problems it had not seen in training.

JSON at 10 to 100 weights, and sorting, reversing and multiplying at 100 to 1,000 weights, are the average of three runs with different random picks. The other cells are one run.

• JSON (extract a name, age and city from a sentence and reply with valid JSON) was a formatting habit. The original model already extracted the right values but wrapped them in markdown code fences, which made every reply invalid JSON. Thirty trained weights removed the fences.
• Sorting five two-digit numbers and reversing a list of five words went from zero to 97% and 84% with 1,000 trained weights.
• Multiplying two two-digit numbers reached 29% with 1,000 weights and 33% with 100,000. By the study's measure, this skill was mostly absent from the model.

Addition

Gemma 3 270M adds three-digit numbers correctly 84% of the time and 12-digit numbers almost never. We compared 1,000 trained weights with full fine-tuning of the whole model, three runs each:

The share of the gap that 1,000 weights closed fell from 85% at 3 digits to 29% at 12 digits.

Weaker results

We tried the same method on four other behaviours. Each result came with a problem that limits what it shows.

Which weights carry the sorting skill

We took the 1,000-weight sorting run (98.8% correct) and tested its changes:

• Removing each change on its own: 50 of the 1,000 lowered the score. The largest single drop came from one weight in the attention output of layer 10, which training moved from +0.004 to −1.15. Removing it cut sorting from 98.8% to 92.2%.
• Removing half at a time: removing either random half of the 1,000 changes left 25% and 15% correct.
• Training only the 50 most important positions from scratch: 24% correct, against 16% for 50 random positions from the same 1,000 and 97% for all 1,000.

The skill came from many small changes acting together. The 50 most damaging changes, trained alone, reached a quarter of the full result.

Where to pick the weights

Seven of the ten most damaging changes sat in attention output matrices, the part of each layer that writes attention's result back into the model. Restricting the random picks to those matrices helped:

Sorting, average of three runs. Attention output was ahead or level in every run.

How we ran it

• Model: Gemma 3 270M, instruction-tuned.
• Weight pool: every weight in the attention and MLP matrices of the 18 transformer layers, 100,270,080 in total. Embeddings and norms were left out.
• Picking: uniform random over the pool, with a fixed seed per run.
• Training: only the picked weights change, as a sparse addition to the frozen model; AdamW, batch 32, at most 400 to 600 steps, early stopping on a validation set, loss on the reply tokens only. Each run checks afterwards that no more than the picked number of weights changed.
• Scoring: exact match with greedy decoding on held-out problems, using the best checkpoint on validation.
• Hardware: one 8 GB consumer GPU, 1 to 2 minutes per training run.

Scope

• One model, Gemma 3 270M, and mostly synthetic tasks with exact answers.
• At most three runs per setting, and one run for many cells, so gaps of a few points may not hold.
• Test sets of 500 problems (200 for the prompt test) carry about 2 to 4 points of sampling noise.
• Learning rates came from a small grid (0.01, 0.1, 1.0), not tuned per setting.

Research prototype for reference only, not an official DeepSeek product. This code grew out of our team's day-to-day development work, with much of the engineering done by coding agents. Treat it as research code, not production-ready software. It may still contain bugs, and its coding style and engineering practices may be inconsistent. Review and adapt it before using it in your own environment.

We have adapted it for publication, so it is not identical to our internal version or the implementation described in Exploring Super-agent Design: Towards a More Autonomous Agent System. Some designs discussed in the article are directions for future development and are not implemented in this prototype.

dsh-superagent is a proactive coding agent for one GitHub repository, packaged as a bundle for DeepSeek Harness. It listens to GitHub and, optionally, Feishu. When someone allowed asks the agent for work, it opens a work item, works on it in its own agent session and directory, and answers in the original conversation, for example with a pull request. A web panel shows what needs a person.

It is also a way of working with an agent: take as little human attention as possible, and give as much value as possible. An agent that can act for a whole team can also waste a whole team's attention, so it acts only when asked or when a rule the team wrote allows it, and speaks only when someone must act or can use a result. docs/000-philosophy.md explains the principle and what it asks of the operator.

Setup

Setup has many steps, so an agent does it with you. Ask your coding agent:

Read SETUP.md in this repository and set up dsh-superagent with me.

It asks one short question at a time, shows the full plan once, and then installs and configures everything. SETUP.md is what it follows.

Documentation

Read the numbered documents in order; each builds only on the ones before it.

• 000 Philosophy
• 001 Domain model
• 002 Authorization
• 003 Execution
• 004 Prompts
• 005 Panel
• 006 Deployment: manual setup, configuration, upgrades and troubleshooting
• 007 Architecture
• 008 Cookbook: adding a Source

Development

pnpm check runs the typecheck, the tests and the build.

License

MIT

Introduction

Today, we are releasing Ampersand Tutor (&Tutor), a model trained to teach, available via API.

&Tutor learns to teach by training against Persimmon, our large-scale user model, which plays the learner. &Tutor achieves state-of-the-art performance on TutorMoments [1], a benchmark built from real tutoring moments.

A core belief at humans& is that models can better understand us and optimize for our long-term goals by learning from long human interactions. Teaching brings this belief into sharp focus. It requires understanding what a learner knows [2] and building on that understanding to help them learn [3]. Our evaluations examine how this training changes &Tutor’s teaching behavior and how those changes are driven by Persimmon.

In this blog post, we describe how we built and evaluated &Tutor. Then, we describe the evaluations we designed to validate Persimmon’s ability to model human learners. We are making &Tutor available to the public today via API.

&Tutor

A good tutor helps learners develop lasting understanding. We built &Tutor’s training environment around this principle, rewarding the tutor by how much the student learns. We train against Persimmon as a learner model. Isolating these effects, &Tutor is not trained from Ampersand, although we plan to integrate it into future Ampersand training. Our evaluations below show that this training environment encourages many downstream teaching behaviors associated with good pedagogy.

Next, we go into detail on our evaluations of &Tutor.

TutorMoments

TutorMoments [1] is built from real tutoring sessions with real students. Experienced math teachers pick out key moments where a tutor has to decide whether to offer support or push the learner to think harder, and note what each moment calls for. Models continue each conversation from that moment, with an AI model playing the student, and an LLM judge scores their responses against the teachers’ annotations. We evaluate tutor models under both a plain prompt (which just asks the model to tutor well) and an evaluation-aware prompt (which explicitly describes the balance between support and rigor).

Results. Under the plain prompt, &Tutor scores 0.754 overall on TutorMoments, compared with 0.500 for the GLM-5.3 model it was trained from. It outperforms leading frontier models, including Claude Sonnet 5.5 (0.667) and GPT-6 Astra (0.634). With the evaluation-aware prompt, which raises every model’s score, &Tutor scores 0.919, just behind GPT-6 Astra (0.926), and has the highest score of any model on appropriate rigor (0.931).

Importantly, none of these behaviors are trained for directly. &Tutor is rewarded based on how much the student learns. The pedagogical skills of knowing when to scaffold, when to push for rigor, and when to hold back all emerge implicitly from training.

Our results also suggest that the learner model plays a key role in the training environment and determines the quality of the resulting tutor model. We ran an ablation training a tutor model with the same recipe but with GLM-5.3 as a learner instead of using Persimmon. This ablation significantly underperforms &Tutor trained against Persimmon. This result is explained by the fact that, as we later show in the learner evaluations, Persimmon is a much stronger model of a learner than GLM-5.3.

The real test of a model like &Tutor is how well it helps people learn over time in real-world settings. We look forward to collaborating with teams working in education to evaluate it in those settings.

Simulated learners have long been a research topic in education [4, 5], used to give teachers practice, evaluate teaching methods, and study how students learn, and language models have recently made them far more capable [6, 7, 8, 3, 9, 10, 11, 12]. Good simulated learners can in turn be used to train tutors [13, 14, 15]. Scaling this approach depends on how well the simulated learners model real learning behavior. If they behave differently from real people, &Tutor may learn to exploit those differences, producing gains in simulation that fail to translate to real-world teaching. The quality of the learner model is therefore central to training &Tutor.

We used Persimmon as our model of learners after extensive validation of its ability to faithfully model the behavior of real human learners. We specifically show that Persimmon is better able to model the distribution of real learners’ behavior over many dimensions relevant to learning, including conversational flows, affect, and error patterns.

To evaluate how closely simulated learners resemble real people, we generate learner responses using real tutoring conversations as a starting point. We then compare the simulated and real learners across four datasets.

Two datasets capture learners working with AI tutors. These are SimulatorArena Math [16], where people work through competition math problems, and StudentBench [17], one-hour GRE tutoring sessions. For these datasets, the simulated learner continues from the real learner’s first message over multiple learner turns, interacting with the same AI tutor prompt and model, or the closest available version where the original model has been retired.

The other two, Eedi [18] and TutorMoments [1], capture learners working with human tutors. Since we can’t replay a human tutor, we generate a single learner turn at a time, using the real conversation up to that point. We repeat this at each learner turn in the original conversation. In both settings, we match the original number of learner turns.

To produce a model that can teach many kinds of human learners, we need to train it against a learner model that can reproduce the distribution of human learning behavior. For this reason, we evaluate Persimmon’s ability to match the overall distribution of real learning behavior. We report results using the multi-user Turing test methodology we introduced in our previous blog post.

For the evaluations below, we use open-coding methodology [19], automated with language models as in recent work [20], to build taxonomies of learning behavior across different dimensions.

For each dataset and dimension, we sample 3,000 real learner turns uniformly from conversations that are not in the evaluation set, and hold out every turn from 20% of those conversations. A model (Gemini 3 Flash) reads the preceding conversation and describes, in a few words, what the learner is doing along that dimension. A second model (Claude Sonnet 4.6) groups these descriptions into a small set of mutually exclusive categories. For conversational moves, these describe actions such as answering incorrectly or tentatively, requesting an explanation, proposing a strategy, or challenging the tutor.

Three independent annotator models (Gemini 3 Flash, Claude Sonnet 4.6 and Claude Haiku 4.5) then label every sampled turn, and the two-of-three majority vote is the label. Categories with fewer than 3% of the majority labels are folded into “other”, repeated until none remain. We then refine for up to three rounds: categories the annotators often confuse with each other are merged or have their definitions sharpened, stopping once agreement on the building split reaches κ = 0.72. We report agreement on the held-out conversations, which played no part in building the taxonomy. Held-out Fleiss’ κ is shown below.

Learners exhibit different conversational “moves” [21, 22, 23, 24, 25]. The moves learners make in a conversation, such as answering, guessing, explaining their reasoning, asking for help or disengaging, reflect how they are engaging with the material and are associated with how much they learn [26].

Results. Persimmon's mix of conversational moves is the closest to real learners' on all datasets, at a distance of 0.06–0.24 from the real learners, against 0.10–0.65 for the other models.

Real learners move between confusion, frustration, engagement and boredom [27]. We open-code each learner's turn for the affect the learner expresses, build a dataset-specific taxonomy of affective states, and compare the real and simulated distributions.

Results. Persimmon also comes closest to the affect real learners express on all datasets, at a distance of 0.06–0.22, against 0.12–0.73 for the other models.

A realistic learner should be wrong about as often as real learners. We label every learner turn on all four datasets as a correct claim, an incorrect claim, or no checkable claim.

Results. Persimmon's mix of correct, incorrect and non-claim turns is the closest to real learners' on all datasets, at a distance of 0.03–0.05, against 0.16–0.33 for the other models. Real learners are wrong on 25–30% of their checkable claims, Persimmon on 19–33%, and the other models on 15% or fewer.

Conclusion

We built &Tutor by training it to help learners understand, with Persimmon playing the learner. This idea extends beyond tutoring. Helping people understand is central to good communication and to giving them more agency. Both are at the heart of the AI we want to build. We’re excited about how far this can go. As our models of people improve, so will the AI we can train to help them.

References

1. Albert Zhang, Alexis Ross, Kajal Patel, Julian Bernado, Rebecca Bowie, Ana Trindade Ribeiro, Daniel Halper, Haripriya Valayaputtur, Jacob Andreas, Susanna Loeb, Li Lucy, Kyle Lo, and Ryan Knight. 2026. When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle.
2. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J. Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. In Advances in Neural Information Processing Systems 28.
3. Alexis Ross and Jacob Andreas. 2024. Toward In-Context Teaching: Adapting Examples to Students' Misconceptions. In Proceedings of ACL 2024.
4. Kurt VanLehn, Stellan Ohlsson, and Rod Nason. 1996. Applications of simulated students: An exploration. International Journal of Artificial Intelligence in Education 5, 135–175.
5. Noboru Matsuda, Evelyn Yarzebinski, Victoria Keiser, Rohan Raizada, William W. Cohen, Gabriel J. Stylianides, and Kenneth R. Koedinger. 2013. Cognitive anatomy of tutor learning: Lessons learned with SimStudent. Journal of Educational Psychology 105(4), 1152–1163.
6. Megha Srivastava and Noah Goodman. 2021. Question Generation for Adaptive Education. In Proceedings of ACL-IJCNLP 2021.
7. Julia M. Markel, Steven G. Opferman, James A. Landay, and Chris Piech. 2023. GPTeach: Interactive TA Training with GPT-based Students. In Proceedings of Learning @ Scale 2023.
8. Eric Zelikman, Wanjing Ma, Jasmine Tran, Diyi Yang, Jason Yeatman, and Nick Haber. 2023. Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency. In Proceedings of EMNLP 2023, 2190–2205.
9. Alexis Ross and Jacob Andreas. 2025. Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors.
10. Alexis Ross, Megha Srivastava, Jacob Blanchard, and Jacob Andreas. 2026. Modeling Student Learning with 3.8 Million Program Traces. In Proceedings of AIED 2026.
11. Rose Niousha, Minwoo Kang, and Narges Norouzi. 2026. INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators. In Proceedings of COLM 2026.
12. Rose Niousha, Mihran Miroyan, Abigail O'Neill, Joseph E. Gonzalez, Gireeja Ranade, John DeNero, and Narges Norouzi. 2026. ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation.
13. David Dinucu-Jianu, Jakub Macina, Nico Daheim, Ido Hakimi, Iryna Gurevych, and Mrinmaya Sachan. 2025. From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning. In Proceedings of EMNLP 2025, 272–292.
14. Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, and Andrew Lan. 2025. Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues.
15. Ke Yang, Chenglong Wang, Michel Galley, Chandan Singh, Jeevana Priya Inala, ChengXiang Zhai, and Jianfeng Gao. 2026. StudentSim: Training LLM-based Student Simulators.
16. Yao Dou, Michel Galley, Baolin Peng, Chris Kedzie, Weixin Cai, Alan Ritter, Chris Quirk, Wei Xu, and Jianfeng Gao. 2025. SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? In Proceedings of EMNLP 2025.
17. Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner, and Jonas Mueller. 2026. StudentBench: AI and Human Tutoring Yield Equivalent GRE Learning Gains.
18. Matthew Zent, Digory Smith, and Simon Woodhead. 2025. PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialogues. In Proceedings of EMNLP 2025.
19. Anselm Strauss and Juliet Corbin. 1990. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Sage Publications.
20. Michelle S. Lam, Janice Teoh, James Landay, Jeffrey Heer, and Michael S. Bernstein. 2024. Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM. In Proceedings of CHI 2024.
21. John McH. Sinclair and R. Malcolm Coulthard. 1975. Towards an Analysis of Discourse: The English Used by Teachers and Pupils. Oxford University Press.
22. Arthur C. Graesser and Natalie K. Person. 1994. Question asking during tutoring. American Educational Research Journal 31(1), 104–137.
23. Abhijit Suresh, Jennifer Jacobs, Charis Harty, Margaret Perkoff, James H. Martin, and Tamara Sumner. 2022. The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves. In Proceedings of LREC 2022.
24. Dorottya Demszky, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, and Tatsunori Hashimoto. 2021. Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions. In Proceedings of ACL-IJCNLP 2021, 1638–1653.
25. Dorottya Demszky and Heather Hill. 2023. The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), 528–538.
26. Michelene T. H. Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist 49(4), 219–243.
27. Sidney D’Mello and Art Graesser. 2012. Dynamics of affective states during complex learning. Learning and Instruction 22(2), 145–157.

Postgres 19 most likely won't be released until November and could possibly be postponed further. Several large features were reverted quite late in the beta period. AI has made it easier to identify bugs, and this has resulted in a lot more bug reports that the PostgreSQL committers have to work on. This has taken the focus off development of new features, as the project cares about quality.

A proposal: how to hire junior engineers in the age of AI

About six weeks ago I wrote We are all Product Engineers now, an exploration of what LLM-written code is doing to the job of software development. A few weeks after that I turned it into an AI Engineer New York talk, which at a quarter-million views is now by far the most-watched talk I’ve ever given.

The talk and the post both end with an exhortation: we have to figure out how to hire new engineers to do the vastly-changed job of software development now that “programming” isn’t part of that job description. This is a monumental shift and there is an entire cohort of people who got into college three and four years ago who are emerging into a job market that has absolutely no use for the skills they learned.

Since then I’ve been thinking about how, exactly, one could go about hiring junior engineers who would be good at the new way of doing things, specifically about how to interview them. This post is about my best idea so far.

Junior hiring has collapsed, and old tech interviews went with it

Back in July I noted that hiring of junior engineers has collapsed starting in roughly 2023, probably not coincidentally when AI accelerated coding took off. Things continue to get worse for them. Stanford's "Canaries in the Coal Mine" research found employment of software developers aged 22 to 25 is down nearly 20% from its late 2022 peak, while hiring of senior engineers has continued to grow. Anthropic, surely at the forefront of AI-driven software development, has said that the share of engineers it hires who are junior has fallen from 27% to 9% since 2025.

A big driver of the collapse, I believe, is that there’s no way to know if a junior person is going to be any good at being what I termed a product engineer, especially if even the limited experience they gained at school was about old-fashioned programming, algorithms and data structures and other things the agents take care of now. Hiring a junior is always a risk, but the risk is greatly magnified when we only just defined the job and haven’t figured out how to interview for it.

Our previous tool for selecting promising juniors was the coding interview. I have said for more than a decade that coding interviews are stupid but the final nail is definitely in that coffin now. People are certainly trying to find replacements for it. In a much-noted blog post, an engineering manager at Anthropic said they have stopped giving coding interviews on their team (frequently mis-reported as being a whole-company policy). Canva replaced its computer science fundamentals round with a mandatory AI-assisted one in 2025, and Meta swapped one of its two onsite coding rounds for an AI-assisted version. Google is piloting code comprehension and AI-assisted rounds.

But nobody is trumpeting a solution yet, so it’s fair to say we’re all still figuring this thing out, and the observed reality is that a lot of companies are just opting out entirely and only hiring senior developers, a totally unsustainable approach.

Hiring seniors is easier, but there are only so many of them

The argument I made in the talk is that we have to hire junior engineers or the industry will shrink and die without them, and that’s true, but that’s an altruistic, big-picture argument that doesn’t work when each individual hiring manager is trying to work under capitalism, a point others have already made. Everyone benefits from a supply of trained seniors, but everyone would prefer that some other company paid to train them, and in the current environment, figure out how to train them at all.

Luckily, a selfish, capitalism-friendly argument can be made for hiring junior product engineers, which is that they are a lot cheaper than senior engineers, who are growing ever-more expensive as 71% of the growth in software job postings between May 2025 and May 2026 called for them. They are also simply available, while senior engineers are all extremely employed already.

So if you can find good product engineers, you can get them cheaply and in quantity, and that’s good because we’re about to need enormous numbers of them. Product engineering is not rocket science. We didn’t stop making people who can do this job, we just haven’t figured out how to identify them, because the job changed so much so quickly.

Internships are not a solution to the hiring problem

One solution a couple of companies have hit on to try and deal with being unable to use coding interviews to figure out who’s going to be good at this new job is paid internships: simply admit the interview doesn’t tell you much, hire a ton of interns, and see who actually performs on the job. Shopify grew its internship program from about 100 a year to over 1,000, and Cloudflare set a goal of 1,111 interns in 2026.

That’s certainly a solution, but it is both slow and expensive and totally unavailable to startups, who lack all of the time and the money and the level of adult supervision required to hire junior developers and see if they sink or swim in six months. Most companies can only afford a few hours of interviewing per candidate. Even if you’re hiring interns, the program is going to go better if you have an interview process that selects better candidates and doesn’t just randomly pile juniors into the room.

You have to interview for something, so what?

The usual shortcuts for picking juniors are just bias in a hat and glasses

Another way is to hope that somebody else has already done it for you, by hiring from a well-known elite school. This too is already happening: the Pragmatic Engineer found that an increasing percentage of new grads in software engineering are coming from elite schools. Hiring for a fancy school is also something I have railed against in the past: credentialism is a filter that simply rewards being born with rich parents and does not predict that they will be good at any job, including that of a product engineer.

Other recently touted approaches include hiring for things like “hunger” and “taste”. I don’t know what those things mean, and neither do the people hiring for them. “Hunger” sounds like a code-word for “youth” which is a) an illegal thing to hire for and b) not helpful when all your junior candidates are roughly the same age anyway.

(Did you know IQ tests are effectively illegal for hiring in the US because they disproportionately screen out minorities? I learn things when I research these posts.)

So a lot of people are falling back on “just have a free-flowing conversation, I’ll know it when I see it” which has been scientifically measured as one of the weakest possible approaches: Sackett and colleagues in 2022 put the predictive validity of unstructured interviews at .19, compared to .42 for structured ones. Google learned this the hard way with its famous brainteasers, which its head of people operations eventually admitted predicted nothing at all.

So the interview has to be structured: everybody gets the same interview. What are we even interviewing for?

We have to interview for the actual things that make you good at the job

To try and figure out what we can actually interview for in a couple of hours, I tried to more concretely break down what goes into product engineering. Here’s what I came up with:

Product sense: given an amorphous mass of unstructured information, can you figure out what the actual problem is that needs to be solved? This involves pushing back on what the user is explicitly asking for, which is often the wrong thing, and finding hidden assumptions and constraints that they don’t mention at all.

Intelligence: you do in fact need to be a particular kind of smart to be good at building systems, so go looking for that kind of smart rather than an IQ test. Can somebody hold a whole system in their head, spot problems, and figure out what needs to change when constraints shift?

The ability to explain yourself in writing: this is always something I’ve prized in hires, but it has suddenly become most of the job. If you can’t explain yourself in natural language, you cannot prompt an agent effectively. I expect this will garner some pushback, since there are a lot of experienced programmers who are notably bad at explaining their work, but the job has changed.

The best evidence of all of these, of course, is a portfolio of existing work where a candidate has done all of these things, and then getting them to explain it to you in writing. Every time you build a system badly you figure out how to build your next system better, so the more you’ve built, the better you’ll be. But most juniors don’t have a portfolio, and selecting for the ones that do is just our old friend Rich Parents again, selecting for kids who had a lot of free time and resources.

So now we’re homing in. It has to be the same fair, job-related task given to everyone who applies. What does that look like? Here’s what I’ve come up with.

Spend an hour figuring out what to build and then build it

The old way would have been to give a developer some requirements and get them to write the software. But with product engineering, figuring out what the requirements are is most of the job.

Luckily, the same thing that wiped out the coding interview can come to our rescue: we can give a big pile of requirements, constraints and exhaustive detail to an LLM, then get the LLM to role-play as the customer.

The candidate gets an hour (or two?) with a model that has everything they need to know to build the system, and a second model (it needs to be the same model for everyone) that will write whatever software they tell it to build. Their job is to get the information out of the customer and into working software, through the interface of natural language.

In the old way, the software built would have to be something very simple to fit into the time constraints of an interview. With LLM-driven development, we can demand much more: real, working software, possibly even deployed to a supplied target platform.

There are of course some rules.

• You can keep asking the customer questions as you go. Real life works that way. You can even show them a half-finished version and see if they like it.
• You can throw away your whole approach and start over, several times if you want to. The worst product people I’ve seen pick an initial approach and then stick to tweaking it when starting over would be the right call.
• You can’t jailbreak the customer. Kudos for thinking of that approach, very Kobayashi Maru of you, but real customers cannot be tricked into dumping a requirements document to the console.

The good thing about this approach is that it can be extremely domain-specific. It can be based on the real working system, or you can just make up a brand new problem. It can be arbitrarily detailed and complex, it can have edge cases galore, it can be like the real world as much as possible, and it will reward candidates who think of edge cases, approach things sideways, and think about whole systems at once. In short, the people who are good at product engineering.

It’s also relatively cheap, because you can build another agent to score the output. You give your judge agent the full requirements and the software, and get it to build and run as many tests as it can think of and report back to an actual human for final scoring. (Is this an eval, being recommended by a guy who currently works at an eval company? You bet your ass it is. Evals are super useful, which is why I work there.)

For bonus efficacy, maybe tack on another hour where the candidate and a human interviewer discuss the resulting software: what works, what doesn’t and why.

This isn’t actually a new approach

While thinking through this approach I discovered that it isn’t actually a novel idea in the least. In the world of medicine, this has been standard practice since the 1960s. In the OSCE, medical students take a history from a standardized patient: a trained actor portraying a specific complaint, who answers consistently from student to student so the assessment is fair. Students are scored on a checklist of the questions they should have asked, which isn’t very far away from being judged on how many requirements they found in the table.

Is anybody already trying this in tech? Not that I can find. Palantir's decomposition interview has long asked candidates to break a vague real-world problem into parts with little or no code, but it doesn’t have the building part. There’s also a startup called Promptster, which gives candidates an incomplete ticket and AI teammates who hold the missing context, then shows reviewers how many hidden requirements each candidate recovered, which isn’t the same but isn’t a million miles away either.

Don’t try to automate the whole thing

You’ll note that I gave an agent the task of writing tests but that the final judgement is given to a human. That’s very much deliberate. A strong model judge agrees with humans about as often as humans agree with each other but a 2026 evaluation of 21 judges found some that gave the same answer every time while still being badly biased. Using fully automated scoring would also be against the law in some places: New York City requires annual bias audits of automated hiring tools, and you don’t want that heat.

If there is one thing I have learned working in the world of evals, it’s that your judges have to be calibrated by human judgement.

Will this be good enough?

This entire approach is, right now, theoretical. I’m not hiring for software developers right now and I haven’t yet tried to persuade anybody else to give this a trial. So I don’t know if it will work, but I’m optimistic that it’s a step in the right direction.

The company that figures this out will have a huge advantage: a ready supply of affordable, capable software developers who can just figure out what to build for themselves, just when demand for that kind of developer is exploding. If you do give it a try, I would love to hear how it goes.

Digital minimalism

For a long time I have been a fan of minimalism in my physical life: have few physical possessions, and try to do more with less. Four years ago, I wrote this blog post describing what I had in the 40-liter backpack that I travel around with.

Physical minimalism, however, was an idea that came of age in the 2000s and 2010s and was responding to the challenges and the opportunities of the 2000s and 2010s. I continue to believe that it is valuable - though I recognize some arguments in the other direction, notably that having the right kinds of physical things can make more practical to have more self-sovereignty over your life. But I also think that in the modern era, we need a new kind of minimalism: digital minimalism.

Digital minimalism is about being intentional about ways that you access digital technology, making each use a conscious choice rather than something that you get sucked into by default. In this post, I will describe some experiments that I've done in this direction.

Social media minimalism

Twitter (aka X) has always had a strain of toxicity to it, that has been increasing in recent years. The platform is definitely engaging, but it's often far from informative, and the ways in which it engages often do not make you feel good. But this all seemed to hit a particular crescendo for me around the end of 2024 and beginning of 2025. Many things were happening at that moment, but if you spent a lot of time on X you will likely remember a particular one: everyone seemed to be suddenly becoming fans of Adolf Hitler.

This is not even mentioning the time that Grok itself briefly turned into "MechaHitler".

Around the same time, "Crypto Twitter" was spiraling into a mood that I saw described as "financial nihilism": roughly, that the natural end state of blockchains is gambling, this is good, and if you dislike this and wish for something better then you are a stuck-up elitist prick.

Naturally, I had (and continue to have) my intellectual disagreements with both of these trends. But on top of that, I also started to more seriously pay attention to the fact that while the style of discourse on the platform was engaging, it was not enlightening and not making me feel good. And so for much of this year, I have been doing an experiment in distancing myself somewhat from this chaos. I did not leave X the platform, but I did leave X the interface. Instead, I have been doing most of my tweeting through Firefly:

Firefly is a multi-client that combines together X, BlueSky as well as Farcaster and Lens (both partially-Ethereum-based decentralized social media platforms). You can see tweets from all four platforms, and you can make a tweet that gets auto-posted to all four platforms.

My primary goal in doing this was to support decentralized social media, and to continue to engage with X but in a way that distributes network effect rather than locking it in. But in my months of using Firefly, I found that it also has a different effect: I feel much better using it.

I suspect the primary reason is that while Firefly allows you to tweet, and see other people's tweets, it is missing one central part of the X interface: The Algorithm™.

In particular, I suspect that in my time using the X interface, I fell into the following trap:

1. See someone making a tweet supportive of Adolf Hitler.
2. Click on the tweet and look around its quotes and responses, the same way that someone would morbidly stare at a car accident.
3. The Algorithm™ thinks: "This user seems to be a fan of Adolf Hitler. Let me mark that down as one of his interests."
4. See ten tweets supportive of Adolf Hitler.
5. Click and look around with morbid curiosity, the same way that someone would morbidly stare if they saw for the first time a car accident where ten cars were crashed and burning at the same time.
6. The Algorithm™ thinks: ... well, you get the idea.
7. See a hundred tweets supportive of Adolf Hitler.

Now, in fairness to X and Elon and its hard-working employees, the Adolf Hitler fandom does seem to have greatly wound down in the one and a half years since then. I suspect part of this is because such movements inevitably burn out once people realize how immoral and stupid they are. But another part is undoubtedly genuine improvements in how The Algorithm™ works.

But even still, in those moments when I go back into the X interface, I find myself logging back out and going back to Firefly. I also find that my mind has changed. I find myself less interested in the various battles that the intellectual titans on the platform seem to be sucking every day of their lives into. Recently, I logged back in, and I learned that the culture war issue du jour was that people were stridently arguing about whether or not Mao was evil. When I saw this, I had an emotional response that was sort of similar to what I think is the common experience of being an adult, seeing some food or TV show or game that you used to love as a kid, and realizing that you now find no interest in it at all. I smiled, and immediately went back to enjoying my day.

My verdict: I recommend more people try this. Explore ways of greatly decreasing or scoping your social media use: use something like Firefly, access X only on Tuesdays, whatever works for you. Get the information benefits of the platform without getting sucked in to the vortex, which does operate on you in powerful subconscious ways that you may not realize.

Cell data minimalism

For a period of 100 days, I embarked upon a second experiment: I lived my life with my phone always on airplane mode. I still used wifi, but not my cell connection.

I had two goals in doing this:

1. Privacy. A phone connected to cell is an always-on tracking device. Cell companies are notoriously bad at keeping that data secure, and sometimes they themselves sell it. So there is a large chance that a trail of your location will end up in the hands of intelligence agencies, military contractors, advertising companies, and more. Anonymous SIM cards are not a sufficient solution, for two reasons: (i) your phone can still be linked across its entire usage lifetime through its IMEI number, and (ii) you leak all kinds of data that can be used to reconstruct who you are from your fine-grained movement patterns, internet access patterns, and modern AI makes it very practical to uncover these patterns. Hence, the best location privacy solution is not having a cell connection at all (also, use GrapheneOS, and be very limited in what apps you give location permissions to; I had already been doing both).
2. Mental health. Not having the internet at all for significant periods of time (when I'm outside) has been a good way of (re-)building the habit of surviving without it, and getting more mental peace and the space to think about other more healthy and productive things.

On the whole, I consider the experiment to have been a success. During those 100 days, I violated the "no cell data" rule once literally but not spiritually - I turned it on briefly to receive a confirmation code that I needed in order to maintain that phone plan itself, and once spiritually but not literally - when going on a remote and slightly dangerous hike, friends insisted that I bring along a few communication devices that they had for emergency comms and use them to check in once every few hours.

My largest inconvenience was a stretch of time I was in Singapore, when my inability to receive SMS confirmation codes meant that I could not log into Grab (the regional Uber equivalent). As a result, my food options were more limited, and I had to take the subway instead of a car even in situations where that was much more inconvenient.

Since those 100 days, I have opened the cell connection occasionally, but so far I have broadly continued an "airplane mode by default" lifestyle.

For navigation, I used CoMaps, a local-first mapping application that downloads OpenStreetMap data and then shows you a map, gives you directions to get where you need, etc locally. My general review of CoMaps is that I find it worse than Google Maps for navigating the human world (restaurants, stores, opening hours...) but better for navigating the natural-ish world (it knows about many trails that Google Maps does not, though it does occasionally fail by claiming trails that do not actually exist, leaving me on one occasion pushing through hundreds of meters of shrubbery to get back to civilization).

I also became interested in the concept of local world knowledge apps. In principle, the data size of meaningful packs of information about the whole world is not that large. Compressed English wikipedia (without media) is only 27 GB. OpenStreetMap's planet.asm is 2.2 TB, though you can doubtlessly prune that down to 1-10% of its size and have everything you might conceivably need.

This month there was a bounty (organized by Statuette; I contributed as one of the donors) to create local phone apps that try to make useful knowledge easily accessible offline.

Here are two of the apps: FieldAtlas and Commonplace.

My general impression is: right now, we are exactly on the cusp of these kinds of apps starting to make sense. Most of the participants in the competition were trying to use on-device text LLMs to think for a long time before coming up with an answer. My impression is that this approach was too slow: answering a single question would take anywhere from 1 to 22 minutes. The best apps, on the other hand, used some kind of semantic search to show the user a search result from the local database immediately, and then followed that up with a very small LLM writing a text answer. This gives actually-good answers in a very reasonable timeframe.

Finally, when you're out and about, especially if you have meetings, sometimes you do need to have internet when you're on the go. When you arrive at a place, you often need to contact a person you're meeting to confirm their exact location or to get them to open the door. When you're at one meeting, you often need to re-confirm details about the next. If you don't have cell data, your only option is public wifi - and importantly, the public wifi hotspot must allow you to connect without providing an SMS confirmation code.

Different places are very different in terms of how friendly they are. Here is a quick tier list, taken from places I've traveled to while I was paying attention to public wifi availability:

1. Taiwan (or at least, the Taipei area). Taipei has a huge number of public wifi hotspots. When I opened my phone to check when walking outside (or even when inside an Uber), there was literally an over 25% chance that there was a hotspot I could connect to that was in range at that moment. Usually, these were either municipal government wifi, or public wifi run by local convenience store chains.
2. Ukraine, USA, Canada, Singapore. Ukraine was the most surprising for me. In a surprisingly large number of places, across multiple cities, there were open hotspots available, some of which did not even require a clickthrough. USA, Canada and Singapore were all similar: they had a very high number of theoretically-public wifi hotspots in general, but a large portion of those hotspots were gated behind either SMS or some kind of login. But even still, a very significant number of open wifi hotspots were available.
3. Austria, Hungary. Both halves of Charles I's old empire are similar. There are many hotspots, but the great majority of the public ones require an SMS confirmation code to log in. I could find properly open wifi, but on several occasions it took trying quite a few hotspots and walking around for a while.
4. Switzerland. Despite its general reputation for privacy, Switzerland ranked at the bottom for me. Almost all public wifi hotspots require an SMS confirmation code. In my first 1.5 kilometers walking through the downtown, I found only two that do not: a random shoe store, and Starbucks, which lets you connect for 10 minutes, and then longer if you enter a confirmation code that you receive by email.

Web minimalism

Recently, I have been starting to do more and more things without relying on user interfaces inside of websites at all. The last few times I updated my IPFS hash on ENS, I did so through a command-line tool that an AI wrote for me. Many of the payments that I make through privacy protocols, I now make through the kohaku-cli tool.

Websites and web applications, especially web applications for doing crypto things, are dangerous for several reasons:

1. If you access a web application, you end up re-downloading the application from the server thousands of times, and you are fully trusting the server each time. If the server gets hacked even one of the thousands of times that you open the site, you might be screwed. Regular old-fashioned applications (where you explicitly click "download") are better than this: you only need to trust the server to not be malicious or hacked at that exact moment once.
2. Websites and web applications are by default very bad for privacy. When you access a website, it can see your browser (including version number), your operating system, your screen size, and once you start acting hundreds of other data points (eg. mouse movements, button click patterns) that quickly identify you. Websites often send user data (and metadata) to all kinds of third-party providers, from CDNs serving scripts to analytics companies, and this all happens by default.
3. As we've seen with social media, web applications crafted by third parties with commercial interests may not have your interests at heart. Fine-grained UI decisions may be dark patterns optimized to extract, whereas what you want is bright patterns that are optimized to enlighten and educate and protect you.

So what's the alternative? Basically, bring back the Unix Philosophy, but make it better and much more user-friendly with local AI: speak English to tell your computer what to do rather than Bash.

At its core, the Unix philosophy was simple: make simple, modular tools that each do one thing and one thing well, and that are designed to talk to each other through the most universal interface (text). You as a user have extremely fine-grained choice about how to make the different programs fit together. UIs, on the other hand, combine many functionalities into one single opaque tool produced by a single developer. At their apex, UIs turned into "super-apps", which greatly concentrate power in the application developer and have complex and unintelligible internal workings, magnifying security risks. With LLMs, we can go back to the Unix philosophy approach, putting the fine-grained decision-making power back into the user's hands, but without the downside of everyone needing to become a programmer.

Conclusion

Physical-world minimalism, of the style that became popular in the 2000s and 10s, was made possible by a new wave of technology: computers and the internet. A single computer could substitute for dozens of physical things. As a result, you could live a life of much greater simplicity and freedom, with just a laptop, a few sets of clothing, and basic charging and hygiene equipment. Physical minimalism was much harder before the internet; it was doable, but would entail a much more complex lifestyle commitment. After the internet, the costs of a more physically minimalistic lifestyle greatly dropped.

With the last ten years of progress of software quality, together with modern AI, I would argue that we now have the wave of technology needed to similarly enable the next wave of minimalism: digital minimalism. We need social media less than before, because you can talk to the bot for information instead (and for human connection, there's still the old-fashioned solution: going out and talking to people in real life). We need the internet less than before, because there's so much more you can do purely locally and offline. And we need the web less than before, because you can do much more, and more conveniently, though either command-line tools or custom-made user interfaces, and AI can help you use or craft both. And these things are all happening at exactly the time that the flaws of these previous paradigms - and, for that matter, the flaws of using AI in the wrong way - are reaching their apex, and so a digital minimalism is becoming more and more necessary.

Each person's path to digital minimalism will be different, as each person has different needs. But I highly recommend everyone try reducing their digital clutter in at least one important way, and if you like the result then keep going.

Today, Cloudflare services for private origins is generally available. Point a proxied DNS record at a private IP, and Cloudflare reaches the origin over your existing Cloudflare One connection. Your origin stays private, while web application firewall (WAF), caching, bot management, and Workers work just as they do for public origins.

In June, we launched this capability in closed beta to address a problem we kept hearing from customers: Their internal APIs, AI agent backends, MCP servers, and operational tools weren’t built to face the Internet, but those services still need to accept requests from partners, customers, or agents over the Internet. Before the beta, putting Cloudflare in front of these services meant exposing a public IP, opening a firewall port, or running cloudflared alongside the origin.

Those requirements often left private applications without the same protections and controls they used for public applications, including WAF, bot management, rate limiting, caching, traffic acceleration, rewrites, and Workers.

What is generally available today

During the closed beta, eligible customers requested access through their account team. Today, we’re opening access beyond that beta group.

Point a proxied A or AAAA record at a private IP, and Cloudflare’s application services work as they do for public origins:

• WAF and managed rules
• Bot management
• Rate limiting
• Caching
• Transform and rewrite rules
• Workers

Some limitations remain. Argo Smart Routing and Dedicated CDN Egress IPs aren’t supported, and origin health checks require Load Balancing. See the developer documentation for the full list of current limitations.

Your origin keeps its private IP, with no inbound firewall exception required. With Cloudflare WAN, no connector software on the origin is required. With Cloudflare Mesh, a connector runs within your private network and can serve multiple origins.

How it works

Create a proxied A or AAAA record pointing to your origin. For supported private IP ranges, use private network routing is enabled automatically. If your origin uses a public IP that’s reachable only through your private network, you can enable the setting manually.

The rest of the request flow stays the same: TLS termination, WAF, rules, and caching continue to run on Cloudflare’s network.

The setting turns on automatically for RFC 1918 private IPv4 addresses, RFC 6598 CGNAT addresses (100.64.0.0/10), and RFC 4193 unique local IPv6 addresses (fc00::/7). It can’t be turned off for these ranges; attempts to disable it through the API return an error.

What the API looks like

For customers automating deployments through the API, private routing is simply an additional attribute on a standard DNS record.

POST /zones/{zone_id}/dns_records
{
 "type": "A",
 "name": "app.example.com",
 "content": "10.0.0.50",
 "ttl": 300,
 "proxied": true,
 "private_routing": true
}

Behind the scenes, Cloudflare's proxy platform determines where to send traffic for app.example.com by querying Cloudflare's Origin API. The response includes metadata indicating that the destination should be reached through a private network path:

{
 "zone_name": "example.com",
 "ipv4_addresses": ["10.0.0.50"],
 "private_routing": true
}

The private_routing flag is the key signal. When our proxy sees it, instead of attempting to connect directly to the private IP address over the public Internet, it hands the request to our private networking layer, which then routes the connection across the customer's existing private network connectivity, whether that's IPsec, GRE, Cloudflare Tunnel, CNI, or Cloudflare Mesh.

The beta post covers the routing layer in more detail.

Get started

Before you begin, you’ll need:

1. The Private Origins entitlement on your account. Ask your account team to turn it on.
2. Cloudflare One connectivity using IPsec, GRE, CNI, Cloudflare Tunnel, or Cloudflare Mesh.
3. Unified Routing enabled on your account, unless you connect over Cloudflare Tunnel (Tunnel doesn't need it). New and existing accounts can upgrade through their account team.
4. If you connect over Cloudflare WAN or Cloudflare Mesh, configure a return route for Cloudflare’s source ranges: 100.64.0.0/12 for IPv4 and 2606:4700:cf1:5000::/64 for IPv6. Then, create a proxied A or AAAA record for the private IP and confirm that Use private network routing is enabled. Your existing WAF, cache, and bot management configuration applies to that hostname without further changes.

Then, create a proxied A or AAAA record for the private IP and confirm that Use private network routing is enabled. Your existing WAF, cache, and bot management configuration applies to that hostname without further changes.

One note from the beta: if requests time out and nothing arrives at the origin, check the return route first. A missing route for Cloudflare’s source range was the most common setup failure we saw.

Once enabled, follow our developer documentation, which walks through the full setup.

What comes next

Today’s release covers public traffic reaching private origins. Next, we’re working on private-to-private traffic: users, services, and AI agents on private networks reaching applications on other private networks through Cloudflare’s application services.

Engineering work is underway, and we’re targeting a closed beta in early 2027. If you’d like to participate, talk to your account team.

The goal is to give private applications the same protections and controls as public applications. An employee opening wiki.company.internal, for example, could benefit from WAF, rate limiting, and bot management. Service-to-service traffic between clouds and data centers could use those controls too, even when neither side touches the public Internet.

For now, today’s release brings the WAF, caching, bot management, and Workers you use for public applications to private origins over connectivity you already operate.

Questions or feedback? Join the conversation in our community forums or reach out to your account team.

How to troubleshoot GKE workloads faster with VPC Flow Logs

Having visibility into your network traffic is essential to operating Kubernetes environments effectively. When a service slows down or loses connectivity, knowing who is talking to whom lets you narrow the problem down quickly. In a standard cloud network, flow logs provide that view by recording connections between IP addresses, ports, and protocols. In Kubernetes, however, pods are temporary. Their IP addresses change constantly as pods scale, crash, or move between nodes, so a raw log entry showing 10.4.2.18 talking to 10.4.8.44 tells you little about which workload (and pod) owned that address, what namespace it was in, or what application it was trying to reach. By the time you search your cluster, the pod may already be gone!

In a Google Cloud environment, VPC Flow Logs closes this gap at the network layer. Google Cloud's virtual network already sees every connection a pod makes, so each sampled flow is recorded with the Google Kubernetes Engine (GKE) metadata behind the IP address: pod, workload, namespace, and service. You get workload-level attribution for L3/L4 traffic without deploying extra agents or sidecars, and it complements in-cluster Container Network Interface (CNI) and dataplane observability tools you may already run when you need deeper detail. Let’s take a deeper look at how VPC Flow Logs and GKE work together, and how to get started with it.

How GKE metadata enriches VPC Flow Logs

VPC Flow Logs samples traffic directly inside Google Cloud's virtual network. When flow logs are enabled for a subnet that hosts a GKE cluster, every record of pod traffic that passes through the VPC network is annotated with the Kubernetes context that owned the IP address at the time of sampling. This helps ensure the context is preserved even after the pod is gone:

• Cluster: The cluster name and location.
• Workload: The pod name and namespace, along with the controller that owns the pod, such as a deployment, StatefulSet, or DaemonSet.
• Service name: The Kubernetes service that received the traffic.
• Node and zone: The standard VPC fields, including the node VM and its zone on both ends of the flow.

If you register your GKE workloads in App Hub, each record also carries application-level labels (apphubSource and apphubDestination) that identify the business application, environment, and criticality of each endpoint. This is especially useful when GKE microservices talk to App Hub-registered resources outside the cluster, such as Cloud Run services.

Before you start

Upgrade your log bucket to Observability Analytics (formerly Log Analytics): Flow Analyzer reads only from an upgraded bucket, typically _Default. Note: This upgrade is permanent.
gcloud logging buckets update _Default --location=global --enable-analytics

Enable VPC Flow Logs organization-wide: One organization-level configuration covers every subnet in every VPC network, including future clusters (requires Network Management Admin role). Keep include-all-metadata on so every record carries namespace and pod details. To balance visibility against cost, you can add a secondary sampling rate (--flow-sampling), an aggregation interval (--aggregation-interval), or a filter expression (--filter-expr) to view only the traffic of interest.
gcloud network-management vpc-flow-logs-configs create CONFIG_NAME --location=global --organization=ORGANIZATION_ID --metadata=include-all-metadata

Turn on intranode visibility: Autopilot clusters already have it, but you need to enable it on Standard clusters. Note: Enabling it restarts control plane and node components and adds some per-node overhead, so test it on a non-production cluster first.
gcloud container clusters update CLUSTER_NAME --location=LOCATION --enable-intra-node-visibility

Now you’re ready to get started.

Analyze traffic visually in Flow Analyzer

VPC Flow Logs write every record to Cloud Logging, you can search and query it in Logs Explorer or Log Analytics. For day-to-day work, Flow Analyzer, part of Network Intelligence Center, turns the same records into an interactive chart and table, so you can group and filter traffic by cluster, namespace, workload, pod, or service in a few clicks.

The walkthroughs below use Online Boutique, an open-source microservices demo, on a GKE Autopilot cluster in us-central1. A loadgenerator deployment sends synthetic shopper traffic to the frontend, which calls productcatalogservice, cartservice, currencyservice, recommendationservice, shippingservice, adservice, and checkoutservice.

Case 1: Turn IP addresses into workload names
Whether you’re investigating an incident or assessing an architecture change, the first question is always the same: Which endpoints are communicating, and at what volume?

Rather than presenting ephemeral IP address pairs, GKE metadata allows Flow Analyzer to answer this question directly using Kubernetes workload names.

Steps in Flow Analyzer:

1. Open Network Intelligence Center > Flow Analyzer and set the time range to last 1 hour.
2. Under Source and Destination, set Organize flows by to GKE workload.
3. In the filter bar, add the GKE workload is known filter so only pod-to-pod traffic is shown.
4. Click Run new query.

Key insights: Sorted by volume, the table is a ready-made application dependency map: frontend handles incoming requests from loadgenerator (representing user traffic), while productcatalogservice answers calls from both recommendationservice and frontend. Because each row shows workload names instead of raw IP addresses, any unauthorized or unexpected workload-to-workload communication immediately becomes apparent.

Case 2: Drill into a single service
When a service is slow or about to change, you need to know who depends on it and how heavily.

Steps in Flow Analyzer:

1. Change Traffic aggregation to Client ↔ server (bidirectional), so requests and responses between a pair are counted together.
2. Under Server, add the filter GKE workload = productcatalogservice.
3. Click Run new query, then click Details on the top row.

Key insights: You get every caller of the catalog service, each caller's share of its traffic, and a timeline for every pair, so if one caller's line climbs while the others stay flat, you know where to look first. When the question is latency rather than volume, switch the display mode to latency to rank the same callers by round-trip time.

Case 3: Find the pods that talk across zones
In a regional GKE cluster, chatty services that land in different zones send every request across a zone boundary, adding inter-zone data transfer charges. Because VPC Flow Logs record the zone at both ends of every flow, the pod pairs driving that traffic are easy to spot.

Steps in Flow Analyzer:

1. Switch from Basic filters to SQL filters, which let you compare two fields of the same flow.
2. In the WHERE box, enter src_gcp_zone <> dest_gcp_zone.
3. Set Organize flows by to Source GCP zone, Source GKE pod, Destination GCP zone, and Destination GKE pod.
4. Click Run new query.

Key insights: The table displays cross-zone traffic ranked by volume, identifying the exact pods and zones on each end. In our cluster, the top flows occur between a frontend pod in us-central1-b and a currencyservice pod in us-central1-c, followed by checkoutservice, shippingservice, and productcatalogservice pairs. To investigate further, click Explore in Log Analytics to open the underlying SQL query for additional customization, such as adding joins or time buckets.

What to keep in mind

• Pod-to-internet traffic: VPC Flow Logs carry no pod annotations, because the masquerade agent replaces the pod IP with the node IP before the flow is sampled. Keeping internal destinations in the non-masquerade list preserves pod names for those flows, and if you run ip-masq-agent with intranode visibility, that list must also include the cluster's node and pod IP ranges to avoid connectivity problems.
• Blocked traffic: Traffic blocked by a network policy never reaches the VPC network, so use NetworkPolicy logging to investigate it.
• Sampling: Flow logs are sampled, which makes byte and packet counts well suited for ranking and trending rather than per-connection accounting.

Next steps

If your clusters already have VPC Flow Logs turned on, upgrading the _Default bucket and opening Flow Analyzer is all it takes to reproduce the three walkthroughs above. To learn more, see the documentation:

• Using VPC Flow Logs
• VPC Flow Logs record format for GKE
• Analyzing network traffic with Flow Analyzer
• Best practices for GKE network observability
• Troubleshooting GKE network observability

Serverless should be the first execution model teams evaluate for AI agent workloads. Many agents spend much of their lifetime waiting, and always-on compute forces teams to pay for every idle minute. In a study of more than 35,000 production coding-agent sessions, the median session ran LLM or tool calls for only 20% of its lifetime. The rest of the time, it sat idle between steps, with its sandbox still allocated.

According to Gartner’s 2026 CIO and Technology Executive Survey, AI agents have the most aggressive adoption curve of all technologies measured, but momentum is outpacing platform maturity, making it difficult to run agents at scale. When Google Cloud surveyed over 1,400 senior IT leaders for their 2026 State of AI Infrastructure report, 83% said they would need infrastructure upgrades to support production-grade agentic AI. Teams can interpret a readiness problem as a reason to provision persistent compute, adopt a new cluster, expand an existing platform, or even choose a different deployment topology. Serverless offers a less invasive path to scale agentic AI.

In a serverless model, the platform provisions capacity as requests arrive and scales to zero when they stop, so teams can add agent workloads without managing a cluster. While some agents require dedicated capacity, many of them can run serverless. Longer runs can use durable serverless workflows to save progress and release compute during a wait, then resume once the next step is ready.

In this post, we’ll explain how to test the serverless default against your own agent workloads and adjust your approach as production data becomes available. We’ll cover how to:

• Determine whether an agent uses compute intermittently
• Compose longer or resumable agent runs as a durable serverless workflow
• Compare its production behavior and cost with warm or dedicated capacity
• Use production evidence to move only the steps whose latency or runtime requirements justify a different workload placement

Estimate whether your AI agent workload needs continuously available compute

Treating agent workloads as if they all belong in the same category can cause teams to provision continuously available compute before they know whether the workload requires it. By creating a workload execution profile, you can determine if the agent needs dedicated workers or can run safely on demand.

Create a workload execution profile for the placement decision

An execution profile records the workload characteristics and runtime requirements that inform where and how the agent’s workload should run. Workload placement describes the process of choosing the execution model for an agent workflow or an individual step, such as on-demand compute, minimum instances, a warm service, or a dedicated worker pool.

Creating an execution profile clarifies which resources must remain available when no agent step is actively running. For example, if only durable state and a trigger for the next step need to be available, a serverless workflow may be the best fit.

We recommend including the following inputs in your execution profile:

• Trigger and arrival pattern
• Response pattern, such as synchronous, asynchronous, or streaming
• Peak arrival rate and burst duration
• Peak concurrency
• Response-time objective, such as the maximum acceptable time to first output or completion
• Active compute time per run
• Total elapsed time per run
• Compute utilization over the expected operating window
• Maximum step duration
• Memory and CPU requirements
• Accelerator, local storage, persistent connection, and private network requirements

The inputs in your execution profile show you when demand arrives and how quickly it accumulates. If you only rely on daily averages, you may choose an execution model that doesn’t suit the workload.

Separate active compute from total elapsed time

The ratio of active compute time per run to total elapsed time per run is an early workload placement signal. A low ratio strengthens the case for on-demand execution, while a high ratio can indicate that dedicated workers may be more economical, especially when paired with steady arrivals.

For a new agent workload, build the initial execution profile from expected traffic, the representative prototype, or load-test runs. If you’re using Datadog, Agent Observability and APM traces can help you measure the total elapsed time per run and the duration of individual steps at the model, tool, and application level. Serverless Monitoring can measure the execution duration of each serverless component, which contributes to the run’s active compute time. It can also report invocation volume, concurrency, CPU, and memory. If you’re using AWS Lambda, enhanced metrics also report CPU utilization and CPU time. After deployment, update the execution profile with production measurements.

Compose longer or resumable agent runs as durable serverless workflows

An agent run that completes within a single invocation may not require durable orchestration. In those cases, using a function or serverless container avoids unnecessary cost and complexity. If runs span multiple steps or need to pause and resume, a durable serverless workflow can be the optimal choice.

A durable serverless workflow combines components with different roles. For example:

• An orchestrator records progress and decides when each unit is eligible to run.
• Serverless execution primitives, such as functions, serverless containers, or managed jobs, perform the active work.
• External stores retain artifacts and application data that outlives the execution process.

We recommend dividing the agent run into bounded units of work. Each unit of work should have defined inputs, outputs, and completion or failure conditions. After each bounded unit of work finishes or a workflow pauses, the orchestrator can record its progress. The workflow can later resume from the checkpoint instead of restarting the entire run. If a step fails, the orchestrator can retry it without rerunning the steps already completed. Design each step so that a retry doesn’t repeat external side effects, such as sending a duplicate email or charging a customer twice.

To decide where the workflow can pause, map the agent run from trigger to completion and classify each wait as a tool response, rate-limit backoff, or human approval. The workflow can release compute at a wait if it can save its state and resume from a timer, event, or callback. If a step must keep a process running, such as to stream a response or maintain a persistent connection, it can’t pause and may require warm capacity instead.

Separate the agent framework from the workflow’s durable orchestration

Many teams use agent frameworks, such as LangChain or Vercel AI SDK, to accelerate deployment. Datadog’s State of AI Engineering report found that agent framework adoption has nearly doubled from 2025 to 2026. Agent frameworks can vary widely in scope, with some managing model calls, tool selection, and the agent loop, while others support checkpointing or persistent workflow state.

When an external durable orchestrator coordinates the workflow, we recommend that it owns the workflow state, scheduling, and resumption, while the agent framework handles model and tool interactions within each bounded step. This separation of duties allows the framework process to end without losing the workflow’s progress and prevents an in-agent loop from staying alive for the entire workflow.

Compare on-demand execution with warm and dedicated capacity for AI agent workloads

The execution profile can help you compare each workload placement option against the same demand. On-demand costs generally increase with active compute time and workflow activity, while warm and dedicated capacity incur costs while provisioned compute is idle.

When comparing on-demand execution with warm and dedicated capacity, include compute duration, orchestration transitions, state storage, network transfer, and telemetry data. You can exclude model and tool charges if they remain unchanged across placements, but include the charges when retries, failures, or repeated work cause one placement to generate more calls.

Instead of invocation prices, compare cost per successful run. Add the costs generated by every attempted execution during the same operating window, including successful, failed, and abandoned runs. Then divide the total by the number of runs that reached the workload’s completion condition.

Use production evidence to change the placement of your AI agent workload

Once in production, monitor the agent’s behavior as it executes in a serverless workflow. By comparing production metrics with your execution profile, you can identify when the architecture should change to address emerging latency or runtime constraints.

Before you change a step’s placement, confirm the source of the problem. For example, if a trace shows that unnecessary retrieval calls or model turns account for most of the delay, keeping the worker warm won’t address that source of latency. By contrast, if compute startup consumes a substantial amount of the response-time budget, warm capacity can help.

Match each operating signal to a placement response

When reviewing signals, analyze startup latency, active compute time, sustained concurrency, and cost per successful run together. Interpret each signal according to the workload’s response-time objective and relevant capacity or cost limits.

Move only the steps that require a different placement

The operating signals can apply to one step rather than the entire workflow. If startup latency, sustained compute use, or a runtime constraint affects a specific step, move that step to warm or dedicated capacity while keeping the durable orchestration and intermittent steps serverless.

Key takeaways for running AI agents on serverless

Serverless is a strong starting point for agent workloads that use compute intermittently. Durable execution allows workflows to preserve progress during pauses and release compute between steps. Warm or dedicated capacity becomes appropriate when measured latency, sustained utilization, or runtime requirements justify it.

Begin with one representative agent workload and create its execution profile. Use that profile to evaluate serverless placement, then compare production behavior and cost per successful run with the alternatives. When a constraint affects an individual step, change that step’s placement while keeping the remaining workflow serverless.

Full article content is not available for inline reading.

Read the original article →

How GKE networking scales to support high densities of concurrent AI agents

What do you need to orchestrate high-density concurrent AI agents? As we transition from static requests toward autonomous agentic workflows, the need for secure, high-density sandboxes is pushing conventional infrastructure far beyond its architectural breaking point. This transition is driving top-tier research labs to demand infrastructure at unprecedented scale — often up to one million concurrent code-execution sandboxes!

At this scale, the efficiency of your Kubernetes networking infrastructure directly impacts the economic viability of an agentic initiative. In advanced workflows like reinforcement learning (RL), extreme pod churn can mean thousands of isolated sandboxes spinning up and down every minute. To make sure that valuable GPUs and TPUs are spending their time executing models rather than waiting on sandboxes to initialize, you need sub-second provisioning and extreme compute density.

By providing secure, lightweight isolation, Agent Sandbox on GKE is uniquely positioned to power these highly dynamic execution environments. Furthermore, for next-generation workloads with millions of concurrent idle sessions, Agent Substrate on GKE builds on these primitives to deliver even greater density and sub-second state rehydration.

In this post, you will learn how GKE Dataplane V2 optimizes network identity management, solves identity scaling limits, and enables sub-second provisioning to power high-density, large-scale AI agent workloads.

A common scale challenge: network identity management

Operating with massive concurrency introduces fascinating new engineering challenges, particularly in how you manage network identities. Traditionally, custom controllers and workload orchestrators often assign unique, dynamic labels to each pod (such as agent-id, timestamps, or claim-uid). GKE Dataplane V2, powered by eBPF technology, helps enforce robust network security policies by mapping pod labels to a numeric identity. Because agentic workflows generate highly unique labels for every pod, the system proactively provisions a distinct identity for each sandbox. In a rapidly changing environment, this serial allocation process can briefly pause pod creation as the system securely updates eBPF maps, occasionally leading to slower-than-desired initialization. Additionally, sophisticated workloads can approach the cluster-wide limit of 65,536 network identities, either through this rapid generation or overall scale of the workload itself.

GKE network dataplane innovations for an agentic world

Providing a reliable and highly secure platform is fundamental to everything we do. As such, we needed to find a way to provide the same underlying identity and security controls while providing higher scalability and performance. That's where the architectural innovations in GKE Dataplane V2 come in. A customer has testified to the success of Agent Substrate on GKE to orchestrate hundreds of thousands of secure environments across clusters and handle spikes of over 30,000 sandboxes on a single cluster.

To achieve this, GKE relies on a two-pronged networking approach:

Migrating high-cardinality data

Google's GKE Dataplane V2 separates "identifying" data from "functional" metadata. Ensure your fields that do not actively participate in service selector matching or core resource grouping are migrated from the Pod's Labels set to the Annotations set. GKE Dataplane V2 intelligently calculates network identities based on labels rather than annotations, moving this high-cardinality metadata out of the labels completely bypasses the need for per-Pod identity generation. This elegant shift natively resolves identity scaling limits without sacrificing visibility.

Advanced Cilium label filtering

In addition, applying precise label exclusion filters (e.g., using the cilium-config-emergency-override ConfigMap in kube-system) by configuring rules like !agents.x-k8s.io/sandbox-name-hash or !timestamp, these explicitly instruct GKE Dataplane V2 to gracefully ignore these dynamic labels during security identity calculation.

This means that even if a developer framework inadvertently introduces high-cardinality labels, configuring appropriate exclusion filters allows Google’s network dataplane to drop them automatically.

As a result, identical agent types continue sharing a single Cilium pod identity, maintaining the ultra-low-latency startup times critical for dynamic AI workloads. By preventing linear identity growth and avoiding Cilium’s 65,536 identity limit, GKE provides leading speed and efficiency for fully isolated agents with 300 sandboxes per second at sub-second latency.

Looking ahead

For workloads that require extreme performance, scale, and density, Agent Substrate natively integrates these GKE Dataplane V2 recommendations while bypassing Kubernetes scheduling limits, pushing GKE’s performance and scale even further. In benchmark validation running 200,000 agents, Agent Substrate sustained 500 activations per second at less than 500 ms suspend and resume latency. You can deploy it on your existing GKE clusters today to minimize idle compute costs for your most demanding AI workflows.

By optimizing network identity calculation and filtering high-cardinality metadata, GKE Dataplane V2 addresses critical infrastructure bottlenecks in large-scale AI workloads. These architectural improvements reduce sandbox provisioning latency to sub-second levels, minimize expensive GPU idle times, and help maintain high compute density. Ultimately, these efficiencies let you scale your agentic AI operations reliably while controlling operational costs.

To learn more, explore Agent Sandbox and Agent Substrate on GKE.

A voice agent can return the right words and still deliver a poor experience. It might leave a caller in silence, speak over them, or stop midway through a response. A transcript may look correct even when the audio the user heard was delayed, incomplete, or different from the text on screen. These problems are difficult to diagnose from text traces alone.

Voice conversations also involve more than a single request and response. Audio streams, transcripts, model interactions, and tool calls all contribute to each turn. To understand why a conversation failed, teams need to examine those signals together and in the order the user experienced them.

Datadog Agent Observability brings audio into traces for supported voice agents. With these signals in one trace, you can listen to captured audio and connect a user-visible problem to the model or application behavior behind it.

In this post, we’ll show how you can:

• Listen to captured voice turns alongside trace data
• Find slow or high-usage voice interactions
• Investigate interruptions and incomplete transcripts
• Start tracing supported voice agents

Listen to captured voice turns alongside trace data

A transcript tells you what text was recorded, but not always what the caller heard. The agent might have selected the right response but delivered it after an uncomfortable pause. A user might have interrupted the response, or the generated audio might differ from what the transcript suggests. Investigating these cases requires the audio and the surrounding agent activity in one place.

For supported applications built with the OpenAI Realtime API, automatic instrumentation into Agent Observability captures each turn’s input and output audio, transcripts, token usage, and tool calls. Datadog converts raw PCM16 and G.711 audio to WAV so you can play it directly from the trace. Agent Observability also adds tracing support for applications built with Amazon Nova 2 Sonic in the Python and Node.js tracers.

From an affected turn, you can listen to the exchange while inspecting its transcript and tool calls. This helps you determine whether the agent chose the wrong action, returned unexpected content, or was interrupted before it finished speaking.

Audio also preserves details that a transcript cannot, including pauses, overlapping speech, and the point at which a response was interrupted. Viewing those details with the rest of the trace lets you investigate the conversation as the user experienced it instead of treating each model response as an isolated text completion.

Find slow or high-usage voice interactions

Response latency is immediately noticeable in a spoken conversation. A delay that might be tolerable in a text interface can leave a caller waiting in silence. Uneven audio delivery can also make an otherwise correct response difficult to follow.

For OpenAI Realtime applications, Datadog’s automatic instrumentation emits the ml_obs.trace.time_to_first_agent_audio distribution metric, which measures how long a user waits for the agent’s audio response. Charting this metric over time can show whether slow initial responses are isolated or widespread. When percentile aggregations are enabled for the metric, you can also compare values such as p50, p95, and p99.

When the metric shows a latency increase, use the “Average time to first agent audio” facet in Agent Observability to scope the span list to turns with the slowest initial audio responses. You can then open an affected trace and inspect the surrounding turns and tool calls to determine whether the delay is associated with a model interaction, a downstream action, or another part of the conversation.

Voice agents can also have different usage patterns than text-only agents have because providers count audio and text tokens separately. Agent Observability captures token usage from OpenAI Realtime interactions, so you can review high-usage turns alongside latency and other operational signals. Opening the associated traces helps you determine whether higher usage is connected to longer conversations, particular workflows, agent routing, or external tool calls.

Investigate interruptions and incomplete transcripts

Interruptions are part of natural conversation, but repeated interruptions can also point to a problem. An agent might give answers that are too long or that begin with irrelevant information, or it might rely on voice activity detection that mistakes a pause for the end of the user’s turn.

Voice traces preserve the sequence of events around these interactions. When a user interrupts an OpenAI Realtime response, you can open the corresponding turn and listen to what the agent was saying at that moment. The surrounding context helps you distinguish normal turn-taking from a pattern that warrants further investigation.

Audio context is also important when you review transcripts. OpenAI Realtime input transcription runs separately from the model’s processing of the audio, and an output transcript can be incomplete when a user interrupts a response. Comparing the transcript with the corresponding audio helps prevent an incomplete transcript from being treated as a complete record of the interaction.

Across traces, these signals help you identify conversations that need closer inspection and determine whether a problem originated in the model response, tool execution, turn-taking behavior, or another part of the application.

Start tracing supported voice agents

Datadog provides automatic multimodal instrumentation for supported voice applications through the Agent Observability SDK. Python applications that use the OpenAI Realtime API with ddtrace 4.13.0 or later can automatically capture per-turn input and output audio, transcripts, token usage, and tool calls without manually constructing audio spans. ElevenLabs Realtime API support will be available soon with ddtrace 4.17.0.

Agent Observability also adds tracing support for applications built with Amazon Nova 2 Sonic in the Python and Node.js tracers. To configure Agent Observability, follow the Agent Observability instrumentation documentation. For supported audio formats, SDK versions, automatic OpenAI Realtime capture, and media size limits, see the multimodal instrumentation documentation.

Applications that already emit spans that comply with the OpenTelemetry GenAI semantic conventions (version 1.37 or later) can also send supported audio message parts to Agent Observability. Datadog’s SDK provides the automatic OpenAI Realtime instrumentation described in this post.

Bring voice context into your agent investigations

Agent Observability connects the audio a user experiences with the model, tools, latency, and token usage behind the conversation. Instead of reconstructing an exchange from separate transcripts, metrics, and logs, you can move from a latency signal or reported problem to the specific turn and trace that produced it.

To get started, read the Agent Observability documentation and the multimodal instrumentation documentation. If you’re new to Datadog, start with Agent Observability for free. The free plan includes up to 40,000 LLM spans per month.

OpenTelemetry eBPF Instrumentation reaches v1.0.0

We’re excited to announce v1.0.0 of OpenTelemetry eBPF Instrumentation—OBI!

OBI brings OpenTelemetry observability to running Linux applications without requiring changes to their source code. It uses eBPF, together with runtime-specific instrumentation, to capture application metrics and traces across services, languages, and frameworks. You can start with request rates, errors, latency, and service dependencies, then build deeper visibility as your needs grow.

This release fulfills the flagship goal we set for 2026: establishing a stable foundation for OBI. It represents a year of expanding instrumentation coverage, improving correctness, strengthening release tooling, and building the compatibility and telemetry migration practices that users need to operate OBI over time.

For users, the most important change is predictability: knowing what to expect from an upgrade and how to move forward when something changes.

OBI 1.0 establishes documented compatibility commitments for configuration and deployment, including Config v2, its generated schema, documented deployment interfaces, and supported environments.

For telemetry, the commitment is a reliable migration path as OBI and OpenTelemetry evolve. OBI emits a mixture of telemetry that follows stable semantic conventions and telemetry whose conventions are still developing. The latter can change as those conventions mature. We will also correct bugs in telemetry that follows stable conventions when its actual behavior does not conform to them.

To make that evolution manageable, OBI describes its telemetry in a Weaver-compatible schema registry. OpenTelemetry Weaver helps us define, validate, and review that telemetry against the conventions it should follow. Published telemetry schemas describe transformations between versions where those changes can be represented, while release notes and migration guidance explain changes that need additional action.

For teams maintaining dashboards, alerts, and processing pipelines, this provides a way to plan upgrades and adapt deliberately. The promise is a supported path forward as telemetry evolves, with the definitions, tooling, and guidance needed to understand and manage that change.

That foundation comes with substantial progress across the project.

• Broader visibility into application dependencies. Across the releases leading to 1.0, OBI added instrumentation for systems including MQTT, NATS, AMQP 1.0, MSSQL, Couchbase, Memcached, and Aerospike. For teams operating mixed technology stacks, that means more opportunities to understand database and messaging activity through one instrumentation approach. The support matrix documents coverage, trace propagation capabilities, and limitations for each protocol.
• Better traces and more useful service detail. Improvements to HTTP/2 and gRPC propagation strengthened handling of existing connections, multiplexed traffic, and individual stream contexts. Framework route discovery expanded to Django, FastAPI, Flask, Rails, .NET, and several PHP frameworks. Route templates help teams distinguish slow endpoints without turning every unique URL into a separate metric series.
• A path from automatic instrumentation to application-specific insight. OBI can capture application-authored spans through supported OpenTelemetry Go Trace API and Auto SDK configurations. Node.js also gained opt-in manual span capture for applications without a registered SDK. These capabilities let developers add detail about business operations while using OBI for the surrounding automatic instrumentation, subject to the documented runtime requirements.
• More context for investigating incidents. Trace-log correlation can enrich supported application logs with trace context, helping teams move from a request’s path to the logs associated with it. Mattia Meleleo’s walkthrough of zero-code trace-log correlation shows it working across two services and explains how to enable it. OBI also added runtime metrics across Go, Java, .NET, Node.js, and Python, alongside network measurements such as TCP retransmissions and failed connections. Runtime metrics remain experimental, with availability depending on the runtime and enabled features.
• Instrumentation for AI applications. OBI’s coverage expanded to model providers, embeddings, reranking, vector retrieval, and MCP interactions. The work includes tool-call and retrieval instrumentation, along with OpenAI-compatible gateways and native Ollama support. These capabilities help teams investigate AI dependencies alongside the rest of their application. Detailed extraction requires explicit payload-capture and feature configuration; it is not enabled automatically for every deployment.

Much of the work behind 1.0 is less visible than a new protocol, but essential to operating instrumentation. Contributors hardened parsers and corrected lifecycle and export behavior, addressed buffer-safety and connection-disruption bugs, and improved trace-parent relationships. Release tooling added signed artifacts, checksums, and software bills of materials, while validation expanded across kernels and architectures. These investments help users evaluate and operate the software they deploy.

For existing users, Config v2 is the configuration contract for 1.0. Available since v0.11.0, it includes a generated schema and the obi config validate and obi config migrate commands. Config v1 remains supported for backward compatibility, but is frozen and receives no fixes or new features. Follow the configuration migration guide, review configuration supplied through environment variables, and compare telemetry in a canary deployment before rolling out broadly. Users embedding OBI in a Collector should also review the receiver’s separate v0 release line when selecting versions.

If you’re new to OBI, start with a representative service on a supported Linux host or Kubernetes cluster. Export to your OpenTelemetry Collector or compatible backend, inspect the resulting metrics and traces, and expand from there. Language agents and manual instrumentation remain valuable when you need application-specific context beyond what OBI can observe automatically.

OBI began with the donation of Grafana Beyla and reached its first alpha release through collaboration across organizations and the wider OpenTelemetry community. That collaboration has carried the project to 1.0.

We want to give special thanks to contributors whose sustained work helped make this milestone possible:

• Nikola Grcevski and Mario Macias, for developing OBI’s instrumentation foundation and extending its runtime support, service discovery, framework integration, and configuration.
• Nimrod Avni, for configuration schema generation, Collector integration, protocol coverage, and the telemetry registry and Weaver validation work.
• Mattia Meleleo, for trace-log correlation, trace-context handling, Go instrumentation improvements, and release preparation.
• Rafael Roquetto, for eBPF infrastructure, buffer handling, parser hardening, and performance improvements.
• Giuseppe Ognibene, for network observability, BPF verifier testing, runtime metrics, and instrumentation and exporter correctness.
• Stephen Lang, for strengthening CI, integration tests, and ARM coverage, alongside application instrumentation improvements.
• Marc Tudurí, for expanding protocol instrumentation, Python asynchronous tracing, and Python and .NET runtime metrics.
• Haibin Zhang, for expanding GenAI instrumentation, including embeddings, reranking, Qwen, and MCP.

Thank you, too, to every community member who contributed code, reviewed changes, improved documentation, reported a bug, or shared experience running OBI. Reproductions, deployment feedback, and questions about unexpected behavior have all helped us build a better project. This release belongs to everyone who helped get it here.

With 1.0 released, we’re turning that momentum toward planning OBI’s work for 2027—and we want you to help shape it.

If there’s an issue you’ve been following, a protocol or runtime you want supported, or a feature that would make OBI more useful in your environment, tell us. Add your use case to an existing issue, open a feature request, or bring your ideas to a SIG eBPF Instrumentation meeting. Tell us what you’re trying to accomplish, what gets in your way today, and what a successful solution would enable.

Try OBI 1.0, share your experience, and help us choose what to build next.

DuckDB 2.0 adds client-server and AI features, while Polars improves streaming and processing of larger-than-memory datasets. On a 23 GB dataset with 2 GB RAM, Polars 2.0 ran out of memory, but 2.1 finished in 438 seconds, nearly twice as fast as DuckDB's 837 seconds.

StarTree adapts Pinot's HNSW vector index for Iceberg tables, keeping Parquet source records in object storage while storing and serving a derived index alongside them. This enables hybrid similarity and SQL-filtered queries without a second copy of the records. The trade-off: indexes are roughly as large as the embeddings, and freshness follows Iceberg snapshot synchronization rather than real-time ingestion.

Full article content is not available for inline reading.

Read the original article →

700 MB/s of Kafka throughput, on Postgres

Kafgres is a Postgres extension that embeds a singleton broker directly inside Postgres, allowing both deep integration with DB transactions, as well as allowing external systems to interact with the database, just as they would with real Kafka.

In a previous article introducing Kafgres, I had mentioned that Kafka becomes the suitable choice for demanding architectures of 100K+ evt/s. However, I knew that in theory, Kafgres should be able to handle this load easily, considering it has to do a fraction of the work of a Kafka cluster doing the same throughput: Postgres-style replication is used, rather than a quorum. With this in mind, I rented a large box from Hetzner, and got to work profiling and poking around the extension to drive up data throughput and records per second.

On an i9 box with PLP-enabled NVME drives (~$0.25/hr via Hetzner server auction), I was able to achieve smooth write throughput of ~700 MB/s, and 600k evt/s.

How it works

Kafgres lives inside Postgres, in that it is installed in the DB as a Postgres extension, and largely uses Postgres's extension APIs to interact with the OS. However, while taking advantage of a lot of the machinery Postgres provides (to avoid having to write our own WAL for metadata, or completely reimplement leader/follower failover), Kafgres largely works independently of the rest of the database. The vast majority of the Kafgres work is handled by an independent Postgres-spawned background worker, and under default configuration, actually writes topic bytes directly to disk, skipping Postgres internals and overhead for the vast majority of data. This allows us to achieve theoretical performance quite close to that of a regular Kafka singleton running on the same machine.

Identifying the bottleneck

On the i9 box (and for a few of the profiling rounds, a cheaper i7 box), I drove multiple producers against Kafgres 0.1.0. At a throughput of 100-200MB/s this began to stall the engine (identified by a latency spike). The CPU core for Kafgres was nearing 100%, suggesting the stalling was not from blocking on the drive, but by CPU time.

We benchmarked the drive on this box, which achieved a throughput of ~1911MB/s, and an fsync rate of ~68K/s. Meanwhile, our broker was performing less than 600 fsyncs/s, which confirmed there was a large amount of drive-write headroom.

Profiling

In a previous blog, I have detailed profiling a previous Postgres extension to improve throughput. I used a similar technique to profile the saturated Kafgres broker.

For the on-cpu time (the dominating bottleneck), 40.7% of our time was spent in parse/analyze/plan, which is quite a lot considering only metadata is carried in tables, and topic data gets written directly to disk.

For off-CPU, roughly a third of blocked time was spent in commit. This was an issue on the weaker machine I used, with drives with no power-loss-protection, but was less of an issue on the i9 box.

Pulling the levers

The first change I made was to attack the nearly half of CPU time spent repeating the same, cachable work. A relatively simple change was to use SPI_keepplan for the metadata statements, allowing Postgres to maintain a generic plan for our statements, to reuse rather than to plan per-execution.

-    Spi::get_one_with_args::<i64>(
+    crate::plan::get_one::<i64>(
         "SELECT log_start_offset FROM kafgres_partitions
           WHERE topic_id = $1::oid AND partition = $2",
         &[topic.into(), partition.into()],
     )

The next change I made was to tackle the commit time. Technically, with the configuration we were testing, non-transactional produce did not guarantee the actual topic data had been flushed to drive by the time it acked. However, we were guaranteeing offsets had been synchronously committed. Taking a similar default-relaxed posture towards the offsets would allow more metadata to accumulate in the WAL before flushing to disk. Introducing the GUC relaxed_produce_commit allows this to be turned on/off, and improves throughput.

pub fn relax_commit_durability() -> Result<(), pgrx::spi::Error> {
    pgrx::Spi::run("SET LOCAL synchronous_commit = off")
}

An additional change we made was not to optimize throughput per se, but to decrease impact on regular Postgres usage. Reaching these high throughputs is less appealing when it's at the cost of your database performance. The i9 machine we were testing on had 2 NVMEs, so we added a config field to change the directory that kafgres topic data is written to. When that is moved to a different drive than the rest of the database, a throughput of 100 MB/s has very little (~1-5%) impact on the performance of pgbench running against the same database.

fn log_root() -> PathBuf {
    match crate::log_directory() {
        Some(dir) => {
            let p = PathBuf::from(dir);
            if p.is_absolute() { p } else { PathBuf::from(data_directory()).join(p) }
        }
        None => PathBuf::from(data_directory()).join(LOG_DIR),
    }
}

These changes (alongside a few others, not worth mentioning in depth) all modestly improved throughput, in total from 113 MB/s -> 197 MB/s.

It was at this point I noticed there was a performance disparity between running the benchmark over docker proxy and hitting the Kafgres sockets directly via host networking. This led me to look into how I was handling work queuing up, and socket readiness. The naive approach opted for in 0.1.0 was to have a 5ms tick-loop, allowing bytes to accumulate during this 5ms of blocked time, and waking the background worker on a schedule to handle the bytes that had accumulated.

The correct way to do this, as we move on from version 0.1.0 of Kafgres, was to use Postgres's own WaitEventSet. This lets us block not until some period of time has passed, but instead until some socket is ready for us to start work.

for &(fd, _) in want {
    pg_sys::AddWaitEventToSet(self.set, pg_sys::WL_SOCKET_READABLE,
        fd, null_mut(), null_mut());
}

-// epoll set holds the latch self-pipe and postmaster-death pipe. Nothing else.
-// Every request waits for the next timer edge.
-while BackgroundWorker::wait_latch(Some(tick)) {
+// Every client socket is in the wait set; `tick` is now a ceiling, not a cadence.
+while readiness.wait(tick) {
+    readiness.sync(listener_fd, &conn_fds);

This change, when we use host networking rather than the docker network, is responsible for the vast majority of the improvement in throughput.

With record sizes of 256 KiB, we have achieved a stable, low-latency throughput of 700 MB/s produce.

In comparison

With these major changes, alongside a handful of other performance tunings that landed, Kafgres now holds its own against vendor Kafka.

According to many SaaS Kafka vendors, the vast majority of Kafka workloads are small, estimated at 56% having <= 1 MB/s throughput. Those normally scaling into hundreds of MB/s are mostly large enterprises, who can have a dedicated team for managing Kafka, or can pay fees to vendors to have their Kafka managed for them.

On AWS MSK, a regular, 3-node (considered the floor to most) Kafka cluster that could handle ~700 MB/s would likely be made up of kafka.m5.8xlarge nodes, coming in at $3.36/hr each. This comes out to $7,357 monthly in broker-compute, before accounting for storage and data transfer costs, which at usual AWS rates would cost tens of thousands more ($18,000/mo in storage for just 24 hr retention for this data volume).

Of course, where MSK (or really, any other vendor) wins is replication. Kafgres can failover to a secondary database, just like the Postgres DB itself, but cannot seamlessly shrug off a zonal outage in the way a 3-zone Kafka cluster can. Additionally, if one chooses to host their Postgres deployment in the cloud, many of the costs incurred for MSK (storage, networking) would still be incurred for the Postgres instance.

Regardless, this strongly suggests that for the vast majority of use cases, the Postgres-resident Kafgres is sufficient. By the time you hit Kafgres's limits, an equivalent vendor-provided "real Kafka" deployment will cost hundreds of thousands per year.

Kafgres 0.2.0 has been released, which vastly closes the gap between "real" Kafka 4.0 and Kafgres.

Nodal Analytics

Agent skills for building governed analytics context, planning data questions, and verifying the answers.

Analytics agents can write valid SQL and still give the wrong answer. They need the definitions an experienced analyst carries around: what a metric means, which table is canonical, which filters are mandatory, where joins fan out, and when a result should be escalated instead of trusted.

Nodal turns that knowledge into reviewable context and reusable agent workflows. An analyst remains the authority: the agent drafts choices where it finds uncertainty; the analyst chooses the correct interpretation, then the agent verifies, records, and evaluates the context.

Who Nodal is for today

Nodal is currently best suited to teams that already have:

• a queryable analytics warehouse or database;
• an analyst or other domain expert who owns business definitions; and
• a supported local agent that can receive approved, read-only access to the warehouse.

The person operating this open-source package should be comfortable working in a local agent project and coordinating data access. They do not need to write Python, and the domain expert who confirms definitions does not need to know dbt or use a command line. dbt, historical query access, and BI dashboards improve the evidence available to Nodal, but they are not all required.

Large organizations are in scope. Nodal is designed to roll out one analytics domain at a time: establish the domain's definitions, failure cases, and eval seeds; verify that its governed answers work; then expand to the next domain. This creates a reviewable governance boundary at every step instead of requiring a company-wide semantic migration before the first result.

Nodal is not yet optimized for organizations without a warehouse or technical data owner. We are interested in extending it both upstream to more complex enterprise environments and downstream to earlier-stage teams, while keeping the current domain-by-domain validation model.

Two ways to start

• Use your own stack. Start with one bounded business domain where a domain expert can confirm the definitions. Prove the workflow there before adding another domain.
• Look at an example company. See Shorelane Commerce, a fictional company with a public warehouse and deliberately ambiguous revenue definitions.

Let your agent guide installation

If you already use an AI coding agent, paste this prompt into a project where you want to evaluate Nodal:

Help me understand and set up Nodal Analytics for this project. Read
https://github.com/nodal-data/nodal-context/blob/main/docs/agent-guide.md
first. Explain what will change, choose only one supported installation method,
and ask before installing anything or editing configuration. Then walk me
through it one step at a time.

Where your data flows: nowhere new

Nodal's open-source package is a set of skills—instructions and local scripts that run inside your own agent, on your own machine. There is no Nodal service in the loop:

• No data reaches Nodal. Schema, query results, query history, dashboard captures, and the generated context all stay on your machine and in repositories you control. The package makes no network calls to Nodal and contains no telemetry.
• No credentials reach Nodal. Warehouse access happens through an MCP server you configure and authenticate yourself, using a read-only identity.
• Your existing agent agreement is the only data flow. Whatever your agent reads while answering a question is processed by the model provider behind your agent host—for example, Anthropic when you run on Claude Code—exactly as in any other use of that agent. Nodal adds no new data recipient.

What you need before the first interview

Nodal never handles warehouse credentials or sees your data. Connect your agent to the warehouse through an approved MCP server using a read-only identity.

For the best initial context build, prepare:

• A domain expert. An analyst or other owner must confirm definitions. Nodal does not silently promote generated documentation to truth.
• Read-only warehouse access. The connection must support SELECT and metadata inspection. Nodal does not run DDL, DML, grants, or procedures.
• Historical query-log visibility. Full Stage 0 discovery needs access to query history across the relevant people, BI service accounts, and workloads.
• Your dbt project, if you use dbt. A local sibling checkout is recommended so the interview can draft from real models, tests, and metric definitions.

The workflows

The skills are small and composable. Use one directly, or let the agent route to the appropriate workflow.

Build and maintain context

• setup-nodal configures project-local context sources, read-only warehouse capabilities, and optional browser access.
• context-interview interviews an analyst to build or improve an Analytics Context Format (ACF) repository.
• analyst-handoff captures critical domain knowledge when an analyst changes roles or transfers ownership.

Ask and verify

• analytics-plan translates a business question into a reviewable plan before any SQL runs.
• verify-result checks executed SQL and results against the approved plan for metric, filter, grain, join, and time fidelity.
• challenge-result gives a completed answer a skeptical second review when the user questions it or asks for another take.
• dashboard-verify reads a named BI dashboard in the user's local browser, capturing both values and active filters for reconciliation.

Why these skills exist

1. The agent knows the schema but not the business

Warehouses encode what can be queried, not what the company means by “active customer,” “revenue,” or “conversion.” Nodal interviews the people who own those definitions and stores the result in files the team can review by pull request.

2. Auto-generated context repeats existing ambiguity

Raw schemas, prior SQL, and BI metadata are useful evidence, but they are not authority. Nodal uses schema, dbt, dashboards, and query history to create drafts and surface conflicts. A human decides which interpretation is correct.

3. A successful query is not a verified answer

SQL can execute successfully at the wrong grain, omit a mandatory filter, use a noncanonical metric, or fan out through a join. analytics-plan makes intent explicit before execution; verify-result checks whether the query and result actually satisfy that intent afterward.

License

Apache-2.0. The format and the interview-built context are yours to keep.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Picture this: you are running automated daily Machine Learning pipelines at scale, and it is time for a critical environment upgrade, such as moving to a new Python version. Consequently, you need to upgrade your Databricks Runtime from LTS 14.3 to LTS 17.3. However, the upgrade cannot proceed because the logs show a cryptic error:

TypeError: code() argument 13 must be str, not int.

This was the exact challenge our team faced recently during a Databricks upgrade that uses MLflow. At our core, we manage daily automated pipelines spanning data preparation, training, and inference across separate Dev, Staging, and Prod Databricks workspaces managed via Declarative Automation Bundles (DAB).

This post covers our journey identifying the root cause of Python version serialization failures, our "Load and Repack" technique to recover production models, and the architectural refinements we made to ensure our MLflow artifacts are forward-compatible.

When cloudpickle meets a Python upgrade

Our upgrade path was clear: transition our machine learning workflows from DBR 14.3 (Python 3.10) to DBR 17.3 (Python 3.12). Python 3.12 introduces excellent language features, such as PEP 695's simplified generic types. This syntax is fully embraced by Pydantic V2 to enforce data validity at runtime using type hints, which prompted us to implement more comprehensive type hints across our codebase.

However, modernizing the runtime introduced a breaking change for our legacy machine learning models in the registry. The root cause is how MLflow traditionally serializes Python models.

When MLflow logs a Python model, it uses cloudpickle to freeze the entire wrapper object, including inner models like CatBoost classifiers or PyTorch modules, into a single serialized artifact. Crucially, this process embeds Python version metadata and bytecode directly into that file. Because bytecode structures shifted significantly between Python 3.10 and 3.12, the new interpreter could no longer safely parse the older pickle files. This resulted in an immediate TypeError whenever the upgraded inference logic attempted to pull an existing model from the registry.

Mapping our ML landscape

To understand why a naive approach like "just retrain all the models" was not viable, consider the scale we operate at: our product classification system alone uses multiple TB of training data.

In our case, that system processes and classifies product data across a large and continuously growing catalogue. For every active product category, our system trains and runs five distinct model types in parallel:

• CatBoost: a tree-based classifier.
• TF-IDF Regression: a logistic regression model operating on cleaned raw text.
• Neural Network (NN): a PyTorch Lightning model that processes dense vector representations from all-mpnet-base-v2, an open-source sentence embedding model.
• Image Model: another PyTorch Lightning classifier that processes dense image embeddings, trained only for image-eligible categories and integrated into the voting ensemble via late fusion with a configurable weight parameter.
• Voting Ensemble: a final logistic regression model that aggregates the predictions of the base models into a single, optimized binary tag assignment.

Our infrastructure uses Databricks to train these category models concurrently. Heavy NN, Image Model and CatBoost tasks run on dedicated GPU clusters, while TF-IDF and Voting tasks run on CPU clusters. Models are governed inside the Databricks Unity Catalog using secure Service Principals and strictly named aliases (like @Production) rather than numeric version tags.

Retraining this entire matrix across all environments and categories from scratch would have consumed immense time and compute resources.

The "Load and Repack" strategy

Bypassing the costly retraining phase, we developed a methodology for migrating and relogging our models to ensure operational continuity. We orchestrated this recovery across two foundational stages.

Stage 1: Load

We created a temporary, one-time migration job using an environment that matched DBR 14.3. By installing the old libraries directly into this environment, we avoided version conflicts. This allowed the system to easily read the problematic pickle files and separate the raw model weights from their restrictive pyfunc wrappers.

Stage 2: Repack

With the models loaded, we unwrapped the inner artifacts from the old pyfunc wrapper. We encapsulated the loaded model artifacts within a streamlined, file-based wrapper class in our model_wrappers.py module. This architectural shift prioritized pure Python logic, removing dependencies on outdated pickle structures to ensure native compatibility with Python 3.12.

By relogging this refined wrapper to MLflow with dependencies managed via uv, we preserved the trained artifacts while promoting the upgraded model back to the @Production alias.

Each of our model types originally used different serialization patterns, so our migration tool applied specialized extraction strategies per model type. With new dummy DataFrame inputs, we could also infer fresh MLflow model signatures.

Future-proofing: long-term architectural wins

While our migration tool was designed to run only once per environment, solving this immediate blocker yielded two architectural improvements for our production ML systems going forward.

Standardized wrapper interfaces

Previously, models were logged using framework-native MLflow flavors with ad-hoc wrapper logic. We introduced a dedicated model_wrappers.py file, establishing consistent mlflow.pyfunc.PythonModel subclasses (e.g. CatBoostModelWrapper, NNModelWrapper). This ensures a uniform predict() interface regardless of the underlying model type. Every model developed post-migration natively adopts this pattern.

Self-contained model artifacts

In our legacy workflow, MLflow artifacts lacked embedded code dependencies and explicit manifest files. To fix this, we pinned package versions explicitly and included model_wrappers.py directly via code_paths at log time. We have since adopted uv for dependency management. Since MLflow 3.11.1, MLflow auto-detects uv.lock and pyproject.toml and exports a precise, reproducible set of requirements at log time via uv export.

By embedding specialized dependencies and internal deserialization logic directly within the artifact, every logged model is now fully autonomous. This architectural shift ensures that future environment transitions or runtime upgrades proceed smoothly, eliminating the need for costly retraining or manual recovery efforts.

Next iteration: eliminating pickle

While our "Load and Repack" strategy solved the immediate DBR upgrade issue, encapsulating models inside custom PythonModel subclasses still relies on cloudpickle to serialize the wrapper object itself. To ensure we never face a version boundary issue again, we are evolving our architecture to eliminate pickle entirely through two advanced MLflow patterns:

Native serialization formats

Instead of wrapping inner models, we can migrate to MLflow's native serialization flavors, which avoid cloudpickle:

• mlflow.catboost.log_model() saves directly to CatBoost's native .cbm format
• mlflow.pytorch.log_model(..., serialization_format="pt2") utilizes torch.export.save (available in PyTorch ≥ 2.4)
• mlflow.sklearn.log_model(..., serialization_format="skops") secures TF-IDF and Voting ensemble models natively

Adopting "Models from Code"

To maintain our unified predict() interface without pickling the wrapper, we can transition to MLflow's Models from Code concept. Rather than logging an instantiated object, MLflow stores the wrapper as a raw Python source file. At runtime, the wrapper script uses the load_context() method to fetch the native model artifacts (like the .cbm file). Because the wrapper is executed as pure source code and the weights are native, the entire artifact becomes independent from future Python bytecode changes.

Lessons learned

Coordinate library shifts early. Upgrading a major runtime means foundational open-source libraries change under the hood as well. Always align early with your Applied Scientists to review respective package lists before scheduling data engineering upgrades.

Prioritize rigorous dependency management. Establish a disciplined workflow when configuring your environments. By thoroughly reviewing LTS documentation and modernizing your manifests, such as updating pyproject.toml and running uv sync --upgrade, you ensure that your Python, PySpark, and Delta Lake layers remain harmonized and resilient against system-wide changes.

Embrace native formats over pickles. Whenever possible, utilize native framework serializers (like .cbm or pt2) and avoid serializing Python objects. The closer your artifacts remain to pure configuration and native weights, the easier your next runtime upgrade will be.

By pivoting from a disruptive serialization bottleneck toward a unified architectural standard, future runtime transitions are now engineered to be as seamless as a simple version increment.

Your backups are green. That does not mean you can restore.

Most teams I work with have a backup dashboard, and most days it is green. The job ran; it finished; nothing failed. That green tick feels like safety, and it is the reason nobody thinks about backups again until the day they need one. Then something breaks, someone asks how long it will take to get the database back, and the room goes quiet. Nobody knows. The backups have been running for a year, and nobody has ever brought one back.

I am writing this because that gap, the space between having backups and being able to recover, is the most common serious problem I find in production PostgreSQL, and hardly anyone is measuring it.

A Monday that goes badly

Here is the shape of it. I have put this timeline together from the kind of incident customers keep running into, so treat it as a pattern rather than one company’s story.

• At 09:38, a new feature is rolled out. One statement drops a column that still holds live data.
• At 09:44, support tickets start coming in because customer records look incomplete.
• At 10:05, after trying a few smaller fixes, the team decides to restore to just before 09:38.

The backups have been green for fourteen months, so everyone is fairly relaxed.

• At 10:20, they find out there is no restore host. Nobody had ever needed one. Someone starts building it.
• At 13:50, the restore finishes and recovery stops earlier than it should, because a WAL segment from months ago is missing from the archive.
• At 16:30, they give up on the exact point in time, recover to a copy that is six days old, and spend the next two days putting the difference back by hand.

The business had expected this to take about an hour, with no visible data loss. Instead, the restore took more than two days, and data loss was certain.

What a green backup actually tells you

A successful backup job is not a lie. It is just a much smaller claim than people think.

It tells you a process started and finished without an error. It tells you bytes were read from one place and written to another. It tells you the backup catalog has a new entry, and that a retention rule ran. All useful. None of it is about recovery.

Here is what it does not tell you. It does not tell you whether all the files outside the data directory that a restore needs are actually in the backup. It does not tell you the WAL needed to bring that backup to a consistent state still exists. It does not tell you the pages inside are healthy. It does not tell you that you can stop at a chosen moment in the past. And it says nothing at all about whether any of this fits in the time you have promised somebody.

Analogy: The fire extinguisher on the wall

There is probably a fire extinguisher near you. The pressure gauge is green. It has an inspection tag signed every month for years. Nobody has ever pulled the pin.

The gauge measures pressure. It cannot tell you whether the nozzle is blocked, whether the powder has gone solid, or whether the person nearest the fire knows how to use it. Your backup dashboard is that gauge. A restore test is pulling the pin.

A backup is only useful if you know you can actually restore it when you need it.

Ways a green backup fails when you need it

These are the ones customers hit most often.

• The WAL chain is broken. Archiving quietly failed for a week. The base backup is perfectly fine. The path from it to the present is not.
• Corruption was copied faithfully. A damaged page has been in every backup for months, and your backup tool did its job correctly every time by copying it.
• Retention does not line up. The WAL expired before the backup it belongs to, or a full backup expired while its increments are still sitting there.
• The surroundings are missing. Roles, configuration files, extensions, tablespaces, certificates. None of that lives in the data directory.
• The key is gone. The backup repository is encrypted, and the only copy of the key lives in a system that is also part of the outage.
• There is nowhere to restore to. No machine with the disk, memory, and network to take two terabytes at short notice.
• There is nobody who can. The engineer who built it has left, and the runbook is a wiki page on a server that is down.

The corruption that gets backed up perfectly

Picture a cluster that was created years ago with data checksums turned off, and has since moved across two generations of storage. One day a report that only runs every few months starts failing. A single page has been damaged for a long time, and nothing noticed because the normal workload never reads that part of the table.

Now check the backups. Every one of them copied the damaged page correctly. The oldest backup still inside the retention window already contains it. There is no clean copy left to go back to.

This is why I push people to think about retention differently. Most people read thirty days of retention as thirty days of protection. Against corruption, it is the opposite. It is thirty days to notice, after which your last clean copy is gone, and the damage is in every copy you hold.

The archive gap nobody was watching

This one is common enough that I look for it first in every assessment.

Archiving to object storage is set up and working. One day a credential gets rotated, and archiving starts failing.

It fails silently because a broken archive does not stop the database and does not stop the backup job.

PostgreSQL just keeps the segments and retries. The first thing anyone sees is disk pressure on pg_wal, and the natural reaction to disk pressure is to free space, which is the one action that turns a temporary problem into a permanent issue.

The alerting was watching the backup job. Archiving is a different mechanism with its own failure mode, and nothing was watching it. A week or two later, the dashboard is green again, and nobody knows there is a stretch of time you can no longer recover to.

Watch three things separately: the archive failure count, the age of the newest successfully archived segment, and the size of pg_wal.

There is only one test that proves anything

Restore the files. Start the cluster. Replay to a chosen point in the past. Then prove the data is correct. Everything else is a stand-in for that, and stand-ins drift.

Where to start

If you only do a few things after reading this, do these.

Check that your archive chain is monitored separately from your backup job. Confirm your retention window covers what you promised, for backups and WAL together. Write down everything you would need that is not in the data directory, and where it is kept. Then pick one database and restore it to a timestamp from last week, on a clean machine, with somebody else at the keyboard.

The goal is not to feel confident about your backups. It is to be able to show somebody why you are confident, and to know the date it was last true.

Affinity’s new all-in-one creative app is finally coming to the iPad, nearly a year after Canva launched the redesigned desktop version for Mac and Windows. Creatives have been asking for the iPad release ever since, especially because files created in the new Affinity app cannot be opened in the older V2 iPad apps.

Affinity already has Photo, Designer, and Publisher apps on iPad, but the new release combines those tools into the same unified app that Canva launched on desktop last year.

We were going to count how many times you asked about iPad, but we ran out of numbers somewhere around “iPAAAAAAAAAAAAAAD”. The truth? We were listening and we wanted to get it right 💚 Pre-orders for the brand new Affinity for iPad are open.

Affinity makes a lot of sense on iPad

The iPad is already a natural device for drawing and visual work because you can work directly on the canvas with an Apple Pencil instead of using a mouse and separate drawing tablet. Affinity takes advantage of that by supporting pressure, tilt, and barrel roll, so an idea can start as a sketch and then be refined into proper vector artwork in the same file.

Affinity says the iPad app can also handle RAW photo editing, detailed vector work, typography, and print-ready page layouts. For a designer, this opens the possibility of taking a project from the first drawing through to finished artwork without moving back to a Mac or PC just to access the full set of tools. It also supports PSD, AI, INDD, IDML, PDF, SVG, and other common creative formats, making it easier to bring existing professional work onto the tablet.

The app is free despite the “pre-order” label

Affinity for iPad is free for individuals. Its core photo, vector, layout, and sketching tools do not require a subscription, although Canva AI Studio features require a premium Canva plan. The software will be available to download from November 3, 2026.

It’s the last day of Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories.

On the morning of January 13, 2018, residents and tourists in Hawaii saw a frightening message on their phone screens:

In addition to being sent to 1.4 million people, a version of the alert also interrupted local TV and radio programs.

Quickly, the officials realized that something went wrong and seeked to undo what ended up being a false alarm. The siren warning system that would normally follow wasn’t activated, and various channels were used to inform people to ignore the alarming message – however, a second phone alert to follow the original one wasn’t sent out for 38 more minutes:

In the meantime, many Hawaii residents sought shelter, flooded 911 lines, and called their families to say good-bye.

What was the reason for the mistake? Industry veteran Jared Spool wrote a post explaining it in detail. We are not 100% sure, but the interface to deploy alerts likely looked like these two examples:

First, it seems like it was too easy to send a message that wasn’t a test. There was no friction for the consequence this heavy – not even a dialog box to click through, let alone something that more deliberately slows the user down, perhaps even spelling out the consequences (“you are about to send an all-caps message to 1.4 million people”), and generally feeling heavier and more cumbersome to complete than drills or local messages normally sent out.

But the true root of the problem happened earlier than that. Spool:

This wasn’t a thoughtfully designed menu, any more than any random collection of user-named objects are.

In the menu, it wasn’t just that the ordering felt completely random and the capitalization and labelling inconsistent, so neither could be used to help guide the user (for example, by putting drills above real messages, or by using all caps only for most serious, non-test commands). It was also the fact that PACOM and DRILL - PACOM were really hard to distinguish from one another.

In the end, it wasn’t just about stress and panic that engulfed the Hawaii islands for a better part of a morning hour. It was also about undermining the trust in the entire system, as any “text message that cried wolf” will see people become desensitized, or even opt out.

You might never be tasked with creating an interface to do this kind of stuff, but there are some universal lessons in here about information architecture, labelling, and friction.

At some point when I was at Medium, I fought with this kind of a thing, where three similar words to popular destinations appeared in the menu right next to each other:

And today, the general industry flattening of visual style leads to an occasional conflation like this one:

The consequences of this design doppelganging won’t be nearly as severe as in Hawaii, but if you’ve read Unsung even for just a bit, you will know what is my general point of view on this: we are better equipped to understand big events like the one in 2018, but it’s harder to see and impossible to tally up the frustrations of everyday small confusions and paper cuts adding up over the years.

Slow design system components make every app built on them slow, so performance belongs next to accessibility on the team's priority list. Timing ten popular systems on a 249-country dropdown and 100 checkboxes found Fluent and Carbon quickest to open the dropdown and Ionic by far the slowest. Playwright tests with explicit budgets, virtualized long lists, and published performance numbers keep components fast by default.

A library of patterns and principles for inventing new interfaces

A designer may have solved your interface problem decades ago. Your AI can connect to this library and show you their solution, demo included.

Drag the list down and let go.

Before Pull-to-Refresh there was Inertial Scrolling, that smooth momentum-based scrolling the iPhone introduced in 2007.

Definitely a pleasant effect, but without meaning. Pull-to-Refresh changed that. It repurposed the gesture into a trigger to reload data.

In Interface Lineage we trace how those interactions connect and evolve.

Notice how resistance increases the further you drag down, like an actual rubber band would feel when being pulled. Then when released it leaves the space of one list item above for the loading animation.

Open a menu and slide toward the submenu.

The submenu suddenly closes. Super frustrating … Why does every other popular app fail this test?

Maybe the makers didn’t test or didn’t care.

But you do care; otherwise, you wouldn’t be here. Knowing about the pain and knowing how to cure it are two different things.

The cure here is called “Safe Triangle”. You can build better interfaces just by knowing and prompting this name. But if you dig a little deeper and read the background stories in this library, you can become an inventor of new interfaces.

Despite being solved way back in 1987, broken submenus are still everywhere today.

Spin the wheel.

A flat list turned into a drum you spin with one thumb. Spin faster and it flies, ease off and it slows.

The iPod wheel came from a Bang & Olufsen cordless phone. The iPod is gone, but you still spin through lists with the Apple Watch’s Digital Crown.

Good mechanisms outlive the hardware that introduced them.

Common interface problems, and the patterns that solve them.

• A reply meant for one person shows up for everyone in a group chat or on a mailing list. Threads Contain the Explosion
• Someone dragging a file or a window has to guess where it will drop on release. Drop-Target Feedback
• Undo Users who click Send or Delete see the mistake a moment later, after the action is done. Grace-Window Undo
• Users waiting for a page or a reply wonder whether the app is still working or has stopped. Honest vs Theater Progress

What you’ll learn

The principles interfaces prove or break.

Working memory is small: Google Authenticator makes you carry the code in your head between apps; iOS AutoFill offers it above the keyboard.

Undo absolves the warning: an Undo snackbar dropped into a mail design, in place of an “Are you sure?” dialog.

You can’t prompt what you can’t name.

Designers often know when something isn’t working but lack the vocabulary to explain why. Without the words to name a problem, it’s difficult to translate design intuition into clear instructions AI can act on.

What is Interface Lineage?

An interlinked library that takes interfaces apart for the people who make them. Every entry names the mechanism, traces it back to the people who built it, and connects it to the ideas it grew from. Many come with a demo you can operate and practical takeaways.

Who is it for?

Designers, engineers and students who already feel the difference between a good interface and one that just works, and want the words to name it and the judgment to defend it. It is not a tutorial, a set of components or a course. It’s a library where you can bring your own librarian.

Who’s behind this?

I’m Johannes Mutter. I am a design engineer from Germany. This library started in 2011 as my personal collection of interface teardowns. I completely redesigned it this year so that it makes sense to you too.

What does it cost to read?

Nothing. Every page is free to read, no account needed. The project is funded by Founding Members: one payment of €199. Membership lets you connect your AI with a personal MCP token.

How was this made?

I had notes, voice memos and Notion databases about interfaces, principles and patterns, plus an ontology I had modeled on top. I could cut it down to a handful of mechanisms like “Slide to Unlock” or “Pull to Refresh” and write a book, or publish the full network. A mechanism is worth less on its own than linked to hundreds of neighbors. That volume needs language models: for drafts, research and most all of the demo code.

Is the prose written by a language model?

Language models write the drafts. I decide which interface and mechanism each entry is about, and the argument comes from my notes and voice memos. Each draft goes through several rounds of human and AI revision against a shared authoring guide. Sources are the makers’ own words: interviews, talks, oral histories. A quote comes from an identifiable primary source, or it is cut.

I missed an internet with more personality. Building a little piece of it involved Claude, 4,096 colors, and a surprisingly difficult pair of legs.

My personal website now has a monster called Jirasaurus. It attacks with Scope Creep and Sprint Planning. Halfway through the fight, it performs a Workflow Migration.

This is still the website where you can read my blog and find my email address.

I miss the old internet. I miss landing on someone's website and getting a sense of the person who made it. Their interests, their sense of humor, the things they cared enough about to spend an unreasonable amount of time on. So much of the web now feels bland and bleak by comparison.

And my own site was playing it safe. I'd spent all this time teaching Impeccable how to be more creative, then left my personal website looking perfectly respectable and a bit boring.

It was time to go all in. I wanted people to enjoy spending time there. To get drawn into something, find a surprise, and come away feeling like they'd visited my corner of the internet. I thought about what I wanted that visit to feel like before getting into what we should build.

So I opened Claude Code and asked for something “extremely over the top.” Ten days later, there was a boss fight.

The result is a little world rendered live in your browser. My projects occupy a solar system. My writing lives on floppy disks. There's a rooftop panel above a city, a road through my career, and a speaking page where my talks play on the screen of a pixel-art auditorium. Three pages hide playable games.

Underneath all of it, Ghost still publishes the posts and sends the newsletter. The whole thing ships as a theme zip.

Let me show you how we got here, including the bits that went spectacularly wrong.

The experience starts before you get there

One experience I keep coming back to is Secret Cinema in London. I went to their production of The Empire Strikes Back, and the immersion started as soon as I'd booked. Emails arrived that treated me as someone who belonged in the story. I was already participating before I got to the venue.

That whole production made a huge impression on me. Ever since, when I organize an event or build something, I find myself thinking about the experience from the very first contact. What someone encounters on the way in matters. So does what happens between the big moments.

I'm a little obsessed with this. I want to get absorbed in a place, and I love creating that feeling for other people.

For this site, I started with how I wanted people to relate to me. I wanted them to spend time in a world that reflected my interests and sense of humor. Even my frustrations could become something we enjoyed together. If you've ever had your day swallowed by meetings and process, there's something satisfying about jumping over a meeting invitation.

That gave the redesign a direction. The scenes, the music, and the way you move between pages all contribute to the same visit.

Giving that world a personal point of view

One early hero put my portrait over a forest. It was attractive, but I couldn't see myself in it. My feedback was simple: “it doesn't really have anything to do with me.”

The forest became a rooftop hack night. I'm on a panel, wearing my cream jacket, with a city behind me and a crowd in front. That's a much more specific introduction to someone who spends a lot of his time building things and talking about them.

That became a useful test for the rest of the site. Each scene had to tell you something about me, and give you a reason to spend a little time with it.

The career timeline became a road with milestone billboards. On the speaking page, choosing a talk dims the house lights and plays the video on the stage's big screen. The mission page turns the problems I care about into monsters you can fight.

The demoscene gave us a way to make these ideas belong together. A demo is a real-time audiovisual production; a megademo strings several parts together. My site already had parts. We gave each one a scene and a tune, then built the machinery to travel between them.

First, take away most of the colors

“Retro” leaves a lot open to interpretation. We needed rules precise enough to make decisions with.

The site's world uses the Amiga's 4,096-color palette: sixteen levels each of red, green, and blue. Edges stay hard. Gradients become stepped bands of color. Shading uses patterns of pixels. Nothing glows.

Most of the scenes are drawn into small Canvas 2D buffers, then enlarged by whole-number scales. One source pixel becomes a square block of screen pixels. The voxel logo uses WebGL2, but it follows the same visual direction.

These constraints do a lot of the work. A rooftop, a spaceship, and an office can be wildly different subjects and still look as though they came out of the same machine.

Look closely at a shaded area. The intermediate tone comes from alternating pixels in a repeating pattern. That's ordered dithering. Zoom out and your eye blends the pattern; zoom in and you can see the construction.

This gave Claude something concrete to preserve across iterations. “Make it feel more retro” could go almost anywhere. A fixed palette, a pixel grid, and an approved reference gave us a shared target.

Teaching myself to walk

I've worked in the video game industry and built a game engine before, so sprites are familiar territory. I approach an animation thinking about its playback rate, the work that has to fit inside each frame, and where the character should be from one frame to the next. Those decisions shape how you build it.

Turning me into a sprite started with a photograph: dark hair, burgundy glasses, short beard, cream jacket. Image-model explorations helped us choose a direction, then the approved artwork was reduced onto a real pixel grid and into the site's palette.

Getting a recognizable little person was manageable. Getting him to walk was an ordeal.

Individual frames looked plausible. In motion, I squished, slid, or hopped forward. Some iterations wandered away from the character I'd already approved. At one point my feedback was: “in this sprite, I now have three hands.”

The walking problem needed a different method. Claude built a rig that kept the approved head and torso and calculated the legs as two connected bones. The boots pivoted around the heel or toe. The animation followed the phases of an actual step, and the movement of the ground matched the stride.

The foot is the giveaway. While it's planted, it needs to stay attached to its spot on the floor. If the scenery and the animation disagree, you feel the mistake immediately, even if you couldn't explain the math behind it.

We started checking the walks as animations at their actual timing. A screenshot had been answering the wrong question. Claude could produce the code and imagery quickly, but its visual understanding wasn't reliably catching what happened between those plausible frames. My experience made the mistakes easy to recognize; we still needed a process that caught and corrected them.

That's something I'd run into while building Radiant, too. An effect can survive a code review and look convincing in a still image, then fall apart when you watch it run.

The browser plays the soundtrack

The music is synthesized live with Web Audio. The site's soundtrack and effects don't come from recordings: oscillators, noise, filters, and envelopes make the sounds as you hear them.

There are twelve tunes in the shipped site. Arriving at them involved a lot of listening.

We built a local sounds lab where I could rate candidates and leave notes. Across 95 candidates, I marked ten “love,” 26 “maybe,” and 59 “no.” My notes contain a remarkable number of variations on “weak melody.”

The arrangements could sound pleasant while leaving me with nothing to remember. The lab made it possible to compare directions, explain what was missing, and return to the ones worth developing.

The finished site has a hidden studio that exposes this machinery: notes arriving in a tracker view, scopes, mute and solo controls. You can listen to the parts separately. There's also a tool that turns a hummed melody into notes and arranges a backing underneath it, all processed in the browser.

The soundtrack also shaped navigation. A normal page load would interrupt it, so the theme fetches the next page behind a closing shutter, swaps the main content, and starts the new scene. The audio context and song clock stay alive. Tunes can change at a bar boundary.

A link click becomes a scene change. The music carries you through it. This is exactly the kind of in-between moment I care about: you should still feel inside the world while moving to another part of it.

A blog header gets out of hand

The writing page takes place on Floor 37, an office where a small version of me dodges meeting invitations, jumps onto an URGENT stack of work, and slides under red tape.

Click the scene and it grows into a playable platformer. You have a working day to collect floppy disks and get through the office. Leaving the game shrinks it back into the page where you started.

The mission page grew a battle system. The speaking page gained Keynote, a rhythm game about delivering a talk: advance the slides, hold for the laughs, recover when the live demo crashes. Claude proposed that last one, and it fit the page well enough to build.

The games carry the same subjects as the pages around them. Floor 37 makes bureaucracy literal. Keynote lets you experience a talk from the other end of the clicker. Jirasaurus is probably self-explanatory to anyone who has survived a workflow migration. They're a way to share a frustration and have a laugh about it together. You get to know something about me by playing.

This was the feeling I wanted to bring back: you click something because it looks interesting, and discover that someone cared enough to make it do something delightful.

They also stay optional. You can come for a post, read it, and leave without fighting anything.

It still has to be a website

A site can look spectacular in a design-awards gallery and still be frustrating to use. I wanted people to get absorbed in this one, but someone arriving to read a blog post should be able to just read the blog post.

An early version had a boot screen in front of the site. It fit the aesthetic. It also made you get through an interstitial before you could do what you came for, and I generally dislike interstitials.

We removed it. A brief nudge now points at the sound control. The site opens immediately, and sound stays off until you choose it.

Thinking about the whole visit also means noticing when you're making it harder for someone to enjoy themselves. An elaborate entrance can wear out its welcome before they've even reached the site.

Titles, descriptions, and links remain real HTML. The canvases draw around them. Reduced motion gets still scenes and immediate transitions. Scenes stop drawing when they're off screen, and games load separately from the pages that contain them.

Responsive design was a big part of this. The experience had to hold together across screen sizes, including the awkward ones. A phone turned sideways gives you a very different space to work with from a tall desktop window.

Try opening the homepage on a desktop and resizing the browser. Watch the rooftop as the window gets narrower. The browser redraws the scene for the space it has, adjusting its framing along with the page layout. At narrower sizes, the rooftop becomes a window onto the panel, with the title and copy arranged around it. Further down, the project hall gives way to individually presented monitors.

There's a nice technical detail here. When a canvas changes size, the engine recalculates its drawing buffer and lets the scene respond to the new dimensions. In the narrow rooftop view, the layout and renderer share a whole-number pixel scale, so the framing can change while the pixels stay crisp. The page is deciding how to present the same world in the available space.

That took real attention. The spectacle only works if you can still see what matters and reach what you came for, on the screen you're actually using.

The new scene engine is plain JavaScript and CSS inside a Ghost theme. There is no framework or development build step. Packaging a release minifies a copy of the assets, then produces the zip I upload through Ghost Admin.

Even with a ticket-management kaiju in it, the publishing workflow is familiar.

What working with Claude actually looked like

Claude wrote the code, built the art pipeline, and composed the music. I supplied direction, chose between versions, and used the thing repeatedly on my laptop and phone.

Most of my feedback came in batches: this doesn't feel like me; that walk is wrong; this tune needs a stronger melody; the phone layout is hiding something; let's see whether this scene could be playable.

Claude ran browser checks and automated playthroughs. Those were useful for catching broken behavior. They couldn't settle whether a tune was memorable or a character's walk felt natural.

The sprite work makes the tradeoff especially clear. Building those animations myself, I would have accounted for timing and motion from the start and avoided much of that particular rework. Producing all the art, code, and variations by hand would also have taken me much longer.

Working with Claude gave me very fast production and experimentation, with less reliable first attempts in areas like animation. The short feedback cycles were how I made use of that speed. I could try a direction, watch it fail, explain exactly what was wrong, and try again while the idea was still fresh. Knowing how the underlying system should work helped me decide when another iteration was worthwhile and when we needed to change the method entirely, as we did with the walk rig.

We also had to protect decisions we'd already made. A later “polish” pass could drift away from an approved design. One made the Impeccable diamond look like a strawberry. We restored the version that worked.

The practical lesson for me was to keep the references close, make feedback specific, and review each thing in the form people would experience it. Listen to the music. Watch the walk. Try the site on a phone. Keep the good version around so “better” has something to be compared against.

AI made an absurd amount of this feasible over the course of ten calendar days. My part was deciding what belonged on my website, and then being quite particular about it.

What stayed with me after Secret Cinema was the feeling of having spent time inside another world. That's the feeling I'm reaching for here, on the scale of a personal website. I hope you come away having enjoyed yourself, with a better sense of the person who built it. Possibly also a high score.

Full article content is not available for inline reading.

Read the original article →

GPT-Rosalind-6 is a frontier reasoning model for biology, drug discovery, and translational medicine. We offer it as GPT-Rosalind-6 Research to qualified scientists, research institutes, companies and government partners. Rosalind has fewer refusals on dual-use biological queries but is only made available via trusted access. We review applicants’ proposed uses, security controls and governance practices before granting access. We also continue to have other deployment safeguards in place.

GPT-Rosalind-6 uses the same weights as GPT-6 Astra, our flagship reasoning model. It is designed to support legitimate biological research, including some dual-use topics, while still refusing requests for biological weaponization and retaining refusals on other safety topics.

Under our Preparedness Framework, we are treating this launch as High capability in the Biological and Chemical domain, below the threshold for Critical. We rely on GPT-6 Astra’s capability assessments for other Preparedness risks and treat GPT-Rosalind-6 as Critical in Cybersecurity and below High in AI Self-Improvement.

This addendum describes GPT-Rosalind-6’s biological capability assessment, additional safety and alignment evaluations, and deployment safeguards. We include results for GPT-Rosalind-5.5 where available for comparison. This addendum should be read alongside the GPT-6 Astra system card, which describes the parent model’s broader capability and safety evaluations.

Anthropic appears to be preparing Claude Mythos 5.5 for an upcoming release after a reference to the new version was spotted on its official Mythos webpage. The change appears recent and currently covers only the version number, with no new specifications or announcement. This could point to a launch as early as next week.

Meanwhile, multiple reports suggest Anthropic has been A/B testing updated Claude Fable 5.5 checkpoints with selected users. Early impressions describe state-of-the-art performance, with some claims suggesting the model even outperforms Google’s unreleased Gemini 4 Carbon. However, these comparisons remain anecdotal, with no public benchmarks available to support them.

Availability could be a major limitation. Anthropic previously restricted Mythos 5.1 to verified cybersecurity and life sciences researchers, while Fable 5.1 offered the same underlying model with additional safeguards for general use. If the company follows a similar strategy, Mythos 5.5 may initially remain exclusive to selected cyber defenders.

Earlier Mythos announcements also triggered sell-offs among cybersecurity stocks as investors questioned the future of traditional security software. The market could react differently this time, however. Recent cyberattacks and vulnerability disclosures demonstrate continued demand for cybersecurity solutions. At the same time, restricting advanced defensive capabilities to selected organizations raises questions about how the broader public can benefit from these advances.

Another question is whether Claude Fable 5.5 will arrive alongside Mythos 5.5. Anthropic previously released both model variants together, making another simultaneous launch plausible. Rumors about Fable 5.5 testing have also circulated for weeks, although no release date has been confirmed.

The timing is particularly interesting given Anthropic’s upcoming IPO. The company filed for a Nasdaq listing under the ticker ANT on October 26, and a major model release ahead of its market debut could attract considerable investor attention. Whether both models arrive next week, and who gets access first, remains to be seen.

Sources and related context

• Anthropic introduces Claude Fable 5.1 and Claude Mythos 5.1: Supports: Anthropic confirms that Fable 5.1 and Mythos 5.1 share an underlying model, with general access for Fable and trusted access for Mythos.
• Anthropic expands its Cyber Verification Program: Related context: The program explains how qualifying security teams can apply for advanced cyber capabilities and access to Mythos 5.1 and future models.
• Anthropic files S-1 for a proposed IPO: Supports: Anthropic confirms its October 26 filing and intended Nasdaq listing under the ticker ANT.

There’s a big uproar about AI agents taking over commerce and orders, but that might not be the case just yet. Delivery company DoorDash revealed during its third-quarter earnings call that it has only seen “several hundred” orders from third-party agents like Muse, Instinct, Fo, and others. For comparison, the company said in its 2025 report it completes an average of 3 million deliveries daily.

“I would say that this [using agents to order] is something that is still a very dynamic area. And my take on all things technology is you never want to fight gravity. But at the same time, you don’t want to be delusional,” DoorDash CEO Tony Xu said in an answer to an analyst’s questions about the topic. “What I mean by that is if you look at where AI agents are today, we’re seeing maybe several hundred orders in total from some of these third-party agents,” he said.

Xu mentioned that users are still predominantly placing orders through DoorDash’s app. However, the company wants to learn more about agentic use cases and develop a healthy relationship with third-party agent makers. He added that, in the future, there could be an opportunity for it to advertise within these agents.

DoorDash has its own agents, including a text-to-order agent it launched in September, a CLI agent introduced in July, and a chatbot called Ask DoorDash it launched in June, which allows users to place orders through prompts and photos. The company also released an MCP to enable ordering from other chatbots for corporate ordering.

The space of agent-based ordering is evolving, as websites are still deciding whether to allow them to access their content or not. Amazon has been blocking such personal agents, but Walmart has laid out a red carpet for them in order to snag some business from its competitors.

Supporting a shared AI training control for websites

Today, we're announcing our plans to support a shared AI training control in robots.txt. This will help websites express their AI training preferences across multiple AI providers in a single step and is based on work from the AIPREF working group at the Internet Engineering Task Force (IETF). This builds on our long history of providing website owners with clear controls, including through Google-Extended, the AI training control we've offered since 2023.

How the new control will work

Instead of managing separate instructions for every AI developer, website owners will be able to use a shared 'policy token' within robots.txt to set broad preferences across participating AI providers.

When a website uses this policy token to opt out of training in its robots.txt file (and Google-Extended is not explicitly specified), Google plans to interpret that as if the publisher had disallowed Google-Extended. Other AI providers can handle it similarly, making it easy for website owners to opt out across all providers, or to pick and choose.

Existing company-specific controls will continue to function alongside this new joint signal. So website owners can easily express a broad choice using the industry signal, while also tailoring their instructions to individual AI developers like Google through controls such as Google-Extended. Websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps.

The new mechanism brings website owners the benefit of the AI training definition developed at the IETF while multistakeholder discussions continue on the rest of the AIPREF vocabulary. As more industry players adopt this mechanism, it gives website owners early and expanding access to the preference signal.

This draft proposal reflects feedback from publishers, civil society groups, regulators and other stakeholders that have called for industry-wide controls on AI training. Support for this proposal across major AI providers will enable a future standard that is cross-industry, robust, and durable. This proposal is consistent with the EU Code of Practice for General Purpose AI (GPAI) Model Providers under the EU AI Act that recognizes robots.txt as a primary method for web publishers to express machine-readable preferences. We'll continue to work with these stakeholders to refine and standardize this through the IETF working group.

We aim to support this new control along with other AI providers in 2027, and will provide updates on our progress.

In a world where youth unemployment is growing as AI replaces more and more entry-level jobs, Cisco president and chief product officer Jeetu Patel is going for a different approach. According to a report by Fortune, he's given a 27-year-old ex-startup founder — brought into the company by an earlier acquisition, and previously languishing in a role five levels below — a new job to refactor 98% of the company's legacy code within 24 months.

The employee, Padmanabhan Krishnamurthy, joined Cisco after it acquired EzDubs, a translation startup that Krishnamurthy helped build in 2023. While having executive experience, Patel found that "Paddy" was left "buried away in the organization doing something unstrategic."

According to the report, Patel spoke to Krishnamurthy in Cisco's office in San Francisco, where Krishnamurthy told him that he could rewrite Cisco's legacy code in a team of just two people, and that "even the most AI-forward people on [Patel's] own team weren't thinking that way."

As a result, he's now been tasked with rewriting 98% of Cisco's legacy code within 24 months, with four weeks as a proof of concept. "I've given him unlimited tokens because he's going to refactor code," according to Patel, telling him "you can go nuts."

AI coding has skyrocketed in recent months, with GitHub forced to deploy 4 million extra CPU cores and 160 petabytes of storage to cope. However, the practice is not without its pitfalls, with rogue AI agents notorious for taking unsolicited actions including deleting entire company databases, TV show edits, and production setups.

Patel hopes that there are more young hires in the Krishnamurthy mould that he can find, both in the company and externally to hire. He's looking for hires to "challenge the system and the status quo," to be "an energizer bunny that just goes and kind of revs everyone up."

Patel believes that current corporate structures work against this kind of thinking. "If you don't hire the early-in-career [workers], and you just let your workforce continue the way that they were, where do fresh ideas get injected? How do you challenge the status quo?"

He acknowledges that this will create friction, putting Gen Z employees among decades-old employees. "Tension is good.... I actually try to create tension in the teams because you want to have enough friction on the ideas. You don't want to have friction with the people and personalities."

Patel believes that a combination of experienced, scale operators at Cisco, alongside external hires and "entrepreneurial founders, CEOs and engineers who are AI-native" will bring results. "Either of these groups by themselves aren't nearly as potent as when all three work together. It's what creates magic."

OpenAI's latest funding round received tepid interest from investors. The deal values the AI lab at $1.25 trillion, less than the $1.4 trillion valuation it had earlier hoped to achieve. Investors appear to have grown pickier about which companies they want to back and on what terms. The industry recently experienced a brutal selloff tied to disappointing earnings results from giants, including Google, Amazon, and Intel.

Meta reportedly built a humanoid robot prototype that can do household chores

A report said the company is already working on the second version that’s codenamed Max 2.

As part of its push into robotics, Meta has built a prototype humanoid robot that's meant to help out around people's homes, according to The Information. The report said that the robot, internally codenamed Max, was built with the help of a Chinese robotics company called Fourier in roughly three months.

According to The Information's sources, the robot weighs roughly 106 lbs and is about 1.5 meters tall, or approximately five feet tall. On the outside, the prototype reportedly has a display that can show facial expressions, a built-in speaker and hands designed to pick up small objects. Internally, Meta went with NVIDIA's Jetson Thor chips to power the robot, according to the report.

The Information's report said there's no launch date set for a mass-market version of Max, but that Meta has already been working on a second prototype, codenamed Max 2, since September. The report added that the tech giant is including updates to the design of the robot's torso and hands, as well as adding a projector and additional sensors to help it handle stairs better.

Meta's efforts to break into humanoid robots began as early as last year after Bloomberg reported that the company created a new team tasked with building robot hardware with the eventual goal of being able to do household chores. Following that report, Meta acquired a startup that's known for building artificial intelligence specifically for robots, earlier this year.

Full article content is not available for inline reading.

Read the original article →

Shopify Shop Pay now works inside personal agents and for brands that are not on Shopify.

GPT-6 Astra can outperform several specialized robotics models despite not being designed as a robotics foundation model.

Full article content is not available for inline reading.

Read the original article →

Toyota will build a custom version of its bZ4X electric SUV with the Waymo Driver integrated right on the line.

Spacelift Brought Scale to Your IaC Provisioning. Now We’re Automating Everything That Happens After.

For two decades, the way we provision and configure infrastructure has kept getting better.

Puppet and Chef turned server configuration into code. Terraform made infrastructure declarative and repeatable across clouds. OpenTofu, which Spacelift helped found, kept that foundation open and is now driving IaC forward, shipping features like native state encryption and OpenTelemetry support, and setting the new standard for how teams define infrastructure.

Each step gave platform teams more speed and more control over how infrastructure gets created and changed. Spacelift has been at the forefront of this effort, helping teams centralize, govern, and scale IaC provisioning across every tool and cloud they run. What happens after provisioning is a different story, and it’s where most of your team’s time actually goes.

The neglected 80%

Provisioning is a moment. Everything after it is where most of the work happens – patching, rotating certificates, responding to incidents, granting access, cleaning up idle resources, and working through the request a developer filed last Tuesday. Platform teams spend roughly 80% of their time there, and in most organizations that work still runs on a patchwork of cron jobs, scripts, and Lambda functions.

Ask a platform team what automations are running in their environment and you’ll usually get a pause. Nobody has the full inventory. The script that rotates credentials was written by someone who left a year ago, and no one wants to touch it. When something breaks, production is the one that tells you. New automation requests turn into big engineering projects, so they sit at the bottom of the backlog and take weeks to build once they finally reach a sprint.

AI is making this harder. Developers are writing code faster than ever, which means more infrastructure reaching production and more Day 2 work landing on the same platform team. Agents want in on the infrastructure action too. Today they often run on an engineer’s credentials or a shared key, with no audit trail of what they changed.

At Spacelift we brought order to provisioning. Unfortunately, Day 2 remains the wild west.

Today that changes

Today we’re launching Spacelift Flows, a new part of the Spacelift platform that brings the same rigor, observability, and control to Day 2 operations that we brought to IaC. Flows is visual workflow automation built for platform and DevOps teams.

With Flows, you can see every automation. Each automation runs on a visual canvas with a step-by-step execution trail, so you can watch live, replay in whole or from any step, and know exactly what ran and what failed.

You can stand up an automation in an afternoon instead of weeks. Start from pre-built blocks for the tools you already use, write code, or describe what you want and let our AI assistant build it. There are no containers or hosting to manage, and every automation can also be managed as code through our Terraform provider.

“With Flows, we had something working in a few hours, instead of four weeks later and still struggling with auth, which is usually the pattern,” said Parker Cameron, senior manager of cloud engineering at Discovery Education.

Flows provides agents a governed path into your infrastructure, by exposing internal systems as Model Context Protocol (MCP) servers with private network access and approval gates. Each agent authenticates as itself and appears in the audit trail as its own user, so every action is attributable to the agent that took it.

Developers get real self-service. A developer fills out a form, Flows checks budget, quota, and policy, and the request is done in minutes instead of turning into a ticket for your team.

The Spacelift Full Stack Infrastructure Orchestration Platform

The Spacelift Platform now offers full stack infrastructure orchestration. Alongside Flows, the IaC and GitOps capabilities you know and love as Spacelift now have a name of their own, Spacelift Deploy. Nothing changes about how it works, your contract, or your access.

Spacelift Deploy, Spacelift Flows, and Spacelift Intelligence form a single control plane for platform teams. A deployment can trigger a Flows automation, and an automation can trigger a deployment. The same policy, role-based access control, and audit history apply to a provisioning run and to the Day 2 automations that respond to it. And with Spacelift Intelligence, all of it runs through a single assistant, so you can understand, design, deploy, and govern your infrastructure in plain language.

For the first time, platform teams get speed, control, and observability across the full time continuum of their infrastructure, from the first plan through every change, patch, and incident that follows.

If you run Spacelift, your infrastructure is already defined in code, governed by policy, and auditable. Flows extends that investment to the work that follows, using the same policies and the same audit trail, instead of adding another disconnected tool to your stack.

See it for yourself

Join our live demo webinar on November 5 https://events.spacelift.io/from-deploy-to-destroy, or explore Flows at https://spacelift.io/platform/flows.

I hope you are as excited about this next chapter in infrastructure automation, and Spacelift, as I am.

Solve your infrastructure challenges

Spacelift is an infrastructure orchestration platform built for IaC. It brings collaboration, automation, and governance into a single workflow, so your team can provision cloud infrastructure faster without losing control.

Grafana 13.3 adds cross-dashboard variables and Notebooks, both in public preview, plus the first phase of a visual refresh. It also brings Assistant dashboard plans and a generally available Undo turn option for reverting Assistant edits.

Vorssaint

One menu bar icon doing the job of a dozen paid Mac apps. Install only what you use.

For anything private, email hello@vorssaint.com.

Per app volume, a real system monitor, a better app switcher, window snapping, Dock previews, clipboard history, text snippets, a file shelf, an uninstaller. The utilities Mac users usually buy one by one, together behind a single menu bar icon, with no account, no telemetry and no subscription.

Install only what you use

Choose individual features or start with a preset. Uninstalled features stop loading and disappear from the interface; reinstalling restores their settings. Setup asks only for the permissions your choices need.

Reorder or hide panel sections, choose a compact layout, and export settings to another Mac. The app supports more than a dozen languages.

Everything it does

Sound

• Volume mixer. Set volume per app, boost quiet audio past 100%, and pin or reorder favorites. No audio driver required.
• Per app output. Play music through speakers while a call uses your headset.
• Output switcher. Switch audio outputs with a shortcut and lower the volume when headphones disconnect.
• Microphone tools. Choose a preferred input, adjust its level on supported devices, and apply available microphone mute or input-volume controls with one shortcut. Siri and some apps may still hear you; Mic Mute is not a privacy lock. Its indicator shows that the feature is on, not that every app receives silence.
• Music app blocker. Prevent unwanted Music launches after detected playback-button presses. Requires Accessibility.

Know what your Mac is doing

• System monitor. CPU, GPU, memory, temperatures, battery health and power use, with history graphs and a view of energy-hungry apps.
• Fan Control (beta). Set manual fan speeds or temperature curves and monitor live RPM.
• Menu bar readouts. Put your chosen readings, usage bars, battery time or fan speed directly in the menu bar.
• Network. See live traffic, session totals and your local IP address, or run a speed test.
• Alerts. Get notified about sustained CPU load, high temperatures, memory pressure, low disk space or low battery.

Windows and the Dock

• App switcher. Switch between apps and windows with live previews, search and display filters. A simpler mode works without screen capture.
• Window layout. Snap windows into layouts, move them between displays and restore earlier positions using shortcuts, screen edges or modifier-drag gestures.
• Dock Preview. Hover over Dock icons to preview windows across desktops. Switch, close, move or snap them from the preview.
• Dock clicks. Click an active app's Dock icon to minimize, hide or cycle through its windows.
• Maximize windows. Use the green button to fill the screen without creating another Space.
• Fixed Space order. Keep Spaces in the order you set instead of rearranging them by recent use. Turning it off restores your previous setting.
• Quit on close. Quit selected apps when their last window closes.
• Quit and close protection. Prevent accidental ⌘Q or ⌘W with a hold, double press or extra modifier, per app.

Keyboard and mouse

• Text snippets. Expand short triggers into text with clipboard, date and time variables, or insert snippets from a searchable menu.
• Smooth scrolling. Give your mouse wheel a fluid glide with adjustable speed and response.
• Pointer acceleration. Disable mouse acceleration and restore your previous setting when turned off.
• Linear scrolling. Make every mouse wheel notch scroll the same number of lines, however fast the wheel spins.
• Focus follows mouse. Bring the window under the pointer forward after an adjustable pause.
• Scroll direction. Invert vertical and horizontal mouse scrolling independently of the trackpad.
• Scroll sideways while holding a key. Turn vertical wheel movement into horizontal scrolling while holding a chosen key.
• Side buttons. Use mouse Back and Forward buttons in Finder, browsers and compatible apps.
• Mouse button shortcuts. Assign keys to extra buttons and side wheels, or use button-drag gestures for Spaces and Mission Control.
• Middle click. Turn a three-finger trackpad press into a middle click.
• Apps to leave alone. Exclude chosen apps from mouse enhancements.
• Extra click filter. Ignore accidental rapid clicks from worn mouse buttons without delaying normal clicks.
• Key debounce. Filter repeated letters caused by a worn keyboard.
• Super key. Use Caps Lock or a right-side modifier as a shortcut combination, with a separate tap action and app exclusions.
• Keyboard shortcuts. Manage feature shortcuts in one place, including optional takeover of supported macOS shortcuts while a feature is active.

Clipboard, files and links

• Clipboard history. Search local text, image and file history, pin favorites, preview entries and paste with shortcuts.
• Auto clear clipboard. Clear the system clipboard after a delay, sleep or lock, while keeping saved history.
• Paste as plain text. Paste without formatting while preserving the original clipboard content.
• Shelf. Park files, text and links near your cursor while dragging, then drop or share them later.
• Finder shortcuts. Move files with ⌘X and ⌘V, rename with F2, or paste copied images as PNG files.
• Clean URL. Remove tracking parameters from links, manually or automatically.
• Disk image installer. Install an app from a mounted disk image and eject it, with optional download cleanup.

Everyday tools

• Dynamic Island. Keep music, notifications, calendars, timers, downloads and everyday controls around the camera cutout, or a simulated one on other Macs. Customize sections and shortcuts, with optional lyrics, a live equalizer, camera preview and file tools.
• AI agents. Follow Claude, Codex, OpenCode and GitHub Copilot in the Dynamic Island: plan limits and when they reset where available, tokens, API value, models, projects and live work, with a notice when a long task finishes. Codex's banked resets can be used from there too, and each extra Codex account (CODEX_HOME) can have a limits card of its own. Copilot uses local CLI/app session logs; its token totals update when shutdown metrics arrive, and aggregate API-value estimates use base rates.
• Command Bar. Search apps, windows, files, clipboard history, snippets and app menu commands from one field. Calculate, convert units, find emoji or run saved scripts.
• Quick panel. Open a floating palette of favorite tools with ⌃⌘V.
• Quick toggles. Switch appearance, hide desktop icons, eject disks, empty the Trash, lock the screen and more.
• Quit apps. See every open app in one list and quit them one by one, all at once or all but the one in front.
• Radial menu. Open a customizable wheel of apps, files, shortcuts and tools around the pointer, with profiles and submenus.
• Scratchpad. Keep autosaved notes in tabs, with Markdown preview and export, in a floating window or Dynamic Island.
• Cleaning Mode. Lock keyboard input while cleaning, with a black screen or a small visible indicator.

Capture and create

• Screen capture. Switch between screenshots, recording, text recognition and color picking in one selector with a pixel magnifier.
• Screenshot. Capture an area, window, screen or scrolling page. Annotate, crop, redact, add backgrounds and watermarks, pin captures, send them through the Share menu or share an expiring link.
• Screen recording. Record with separate system-audio and microphone tracks. Trim, cut, add automatic zooms, blur private details and export video or GIFs, or share an expiring link.
• Camera preview. Check your camera in a floating mirror or Dynamic Island before a call.
• Copy text from screen. Recognize text offline from any screen area, or read a QR code.
• Color picker. Copy a screen color as HEX, RGB, HSL or SwiftUI code.
• Media tools. Compress and edit videos, batch-convert images, add watermarks, make GIFs and extract text, all locally.

App management

• App updates. Check store apps, package-managed apps and supported developer feeds in one list. Install managed updates together or open an app's own updater.
• Cleaner. Remove caches, logs and app leftovers, manually or on a schedule.
• Messaging downloads. Review, organize or move messaging downloads to the Trash using retention rules.
• Uninstaller. Remove apps and review their related caches, preferences and helpers before moving them to the Trash.
• Homebrew manager. Search, install and remove formulae and casks without opening a terminal.
• Port manager. Find processes listening on network ports and stop them with Kill Process when installed.

Energy and display

• Keep awake. Keep your Mac working on a timer, with the lid closed, or while selected apps, power or external displays are present.
• Displays. Control individual displays and brightness, with hardware control where supported, half or quarter steps for the brightness keys, optional extra dimming below a monitor's minimum, and software dimming as a fallback.
• Extra brightness. Use a MacBook Pro XDR display's HDR headroom to go beyond its normal maximum brightness.
• Bluetooth on sleep. Disconnect Bluetooth during sleep and restore it on wake only if Vorssaint turned it off.

Install

With Homebrew:

brew install --cask vorssaint

Or grab the disk image from the releases page and drag Vorssaint into Applications.

Builds are signed with an Apple Developer ID and notarized, so macOS opens them without a fuss and your permissions survive updates.

Uninstall

With Homebrew:

brew uninstall --cask vorssaint

To remove Vorssaint completely, including its settings and permissions:

./Tools/uninstall.sh

Private by default

Vorssaint is local-first, with no account, analytics or tracking. The network is touched only by things you can see: update checks, the speed test, Homebrew actions, optional online lyric lookup, temporary screenshot or recording links and feedback you explicitly send.

Permissions get the same treatment. Every one is optional, the app explains each in plain words, shows which features actually use it, and even tells you when a permission you granted is no longer needed by anything, with a shortcut to revoke it.

What you need

• A Mac with Apple Silicon
• macOS 14 Sonoma or newer

Build it yourself

git clone https://github.com/vorssaint/vorssaint-utils.git
cd vorssaint-utils
./build.sh --dev            # build the separate Developer variant
./build.sh --dev --install  # install and launch it

Community

Vorssaint went from first commit to the front of GitHub trending in three days, top of the Swift charts, and issues and pull requests have shaped every release since. Bug reports, feature ideas and translations are all welcome.

Vorssaint is free and will stay that way. If it earned its place in your menu bar, a star helps other people find it.

Acknowledgements

• App icon designed by @divisionseven

License

GPL 3.0 or later, copyright 2026 Vorssaint. The license covers the source code; the Vorssaint name, logo and look are covered separately.

Data products did not disappear. The hype around data mesh cooled because changing an organisation is harder than changing a data architecture. But the business problem remains: people and systems still need data they can find, trust, understand, and use for a clear purpose.

The useful question is not whether data products won. It is whether organisations can turn important data into accountable, reusable offers that support decisions, workflows, and AI. The evidence points to a more practical phase. Data product thinking is moving into existing platforms, marketplaces, catalogs, and governance work rather than demanding a full enterprise-wide data-mesh programme.

Data mesh was a demanding operating model

Data mesh linked four ideas: domain ownership, data as a product, self-service data infrastructure, and federated computational governance. That is a large change. It asks business domains to own the meaning, quality, and lifecycle of data used by others, while a central team provides shared services, standards, and controls.

That combination is difficult to fund and operate. The 2024 BARC study surveyed 197 data and analytics professionals. It reported that 20% had data mesh underway and 24% planned it, down from 35% planning it in 2023. Those figures describe the study sample, not an enterprise-wide adoption rate, but they show how hard it is to move from interest to an operating model. BARC's study also makes clear that data products remain a central part of the conversation.

The difficulty does not remove the need for trusted profitability data in finance, approved customer views for commercial teams, or reliable performance data in operations. It means many organisations are addressing those needs without trying to decentralise every part of data management at once.

Data products moved beyond the architecture debate

A data product is not a table, dashboard, API, or model with a new label. Those assets can be part of a product. The product exists when a known consumer can use a governed output for a stated purpose, an accountable owner stands behind it, and the result can be assessed over time.

This is now a practical delivery discipline, not a demand to choose between data mesh and a lakehouse. Gartner listed highly consumable data products among its 2025 data and analytics trends. Its advice is direct: begin with business-critical use cases, build reusable and composable minimum viable products, and agree KPIs between producing and consuming teams. Gartner's 2025 trend guidance supports that use-case-first approach.

The distinction matters. A catalog can describe the estate. A marketplace presents a curated offering for a consumer need. Both are useful, but neither replaces product management: deciding what to invest in, who owns it, what it promises, and when it should change, merge, or retire.

Demand is established; delivery discipline is not

KPMG's 2025 survey of 250 executives found a sharp gap between belief and realised value. Ninety-two percent said well-constructed data products were critical to organisational success, and 96% said they were moving from legacy assets toward modern data products. Yet only 35% reported extensive value from their initiatives. KPMG's survey is not a universal adoption measure, but it is strong evidence that intent is ahead of execution in its respondent group.

That gap usually appears before the technology fails. Teams document assets, build taxonomies, or publish datasets, but leave basic questions unresolved. Which decision or workflow does this product support? Who owns the outcome? What quality, access, and service conditions apply? How will the organisation know the product is being used and creating value?

The better starting point is narrower: which consumer needs which governed data offer to complete an important task? From there, a team can define the output, owner, quality expectations, access model, and measures of use. The work becomes smaller, testable, and easier to improve.

AI makes the missing context visible

AI agents create a new kind of consumer. An experienced analyst may recognise an odd metric, investigate a table, or ask a colleague for context. An agent works quickly and at scale. If meaning, ownership, quality, or permitted use are unclear, it will make weak decisions faster.

An agent needs more than a connection to a warehouse. It needs to know which source is authoritative, what a measure means, who owns the product, how current it is, what quality it meets, and what it is allowed to do. A governed data product can package that context around a defined business purpose.

This is not just a future idea. The 2026 Data Product MCP research paper describes a marketplace where an agent can discover, request, and query data products while contracts enforce access and policy rules. Its qualitative assessment involved 16 data-governance experts, so it is early design evidence rather than proof of broad adoption. The direction is still important: agentic data access needs structured products and enforceable controls.

Platform roadmaps show the same shift. Internal marketplaces and governed product views are now visible across major data platforms. The value is not another catalog screen. It is a way for a consumer to discover a trusted offer, assess its meaning and conditions, and use it through an approved path.

The product layer needs to be portable

Most organisations already have warehouses, lakehouses, pipelines, APIs, catalogs, data-quality tools, and access controls. The harder management problem is connecting those tools to business intent and keeping a product understandable when it moves across platforms.

This is where an open, machine-readable product definition matters. The Open Data Product Specification gives a data product a portable description of its purpose, owner, access, quality, governance, and other operating conditions. It does not replace a catalog, marketplace, or cloud platform. It gives them a shared product definition to work with.

Portability makes a real difference when an organisation operates across several clouds, catalogs, and AI environments. The product should not lose its business definition, contract, or accountability because a team changes tools. It should also not rely on a model's temporary interpretation of a document as its source of truth.

What to do next

Do not start by declaring a data-mesh transformation or turning every asset into a data product. Start with one important decision, workflow, or AI use case. Identify the consumer, the accountable owner, the governed output, and the evidence that will show whether it is useful. Then make the product's meaning, quality, access, and permitted use clear enough for both people and machines.

Data products have not gone away. They have become more concrete. The organisations that make trusted context and accountability visible will be better placed for self-service analytics and responsible AI.

In most cases, flipping the reward labels in an agent's memory barely changes how it learns. This test hid a rule only the labels could reveal. With true labels, GPT-6 Sol scored 94.4 out of 100 on its last six trials, and with flipped labels 7.8. Claude Haiku 5.5 improved less, and Qwen3 4B barely moved.

Meta has quietly launched Style Sketch, an AI fashion app that helps users put together outfit ideas using clothes they already own. The app connects to Instagram to build a virtual closet from photos you’ve posted, or lets you add items manually by snapping a photo, recording a quick video, or uploading images from your camera roll.

Style Sketch then suggests outfit combinations built from your wardrobe matched to your style. Users can then try on the outfits on a “realistic model matched to your body shape and skin tone,” the app description reads.

The app, which was first spotted by social media consultant Matt Navarra, is available in Indonesia, Singapore, and Thailand on iOS and Android.

Meta did not respond to TechCrunch’s request for comment. It’s unknown whether Meta plans to launch Style Sketch more widely.

Several apps already use AI to help users digitize their wardrobes and visualize new outfits, including startups like Alta. Even Google Photos offers tools that let users turn photos of their clothes into digital closets where they can create new outfit ideas.

To compete with what’s already on the market, Meta will likely rely less on the novelty of the idea and more on the app’s integration with Instagram, where users already have a visual record of their personal style.

Rather than relying on random suggestions, users can also select a specific item from their wardrobe and get fresh outfit ideas built around it, according to the app description.

The app can also help users plan outfits for trips. If they enter their destination, trip length, and planned activities, users can get a day-by-day outfit plan that takes the weather forecast and itinerary into account.

Style Sketch is one of many AI apps from Meta in recent months. Most recently, the company launched an app called Waddle World, which essentially looks like a Tamagotchi clone, but with AI. Meta has also launched Pocket, a tool for vibe-coding games, and Vibes, which lets users quickly generate AI-powered videos.

The wave of new apps is part of a broader effort at Meta to release more apps. The Wall Street Journal reported earlier this year that CEO Mark Zuckerberg told employees the company plans to release far more apps than it has historically, as AI-driven efficiencies have made it faster and easier to ship new apps.

My feed is mostly designers talking about AI, and the temperature of all of it is the same: Something is coming. You need to get out in front of it. You need to be the one doing the changing rather than the one being changed.

I don’t disagree with most of the individual advice. But read enough of it in one sitting and what surfaces underneath is competition — ordinary competition, of the kind that has always been there.

The worry takes two shapes. Either the client no longer needs a designer, because the tool supplies the design. Or another designer, using the tool, takes the job you would have had.

Both are worth taking seriously. But neither is new.

The Permanent Condition

There has always been someone faster than me, someone cheaper than me, and someone better than me. This was true the day I started my career and it will be true on the day it ends. It describes a market, not an era.

Designers — actually, all in the professional services — have a folk theorem for living with this reality: Fast, cheap, good — pick two. I’ve heard it said and read it written countless times. I’ve said it and written it, too, to win arguments and contracts.

Today I got curious about where the saying comes from, and went looking. Nobody knows. The phrase itself gets attributed to something called the Common Law of Business Balance, and from there to John Ruskin — an attribution with no evidence behind it whatsoever. The idea underneath has a slightly better paper trail: an engineer named Martin Barnes drew it as a triangle, with cost, time and quality at the corners, for a project management course in 1969. Even that isn’t an origin. Versions of it were in use by the 1950s, and no one can say who put it that way first. This beaten-to-cliché piece of professional wisdom is a cipher.

Which is useful to remember because it’s also, often, wrong. There are as many designers who are fast and cheap and good as there are those who are slow, expensive, and bad. I have worked with some of each. Thinking back on this now, I am surprised that no one ever responded to the “pick two” challenge by saying, “I want all three, I just won’t be getting them from you.”

Faster and Cheaper

My entire career has been punctuated by tools that sped up production. Desktop publishing collapsed typesetting and layout into something one person could do in an afternoon. Then Photoshop. Then the rest of the Adobe suite, Illustrator and InDesign. Then Sketch. Then Figma. Each one made producing the work materially faster, and each one set off a race to master it before everybody else did.

Each of them may have changed what a day at work felt like, but none ended the profession.

What makes faster feel dangerous is that we convert it immediately into cheaper, and we do that because most designers price their time. If the work takes four hours instead of twelve, charging the twelve-hour price feels indefensible, and somebody else will happily charge for four.

That’s a pricing problem rather than a technology problem, and it’s been the same pricing problem through every acceleration I just listed. A practice that charges for outcomes absorbs acceleration as margin. A practice that charges for hours absorbs it as a pay cut.

As for cheaper, it arrived long before any of this, and it arrived by geography. I work closely with a designer who lives in Eastern Europe. This person is, by some distance, among the most talented designers I have ever worked with. They didn’t go to design school. I went to the Rhode Island School of Design, which bought me a credential the marketplace sometimes reads as shine, and I’m clear-eyed about the fact that — on the measure that actually matters — my colleague isn’t behind me. They can also charge less than I can, because rent where they live isn’t rent where I live.

None of that has anything to do with AI. A cheaper designer has been available to every client I have ever had as long as we’ve had an internet connection. And just in case anyone thinks that professional design competition from overseas is a more recent phenomenon, the first designer I met when I visited Malaysia before moving there in 2004 was born and raised in Penang, had no design degree, and had dozens of clients most of whom were headquartered in the United States. He was fast, cheap, and good. That’s why they paid him, rather than someone else.

What Better Means

There is an objection to what I’ve said so far. I’ll try to put it in its best form rather than a convenient one.

Every tool I listed made production faster without making the thing. Desktop publishing didn’t design the page. Photoshop didn’t choose the image. Figma doesn’t know what the product should be. AI, as a design tool, is different in kind, because it produces the artifact. So the comparison to Sketch and Illustrator is a weaker argument than it sounds.

I think that objection is right about the tool and wrong about the job. It only holds if the artifact itself is better.

Imagine two designers given the same brief — an identity system for the same client. Put the results in front of a panel, blind, with no knowledge of how either came to be. The panel picks a winner. They’re qualified to do it and their reasoning will be sound.

Now ask the client which designer they’d hire again.

It is by no means a given that the panel and the client will agree. The two answers can fork, and when they do it isn’t because the client has poor taste. The panel is assessing an artifact. The client went through something. They were asked questions nobody had asked them before. They had a position drawn out of them that they couldn’t have articulated on their own. At some point they were told that the thing they wanted was the wrong thing, by somebody who had earned enough credit to say so. None of that is necessarily visible in the file — nor should it be.

I wrote recently that expertise is becoming subjective — that where knowledge is cheap, experience is what differentiates. If that’s true, it cuts in both directions. Experience doesn’t only distinguish you; it aligns you with some people and not others. A prospect’s fit with how you’ve worked, what you’ve lived through, and how you talk about it is doing as much work as the expertise itself.

Which means there’s no better in the abstract. There’s better, for them.

There’s also a posture question here. To believe that your prospect couldn’t do better than you is a level of audacity that runs counter to how every designer I know actually thinks — and yet it’s close to what we’re advised to project. In a world this populous it’s absurd to imagine you’re the best at anything, and more absurd still to build a public persona on the premise. That advice persists because it works on a seductive technicality: whether or not you believe you’re the best, behaving as though you are turns out to be convincing to other people. Notice who that advice is sold to. We are the market for it; our clients have never heard of it. Rejecting that is realism, not modesty. The world is too crowded with designers to spend a second ranking yourself among them.

Finally, the same takes on the current state of things also sell a tempo. They essentially argue that AI compels us to move faster in order to stay ahead. But if experience is the differentiator, speed is the thing eroding it. Going slowly through a problem — getting it wrong, trying it on a real client, finding out — is where the differentiation was built all along. The availability and performance of AI doesn’t necessarily run counter to that, of course, but a willful misunderstanding of it does.

Nobody Asks About Tools

When I moved from Sketch to Figma I thought it mattered. I had my reasons. I argued about it. Designers I respected stayed on Sketch for years afterward and we all thought the other was wrong.

In the end, it didn’t really matter. I know designers who still work primarily in Photoshop. In the last twelve months, I actually saw a website design produced in Illustrator. Of course I think these are mistakes, bordering on the absurd. But it’s also none of my business, and as far as I can tell it hasn’t cost them a job. Isn’t that interesting?

But it makes some sense. After all, nobody has ever asked me what tools I use. Except, of course, another designer.

I don’t know what this technology will do to the profession over the next twenty years, and I notice that the people who claim to know are usually the ones with something to sell. What I can see from here is smaller and steadier than that.

Somebody out there is faster than me. Somebody is cheaper. Somebody is better. Again, not modesty, but the realism that comes from decades in the work, and I have met them. All three have been a phone call away from every client I have ever worked with. Any prospect can have them, faster, and at a cheaper price.

But they don’t have me. And they don’t have you.

Be singular.

See what to fix, and how

Every issue, ranked, explained, and pinpointed.

Issues listed worst-first

Every issue is ranked from critical to minor, so you fix what matters most first.

See exactly where each issue is

Click any issue to jump straight to the problem element, highlighted on your PDF.

Guidance to fix every issue

Each issue explains what’s wrong and how to fix it, mapped to the WCAG or PDF/UA rule it relates to.

What gets checked in your PDF

We check every machine-verifiable success criterion of the Web Content Accessibility Guidelines (WCAG), and every machine-verifiable test in PDF/UA (ISO 14289).

Structure

• Is the reading order correct?
• Are heading levels correctly nested?
• Are lists structured correctly?
• Do tables have associated headers?

Content

• Are fonts embedded and mapped to text?
• Do images have alt text?
• Is a valid document language set?
• Does it have a title and bookmarks?

Presentation & interaction

• Does text meet WCAG contrast minimums?
• Do form fields have labels and tags?
• Are links tagged and described?
• Is the tab order set correctly?

Note: These are the most common checks. There are many more we run on top of these.

Inaccessible PDFs fail your users, and put you at legal risk

Easy to overlook, expensive to ignore.

Excluded users

People who rely on screen readers, keyboard navigation, or other assistive technology depend on a PDF being properly tagged. When it isn’t, they can’t read it properly, if at all. Information everyone else can access stays out of reach.

Legal exposure

Major accessibility laws, including the ADA, Section 508, and the EAA, require the documents you publish to be accessible. Inaccessible PDFs put you on the wrong side of those laws and leave you exposed to complaints and legal action.

Expert PDF accessibility audits and remediation

The online PDF accessibility checker catches only what a machine can verify. For everything it can’t check or fix, our accessibility experts have you covered.

Expert audit

A specialist manually tests your PDF for the issues automation can’t judge, like reading order, alt-text quality, and logical structure. You get back a complete findings report.

PDF remediation

Short on time or expertise? Our team remediates your PDF to meet WCAG and PDF/UA, fixing the tags, structure, and content, then hands it back ready to publish.

Frequently asked questions

Is this PDF accessibility checker really free?

Yes, completely free. No sign-up, no email, and no files stored.

Is this an online PDF accessibility checker?

Yes, it’s a fully online tool, with no software to download or install. Upload a PDF or paste a public link and check it right in your browser.

How do I check a PDF for accessibility?

Upload your PDF or paste a public link. The checker scans it against WCAG and PDF/UA, then lists every issue by severity, with guidance on how to fix each one.

What standards does it check against?

Every machine-verifiable success criterion of WCAG (Level A and AA), and every machine-verifiable test in PDF/UA (ISO 14289).

Can I use this as a PDF ADA compliance checker?

Yes. It checks your PDF against the WCAG and PDF/UA standards that underpin the Americans with Disabilities Act, and flags the accessibility issues that affect ADA compliance. But some checks still need to be done manually to determine whether your PDF is fully compliant.

Is this a PDF 508 compliance checker?

Yes, it checks against the same WCAG and PDF/UA standards that Section 508 relies on, so you can find the issues that affect Section 508 compliance. But some checks still need to be done manually to check whether your PDF is fully compliant.

Does a clean scan mean my PDF is fully compliant?

No. A clean scan means your PDF has passed every machine-verifiable check. But some checks can only be done manually, and those need to pass too before a PDF is considered fully compliant. Our expert audit covers that manual testing.

Does it fix the issues, or just find them?

It finds them and explains them. Each issue comes with a plain-language description and guidance on how to fix it. You or your team apply the fixes in your document. If you’d rather hand it off, we offer full PDF remediation as a paid service.

A beautifully lit room can stop a scroll. It cannot, by itself, explain how to get one. That gap is where many premium service websites lose people, and closing it is the core of website design that converts.

Eclectic Lighting is a Dutch lighting consultancy founded in 2020 by Rick Hommel. The studio combines lighting advice, design, product supply and installation, with work across homes, offices, public buildings, shops and showrooms. Its homepage states the philosophy early: light is part of the architecture, from concept to realization.

Our job was to build a premium, image-led website that does more than look good. It had to help homeowners, architects, and business owners understand the service, see relevant expertise, trust the process, and start an inquiry. The first half comes naturally to most web designers. The second half decides whether the site earns its keep.

Why good-looking websites lose visitors before the inquiry

Lighting is unusually visual. A photograph can make a room look exceptional, but it cannot tell a potential client what the collaboration involves. It also cannot show where the process starts or whether the studio works on their kind of property. Visitors admire a project and still leave without getting in touch.

I think of this as the gap between inspiration and action. A premium site needs bridges that close the gap. On Eclectic Lighting, there are four: project filters by sector, a clear four-step process, specific client reviews, and a floor plan upload. Each one answers a question the photography leaves open.

Photography that leads somewhere

Project photography is essential for a lighting studio. It is the closest a visitor gets to standing in the finished room. Every strong image should still lead somewhere.

On the site, each featured project opens into its own page, like the Deza Kozijnen showroom, with a “view project” link placed on the image. A visitor who stops to admire a room has an obvious next click. We kept the imagery immersive and gave it a direction. The same principle runs through the premium websites in our portfolio, from real estate to chauffeur services.

Four steps that make a bespoke service feel approachable

Premium services often feel abstract from the outside. Prospective clients cannot picture what happens after the first call, so some never make it.

Eclectic Lighting’s process has four steps. It starts with an introduction meeting at the office or on site. The design step produces a 2D lighting plan that shows which light goes where. In the presentation, the client sees that plan alongside a fixture book. Realization follows with a no-obligation quote, and the team can coordinate with the architect, interior designer, installer, and contractor.

Laid out in sequence on the homepage, those steps turn a bespoke service into something a visitor can picture. That reduces uncertainty without making the service feel transactional.

A portfolio organized by the visitor’s context

A homeowner renovating a kitchen and an architect specifying lighting for a public building look for different evidence. One wants to see a home like theirs. The other wants proof that the studio can handle scale and coordination.

The portfolio filters projects into commercial, residential, public buildings, and shops and showrooms. That turns a gallery into a set of answers. Visitors can find work that matches their own project and judge relevance before they judge beauty.

Reviews that make expertise believable

Generic quality claims do little on a premium site. Specific client experiences do more. Eclectic Lighting shows a 4.9 out of 5 rating from 28 reviews, along with longer testimonials.

The longer ones carry the weight. Homeowners describe detailed lighting plans and fixtures labeled to match the plan, which made their installer’s job easier. Interior architects describe repeat collaborations and reliable delivery. Those details tell a new visitor what it feels like to work with the studio, in a way no adjective can.

The floor plan upload as a natural next step

Most contact forms ask visitors to “get started” without telling them what that means. Eclectic Lighting asks for something visitors already have. The contact section invites them to upload a floor plan and receive a no-obligation cost indication for their lighting plan.

That request matches the service. People arrive with a renovation, a deadline, and often a drawing from their architect. Asking for the floor plan lets them bring their actual project into the conversation. It also asks more of web development than a standard form, because the upload has to accept PDFs, Word files, and images.

For visitors who prefer a quick question, a WhatsApp help link sits on the page. Small entry points like these are worth checking on every screen size, and good UI/UX designers treat them as part of the conversion path.

The test I use before a premium website goes live

After a visitor admires the work, do they know whether the studio fits their project, and how to begin? If the answer is yes, the site is doing both of its jobs. If not, the photography is carrying more than it should.

Premium design should feel effortless for the visitor, all the way from the first image to the first inquiry. That is the standard we apply to every website and landing page design we take on.

McKinsey's nearly 90% usage, Ramp's 55.9% paid adoption, and Census's 19.8% measure different businesses, behaviors, and time windows.

WhatsApp is adding four features designed to make busy group conversations easier to manage, including permanent message pins and Group Status updates visible only to members of a particular group. A new filter collects unread conversations containing mentions, direct replies, or @all messages so users can quickly find messages that need their attention. Recent message history can also now be shared with someone after they have joined a group, helping new members catch up without exposing older conversations automatically.

An AI video agent that turns your ideas, documents, links, and media assets into share-ready videos. Describe the video you want to make, upload your assets, and Cutbay helps you shape the idea.

Kontrapunkt extended Copenhagen's JUNO the bakery into a restrained identity and packaging system built around white paper, subtle blue accents, simple duotone illustrations, and the bakery's blue peacock mascot. Its Danish Butter Cookies use the same quiet visual language. COPUS and Tommy Campbell carried the identity into JUNO's first online shop, using blue sparingly for highlights, product tiles, and the add-to-cart button to make the digital experience feel like a window into the physical bakery.

Equine Eyes, an oversized horse-head headset, uses side-mounted cameras and a VR display to simulate a horse's nearly 350-degree field of view and blue-yellow color perception.

Japanese-American illustrator Mike Yoshimura channels 80s and 90s anime, retro cyberpunk, and vintage cel animation into solitary figures in neon-lit rain and twilight cityscapes.