OpenAI introduced a full-duplex voice model that can listen and speak simultaneously, handle natural conversational cues, and delegate complex tasks to GPT-5.5 while maintaining the flow of dialogue.

An off switch for dual-use knowledge in AI models

This post describes research conducted by AE Studio in collaboration with Anthropic.

A frontier AI model is, among other things, a large store of knowledge. Some of that knowledge is dual use, meaning it can be used for good or for bad. For example, knowledge of cybersecurity can help patch critical security vulnerabilities, or it can be used to exploit them. Knowledge of virology can help a researcher create a vaccine, but it can also help a malicious actor design a deadly pathogen. Ideally, we would be able to balance three separate goals: first, limiting access to dual-use capabilities in as surgical a way as possible; second, allowing trusted users to access those same capabilities for beneficial purposes; and third, doing all this without affecting the model’s performance on any other task.

Current safeguards are imperfect. We train models to refuse harmful requests and use classifiers to screen inputs and outputs for dangerous content. These layers of protection guard against dangerous outputs—but they don’t change the knowledge stored in the underlying model. Despite our safeguards, a sufficiently determined attacker may still try to jailbreak the model, working past its defenses to access the dual-use knowledge.

A more robust protection against misuse would be to control what the model knows. We’ve explored this before: in earlier work, we filtered information about chemical, biological, radiological, and nuclear weapons out of pretraining data, and later showed that dual-use knowledge can be confined to a removable slice of a model’s weights. But filtering is a blunt instrument. It produces one model with one fixed set of capabilities. Using filtering, if you want a model version that can discuss advanced virology—for deployment in a vetted biosecurity lab, say—and another version that can’t, you have to train two separate models. Especially in the case of frontier models (which are large and very expensive to train), the cost to the developer would be prohibitive.

In new research carried out with collaborators at AE Studio, we explore a new method that could enable the benefits of training many separately filtered models, but at the cost of training only one model. We call it GRAM, for Gradient-Routed Auxiliary Modules. Note that the results of the experiments presented here are preliminary—GRAM has not been applied to any of the production models at Anthropic, and we’re not sure it ever will be.

How GRAM works

The idea behind GRAM is to give a model dedicated, removable compartments for each category of dual-use knowledge, and to update only those compartments when learning from dual-use data.

Concretely, GRAM adds extra neurons to every layer of a standard Transformer (the neural network architecture on which large language models are based). These neurons are divided into groups (or “modules”), one per dual-use category. During training, when the model encounters general-purpose text, it learns in the usual way. But when it encounters text from a dual-use category—virology, for instance—the rules change: the model can use its general knowledge to make predictions, but only the virology module is allowed to learn from that text. The general-purpose weights are temporarily frozen.

The consequence is that virology knowledge accumulates in the virology module rather than diffusing across the whole network. After training, the module can simply be deleted, and the capability goes with it. Or it can be left in place for trusted deployments, when virology knowledge is needed. The knowledge can be tailored very specifically to the type of deployment needed: in our experiments, we defined four dual-use categories, so that one training run with GRAM yielded a model that can be configured 16 different ways (“on” or “off” for each of the four categories).

Testing GRAM

We tested GRAM in three settings of increasing realism.

First, on a synthetic dataset of children’s stories tagged by topic, a small GRAM model could be reconfigured to “forget” any chosen topic, and each configuration performed almost identically to a separate model trained from scratch with that topic filtered out. That is, for the cost of training a single model, we achieved results that would normally require multiple training runs on different datasets.

Second, we trained a larger model on a realistic mix of web text, code, and scientific papers, with four dual-use domains: virology, cybersecurity, nuclear physics, and a niche programming language (to serve as a proxy for specialized dual-use code). The capability associated with each dual-use domain is routed to its own module. Deleting a module removed the corresponding capability about as effectively as never having trained on that data at all. Remarkably, we find that this removal did not degrade general performance.

We also tested whether an attacker could recover the removed knowledge by training on a small amount of malicious data; GRAM resisted this about as well as data filtering did. By contrast, an “unlearning” technique applied after training only suppressed the knowledge—it was easy to restore with a small amount of fine-tuning.

Third, we ran the experiment at seven model sizes from 50 million to 5 billion parameters. GRAM matched the performance of data filtering at every size, and the gap between “module on” and “module off” grew wider as models got larger. In terms of compute costs, attempting to bypass our protections became relatively more difficult and expensive as we scaled.

Conclusions

As AI companies train more capable models, the need to limit access to dual-use capabilities will increase. Today, companies limit access through classifiers and refusal training. However, these safeguards are difficult to make robust without degrading performance on harmless requests. Methods like GRAM offer a potential path toward access control that is more robust.

This is early research, and there are clear limitations. We haven’t tested GRAM at frontier scale or in a production training pipeline. (As noted above, it hasn’t been applied to any of our Claude models.) Our evaluations quantify performance in terms of next-token prediction ability, rather than performance on real downstream tasks. And there’s a deeper open problem that applies to data filtering and methods like GRAM: some dual-use capabilities might be so entangled with general knowledge that no method can separate them cleanly.

For further details on our experiments, read the post on our Alignment Science blog.

Footnotes

1. One technical detail is that the virology module is also sometimes turned on when learning from general-purpose text. We find this helps the modules “work together” more effectively.

OpenAI examined SWE-Bench Pro's construction, model failures, and task metadata, concluding that roughly 30% of its public tasks were broken. The analysis showed how flawed evaluations could distort assessments of coding ability, safety, and model progress.

Today, we’re launching SWE-1.7, the most capable model we’ve trained so far. It reaches frontier-level intelligence at a much lower cost, advancing the cost-performance Pareto curve.

SWE-1.7 is the result of broad improvements across our RL pipeline: better infrastructure, more stable training, higher-quality data, and new techniques for long-horizon tasks. Since SWE-1.7 was trained from a Kimi K2.7 base, which had already undergone extensive RL post-training, the large additional gains from our own training challenge the idea of a ‘post-training ceiling’ and suggest that RL can push capabilities much further than previously believed.

At Cognition, we have been formulating and refining principles for good agentic software engineering both in evaluation, with FrontierCode, and now in training, with SWE-1.7. Our model is particularly optimized for longer-horizon asynchronous tasks, an important component of high-quality software engineering.

SWE-1.7 is available today in Devin (Web, Desktop, and CLI) via Cerebras at 1000 TPS. We encourage you to try it for yourself!

The rest of this post covers how we trained SWE-1.7: the infrastructure, algorithms, and data work behind our model. We cover four important components that stand out.

• Preserving entropy and stabilizing training: Long RL runs face two challenging problems: entropy collapse, and instability due to numerical drift between training and inference. We hunted down and addressed causes of each, which enabled training to keep improving well past where earlier runs stalled.
• Multi-cluster training and fault tolerance: RL doesn’t need all of its inference compute in one cluster. We trained on clusters across three continents, shipped weight updates through object storage, and built fault tolerance so that hardware failures never stalled the run.
• Curating high-quality data: We built an extensive data-quality pipeline that runs each task through automated execution tests, filters out tasks with low learning signal, and hardens tasks to prevent reward-hacking.
• Self-compaction for long-horizon tasks: The model learns to summarize its working state and resume from the summary, extending task horizons past the raw context window. We use an alternating length penalty to incentivize concise output without sacrificing correctness.

Preserving Entropy and Stabilizing Training

We found training stability to be a key contributor to predictable improvement at scale.

When training with asynchronous RL, one of the most problematic issues we encountered was the KL divergence mismatch between inference and training, since the trainer policy is usually different from the sampling policy.

We find that top-p sampling contributes significantly to staving off entropy collapse, where a strong model stops exploring and reward plateaus within a few hundred steps.

This entropy-preservation effect makes top-p sampling desirable in our rollouts. But naively implementing top-p clearly increases the training-inference mismatch — the trainer computes probabilities as a selection from all tokens, while rollouts sample from the top-p subset, so the distributions have higher divergence, leading to collapse after a small number of steps. Thus, we implement sampling distribution replay, where we record a kept-set of tokens available for sampling at rollout time, and renormalize probabilities with those masks in the trainer.

Multi-cluster Training

Cognition is a fast-growing research lab entering an established landscape that is heavily compute-constrained. We aim to train trillion-parameter models, but today, large clusters with 10-100k chips on a single network fabric are a scarce resource. In contrast, smaller clusters around the world are abundant, if used together correctly.

In this setting, the structure of RL works in our favor. RL decomposes naturally across multiple clusters. Only the trainer must live on a single high-bandwidth cluster. The inference engines that generate rollouts are self-contained. They can run anywhere and need nothing but the current weights.

We invested in infrastructure that makes use of this property. Our RL training spans four datacenters across three continents, combining our own GPUs across multiple clusters with additional compute from inference providers like Fireworks.

The central challenge in this setup is keeping all inference engines up to date with the trainer weights after each optimizer step. We want these weight updates to be fast to reduce staleness of trajectories so we can train with more aggressive learning rates.

Naively broadcasting the full model from one cluster to another would be slow and inefficient. Instead, every K gradient steps, we compute and send a compressed weight delta between the current and previous weights, reducing the size of each transfer by over 99%.

Fault Tolerance

At large scale, hardware failures occur continuously, and globally restarting on each failure makes long runs infeasible. Our architecture handles this differently depending on where the failure occurs — the inference engines or the trainer.

Failures on the inference side are cheap by construction. Engines are self-contained and hold no state beyond the current weights, so a dead engine costs only its in-flight sessions. The trainer is the one place where a failure is expensive: it’s the single tightly-coupled component, where one dead node stalls the whole cluster. To make recovery fast, each node checkpoints asynchronously to local disk every step and replicates its shards to peers, so a dead node’s state is rebuilt from replicas in seconds.

Intelligent Self-Compaction for Long-Horizon Tasks

From the start, we built Devin for completing asynchronous, long-running tasks. SWE-1.7 is trained directly in the Devin harness, so naturally we want to train our own model on longer horizon tasks.

• When an agent approaches the context limit, we ask it to summarize its working state, and we resume it from its self-authored summary. During training, the model simultaneously learns (1) to write more informative, succinct summaries, and (2) to better work from and leverage such summaries.
• Rather than applying a length penalty uniformly throughout training, we use an alternating strategy. In unconstrained phases, the model optimizes only for task success. In budget phases, we penalize solutions that exceed a certain budget of our weighted cost function that includes tokens, turns and total time spent in tool calls.

Data Quality

• Verifier quality: A task’s verifier can be wrong in two directions: accepting incorrect solutions (false positives) or rejecting valid ones (false negatives). We devised extensive quality-assurance pipelines to minimize observations of false positives and false negatives in our training.
• Difficulty: On tasks where the model always solves or always fails, we don’t observe any meaningful learning signal. Instead, we curated training data that the model only solves a low fraction of the time.
• Cheating detection and prevention: We employed a variety of defenses against different forms of cheating. For instance, we network-restricted our sandboxes and stripped them of git history and reference artifacts.

Results: Model Behaviors

Due to extensive RL, SWE-1.7 exhibits noticeably different behavior from Kimi K2.7 Code, its base model. Firstly, it is significantly more aligned and trustworthy than K2.7 or other frontier open-source models.

One behavioral difference we noticed in SWE-1.7 is condensed chain-of-thought. Compared to Kimi-K2.7-Code, SWE-1.7’s first chain-of-thought has a much lower function-word ratio and nearly half the average number of words per sentence.

The other major behavioral difference we observed is that SWE-1.7 explores the codebase much more thoroughly before acting, as you can see in the number of tool calls, file reads, and searches the model executes.

SWE-1.7 is much more likely to investigate the root cause of the bug and consider edge cases, hypotheticals, adversarial inputs, and beyond-the-ask requirements than Kimi-K2.7-Code. Through its enhanced codebase exploration, SWE-1.7 also does a much better job understanding the exact design decisions that need to be made.

Introducing Robostral Navigate

Robostral Navigate is an 8B model that enables robots to autonomously navigate complex environments using only a single RGB camera, achieving 76.6% success on unseen R2R-CE benchmarks—outperforming multi-sensor approaches while being more efficient. Built entirely in-house with simulated data and token-efficient techniques, it generalizes across robot types and adapts to real-world obstacles unseen during training. The model combines pointing-based navigation with reinforcement learning for continuous improvement, paving the way for unified embodied AI in robotics.

Today we're introducing Robostral Navigate, our first model built for embodied navigation. It's an 8B model that takes RGB images and a plain-language instruction and moves a robot through an environment:

“Leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf.”

To perform such tasks, other models often employ depth sensors, LiDAR, or several cameras working together. Robostral Navigate uses only one ordinary RGB camera and no depth sensors, yet still achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments) validation unseen, the benchmark for following instructions in environments held out of training. Consequently, it beats the best single-camera approach by 9.7 points and the best system using depth or multiple cameras by 4.5 points, despite using neither.

Navigation

Our model is designed for robotic navigation, enabling robots to autonomously navigate complex environments, including offices, residential and commercial buildings, and outdoor settings.

This technology unlocks numerous applications across manufacturing, delivery, logistics, and hospitality, making it one of the most in-demand capabilities for our customers today. Give Robostral Navigate one instruction and it completes the entire task on its own, moving through a live space full of people and obstacles it was never shown, capable of adapting to any setting.

Highlights

• State-of-the-art performance on R2R-CE
  • 79.4% Success Rate on validation seen
  • 76.6% Success Rate on validation unseen
• Operates from a single RGB camera, with no LiDAR or depth sensors
• 8B model, built in-house and trained entirely in simulation
• Runs on wheeled, legged, and flying robots, and generalizes across robot sizes
• Robust to differences in camera intrinsics
• Token-efficient training via prefix-caching

Navigation via pointing

Given a task and a history of observations, Robostral Navigate predicts where the robot should move next via pointing: it infers the image coordinates of the target location in the robot's current camera view, together with the desired orientation upon arrival. Unlike commands relying on metric displacements, pointing makes the policy naturally robust to changes in camera intrinsics and world scale.

However, this method cannot handle cases where the target location lies outside the current field of view. When pointing does not apply, the model falls back to displacements in the robot's local coordinate frame, such as:

"Move 2 meters forward, 1.5 meters to the left, and turn 25 degrees left."

Built from the ground up

Robostral Navigate is built entirely in-house and does not rely on existing open-source VLMs.

The model is initialized from our vision-language model specialized for grounding tasks such as pointing, counting, and object localization. Navigation emerges as a natural extension of these capabilities: once it understands where things are, it learns how to move.

We built an efficient data generation pipeline entirely in simulation. This enabled rapid iteration on the data, resulting in a dataset of approximately 400,000 trajectories collected across 6,000 scenes.

Efficient supervised training

A key ingredient of Robostral Navigate is an efficient training algorithm based on prefix-caching. Using a tree-based attention-masking strategy, our method compresses an entire episode into a single sequence, enabling training on all time steps in a single forward pass while preventing information leakage between time steps.

Compared to training with one sample per time step, our approach reduces the number of training tokens by 22× while preserving all of the learning signals. In practice, this method transforms training runs that would take months into runs that complete in days.

Online reinforcement learning

We leverage our knowledge of post-training LLMs at scale, using online reinforcement learning, to boost the performance of Robostral Navigate. After the supervised training stage, we further improve the model's performance using CISPO, an online reinforcement learning algorithm. This enables the model to learn from trial and error, recover from failures, and acquire exploratory behaviors, effectively mitigating the distribution shift issue of vanilla behavior cloning. This alone improved the success rate by 3.2%. We are not seeing any plateauing, so we are confident that more training and more experiments will continue to push this number up.

What's Next

Robostral Navigate is only the first step toward a unified embodied agent.

We believe navigation is a foundational capability for general-purpose robotics. By combining large-scale simulation, efficient training, and strong grounding priors, Robostral Navigate demonstrates that state-of-the-art embodied navigation can be achieved with a compact model and a single RGB camera.

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

Full article content is not available for inline reading.

Read the original article →

AI applications increasingly need to search data by meaning, not just by exact fields or keywords. Product descriptions, support tickets, documents, images, and other unstructured data are becoming first-class inputs to analytics and application workflows. But adding semantic search to lakehouse data has usually meant introducing a separate vector database, copying embeddings into it, and maintaining a sync pipeline to keep records, metadata, deletes, and updates aligned.

Apache Hudi's native vector search brings that workflow closer to the table itself. Embeddings can live in the same Hudi table as the rest of the row, represented as a first-class VECTOR column and queried directly from SQL. This lets semantic search compose with structured filters, joins, updates, time travel, and incremental queries instead of requiring a separate sidecar system.

Hudi exposes vector search through two SQL functions: hudi_vector_search for a single query vector and hudi_vector_search_batch for running top-k search across a table of query vectors. Embeddings can be stored as FLOAT, DOUBLE, or INT8, and compared using cosine, L2, or dot product distance. The initial implementation uses distributed brute-force KNN, while an optimized ANN vector index is being developed behind the same SQL interface so applications can adopt the API today and benefit from faster algorithms as they land.

What is Semantic search?

Consider an example of a user interacting with an ecommerce site that has millions of product listings. Typically in most data analytics systems, a user would have to craft a query on exact fields, such as category, brand, price, size, or inventory_status in order to find a match.

But many searches start with intent, and not direct fields. A user instead might search for "comfortable shoes for walking around Europe," even though there is no column called comfortable and no guarantee that the best listings use those exact words. Some products may say "travel sneakers," "all-day support," or "lightweight walking shoes" instead.

To handle this, semantic search is a way to search by meaning rather than by exact fields or keywords. It works by converting text, images, or other content into what are known as "vector embeddings".

A vector embedding is a fixed-size array of numbers that represents the meaning of some piece of data, such as a product description, customer review, image, support message, or document chunk.

Embeddings are typically produced by machine learning models. For text, this might be an embedding model or LLM-based encoder that converts a sentence, paragraph, or document chunk into a vector. For images, it might be a vision model that converts visual content into the same kind of numerical representation.

For example, a marketplace product description like:

Lightweight sneakers with cushioned soles for all-day walking

might be converted into a vector that looks conceptually like this:

[0.12, -0.03, 0.44, ..., 0.08]

The individual numbers in an embedding are often called features. Each one helps place the original item somewhere in a high-dimensional space. For a product description, the model may learn features related to things like product type, use case, style, material, comfort, or activity. For an image, the features may capture patterns related to shape, color, texture, or visual similarity.

These features are usually not directly interpretable on their own. One number does not simply mean "comfort" or "travel." Instead, the full vector captures many learned signals together. What matters is how one vector relates to another. Products with similar meaning should produce vectors that are close together, even if the words are different.

That is what allows a search like "comfortable shoes for walking around Europe" to match products described as "travel sneakers," "supportive walking shoes," or "lightweight everyday trainers," even when those listings do not use the exact same words as the query.

The need for both structured and unstructured data

The marketplace example shows why modern AI applications need both unstructured and structured data. The meaning of a product may live in descriptions, reviews, images, or support conversations, which can be converted into embeddings and searched semantically. But deciding whether a result is useful still depends on structured fields such as price, size, inventory_status, shipping_region, and seller_rating.

This pattern has become more important with the rise of LLM-powered applications. Search, recommendations, RAG, document intelligence, and agent workflows increasingly operate over text, images, and other unstructured data. But these applications still need the guarantees of structured data systems: filters, joins, updates, deletes, freshness, and governance.

This is where the lakehouse becomes a natural fit. In many AI applications, the embedding is not an independent object. It is derived from the row itself: a product description, review, image, support conversation, document chunk, or some combination of fields. When that source data changes, the embedding needs to change with it. When a row is deleted, expired, corrected, or reprocessed, the corresponding embedding needs to follow the same lifecycle.

A standalone vector database can be a good serving layer, especially for ultra-low-latency retrieval at very large scale. But if the source records live in the lakehouse, a separate vector database introduces another copy of the data. That copy has to be kept in sync with the table, including inserts, updates, deletes, schema changes, metadata filters, and backfills. Over time, the hard problem becomes less about computing nearest neighbors and more about maintaining consistency between the source data and the vector index.

Hudi vector search is built around a simpler model: keep the original records, structured metadata, and embeddings together in the same table. In an ingestion pipeline, new and changed records can be processed incrementally, embeddings can be generated or refreshed from the latest source fields, and both the source data and vector columns can be committed together into Hudi. That makes the Hudi table the single source of truth for both analytics and semantic search.

For example, Hudi Streamer can ingest changes from sources such as Kafka or cloud storage into a Hudi table. An embedding step can be added to the pipeline so that when product descriptions, reviews, images, or metadata change, the corresponding embeddings are updated as part of the same table workflow.

How does Hudi represent VECTOR?

Once embeddings are produced, they need to be stored in a table in a way that preserves their meaning. A plain array of numbers can store the values, but it does not tell the table system that the column is a fixed-dimensional vector intended for similarity search.

Hudi represents embeddings using a first-class logical type: VECTOR(dimension) or VECTOR(dimension, element_type). The dimension value defines the fixed length of the vector. For example, VECTOR(768) represents an embedding with 768 values.

A product catalog table can declare a vector column directly in SQL DDL:

CREATE TABLE products (
  product_id string,
  title string,
  category string,
  brand string,
  price double,
  inventory_status string,
  description string,
  description_embedding VECTOR(768)
) USING hudi;

When the element type is omitted, Hudi uses FLOAT by default. This is the most common representation for embeddings produced by text and image models. Hudi also supports DOUBLE and INT8 for workloads that need higher precision or more compact storage.

The storage difference becomes meaningful at scale. A VECTOR(768, FLOAT) stores 3,072 bytes of raw vector values per row. The same vector represented as INT8 stores 768 bytes per row. Across millions or billions of rows, that difference affects table size, index size, and query cost.

At the Hudi table level, both the SQL DDL form and the DataFrame API form describe the same logical column: a fixed-dimensional vector that can be validated, indexed, and used for vector search. At the physical storage layer, Hudi maps the logical vector type onto storage formats in a way that balances efficiency and interoperability. Readers that do not understand the Hudi vector logical type can still access the underlying data, while Hudi-aware readers can validate dimensions and treat the column as a single vector value.

Today, vector columns are supported as top-level columns. They cannot yet be nested inside structs, arrays, or maps. For data models that naturally contain multiple embeddings per entity, such as a product with many variants, the current pattern is to model each embedded item as its own row. Nested vector support can be added in the future as the type system and query semantics evolve.

Querying with Hudi Vector Search

Once embeddings are stored in a Hudi table, they can be searched directly from SQL. Hudi provides two table-valued functions:

• hudi_vector_search for one query vector
• hudi_vector_search_batch for many query vectors

The difference is easiest to understand through the marketplace example.

A single-query search answers a question like: Given this shopper query, find the 10 most similar products.

A batch search answers a larger version of the same question: Given this table of many shopper queries, products, or documents, find the 10 most similar products for each row.

So single-query search is one nearest-neighbor lookup. Batch search is the same lookup repeated for every row in another table, executed as one distributed job.

Single-query vector search

hudi_vector_search takes the table to search, the embedding column, a query vector, and k, plus a few optional arguments: the distance metric (cosine, l2, or dot_product), a filter predicate applied before distance computation, and a max_distance cutoff to drop weak matches.

For example, a shopper might search for "wireless headphones for travel." The application converts that text into an embedding and searches the product catalog:

SELECT product_id, title, brand, price, _hudi_distance
FROM hudi_vector_search(
  'product_embeddings',
  'embedding',
  ARRAY(/* embedding of the shopper query */),
  10,
  'cosine',
  'brute_force',
  'category = ''electronics'' AND in_stock = true',
  0.4
);

The filter is important. Since the first implementation uses brute-force search, reducing the candidate set before distance computation is one of the most effective ways to control latency. Searching 50 million product embeddings is expensive. Searching only the in-stock electronics products may reduce the search space to a much smaller subset.

The max_distance argument is another practical control. It removes results whose distance is above the threshold, so the query does not return weak or irrelevant matches simply to fill the requested k.

The output includes all corpus columns except the embedding column, plus _hudi_distance. Results are ordered by distance ascending, so smaller values are more similar.

Batch Vector Search

Batch search is useful when you do not have just one query. Instead, you have a whole table of things that each need their own nearest-neighbor results.

That is what hudi_vector_search_batch does: it runs top-k vector search for every row in a query table. It takes the same arguments as the single-query form, but searches with a whole query_table (and its query_col) instead of a single vector, returning the top k corpus rows for each query row.

For example, suppose a marketplace wants to precompute related products every night. The input is not one shopper query. The input is a table of products:

SELECT
  query_product_id,
  product_id AS related_product_id,
  title,
  _hudi_distance,
  _hudi_query_index
FROM hudi_vector_search_batch(
  'product_embeddings',
  'embedding',
  'query_products',
  'query_embedding',
  20,
  'cosine',
  'brute_force',
  'in_stock = true',
  0.4
);

The result contains the matching corpus rows, the query row metadata, and the computed distance. Hudi omits the embedding columns from the output because applications usually need the matching records and scores, not the raw vectors.

Batch search also adds _hudi_query_index, which identifies which query row produced each result. This is useful because the output contains many groups of top-k results: one group per query row.

Put simply:

• hudi_vector_search returns the nearest rows for one query.
• hudi_vector_search_batch returns the nearest rows for each query in a query table.

Batch search is especially useful for offline or scheduled workloads: related-product generation, duplicate detection, content matching, recommendation refreshes, and document similarity jobs.

How the brute-force implementation works

For a single-query search, the execution flow is:

1. Spark receives the query vector and broadcasts it to executors.
2. Each executor scans its portion of the corpus.
3. For each row, the executor computes the distance between the query vector and the row's embedding.
4. Each executor keeps only its local top k matches.
5. The driver merges those local top-k results into the final global top-k result.

This keeps executor memory bounded by k, rather than by the total size of the corpus. Even if an executor scans millions of rows, it only needs to keep the best candidates it has seen so far.

The main cost is distance computation. If the query searches 50 million embeddings, Hudi has to compare the query vector against 50 million candidate vectors. If the query first filters the corpus down to 200,000 relevant rows, the amount of distance computation drops dramatically.

That is why the filter argument in hudi_vector_search matters. It lets applications apply structured predicates before vector distance computation runs. For example, an ecommerce query may search only products where category = 'electronics' and in_stock = true, rather than searching the entire product catalog.

Batch search follows the same idea, but repeats it for many query vectors in one distributed job. Instead of asking, "what are the nearest products for this one shopper query?", batch search asks, "what are the nearest products for every row in this query table?"

For example, a nightly recommendation job might take every active product as a query row and find the top 20 related products for each one. Hudi compares each query embedding against the filtered corpus, ranks matches per query, and returns one group of top-k results for each query row.

Because batch search multiplies the amount of work by the number of query rows, the candidate set matters even more. A small query table searched against a well-filtered corpus can be practical. A large query table searched against an unfiltered corpus can become expensive quickly. This is why batch search is best suited for offline or scheduled workloads such as related-product generation, duplicate detection, content matching, and recommendation refreshes.

The max_distance argument is useful in both single-query and batch search. It removes weak matches above a distance threshold before returning results. In batch jobs, this can reduce output size significantly because each query row may otherwise produce up to k results, even when some matches are not useful.

This is why vector search becomes more powerful when it is integrated with table semantics. Many applications do not want nearest neighbors across all data. They want nearest neighbors within a meaningful slice of the table: active products, recent documents, records for a specific customer, listings in a region, or items that pass business rules. It is most useful when combined with the same filtering, pruning, and table layout techniques that data systems already use to reduce the amount of data read.

Future Vector index

The SQL interface is designed to stay stable as new algorithms are added. Today, algorithm => 'brute_force' uses a distributed linear scan. In the future, indexed implementations such as IVF-PQ can replace the scan with an index lookup plan.

In that model, Hudi would use a vector index to produce candidate record IDs and distances, then join those candidates back to the table to return the same output shape. Applications written against the current table-valued function interface should not need to change when switching from brute-force search to indexed search.

Conclusion

For the past few years, adding semantic search to a lakehouse application usually meant adding a separate vector database, copying embeddings into it, and maintaining a sync pipeline between the two systems. That architecture works, but it adds operational complexity at exactly the point where correctness matters: keeping records, embeddings, metadata, deletes, and updates aligned.

Hudi vector search takes a different approach. Embeddings are stored in the same table as the rest of the row, with the same schema, snapshots, updates, and table services. Vector search becomes part of the query layer instead of a separate sidecar system that has to be synchronized with the lakehouse.

The first implementation focuses on making this model simple and usable: a VECTOR type for embeddings, SQL functions for single-query and batch vector search, and integration with Hudi's existing table semantics. This is especially useful for RAG, document intelligence, recommendation refreshes, deduplication, clustering, and similarity search jobs where correctness, freshness, and SQL composability matter as much as raw serving latency.

The best way to understand the model is to try it directly. Start with Hudi 1.2, create a table with a VECTOR column, load a small set of embeddings, and run your first hudi_vector_search query. From there, try adding structured filters, experimenting with hudi_vector_search_batch, or materializing similarity results back into a Hudi table.

If you are building AI applications on lakehouse data, this is the core idea to test: semantic search does not have to live outside the table. With Hudi, embeddings, metadata, updates, and search can move through the same data system.

Why Your Semantic Layer Matters More Than Your AI Agent

How I built a governed semantic model that eliminates silent analytics failures: the ones where the query succeeds and the number is wrong.

A data warehouse knows what columns exist. A semantic model knows what they mean. This is the story of how I built the layer between the two, and why I believe the model, not the AI agent, is the product.

Key takeaways

• The most common silent analytics failure is not a broken query. It is a successful query that returns the wrong number. A semantic model makes this structurally impossible.
• The AI interprets user intent and translates it into a semantic query. A deterministic engine generates the SQL. Same inputs, same output, every time. No LLM is involved in SQL generation.
• The model is organized in two layers: a physical layer that mirrors the warehouse schema, and a model layer that defines what the data means. They change independently, owned by different people.
• Every piece of metadata provided to the AI (disambiguation hints, semantic types, canonical filter values) was added to fix a specific, observed failure. Metadata is infrastructure, not documentation.
• AI can compress semantic model development from months to weeks by generating structured drafts for human review, eliminating the blank-page problem.

The problem nobody solves with better prompts

Every data platform eventually confronts the same question: where does the definition of "revenue" live? Is it in the SQL query an analyst wrote last quarter? In the transformation pipeline a data engineer maintains? In the dashboard someone configured two years ago that nobody wants to touch? In the Slack thread where someone explained to the CFO that "net revenue" doesn't include refunds but "total revenue" does?

In most organizations, the answer is "all of the above." Metric definitions are scattered across queries, dashboards, documentation, and tribal knowledge. The same column is interpreted differently by different teams. Hand an LLM the raw warehouse schema and it will confidently join two tables that shouldn't be joined at the grain of the question, because nothing in the schema says otherwise.

A user asks: "Show me Net Revenue by Consumer Region for last quarter."

Seven words, three layers of ambiguity. "Net Revenue" is not a column. It is a derived metric spanning three tables with different grains. "Consumer Region" requires a historically-versioned join where the correct row depends on when the order was placed, not today's date. "Last quarter" could mean the calendar quarter or the fiscal quarter, depending on which team is asking. An LLM will make a confident choice for each ambiguity. It will be wrong about at least one, and the resulting number will look plausible.

The silent failure mode

A SQL syntax error is visible: the query fails, the user retries. A wrong metric definition is invisible: the query succeeds, the number lands in a slide deck, and a decision is made on a figure that was never correct.

The semantic model exists to make this second failure mode structurally impossible.

More schema documentation does not close this gap. You can annotate every column with a description, and an LLM still will not know that two columns in different tables represent the same concept, or that a dimension table requires a time-windowed join. What you need is not more metadata on the schema. You need a separate layer that defines what the data means, independent of how it is stored.

The approach: separate what changes for different reasons

The central design principle is simple: things that change for different reasons should live in different places.

Warehouse schemas change when data teams add new tables, introduce new columns, migrate to a new warehouse, or restructure data models. Business definitions change when analysts refine what "net revenue" means or a new team needs a different lens on the same data. These are different rates of change, driven by different people for different reasons. Mixing them in the same layer means every schema change risks breaking a metric definition, and every metric change requires touching schema files.

I separated them into two layers.

Two-Layer Architecture

The physical layer is a complete mirror of the warehouse schema: every table, every column, every join path between tables. It is the single source of truth about what the warehouse contains. If a column is not declared here, the system cannot reference it. If a join is not declared here, the system cannot perform it.

The model layer sits above it and defines what the data means. Business domains like "delivery" or "subscription" are modeled as entities that pull from multiple physical tables, define metrics with precise formulas, and compose through inheritance. An analyst thinks in domains, and the model layer mirrors that vocabulary exactly.

What this separation buys you

When the warehouse changes (a column is renamed, a table is migrated): only the physical layer is updated. Every metric and business definition continues to work untouched.

When a business definition changes ("net revenue" now includes a new deduction category): only the model layer is updated. The warehouse schema files are untouched.

When a new team needs a different lens (a Finance perspective on the same delivery data): one new file is added. The core domain model it inherits from does not change.

Each change touches exactly one layer. This is the core design constraint, and it is what makes the system maintainable as the number of metrics, teams, and use cases grows.

Everything is defined in version-controlled configuration files, reviewed in pull requests the same way application code is. A metric definition change is a two-line diff. The alternative (metric definitions stored in a database, managed through a UI) trades reviewability for convenience. I chose reviewability.

How the model solves the hard problems

A key architectural decision: the AI interprets the user's question, translates it into a semantic query (selecting the right metrics, dimensions, filters, and date ranges from the governed catalog) and hands it to the semantic engine, which handles all SQL generation. The AI understands intent. The engine guarantees correctness. Same semantic query in, same SQL out, every time.

When multiple metrics share the same name

A user asks for "revenue." The model contains total revenue, net revenue, gross revenue, and revenue per order. Without guidance, the agent picks one (silently wrong most of the time) or asks a clarifying question when the answer is obvious from context.

I solved this by making every metric carry more than just a formula. Each metric has a certification level (is this verified by the data team or still experimental?), a semantic type (is this a rate that should never be summed, a duration that should be averaged, or a cumulative total?), and free-form disambiguation hints written by the metric author. The agent reads these hints in context and resolves ambiguity without asking. Each of these metadata fields was added to fix a specific, observed failure. They are not documentation; they are operational guidance for the AI.

When joins are ambiguous or historically versioned

In a parcel delivery business, a delivery has two consumers: the person who placed the order and the person who receives it. Both live in the same consumer table. "Show me revenue by consumer region" is ambiguous: which consumer?

And even once you pick the right consumer, their region may have changed over time. The consumer table has multiple historical rows per person. Joining without accounting for the time dimension silently shifts revenue between regions, or worse, multiplies it.

I solved both problems in the model. Every join between tables is explicitly declared with its cardinality, its join logic, and whether it requires time-windowed matching. The query generator does not infer joins from column names or foreign keys. If a join is not declared, it does not happen. For historically versioned dimensions, the system automatically resolves the correct row based on when the transaction occurred, not the current date.

When summary models exist and the AI can't tell when to use them over the fact table

Pre-aggregated tables exist for performance. But they carry restrictions. A daily delivery aggregate might be pre-built excluding cancelled deliveries. If a user asks for "revenue by delivery status" and the system routes to that aggregate, the cancelled row is missing entirely. The total is lower than reality.

I solved this with a strict eligibility check. Before routing any query to an aggregate, the system evaluates whether the aggregate can safely answer the entire question. The decision is binary: the aggregate serves the whole query, or the system falls back to the full table. There is no partial routing, no hybrid mode. It is better to be correct and slower than fast and wrong.

When the AI guesses filter values

"Show me cancelled orders." Is the filter value "cancelled", "Cancelled", "CANCELLED", or "cancel"? A wrong guess produces a valid query that returns zero rows. The user assumes there are no cancelled orders.

I solved this by declaring the exact valid values for every enumerated column in the physical layer. These canonical values are extracted when the model loads and provided to the AI as part of its operating context. The agent matches against exact strings.

When the AI misunderstands how a metric should be aggregated

"What's the total cancellation rate across all regions?" If the agent sums individual region rates instead of computing a weighted average, the resulting number is meaningless.

I solved this by tagging metrics with semantic types: rate metrics should be displayed as percentages and never summed, duration metrics should be compared with averages, cumulative metrics should not be double-counted. These rules flow into the AI's context as behavioral guidance.

Why you can trust it

Errors are caught when the model is defined, not when a user runs a query. Every model goes through structural validation before it can serve a single query. Circular dependencies, missing fields, broken references—all are rejected immediately when the model author saves their work.

The query generator is deterministic and traceable. Given the same inputs, it produces the same SQL every time. No probabilistic reasoning. A human reading the generated SQL can follow every clause back to a specific model definition.

The model detects when the warehouse changes under it. The system compares its definitions against the live database and reports exactly where they disagree: which columns were renamed, which were added, which changed type. This turns an invisible problem into a visible, actionable one.

Getting started: how AI solves the cold-start problem

Building a semantic model by hand is prohibitively expensive. For an organization with hundreds of tables and thousands of columns, this is months of work.

I believe AI can compress this timeline from months to weeks, not by replacing human judgment, but by eliminating the blank-page problem. An AI-powered Studio Agent reads the physical data model and generates a structured draft of the semantic model: proposed metric definitions, inferred join paths, temporal rules, and dimension hierarchies. Every proposal carries supporting evidence.

Building a semantic model from scratch is authoring a document from a blank page. Reviewing a structured, evidence-backed draft is reviewing a pull request. The cognitive load is fundamentally different. The person doing the review still needs domain expertise, but they are correcting and approving, not creating from nothing.

Keeping it clean: preventing metric proliferation

Getting started is one problem. Keeping the model healthy as it grows is another. Without guardrails, teams create variations instead of reusing existing definitions. The guardrails are structural. When a new metric is proposed, the system checks for semantic overlap. Equivalent metrics with different filters get redirected to the existing metric with a filter parameter.

The model is the product

The semantic model is not scaffolding you build so the AI can do interesting work. The model is the interesting work. It encodes institutional knowledge about what the data means, how metrics relate to each other, which dimensions make sense together, and where the edge cases hide. The AI agent is a delivery mechanism for that knowledge. The model is the substance.

Every improvement to the model directly improves every query the system will ever generate. In my experience, improving the model delivers higher returns than any amount of prompt engineering. The model is the leverage point.

"A great model with a mediocre agent produces better answers than a mediocre model with a great agent."

President Donald Trump’s plan to “fill the digital potholes” and use AI to quickly redesign every government website isn’t going very well.

Last August, Trump created the National Design Studio, or NDS, by executive order. A temporary DOGE-like entity that answers only to the president, NDS was tasked with creating new standards to update the US Web Design System (USWDS) and overhaul 27,000 dot-gov websites in just three years. At the end of this so-called “America by Design” initiative, the government’s “design language” would supposedly be more usable and beautiful, Trump expected.

However, that monumental task—assigned to a small team under a short timeframe—was seemingly made harder by DOGE’s deep cuts to agencies previously responsible for improving government websites, including dismantling the 18F technology unit and restructuring the US Digital Service into DOGE.

Those teams knew exactly how hard it can be to get every government agency to adopt new web standards. They had spent years trying to push agencies to update their sites to comply with USWDS standards, yet “only 30 percent of government websites used them as of mid-2023,” NextGov reported. Notably, the USWDS team—which was created in 2015 to ensure government websites were accessible and mobile-friendly—was reduced to one full-time employee after Trump took office.

Most people agree that updating government websites is a worthwhile and necessary endeavor. But about a year into NDS’s existence, the team hasn’t accomplished much.

Its biggest achievement has been modernizing the federal retirement system. However, former government workers have accused the Trump administration of claiming “false victories and overstated credit,” noting that the project was underway before NDS was created.

The group’s other output has been meager, with few launches and substantial backlash from design experts, who argue that the team relies too heavily on AI and has failed to test sites for compliance with the Americans with Disabilities Act. As scrutiny of NDS has intensified, most agencies are now resisting connecting with the team about adopting new web standards, Ars has learned.

Single-page launches, odd redirects

Ars conducted a comprehensive review of launched sites to assess NDS’s progress so far.

Most of the few dozen websites NDS has launched consist of a single page, where visitors can do little more than fill out a sign-up form. The most useful offering may be TrumpRX, which includes a search tool for comparing drug prices. For anything else, visitors must visit a legacy site.

There are also many newly registered domains—like live.gov, onlyfarms.gov, aliens.gov, and why.gov—which currently redirect to legacy sites. Some of those old sites may be updated with NDS’s signature flair, but the ones that don’t look as pretty remain the primary resource for Americans seeking government information or assistance online.

At least one website, 250.gov, which celebrates 250 years of US history, curiously redirects to a dot-org rather than a dot-gov, which is unusual for a government site and could erode visitor trust.

Among the few larger sites that NDS has launched is its own, ndstudio.gov. Currently, that site catalogs the team’s launches, shares a brief timeline of US design achievements, discusses the team’s AI and accessibility efforts, and encourages talented designers to “apply now.”

It also briefly hosted a store marketing a $47 limited edition MAHA poster and a $400 “collector’s edition” with Secretary of Health and Human Services Robert Kennedy, Jr.’s autograph, NextGov reported. The store disappeared after the White House faced questions about where the profits from sales would go. A White House spokesperson told NextGov that the posters were never “actually for sale,” as the store’s items did not include a “purchase button.”

The only other site of similar scope is merrychristmas.gov, which, beyond the homepage, includes one page for each of the 12 days of Christmas. An apparent vanity project rather than a government resource, the site is also a celebration of NDS designs and culminates on Christmas with a page praising the group for building sites reflecting “a belief that thoughtful design can strengthen democracy and improve civic life.”

These sites are supposed to “delight” Americans, Joe Gebbia, the Airbnb cofounder serving as Trump’s chief design officer, told NextGov in February. The next month, he told Fox News he wanted visiting government websites to feel like “an Apple Store-like experience.”

However, the rush to launch sites that look as slick as iPhone ads has not always gone smoothly.

On TrumpRX.gov, for example, NDS was mocked for using AI to generate an image that “showed a child with six toes running towards an American flag without any stars on it,” NextGov reported. (That seemingly contradicts a directive from former President Joe Biden that agencies should never update websites to “alarm or frighten your users in ways that erode trust.”)

A design for CIO.gov was abruptly pulled after critics on LinkedIn panned it as inaccessible and discovered that NDS had seemingly exposed its design system by accident. On X, an NDS staffer boasted that it was “one of our first deployment[s] that is almost entirely generated by our internal AI agent system” end to end.

But commenters pointed to odd coding choices, such as “inconsistent” color labels, as evidence of a rushed rollout, with one LinkedIn user writing, “it’s as if they used an AI with a hangover to generate it!”

Another commenter lamented, “this is clearly a design system for AI agents to replicate the look of vs. for humans to implement or understand. It doesn’t have to be this way though.”

Currently, that page redirects to councils.gov, but the Wayback Machine still has an archived screenshot of the yanked page as of this writing.

Commercial trackers, accessibility concerns

Perhaps even more concerning to the public are the projects NDS seemingly completes and never ships. The Drey Dossier, a YouTube investigative outlet, investigated whether the Trump administration may have ulterior motives with the redesign—like surveilling Americans, generating propaganda (most NDS sites explicitly praise Trump), or abusing data access. They have questioned the status of potentially sensitive but unlaunched domains, like vote.gov or passport.gov.

This weekend, the Guardian published an investigation corroborating some of the Drey Dossier findings, including that NDS had “built versions of services legally assigned to other agencies,” including passport.gov and vote.gov. Regarding the latter, the Guardian reported that “under the studio’s design, voters would be required to verify their identity through Login.gov, the federal sign-in gateway, and to have their citizenship checked against a database run by the Department of Homeland Security.”

Any plan for data retention policies remains troublingly unclear, and there’s seemingly been no privacy impact assessment weighing the privacy implications of centralizing so much sensitive data in the White House, the Guardian reported. Notably, “the commission Congress put in charge of vote.gov has not decided to formally participate in the initiative,” the Guardian reported.

The Guardian’s reporting also confirmed that four federal sites built by NDS—ndstudio.gov, trumprx.gov, realfood.gov, and trumpaccounts.gov—run “commercial visitor-tracking software” that’s “configured to evade the privacy tools many web users install.”

None of those sites “carry the public filings federal privacy law requires under laws including the Privacy Act of 1974 and the E-Government Act of 2002,” the Guardian reported.

The trackers were apparently removed after the Guardian contacted the White House. Liz Huston, a White House spokesperson, told the Guardian that “all National Design Studio personnel comply with all legal requirements in their important work to improve how citizens interact with their government.” But she did not comment on what happened with the data that “was collected from users of the government websites while the tools were live, whether it was retained, and who has custody of the data,” the Guardian reported.

NDS has also seemingly abandoned projects, such as an unlaunched website for the FBI’s Charlie Kirk tipline.

Additionally, NDS has faced ethical questions about why it posted, and then removed, corporate logos and links to websites promoting companies like X and Cloudflare, NextGov reported. Unlike the 250.gov site, which permissibly highlights partners for a community event, displaying corporate logos on a government site “reeks of both corruption and incompetence,” Emily Peterson-Cassin, a policy director for the advocacy group Demand Progress, told NextGov.

Following its investigation, the Guardian concluded that questions remain about who oversees NDS and how it’s funded, noting that “a search of USAspending returns no record of the National Design Studio either as a paying agency or as a recipient of funds.”

However, while criticisms of NDS are widespread, many former government workers have maintained that the biggest concern is NDS sites that do not appear to comply with laws requiring usability and accessibility. The heavy load of some sites could make them harder to access on mobile, for example.

One former federal designer, Ethan Marcotte, criticized an NDS site as an “overbuilt, too-heavy website” that the design-focused Architect’s Newspaper reported “not only fails basic ADA web compliance but ships close to three megabytes of code to boot. For those unfamiliar with web design, this technical cost is comically outsized. Three MB is the kind of payload you’d expect from an image-heavy editorial feature or an interactive map, not from a single page featuring a single style of text.”

NDS’s standards matter, especially when it takes on bigger projects and websites. It’s suspected that NDS may be planning to redesign Recreation.gov, one of the most widely used government websites. And Barbaccia told NextGov that he plans to “build a ‘digital front door’ to the government for taxpayers, among other priorities.”

The design studio also reportedly plans to use Salesforce AI tools to personalize government websites by “creating tailored web experiences using real-time and historical data with AI-driven content and product recommendations.” Imagine veterans visiting VA.gov and the content dynamically adapting to their prior interactions.

Those plans may not offer the best usability, though, as it’s easy to imagine a user becoming frustrated if they can’t reliably pull up the same information.

NDS aims to use AI for everything

NDS did not make a spokesperson available to discuss their work with Ars. However, at a recent panel, Gregory Barbaccia, the federal chief information officer, said NDS is experimenting with AI to produce “complete website redesigns” using NDS guidelines in an effort to accelerate work on the tens of sites still to be updated in the next two years.

Former government workers told NextGov that using AI to launch sites could “theoretically” work, but only with “careful monitoring”—something the six-toed kid image on 250.gov calls into question. Without that oversight, government sites risk more than the embarrassment of publishing an obviously AI-generated image; sloppy AI code could also introduce cybersecurity risks.

It remains unclear if NDS could use AI to create more complex websites. The risk of disrupting critical government resources appears to already be stalling NDS efforts to prioritize “improving websites and physical sites that have a major impact on Americans’ everyday lives,” as Trump’s executive order directed.

For agencies, it’s hard to trust NDS when it doesn’t appear to follow any brand standards across its launched sites.

The purported aim of the “America By Design” initiative is to bring consistency to the government’s online information ecosystem. Yet NDS sites vary radically in appearance. Compare moms.gov to nasaforce.gov or earlycareers.gov, and you’ll find wildly different fonts and font sizes. And while sites like realfood.gov and freedom.gov open with flashy animations, TrumpAccounts.gov and TrumpRX.gov look much more like traditional government sites.

Some critics suggest that any consistency in NDS designs could be an artifact of the group’s reliance on AI. Add in concerns about usability and accessibility, and it’s easy to understand why NDS appears to have no plans to update the USWDS standards any time soon.

Agencies resist working with NDS

NDS’s unimpressive output has reportedly made agencies hesitant to work with the group. Trump gave agencies until July 4, 2026, to share the “initial results” of discussions with NDS to create new USWDS standards. Earlier this month, however, a GitHub page tracking USWDS noted that none of the agencies involved in implementing the standards had responded to repeated USWDS outreach about complying with Trump’s deadline.

Rather than quickly reaching a consensus on standards, the requirement that NDS work with agencies to revise the USWDS appears to have been dropped, at least temporarily, from Trump’s order.

Without those standards, it’s unclear how the sweeping redesign project, whose core goal is to unify the look and feel of government websites, will proceed.

A message posted in the USWDS public Slack channel and shared on the GitHub news page confirms that USWDS was “notified that there’s been a change in direction” and that the section of Trump’s order requiring updates to USWDS is “no longer a requirement.” The Slack message added that the team would work with Trump’s chief design officer to adapt if anything changes.

The White House declined to comment on the update to Trump’s executive order reported on the USWDS GitHub. Instead, a spokesperson shared a statement praising NDS for “doing outstanding work modernizing federal digital and physical services, improving both usability and design across platforms like the Trump Rx, Eat Real Food, and Trump Accounts websites. President Trump has consistently prioritized innovation and efficiency, and he will continue to ensure federal services deliver results and meet the needs of the American people.”

It seems likely that the July 4 deadline will pass without any major launches or announcements, apart from the official July 4th rollout of Trump Accounts, including an app which has already generated complaints from users. Those wanting to monitor NDS’s output can always follow this Bluesky account, which posts an update any time a federal domain is registered or removed.

NDS can’t force agencies to update sites

Unlike DOGE, which seemed unstoppable as it wreaked havoc across the federal government before disbanding, NDS appears largely powerless to direct agencies to update their sites.

The 21st Century Integrated Digital Experience Act and a Biden administration memo are perhaps the only tools NDS has to get agencies on board with “America By Design.”

The law doesn’t mention NDS, of course, instead directing agencies to coordinate with the director of the Office of Management and Budget to assess digital needs and costs and standardize government websites only to the extent practicable. The Biden memo also encourages agencies to use the USWDS standards, which NDS hasn’t appeared to rely on. Most glaringly, the standards the law requires have not taken effect, so agencies do not yet have to comply with them, especially if they don’t have the budget or resources after DOGE’s cuts.

It’s possible that NDS could finish drafting new standards and begin phasing out old systems sometime in the next two years. But even if AI could simplify that transition, migrating thousands of websites from different teams and managers would likely be like herding cats. NDS might need substantial follow-through to get every agency on board once new standards are required, and the design team’s siloed way of working and failure to collaborate with the one staffer left at USWDS don’t inspire confidence that NDS would put in that kind of grunt work.

Right now, NDS’s top priority appears to be recruiting, with a Tech Force site advertising efforts to build “a two-year, White House–backed engineering corps.” Former government workers still bruised from sudden cuts likely aren’t rushing to join NDS or Tech Force, which both dissolve in two years.

In a blog post, the former administrator of the US Digital Service mocked Tech Force as seeking “a silver-bullet solution to all the government’s technology problems.”

Some former staffers who formed a group called We the Builders have warned that anyone looking to rejoin the government should prepare for potential ethical dilemmas, such as projects that may be co-opted for surveillance, invade Americans’ privacy, or extract data without oversight.

“Tremendous amounts of institutional knowledge and work has disappeared, meaning less opportunities to learn from and with seasoned leaders,” We the Builders further warned. And “the environment may be hostile to civic tech values, as we’ve seen decimation of the ideologies behind best practice around user experience, specifically accessibility, language access, and well-researched design systems.”

There’s also uncertainty about what will happen if NDS goes away, as there appears to be no plan to continue updating what, optimistically speaking, could be thousands of websites subject to AI-driven redesigns.

If USWDS survives NDS, that gutted team could be rebuilt under a future administration to continuously evolve the federal web design system. Unlike NDS, that team had been prioritizing updates on the most-visited government sites, and many former government workers have defended the USWDS as a system that deserves to be maintained. On its website, the USWDS stated its primary objective was to use “human-centered design to support human-centered design teams.”

Tossing USWDS would be a waste of taxpayer funds, some critics have said, achieving the opposite of what the Trump administration claimed its goals were when rebranding the USDS as DOGE.

“USWDS is solid,” Charles Hall, an accessibility expert, wrote on LinkedIn. “It represents a significant investment and was created by top talent in 18F and contributed to by top talent via open source. Not using it is already a waste. Making something else is exponential waste.”

SpaceXAI launched Grok 4.5 as its strongest model for coding, agentic tasks, and knowledge work, noting that it was trained alongside Cursor.

Meta has started rolling out Muse Image, its first image-generation model from Meta Superintelligence Labs, inside Meta AI, turning the assistant into a visual creation tool across the company’s consumer apps. The model is now available in the Meta AI app, with Instagram Stories support in the US and WhatsApp image generation in select countries. Facebook, Messenger, more Instagram and WhatsApp surfaces, and advertiser access through Advantage+ creative are next in line.

Muse Image is built for prompt-based image creation, photo editing, multi-reference composition, room redesigns, personalized presets, and social creation using Instagram context. Users can start with text, existing photos, suggested prompts, @-mentioned public Instagram accounts, or sketches drawn directly on top of an image. Meta says the model can render cleaner text within visuals, generate QR codes and plots via coding tools, and use web search to ground images in current or factual context.

The release makes Meta’s image strategy more tightly tied to its own social graph than standalone image generators. Muse Image can draw from public Instagram photos when a user tags an account, while Instagram users have the option to turn off this type of AI reuse. That puts the launch directly inside creators’ existing workflows, from story effects and chat images to event graphics, room makeovers, product concepts, and small-business marketing assets.

Under the hood, Meta frames Muse Image as an agentic media model rather than a basic text-to-image system. It can plan before generation, use search and coding tools, self-refine outputs, scale reasoning at inference time, and work with Muse Spark, Meta’s earlier reasoning model. Meta also says Muse Image ranks No. 2 on Arena for text-to-image, single-image editing, and multi-image editing as of July 5, 2026, behind GPT Image 2 in the text-to-image leaderboard and ahead of models from Reve, Google, Microsoft AI, xAI, and others.

Meta is also adding Content Seal, an invisible watermark for images generated in Meta AI. The company says the signal is designed to remain detectable after cropping, compression, resizing, or screenshots, and it is previewing a detection tool for checking whether an image carries the watermark. Meta says the same provenance system is planned for video later.

Meta is positioning Muse Image as the second major step in its Muse roadmap after Muse Spark, which began powering Meta AI earlier this year across the Meta AI app, WhatsApp, Instagram, Facebook, Messenger, Threads, and AI glasses. With Muse Video already previewed and coming later to creators and Meta AI, Meta is now moving from assistant answers into social media production, ad creative, and generated visuals inside the apps where its users already post and message.

ByteDance Seed has launched Seedream 5.0 Pro, a multimodal image-creation model designed for production design work rather than one-shot image output. The model targets creators, designers, marketers, educators, product teams, and developers who need dense visual layouts, localized text, controlled edits, and reusable design assets from prompt and reference inputs.

The new Pro model brings four core upgrades: complex information visualization, precision editing, realistic imagery and portrait textures, and native multilingual input and generation. ByteDance says it can turn data, concepts, and long text into professional layouts, including infographics, posters, UI mockups, educational visuals, and structured commercial assets. Compared with earlier Seedream models, the company points to stronger image-text alignment, structural coherence, text rendering, and visual aesthetics.

Seedream 5.0 Pro is UNLIMITED on Magnific in 1.5K resolution. Precise image generation powered by @byteplusglobal. Generates text natively in 14 languages. Full infographics in one go. Precision editing straight into your design workflow.

Seedream 5.0 Pro can use point selection, lasso selection, box selection, sketches, color swatches, material references, and multi-image inputs to edit a specific region rather than regenerating the entire image. It also supports layer separation, letting a poster be split into editable assets such as text, subject, background, and decorations, with occluded background areas restored during the process.

Seedream 5.0 Pro is now available on fal. Region-precise editing that changes one element and leaves the rest untouched. Advanced prompt understanding and native text in 14 languages. Suitable for structured designs, posters, product mockups, UI-style layouts, charts.

For realistic outputs, ByteDance is pushing the model toward physical lighting, material behavior, skin texture, photographic motion, and multi-person compositing. The company shows examples covering glass reflections, architectural photography, cinematic portraits, panning shots, and group photos built from multiple source images. For global use cases, Seedream 5.0 Pro supports more than 10 languages, including Chinese, English, French, German, Russian, Japanese, Korean, Spanish, and Arabic, with support for right-to-left layouts and accents.

A new seedream-5-0-pro image generation model from ByteDance is now available on BytePlus APIs for testing. It supports text-to-image, single image-to-image, and multi-reference image-to-image.

Developer access is also taking shape through BytePlus ModelArk. BytePlus documentation lists seedream-5-0-pro as a new image generation model that can generate a single image or produce 2 to 10 reference images from a text prompt. The same documentation section was updated on July 8, 2026, indicating an API-side rollout around the launch window.

ByteDance Seed is the company’s foundation model research unit, working across LLMs, infrastructure, vision, speech, multimodal systems, AI for science, robotics, and responsible AI. Seedream 5.0 Pro sits inside its GenMedia push alongside models such as Seedance 2.0 and Seedream 5.0 Lite, moving ByteDance deeper into creator tools where image generation, editing, layout, and multilingual production are converging.

• SambaNova is now valued at $11 billion after raising fresh financing of $1 billion.
• It's one of a slew of AI chip startups investors are backin to challenge Nvidia.
• SambaNova is focused on inference chips and on-premise AI deployments.

An AI chip startup has raised $1 billion in financing as investors continue to pour money into companies looking to challenge Nvidia.

SambaNova is now valued at $11 billion thanks to the financing, which was led by General Atlantic along with participation from Seligman Ventures, T. Rowe Price and Capital Group.

The latest round of funding, announced Wednesday, comes after the startup raised more than $350 million earlier this year from investors including Intel, with which it also announced a partnership.

"Inference has broken everything open, and so what we're seeing now is that as a standalone company, you have the ability to really move fast and drive the business across a broad range of sectors," SambaNova's Co-founder and CEO Rodrigo Liang told CNBC at the Raise AI summit in Paris.

"We're scaling the business really, really fast, and so the capital allows us to really accelerate the deployments of the racks that customers really want," he said.

Liang added that the company is strongly considering an IPO in 2027, most likely in the U.S.

How SambaNova is targeting AI inference

SambaNova is one of a slew of startups looking to make waves in the market for inference chips. These are semiconductors designed to run large AI models in a quick and cost-efficient manner, which has become a particular focus for the industry as more complex AI agents are deployed.

The company sells its latest chip — the SN50 — as part of a server unit that can be deployed in data centers. This is different to the graphics processing unit (GPU) architecture that Nvidia has sold and has been critical for training huge AI models.

But SambaNova is also focused on so-called on-premise deployments where its server units can be installed at a data center owned by a specific company. On Wednesday, JPMorgan Chase said it would deploy SambaNova's systems for "on-prem inference in our demanding enterprise AI workloads."

"JPMorgan has selected SambaNova to be the inference provider for the bank," Liang said.

"For banks and for other industries where data is incredibly important, bringing this infrastructure on prem, bringing this infrastructure with models that are then under your control with your private data, and having all within your firewalls is an incredibly important aspect of running AI in a very secure and private manner," he added.

SambaNova has said that on-premises inference can mean faster, more secure AI as it's controlled by the company using it rather than a third-party cloud provider or AI lab.

Public market investors have been bullish on the semiconductor sector, which is often dubbed the "picks and shovels" of the AI buildout. The PHLX semiconductor index, which tracks a basket of chip stocks, is up around 80% this year.

But there has been a flurry of activity in the private markets around chip companies that are trying to challenge the incumbents.

On Wednesday, Sunghyun Park, the CEO of South Korean startup Rebellions, told CNBC exclusively that it is gearing up for an initial public offering on the Kospi in the first or second quarter of 2027.

Last year, Nvidia signed a deal to license technology from inference chip startup Groq.

OpenAI wants the work that once belonged to consultants. Its deployment arm has agreed to buy Northslope, an applied-AI firm, the company told Axios in an exclusive on Wednesday. It did not disclose terms, and the deal still needs regulatory clearance.

The purchase marks its second in two months. The OpenAI Deployment Company launched in May to help firms put AI into core operations. Northslope follows its first buy, an AI deployment outfit called Tomoro.

The unit exists to spend. OpenAI majority-owns and controls it, and seeded it with $4 billion for acquisitions. Northslope adds hundreds of “forward deployed engineers” to the bench.

What a forward-deployed engineer does

The job title doubles as the strategy. A forward-deployed engineer sits inside a customer’s business and builds the AI systems around its actual work. They speak tech and business at once, closing the gap between staff who want a model and staff who cannot make it behave.

OpenAI did not invent the playbook. It copies Palantir, which has long embedded engineers with clients to build software around their operations. Northslope’s founders came from Palantir, so OpenAI buys the method as much as the people.

Why it matters

Frontier models keep converging, and raw performance alone wins fewer deals. The next edge lies in adoption: getting enterprises to actually use the tools they pay for. Rivals have spotted it too.

Microsoft has built its own AI deployment business, and Anthropic has launched a services company for mid-sized firms. The shift arrives as buyers grow wary of AI spend, data exposure and security. The pitch no longer stops at a smarter model. It now promises someone who will sit with you until the thing works, the same logic behind OpenAI’s hunt for enterprise expertise.

Introducing GPT‑Live OpenAI finally upgraded the model used by ChatGPT voice mode!

I've had preview access for a few weeks in the iPhone app, and the new model is very impressive. It also has the ability to spin off harder tasks to GPT-5.5:

For questions that require web search, deeper reasoning, or more complex work, it delegates to our latest frontier model behind the scenes and brings the result back into the conversation when it’s ready. While it works, GPT‑Live can keep talking with you and maintain the flow of conversation. At launch, GPT‑Live will use GPT‑5.5 in the background. As we release new frontier models, we’ll continuously update the model used by GPT‑Live.

The previous voice mode in the ChatGPT app was based on a GPT-4o era model, with a knowledge cut-off some time in 2024. I had mostly stopped using voice mode because the age and relative weakness of the model greatly limited how useful it was as a brainstorming partner.

During the preview period I encountered a pretty obscure bug: the model was interrupting me to laugh at things I said, which weren't even intended as jokes! It felt rude and condescending - I reported it to OpenAI and as far as I can tell they made some tweaks and it's now less likely to happen.

From looking back at my transcripts I think it was this bit that triggered the interrupting laugh:

so where are the owls when they're not, like before dusk? The owls exist, right? Are they hiding in holes? Where are they hiding?

My longest conversation with the new model has been a full hour while walking the dog (and taking photos of pelicans). I have not yet managed to take a photo of an owl.

Former GitHub CEO launches competitor designed for the age of vibe coding

In the era of vibe coding, even GitHub is having trouble keeping up with all the traffic. Now, Thomas Dohmke, the service's former CEO, has launched his own Git hosting network to meet the needs of AI agents and those minding them.

His company is called Entire, which the biz has repackaged as an adverb to make the point that it is pitching "an entirely new Git hosting network" based on the 21-year-old version control software.

"The question is not if Git survives through the sheer weight of its ecosystem lock-in," Dohmke mused in a recent post. "The question is how we can expand, rewire, and evolve Git hosting for a world where AI agents are the primary producers of code."

Git's survival hasn't been seriously questioned – it's used by an estimated 93.87 percent of developers and remains the dominant version control system. It's GitHub, Microsoft's hosted Git platform, that's been having problems due to the unanticipated infrastructure stress arising from the proliferation of AI coding agent interactions.

Dohmke admits as much when he acknowledges that his company's proposed fix for Git – decentralization – has been part of Git's design since day one. What Entire really aims to fix is GitHub and other code hosting platforms, which weren't built to accommodate AI agents.

"By design, Git was always meant to be decentralized," Dohmke wrote. "Every clone contains a complete copy of the repository and its history, allowing software to be replicated across many hosts rather than controlled by a single server. But in practice, Git hosting platforms have largely routed developers into centralized systems. This was sustainable until agents came along."

In its initial form, Entire.io allows developers – presently those promoted from the waitlist in the US, EU, and Australia – to mirror their public or private GitHub repos. This creates a parallel universe where AI agents can fumble their way through Entire-hosted code without putting strain on GitHub resources that might be needed for actual deployment. Users can also choose to rely solely on Entire-native branches.

The Entire.io network complements the Entire CLI, a Git-integrated tool designed to collect AI agent sessions alongside code commits. "It hooks into your workflow to track prompts, responses, file changes, and other context from AI coding agents – allowing you to understand not just what changed, but why," the company explains.

That provides some hint at the company's fill-in-the-blanks business model – Git hosting, agent auditing (showing why an agent made a decision), and eventually monetization of whatever data exhaust is relevant to software agents and companies managing them.

The Entire network is designed for scale, low-latency, regional content control, and availability. It's intended to better handle concurrent requests, something that agents may inflict upon repos. As a point of comparison, the company cites performance stats from SpaceX's Cursor, which last month announced its own agent-friendly GitHub challenger called Cursor Origin.

Entire claims that its network can handle 2.1 million pushes per hour and 570,000 clone operations per hour compared to Cursor Origin's 81,000 pushes per hour and 296,000 clones per hour.

The company says in the months ahead it plans to open source its Git network and allow self-hosting.

"We will continue building up and down the stack towards an open, decentralized, independent developer ecosystem for any agent and any human," said Dohmke.

Introducing Meerkat: an experiment in global consensus

Many internal services at Cloudflare need to read and modify the same control-plane state from across our 330+ global data centers. They need guarantees that different readers never see inconsistent state, and that the system remains available for writes even when some data centers or links fail.

But Cloudflare’s network runs across the entire Internet, and the Internet is an unpredictable place. Servers and data centers go down. Queues fill up. Links and cables get cut. These conditions make it difficult to run a globally available data system that guarantees strong consistency (e.g., that all readers are guaranteed to read all prior writes) because hostile conditions hinder distributed system replicas’ ability to reliably synchronize data with one another.

One way to synchronize data safely despite adverse network conditions is via a consensus algorithm, which allows a set of machines to agree on the same sequence of values, such as key-value store put and get operations, as long as a majority remains alive and able to communicate.

Unfortunately, commonly deployed consensus algorithms like Raft suffer in wide-area networks like Cloudflare’s because they rely on leaders and timeouts. The leader is the only replica allowed to make writes, and if it fails due to a crash or network degradation, the system becomes unavailable until some other replica times out and a new leader is elected. And these timeout values are hard to configure in networks with unpredictable latencies.

We have experienced multiple incidents caused by unavailable leaders in consensus-driven systems.

And so, for the past year, Cloudflare’s Research team has been building a new distributed consensus service called Meerkat powered by a consensus algorithm called QuePaxa, published in 2023 by Tennage & Băsescu et al. QuePaxa differs from Raft in that all replicas can perform writes at all times, and progress is never halted due to a timeout, which makes it well suited for Cloudflare’s network. We layer applications, like a transactional key-value store and leasing system, atop Meerkat’s consensus log. To our knowledge, this will be the first industrial deployment of QuePaxa at global scale.

Meerkat is an experimental consensus service that is still in development. It’s being designed initially to manage small pieces of control plane state (e.g., leadership for replicated databases) and so it will be kept internal-only for the immediate future. This post introduces Meerkat and lays the groundwork for the Meerkat-related blog posts to come.

What we need from a global control-plane data system

Many Cloudflare services read and write control-plane data, data that helps those services operate correctly, from multiple machines distributed all over the world. One example of control-plane data is placement information: where certain resources (like an AI model instance) are stored. Another example is leadership information: which machine is currently allowed to perform writes to a database.

Control-plane data must be both strongly consistent and accessible despite particular kinds of faults.

In this section we precisely describe our consistency and fault tolerance requirements for a Cloudflare consensus service. We use a key-value store for a running example of an application running atop our consensus service, though other applications (e.g., distributed leases/locks) are possible.

Strong consistency

A distributed data system’s consistency level describes what kinds of weird behavior the system is allowed to exhibit when it receives concurrent reads and writes. Consider a distributed key-value store that stores a single numeric value x = 6 across multiple nodes. Also consider the following sequence of writes. These writes are submitted to different nodes on a best-effort basis, and could arrive in any order:

1. x = x + 1

2. x = x / 2

A system’s consistency level tells you what values of x a client might see when reading x after these writes. Consider the following sequence of operations and the possible execution orders under different consistency levels:

In a weak consistency level, writes can be re-ordered. In a stronger consistency model, writes can’t be reordered, but reads can. In the strongest possible consistency level, the operations are ordered exactly as they occurred in real time. This property is called linearizability.

At Cloudflare, many services want linearizability. Unlike weaker forms of consistency, linearizability relieves programmers from thinking about all the weird behaviors the data systems might exhibit. Instead, they can reason about the distributed system like they reason about local memory on a single-threaded machine: all reads after a write will see that write. For additional reading material on the dangers of weak consistency, check out this post by Marc Brooker.

(If you’re wondering, Meerkat’s key-value store also provides serializability, which we’ll write about in a future post.)

Fault tolerance

A system’s level of fault tolerance describes what kinds of faults the system can handle before catastrophes happen. Catastrophes are typically violations of properties the system aims to uphold, e.g., that two consecutive reads without an intervening write for the same key never see different values, or that the system remains available for writes. The faults include network failures or delays, machine crashes, and machine restarts. A system will typically explicitly handle some faults but not others (you can’t handle all faults, as the universe could always reach heat-death). For example, some key-value stores might guarantee to remain available for writes as long as two-thirds of the machines in the system can communicate and don’t crash, but make no promises if a machine is compromised and starts sending malicious messages.

Our desired fault tolerance properties are as follows:

First, the data system should remain available for writes and reads from a client located in any of our data centers as long as the following are true:

1. A majority of the machines in our system are alive and can communicate with one another. (Formally, we tolerate f faults in a system of 2f + 1 machines).

2. The client can contact any machine in the system that is connected to a majority of live machines.

This means that a single failed machine, or network degradation on a single link, does not affect availability of the system. This property is not provided by Raft-based systems, as we’ll see later.

Second, the data system remains correct as long as no actor in the system is actively malicious (and, of course, there are no bugs). We define correctness in terms of consensus safety later, but loosely speaking this means no two up-to-date machines will ever disagree about the world (e.g., one thinks that key1=1 while another thinks that key1=2).

To summarize, the system must remain correct even if machines crash, machines restart, networks fail or degrade, data centers go down, and more (though we, like Raft-based systems, do not handle Byzantine faults).

Introducing Meerkat

Meerkat is a consensus service upon which we can build applications that exhibit the above properties (strong consistency and fault tolerance) like a key-value (KV) store. To understand how Meerkat works, we first outline Meerkat’s general architecture, and then describe how Meerkat’s choice of consensus algorithm helps provide strong consistency and fault tolerance.

Developers of services using Meerkat request a cluster of Meerkat replicas. Each replica is connected to every other replica. Each replica participates in the consensus algorithm and can receive both reads and writes. The developer can specify which data centers are allowed to host their replicas, and Meerkat places them automatically.

To interact with their cluster, a developer’s client sends an application-specific request to any replica in the cluster. A single replica may host many kinds of applications, but the simplest one is a key-value store, so the simplest application-specific request type is a KV get or put. The replica responds to the request with an application-specific response (e.g., the records requested with the get). Note that KV reads (gets) are guaranteed to read up-to-date information.

Meerkat’s log

Under the hood, the replica translates application requests (e.g., get and put) into log events. That replica distributes each log event to all other replicas using a consensus algorithm such that all replicas maintain the exact same log of events (in reality, a replica may lag behind, but shall never record different entries). These events are arbitrary — Meerkat’s core doesn’t care what’s in them. Meerkat applications care about log event contents. Each Meerkat replica “hosts” many Meerkat applications (e.g., key-value store) that read the log events and construct state. (Note that each replica belongs to exactly one cluster.)

For instance, the KV Meerkat application constructs an in-memory key-value store from the log events. So when a client sends a write like put k1 v1, the receiving replica places that write into a log event and distributes it to all replicas. If someone else subsequently writes put k1 v11 to a different replica, this event is also distributed to all replicas. Since all functioning replicas have the same log, those replicas can apply the operations in the log in sequence to construct the exact same state. Note that get requests also create distributed log events (for linearizability, as explained in the next section).

How Meerkat’s log enables strong consistency

Meerkat guarantees that if one client executes put k1 v1, a second client subsequently executes put k1 v11, and a third client subsequently executes get k1 (with a consistent read), they will always read v11. It guarantees this even if each request is submitted to a different replica, and those replicas are distributed randomly across the world. This is linearizability. To see how Meerkat guarantees this, we must examine Meerkat’s log in more detail.

The Meerkat log is a sequence of slots. A slot is a box that can contain an event or not. A slot that contains an event is called a decided slot. All slots in the log are decided except the last slot, which is currently being decided. One of Meerkat’s invariants is that if any two replicas decide on the value for a slot, those values are the same. In other words, no two replicas will ever disagree on the value of a decided slot (though one replica may think the last slot is empty while another does not). This property helps guarantee the desired properties we described in the previous section.

To decide on the value of the last (empty) slot in the log, Meerkat replicas run a distributed consensus algorithm. A consensus algorithm allows a set of machines communicating over a network to agree on a decided slot value. Our consensus algorithm works as long as a majority of replicas (more than half) are alive.

So if the log currently contains two entries, and a client submits put k1 v11 to a replica, that replica triggers a consensus algorithm for slot 3. But another client might have submitted put k1 v111 to a different replica for slot 3. The consensus algorithm ensures that only one such proposal for slot 3 wins out. Specifically, it ensures that at least a majority of replicas agree on the same proposal, deciding it for slot 3. The non-majority can never decide a different proposal, but might miss the fact that slot 3 has been decided at all.

To see how this provides linearizability for our key-value store, consider a write followed by a read. One replica Z proposes put k1 v11 and this proposal is decided at slot 3 by a majority of replicas, but NOT replica Y. Subsequently, a reader executes get k1 on replica Y. Replica Y believes slot 3 is empty, so proposes get k1 at slot 3. Critically, a majority of replicas will not agree to place that event at slot 3, because that slot has already been decided. They will force replica Y to decide (by receiving older decisions) put k1 v11 in slot 3, and to propose get k1 for slot 4, thus linearizing the read after the write in the log. (And if that replica can’t contact a majority, it will be unable to complete the read.)

How Meerkat’s consensus algorithm provides higher availability than Raft

Deciding on log entries requires a distributed consensus algorithm. But which one? All valid consensus algorithms would provide the required consistency and correctness guarantees, but not all provide the same availability guarantees.

Specifically, many algorithms that rely on authoritative leaders do not provide our desired availability guarantees, because they can become unavailable when a single machine experiences issues. Consider Raft, one of the most well-known and probably the most implemented consensus algorithm. Raft relies on an authoritative leader: the only replica in the cluster that can drive consensus. As a result, all writes get forwarded to the leader. This design choice helps make Raft “understandable” and, coupled with leases, can make leader-served reads automatically linearizable (since they’re guaranteed to be up-to-date). But it also adds a single point of (temporary) failure.

In general, there are two problems with authoritative leaders. First, if the leader goes down, the system becomes unavailable (all writes block) until a new leader is elected. This is unacceptable for Meerkat. Second, if the leader stays up but slows down, either because it is overloaded or there are network delays, then performance degrades. The leader is a bottleneck because there is no alternative way to perform writes.

The first problem is exacerbated in wide-area networks. Consider that when a leader goes down, most algorithms choose a new leader using timeouts: if a non-leader replica hasn’t heard from the leader in some amount of time, they propose themselves as the leader. At that point, the old leader has been deposed, and the system cannot accept writes until a new leader has been elected. The problem is that when the timeout is shorter than the network delay between the original leader and that replica, replicas will constantly be timing out and thus blocking writes. And when the timeout is too long, the system reacts slowly to a failed leader, during which writes are also blocked. Plus, if multiple replicas propose themselves as leader at the same time, their “campaigns” can interfere with each other, causing them to constantly re-propose themselves as leader — all the while blocking writes. We have seen these exact issues with Cloudflare’s systems that use Raft because our wide-area network delays can and do vary wildly, making tuning timeouts especially difficult.

We chose a different consensus algorithm for Meerkat, called QuePaxa, that aims to avoid the “tyranny of timeouts” imposed by protocols like Raft. QuePaxa is a subtle protocol, but here are the highlights. A client can contact any replica, and that replica can drive consensus for the latest slot. There is a leader, but it is not required — its only advantage is that it can drive consensus with fewer round trips (one) than other replicas (3+). Critically, clients are free to contact multiple replicas concurrently for the same proposal, to increase the chance of the proposal being successful. Concurrent proposals do not destructively interfere: replicas work together to decide one of the proposed values.

In short, QuePaxa has three advantages over Raft for our purposes:

1. Because there is no required leader, the system never becomes unavailable or degraded due to a single replica (the leader) being down, unavailable, or degraded. Clients can perform writes as long as they can contact some healthy replica (anywhere in the world).

2. Because there is no leader, there are no leader elections that degrade the system. And concurrent proposals made by different replicas constructively interfere, unlike Raft’s leadership elections. This is ideal for Cloudflare’s network, in which latencies can vary wildly.

3. QuePaxa was designed for a less reliable network environment (“asynchrony”), and for networks in which an imaginary adversary can launch targeted attacks on replica connections. The authors found that it maintains much higher (~10x) throughput than Raft and Multi-Paxos during such conditions. These conditions more accurately resemble our own network than the conditions other algorithms assume.

We will save the full description of QuePaxa for another post. Major shoutout to the authors of the QuePaxa paper for being available for feedback and questions about their work.

Assessing Meerkat’s performance

Meerkat has limitations. It is not designed to create general-purpose data systems like databases.

All consensus algorithms come with a cost: lots of round-trips. QuePaxa in particular takes one to three round trips (usually, although it can take more) between the initial proposer and a majority of replicas to decide on a proposal and add an event to the log. The difference is with the leader. It takes one if the leader is proposing (+ an extra broadcast to notify replicas of the decision) and three if a non-leader is proposing (+ extra broadcast). If multiple replicas make proposals at the same time, it can take more. These communication costs point to the important performance limitation of consensus algorithms in general: proposal decision latency is proportional to the latency between some majority of replicas. So if your replicas are far from one another, latency will increase — there’s no getting around that.

At first glance, it seems Meerkat’s write and read latency will be quite poor. Especially if all writes and reads (for consistency) must go through the log, and thus require so many round trips.

But there are a few ways to squeeze better performance out of Meerkat:

1. Because developers have control over where their replicas live, they can choose to move replicas closer together, reducing round-trip latency (only applicable for services that don’t need truly global distribution).

2. Writes can be batched. So if a replica receives 10 writes in a span of 10ms, it can place all of those in a single proposal, improving throughput.

3. Not all reads must trigger a consensus round. If a developer is OK with reading stale (but never inconsistent) data, they can read from any replica’s local data.

4. Multiple operations can be bundled into a single consensus round. For instance, our key-value store supports compare-and-swap-style writes in which writes execute only if a value has not changed since it was read. (In fact, it supports general transactions.)

Still, Meerkat’s fundamental latency limitations remain, especially when it is run at global scale, as it was designed to do. These limitations make it perfect, in the short term, for control plane information that is written infrequently but must remain consistent.

What’s next

Meerkat is not deployed to production, but we have run multiple proofs-of-concept with up to 50 replicas distributed around the world, to great success. Leaders in our proof-of-concept clusters constantly fail, and the cluster keeps operating with no increase in error-rate.

We have a lot more to say about Meerkat. Over the course of the next year we’ll be writing Meerkat posts that discuss how QuePaxa really works, how we’re formally verifying some of our Rust implementation, how bootstrapping and cluster management works, how we find optimal replica placement, how we use deterministic simulation testing to find bugs, and more. We’ll also be preparing a manuscript for peer-review!

Follow along on the Cloudflare Blog as Meerkat progresses, and check out more of our projects at Cloudflare Research.

Ownership

Thoughts on ownership I sent to the Amp team in internal note

The following is an internal Slack message I sent to my Amp teammates after I had a conversation with one of them about ownership. It’s only lightly edited.

I shared it before, but not here, because I didn’t think too much of it. Then today, someone said their CTO shared my post with them and I thought: well, now I have to put it in the newsletter, don’t I? So here we are.

Below, after the Slack post, I added some thoughts on juniors on how I see this advice applying to them.

Just had a (great) conversation about ownership and engineering here and I realized that I often use the phrase “ownership” or allude to it, but haven’t explained what “ownership” means to me in a while. So, ownership.

If I ask you “can you own this?” or “can you take care of this?” or “are you on it?” — what I’m doing is I’m asking you to own it, to own the solution of a problem from end to end. From “we have a problem” to “we don’t have to think about it again.”

That means, when you say that you’re owning something, the expectation is that you…

• Think about what the problem actually is. Maybe you already have a solution in mind, without having thought about what we’re actually trying to solve here. Maybe you think “the problem is that we need to migrate from using X to using Y”, but that’s not a problem, that’s a solution. The problem is likely something like “performance is bad”, “it’s not stable”, “it fails for customer x”. Maybe there’s other possible solutions to that? Think about those. What are the tradeoffs? What’s the best solution to go with considering these tradeoffs?
• Think about edge cases. What are they? Which ones are important? Which ones can we ignore?
• Think about failures. Network failures are a given, for example. How do we handle them? Retry? Well, how often? How long?
• Think about data flow. How much data is involved here? Does data need to be migrated? Cleaned up? How can I get my hands on data to properly test this? What invariants are in the data? What assumptions do I have about the shape of the data that I haven’t confirmed yet?
• Think about how you’d test this. How can I know that what I built is correct or not? Are tests enough? Do I need to manually poke at things? Is the difference visible on a screenshot or in a video?
• How would we announce this? How do we communicate it? Can you picture it? How does it fit into the larger picture of our roadmap? Questions or concerns in that area — push back! ask!
• Do the work, with precision, with care, with a sense of urgency, with calmness. Do not half-ass things. Before you merge, ask yourself: am I proud of this? would I show this to John Carmack and say “here’s what I built, under these constraints, with these tradeoffs?”
• Test it manually. Yes, there’s automated tests. But in 99% of cases you can manually test or confirm that what you built works: you can run it yourself, you can ask an agent to run through test scenarios, you can poke at the data before and after, you can take screenshots, you can make a demo. Are you sure that what you did actually solves the problem?
• Make sure it lands in production and works in production. Is it deployed? Did the deploy fail? Do you need to activate a feature flag? Does the feature flag work? Can you use it in production? Can you confirm it’s actually deployed?
• If you think your colleagues needs to know about this change, because it’s new feature they should all test, or it’s a new convention in codebase, or maybe it’s a tricky thing everybody needs to be aware of, or something else: let them know! Do not underestimate peripheral vision: knowing that person X yesterday changed the behavior of how Z works might save person Y three hours of debugging today when a bug report related to Z comes in.
• Do customers need to know? Who reported the bug? Who’s blocked? Let them know.
• Does the world need to know? Announce that it’s out.
• Are there follow-ups? Do you need to check on what you shipped in the logs? A week later maybe?

Yes, that’s a lot. And there’s actually more, because I’m sure I forgot some stuff. But that’s how you build a product in a small team. We don’t have PMs, we don’t have a Q&A department. We’re small, but we’re great, we can do all of that. And it’s always okay to ask for help, it’s okay to ask questions, it’s okay to redo things and triple-check. What’s not okay is to implicitly assume that someone else will do the things here that you haven’t thought about.

“How does this apply juniors? You can’t expect them to really do all of that?”

I’ve been asked these questions, or variations of them, after I shared the thoughts above and here’s my answer.

I do not expect juniors to do all of these things right away. But I would expect them to read through the list and aspire to one day be able to do all of these things. Until then, they can and should ask for help.

In fact, I don’t expect anybody to always do all of these things for everything. It’s a mental checklist of things to consider — problem, edge cases, tradeoffs, deployment, customers, messaging, … — but for quite a few things there aren’t edge cases to consider. Or big tradeoffs to weigh. Or deployment is a solved problem. And maybe someone else does the messaging for you.

And I imagine that most of these things you shouldn’t even consider when you work in a company with, say, 5000 employees. I’ve never worked in a company that large, only startups, so I can’t speak to how to successfully ship a software feature at Apple, end to end.

But when you work in a small company in which there’s only a single department, when you want to build things you’re proud of, when you work with me and you say own something, I expect you to keep these things in mind and run through them before you declare something as done.

TLDR;

I implemented a graph map-reduce using Apache DataFusion. Where possible, I offloaded everything to disk, and designed the algorithms to rely on bulk scans rather than random access. DataFusion handles spillover, sort-merge joins, aggregations, planning and execution, so my code is very lightweight. I tested it in strict mode by running it via systemd-run with a hard memory limit. It works. Of course, I have encountered some issues: for example, I frequently experience deadlocks from FairSpillPool in extreme scenarios, and I have not yet found a way to make SMJ use pre-sorting of the data on disk. But it works. I can compute PageRank on a directed graph with one billion edges (graph500-26 from the Graphalytics dataset) using 5 GB of memory. Alternatively, I can identify all the weakly connected components in a graph with two billion edges (twitter_mpi from the same dataset collection) using 10 GB of memory. Neither NetworkX nor Igraph can do this; most existing graph algorithms require the graph to fit into memory. Previously, I thought you needed Apache Spark and GraphFrames for billion-scale graph analytics. Now, however, I think all you need is a laptop. I have completely changed my old opinion about using Apache DataFusion for graph analytics.

Setup

I tested two tasks.

PageRank

The task is to compute PageRank on graph500-26 from Graphalytics dataset:

PageRank is one of the most popular graph centrality algorithm and is used from search results ranking to anti-fraud scoring. My DataFusion implementation is classical Pregel: bulk-synchronous parallel algorithm (aka Map-Reduce) which I expressed using joins and aggregate. Very similar to what is in the core of Spark's GraphFrames library.

Weakly Connected Components

The task is to identify all the weakly connected components on twitter_mpi from the same dataset:

WCC is the core part of any identity (entity) resolution problem. For example, when you need to do data deducplication from different system through transitive IDs you end up with WCC problem. My DataFusion implementation is based on the "In-database connected component analysis", Bögeholz et al., arXiv 1802.09478. I already implemented the same algorithm for the Spark's GraphFrames so it was an obvious choice.

Results

PageRank

An easy part. I used SMJ just to proove the scalability but it is also possible to use HJ because vertices are small (32M) and PageRank state is trivial: one column rank (f64), one column out-degree (i64), on participation flag (bool). With HJ it is faster. PageRank works over directed edges so it deos not require to symmetrize the graph. Just offload edges to disk and iterate by updating state (and offload to disk as well to break the lineage) until converged.

The compute time is long: around 30 minutes for 15 full iterations. But the setup is about memory, not speed. Give it some more realistic numbers for 1B graph analytics and it will work fast enough (I tested). I checked numbers against the ground truth: 100% match (with 0.0001 tolerance). A lot of optimizations can be done here as well: in theory it is possible to bucket edges by range or do kind of range-partitioing, so the SMJ does not need to sort again the biggest join-side (edges) on each iteration to get triplets. As well I'm not 100% parquet is the best choice here. Also would be interesting to try to fuse join+agg: each Pregel iterarion is like edges <-[join] nodes-state -> group by + agg -> [join] -> nodes-state -> update nodes-state. If I can fuse together the first two stages it can be a huge win from the performance point of view. Meanwhile I do not know yet how to do it in DataFusion: a lot of thing to learn.

WCC

The hardest part. 2B edges twitter graph is already huge (its edges are 30 GB in CSV !!!). But for WCC we need to symmetrize edges (or do a union between src, dst and dst AS src, src AS dst + distinct on top), so at the peak we are crunching almost 4 billion of edges using only 8GB DataFusion pool. After the flow survives the first few iteration, contraction process reduce the amount of edges dramtically and algorithm ends in 10 minutes with low memory pressure.

sem@fedora:~/github/graphframes-rs$ systemd-run --user --scope \
     -p MemoryMax=10G -p MemorySwapMax=0 \
     -p AllowedCPUs=0-1 \
     --setenv=RUST_LOG=graphframes_rs=info,datafusion=warn \
     ./target/release/run-algorithm twitter_mpi-v.parquet twitter_mpi-e.parquet wcc 42 file:///var/home/sem/Downloads/gf_wcc_out 8G 2

Results are correct: Graphalytics provides ground truth and it is easy to check:

memory D SELECT column1, count(*) as cnt FROM read_csv('twitter_mpi-WCC', delim=' ') GROUP BY column1 ORDER BY cnt DESC LIMIT 5;
memory D SELECT component, count(*) as cnt FROM results  GROUP BY component ORDER BY cnt DESC LIMIT 5;

The code

The code is here: https://github.com/SemyonSinchenko/graphframes-rs

I wrote most of the code by myself, not "Claude do it, make no mistakes", so there is no README and code comments can be outdated somewhere. I'm learning Rust and DataFsuion on this project, so no LLM usage in core parts.

Graph representation is almost like in Spark's GraphFrames:

#[derive(Debug, Clone)]
pub struct GraphFrame {
    pub(crate) vertices: DataFrame,
    pub(crate) edges: DataFrame,
}

Core parts:

• Pregel main loop: pregel.rs
• WCC implementation: connected_components.rs

To understand the Pregel notation this can be very useful source: Lecture 8, CME 323: Distributed Algorithms and Optimization

An example of PageRank using the API: pagerank.rs

Semantic search plays a critical role in HubSpot operations and powers a wide variety of use cases, from agents and RAG (Retrieval Augmented Generation) to contact deduplication. Starting from the POC a few years back, it has grown to a scale of tens of billions of vectors, utilized by 38+ teams. With the growth of agent usage, semantic search capabilities have become even more important to ensure agents can quickly retrieve the information they need while maintaining retrieval quality. This article describes the journey of vector DB infrastructure, from a small POC to the tens of billions of entries powering hundreds of use cases across HubSpot.

What is VaaS?

VaaS (Vector as a Service) is HubSpot's centralized vector storage and search platform built on top of a vector database. It provides an API layer and features such as:

• Access control
• Embeddings generation based on the collection setup
• Data versioning
• Feedback collection

It sits in front of the vectors database, in our case Qdrant, and is directly exposed to the consumers. It handles access control, basic validation, embeddings generation and then proxies requests to Qdrant. For read operations, requests are handled synchronously, whereas for write operations the flow is asynchronous with minimal consumer lag.

Why Qdrant?

HubSpot uses the Qdrant vector database for all semantic search operations. Qdrant is an open-source vector database that can be deployed on-premises and provides an extensive API covering most modern semantic search features, such as named vectors, hybrid search, multi-stage querying, and weighted reranking. It uses HNSW (Hierarchical Navigable Small World) as an ANN (Approximate Nearest Neighbor) algorithm, which provides very low latency at the trade-off of higher memory consumption.

Qdrant also supports an extensive set of cost-efficiency optimizations, such as quantization, on-disk storage options for payloads and vectors, and configurable precision. This allows us to tune the setup to find a balance between providing the required level of retrieval quality and associated costs.

Why we need to run Qdrant in-house

HubSpot has a lot of expertise in running self-managed open-source database systems in-house, and over the years our infrastructure team has built numerous solutions, rules, and best practices that we want to take advantage of. Having such an extensive infrastructure layer lets us seamlessly integrate with and benefit from all of HubSpot’s internal tooling, including tracing, cost tracking, rate limiting, and scaling. Running Qdrant in-house enables us to easily fine-tune the infrastructure to our needs and control costs by utilizing our corporate AWS rates. Finally, running in-house gives us full control over customer data and infrastructure, and ensures all of HubSpot’s extensive security best practices are applied to this data store.

Current scale

The VaaS (Vector-as-a-Service) platform is currently used by 38+ teams and has more than 200 indexes that run on a fleet of 140+ clusters across 5 regions and 2 environments. We isolate clusters by product teams to minimize the blast radius from failures affecting the entire platform. Collectively, the underlying vector database stores 20+ billion vectors across different regions, with the largest index holding 9.5 billion vectors and an average index size of 95 million vectors.

The platform handles 5,000+ RPS for writes, with spikes up to 100,000 RPS, and serves 1,000+ RPS for reads across all regions. The traffic can also be spiky, specifically in cases when large backfills run in parallel with the live traffic; therefore, the underlying infrastructure should be scalable and reliable to provide the best customer experience while supporting engineering use cases.

The Early Proof of Concept

Back in 2023, semantic search had just started to take off, and the internal POC was set up to run Qdrant alongside the required pipelines and an API layer on top of it. There was a limited number of consumers and a dozen Qdrant clusters that were relatively easy to operate. As a first solution, Helm was utilized to define per-cluster/environment setup and parameters such as number of replicas, disk space, RAM, CPU, etc., were adjusted manually. The cluster definition and related cluster artifacts (Kafka topics, workers, etc.) were defined in code and created manually. For a small fleet of clusters, it was a great solution that allowed us to iterate fast, utilize overprovisioning to handle traffic spikes, and keep the setup simple. All the clusters were defined as StatefulSets and shared the same namespace. At that time, Qdrant was already handling 2+ billion vectors across ~10 indexes.

Scaling Up the Infrastructure

With more and more features relying on semantic search, and VaaS being at the core of multiple AI capabilities, powering functionalities from RAG (Retrieval Augmented Generation) to agentic experiences, it became clear that it was time to transition from the POC to full production scale to address the growing demand.

Alongside the rapid growth in demand for semantic search, our setup started to evolve in terms of complexity, with additional artifacts required for each cluster and a growing need for quick, consistent cluster setup. Helm is a powerful tool, but it’s purely a templating and deployment tool that cannot:

• Make API calls
• Auto-scale based on external metrics
• Run custom logic for complex state-aware lifecycle management

As a result, the decision was made to migrate from Helm to a Kubernetes Operator pattern to ensure reliability and scalability while reducing operational load on the team by automating standard operating procedures.

Onboarding to Kube-operators framework

HubSpot has an internal framework called `Kube-operators` that implements the pattern. The main idea is to automate the lifecycle management of systems running on K8s by using operators to track and maintain the system’s defined state. It provides a robust toolkit for building operators designed to meet the needs of HubSpot and our infrastructure teams. It also enables seamless integration with multiple HubSpot infrastructure tools, which makes it possible to use them in the Kubernetes Operator’s logic.

Running Qdrant on Kube-operators allows us to instantiate and decommission clusters together with all related artifacts and subsystems. It also automates multiple workflows to reduce operational load on the team.

The main components in the setup are Translator artifacts. Translators watch assigned CR, running in a loop, re-evaluating the state every 60s, applying the required updates, and ensuring that underlying resources are in sync with the defined state. This is an extensible architecture that allows adding/removing translators and therefore onboarding more features to the system. In addition, the translator contains all the logic required for rendering underlying artifacts; therefore, it can be extended with required automation.

Our current setup has 3 translators:

• Cluster translator: responsible for instantiating the individual namespaces for clusters and downstream artifacts.
• Qdrant nodeset translator: responsible for instantiating Qdrant clusters. It instantiates all required K8s artifacts with the specified number of pods, memory, disk storage, parameters, etc.
• Indexer nodeset translator: indexers are the workers that handle incoming write traffic and index it into Qdrant. There are multiple indexers per cluster handling different traffic priorities, etc. This translator is primarily responsible for instantiating the required indexers and creating necessary Kafka topics.

Migrating from the Helm setup, we also updated the K8s namespace strategy and moved towards a namespace-per-cluster approach instead of using a single namespace. This provides the following benefits:

• Isolation: provides higher reliability, since issues such as misconfigurations have a reduced blast radius.
• Namespace-specific configuration: we can set up namespace-specific resources, limits, etc.
• Simplified cost tracking: tracking costs per namespace is much easier, which is important for current scale.

Dynamic cluster definition

Together with onboarding to Kube-operators, we redefined the contract between VaaS (API layer + data pipelines) and the underlying infrastructure to ensure Qdrant clusters are self-discoverable and do not require any additional code changes when a new cluster is created. The important part is that cluster metadata is required for each search/ingest/delete request; therefore, cluster lookup should have minimal effect on end-to-end latency. As a solution, we decided to abstract cluster metadata into a K8s CRD (Custom Resource Definition) to ensure the latency overhead is minimal.

Automated Cluster Maintenance (Operations & Scaling)

Before Kube-operators, cluster maintenance was very difficult. Open-source Qdrant does not support a lot of automated maintenance operations out of the box that other distributed data stores do. Qdrant provides APIs that serve as building blocks to perform actions like shard transfers, but leave it up to the user to implement the full workflow. In practice, this means that horizontal scaling can be very difficult or error-prone. To scale down a cluster horizontally prior to Kube-operators, we needed to:

1. Run an on-demand job to transfer shards off evicted peers and remove peers from consensus.
2. Update cluster Helm chart to lower pod count.
3. Apply the Helm chart.
4. Delete the PVCs for the removed pods.

With Kube-operators, we baked all of the above into our Translator. All we have to do is update the number of replicas in the cluster config; the watching Translator daemon will reconcile, detect that the replica count has been lowered, and automatically run the eviction process - consolidating 4 operations into 1.

Safe eviction of shards from Peers

To scale down a cluster, we needed to transfer every shard off the pods being removed before we could safely remove them from consensus. The eviction process rejects the eviction upfront if the remaining pod count can't satisfy the replication factor, and validates that every shard on the evicted pods has a valid destination before issuing a single transfer.

Automated shard distribution balancing

Some of the clusters in our fleet host collections with billions of points. For performance reasons, we have to utilize custom user-defined sharding. With custom sharding, requests only need to hit a subset of shards defined by a shard key, whereas with auto sharding, requests need to fan out to every single shard. However, using custom sharding may result in shards that are unevenly distributed across the cluster. We designed an algorithm to greedily balance shards - by shard count and by point count - and baked detection of uneven distribution into our Kube-operators translators.

Recovering replication factor for shards

In production systems, we want a replication factor of 3 to tolerate AZ failures in the deployed region. The Kube-operators translator scans every shard per reconciliation loop, compares its current replica count against the target replication factor, and for each under-replicated shard, picks the peer with the fewest shards that doesn’t already host that shard and adds a replica there.

Results

• New cluster spin-up time dropped from hours to minutes.
• Clusters are now provisioned from a single config, with all required artifacts created together.
• Operational load no longer scales linearly with cluster count.
• Lifecycle-aware automation is now unlocked, supporting operations like horizontal scaling, replication factor recovery, and shard rebalancing.

Lessons Learned

• Manual operations don't survive growth. For any rapidly growing fleet, it is worth investing in lifecycle automation before manual operations become a bottleneck.
• Extensible automation beats one-off scripts. We add logic into the translators' reconciliation loop, where it runs automatically on every cycle.
• Even distribution is cost savings. Because Qdrant pre-loads all vectors into memory, the pod with the heaviest skew effectively determines the memory limit requirements of the entire cluster.

Where AI Agents Belong in Data Engineering: The Correctness Layer

With ever-changing models, new and better ones coming out every few months, it’s great if we don’t have to rely on them too heavily. The better your tooling, the less dependent you become on any single model. That’s also why the deterministic harness matters: a correctness layer that lets you reproduce outputs and trace lineage regardless of which model you’re running underneath. This is especially true during maintenance or extending the project, where verification is the real job.

The danger isn’t only a crash or an error message, but a wrong number that didn’t break. It might be a clean query, but it introduces duplicated rows.

In this article, we go through the three levels of AI agents in data engineering, how to structure projects so the AI delivers its best outcomes, and how dedicated agents with a deterministic core help us build higher-quality pipelines — ones we can actually trust. And we look at a practical example of how it works with a blast radius analysis.

The Three Levels of AI Agents in Data Engineering

Why should we use agents for data engineering? And at what levels can agents help us productively? As LLMs will always have some error tolerance, as humans do too, we need a way to be more confident in producing the code.

Chat-phase, Autonomous and Dedicated Tooling

There are different levels of confidence and levels on which the agents can help us.

1. The initial chat-phase: the development where we prompt Claude or ChatGPT. The model tries to understand the context based on what it has access to. It takes a decent amount of tokens, as it needs to scan everything from scratch.
2. The autonomous approach, where Claude Code or Codex also have access to the tools humans have, mostly the CLI on the terminal, making it possible to query Postgres with psql or read from S3 or Parquet with DuckDB to verify queries and data. A much higher quality outcome.
3. Dedicated agents for the task at hand. E.g., for data, the tools know dbt or know how to transpile SQL code deterministically, meaning not from training data only, but with an actual tool that does it much faster and more reliably. Built-in checks and features a “general” agent can’t provide.

Where in the DE Lifecycle Each Level Actually Helps

BI Dashboards vs. Plumbing the Data Pipelines, or Creating Source Ingestions, or Maintaining? For data engineering, the question is not only if there is dedicated agent tooling, but also on what part of the data engineering lifecycle AI agents can help data engineers and analysts the most, and potentially even domain experts?

The lifecycle contains the ingestion part, ETL, or understanding the business in great detail, or is it just to visualize the result? Or should it cover maintenance in case of overnight ETL errors, or the full data lifecycle?

In general, before we go into more details later, agents can help us on the full cycle, but it always depends on who you are and what role you play. Building from scratch with no knowledge or seniority is dangerous. Why? Because they can’t verify if the produced code is correct. Okay for a side project or a proof of concept, but not for actual production.

What’s the Engineering Discipline for Working with AI?

There’s also a part that is less technical, a way of guiding the agents in the right direction. Especially if we want to safely use it in large projects or organizations, we can’t just let it run without guidance.

For that we need:

1. clear project structure in which the agents can flourish. The more is given, the fewer tokens are used for this work, and it will be more aligned across the project. (Another reason a deterministic workflow such as uv init is best, because it will always be the same).
2. build with clear instructions on how the tools are used (basically providing CLIs and API documentation). This is the bulk of the work anyway. That’s the data architecture, the brainstorming with fellow humans before you build something, instead of missing a key insight in the beginning and then letting the agent run down the wrong path. Also, be realistic: prompting “be correct” or “use state-of-the-art” won’t make it more correct or more state-of-the-art than the model was trained on. So if it’s a rather new architecture, it’s a must that you provide these links and hints.
3. set up the project in a modular fashion, so the agents cannot break the whole project if they make a small change, so you don’t end up in a scenario like dependency hell with everything dependent on each other.
4. use a declarative approach, with descriptive configuration that says the what and not the how, so that you can collaborate on these configs with the agents, version them, and easily revert or change something, as well as decouple the implementation logic from the actual business logic.

With these steps, you can get the best out of the agents of today. I’d say the model matters less, but the structure does, and as Mario says, so does the workflow approach. For example, extensively plan (the process before writing a single line) and correct the model before any implementation that could lead down the wrong path is written.

Also, don’t overthink it. But this is only the workflow and learning the soft skills and discipline of working with agents. How does that look in a real-world project?

The key is to get use out of AI, not to get more work.

E.g., most developers used to think about the problem. Today, most drown in PRs. When the AI tooling gets better, AI can provide more quality code that is correct, that needs less review or fewer iterations, which means fewer PRs and less work for the developers to go through.

The Correctness Layer for Data Engineers

A key insight is that AI agents should support the “human in the loop” for correctness, or a correctness layer. And rather than making more work to verify more code, we should be confident in the process and know that the code it produces is verified and ultimately correct.

But how do we get more “correct” work and a layer in which we can verify it? The biggest argument is a deterministic-validation architecture in full. E.g., Altimate Code splits the agent into a probabilistic layer on top and a deterministic Rust/TS layer underneath that does the actual SQL ops such as parsing, validating, and equivalence checks, so that the agent itself never has to be trusted on those questions.

Altimate Code, for example, is built on a probabilistic agent, deterministic harness, and deterministic core. The probabilistic agent with the LLM does the creative work of reading intent, picking a strategy, drafting SQL, summarizing results, and recovering when something goes wrong.

Below the boundary sits the deterministic harness, a TypeScript layer that intercepts every tool call: a dispatcher checks hasNativeHandler before the call runs, and routes it either to a native, deterministic handler or back to the model. Those handlers don’t reimplement logic themselves, they call into the deterministic core, a Rust engine (altimate-core) that exposes SQL operations as pure functions over ASTs and schemas, wired in via napi-rs bindings. Parsing, validating, transpiling, checking query equivalence, diffing schemas, extracting column lineage, diffing rows across warehouses — all of it runs sub-millisecond, and all of it returns the same answer on the same input, every time.

Like a compiler, the agent never decides whether two queries are equivalent or a column exists upstream. Instead, it calls a function that proves it against the parsed AST and the schema, the same way a type-checker proves a program compiles rather than guessing.

That’s the distinction that makes the output easier to review, as factual checks have been run and the output is either correct, or there’s a bug that it can fix directly. The rest a human can re-verify.

There’s another factor, being wrong.

Bare agent use might be cheap, but only until they’re wrong, and then the cost is unbounded.

Improvements for Better Usage of Tokens

Altimate, or data engineering agents that have deterministic functions and integrated understanding of how to work, can help you save tokens and be token lean. Because in large enterprises, token costs are a real budget point.

To slow down the tokens, an easy trick is to instruct the model to use fewer tokens and words itself. There’s a second, less obvious cost: the token itself isn’t a stable unit.

Altimate makes the case that “cost-per-token is the wrong number to optimize”, since the meter itself can move with a vendor’s next model update, and what we should track instead is cost-per-task. I fully agree, and this is where deterministic function calls work around that volatility by not using a model/tokens for every task, making it less expensive.

Typical Use Cases

In this chapter we go through typical AI agent use cases for data engineering. There are many of them. You can use them to educate yourself or your team, build production data pipelines, build data apps, and visualize your data in new innovative ways. But in general, the use cases fit into these approaches:

1. Start a new project from scratch example: Building a data landscape with more open source.
2. Extending an existing project or data warehouse: Adding new data pipelines.
3. Maintaining current setup: Update and verify it still works when changes come in.
4. Migration: Migrate from one database or tooling to the next.
5. Finding the Blind Spots: Two similar-sounding IDs might be wrongly used for a join, or missing data in a column that got missed in a nightly load, or anything in between. If agents can do these checks, that would be super beneficial. With more access to CLI, Model Context Layer, and deterministic tooling, these things are truly possible.

Showcases: Blast-Radius Example

A Blast-radius refers to the potential extent of damage. For example, before you knock down a wall in your house, you want to know if there’s plumbing behind it, electrical wiring within it, or if it’s holding up the floor above.

The same is true for a data warehouse or a data project with lots of ETL. For example, if a data engineer cleans up the table fct_orders by joining orders to order_items and summing order_total. It compiles, the dbt tests pass, nothing errors. But the join changes the grain, so any order with several line items now gets counted once per item, and revenue quietly inflates.

It’s best to know, before you rename a column or add a new join, the downstream (data that comes after the current task) dependencies to the dashboard — that’s what the blast-radius report does.

With Altimate Code we can achieve this. Before any change goes through, it maps out the full impact automatically and produces a detailed blast-radius report with what will break, what’s safe, what needs someone to sign off, and also performs the changes.

Connect a model to Altimate.

Make sure to connect to a model with /connect and choose an existing subscription with API credits, or any other subscription.

Correctness Over Confidence

I hope you got a better understanding of why AI agents can be genuinely useful, especially when provided with the right tools and applied with the right discipline.

You’ve also seen how deterministic tooling, purpose-built for data engineering and analytics problems, gets you both better correctness and better token economics than general-purpose agents alone.

Coming back to where we started: not every task needs a level-three agent. A quick chat-phase agent is fine for exploring a dataset or drafting a query you’ll review yourself. But the moment that output touches production or serious work, a dashboard, a nightly job, a number someone makes a decision on, you want the deterministic core underneath it, not just a model that sounds confident.

That’s the gap Altimate Code is built to close. It runs on deterministic functions purpose-built for DE workloads, it’s open-source via the OpenCode TUI, and for teams wanting more, there’s Altimate Studio — a paid, multi-agent platform with extras like warehouse cost optimization, dbt development acceleration, and migration tooling.

Full article content is not available for inline reading.

Read the original article →

malloyyo

A natural-language interface to any corpus of data, however complex — accurate and consistent, served to any AI over MCP.

Problem: AI + document context + your analytical database = inconsistent results. Pointed at a raw database, an AI writes SQL from scratch — so the same question tomorrow yields a different query and different numbers, with wrong joins, invented columns, or fan-out double-counts that still look right.

Solution: AI + a Malloy semantic layer + your analytical database = consistent results. Measures, dimensions, and joins are defined once, correctly; the AI composes queries against the model instead of writing SQL, so numbers come back right by construction.

Malloyyo is the thin layer that serves that model:

• Thin by design — it sits between the AI and your data, nothing more.
• Develop, then publish — build the model locally and malloyyo publish it (or point Malloyyo at a GitHub repo).
• Claude already knows Malloy — the same way it knows Python — so authoring is incredibly fast and assisted.
• Readable, full-featured queries — Malloy is a complete query language (join, nest, aggregate, filter) that stays legible: you can read a query and see at a glance it's doing the right thing.
• DuckDB built in — query Parquet over plain HTTP (S3, GCS, any web server) with no warehouse required, or attach your own (BigQuery, Snowflake, MotherDuck, Databricks, …).
• Tight control — the AI can only query what's in the semantic model; nothing outside it is reachable.
• A web interface, too — every query is logged, so you can browse, edit, favorite, re-run, and share them in the browser (ltool), and hand any one off to "Explore further with Claude." Because Malloy renders a whole dashboard as a single query, one saved query can be a full report.
• Deploy in minutes — one-click to Vercel, or self-host with Docker.

Try the demo server and "Explore further with Claude" — sign in with any Google account.

Questions, or built something cool? We'd love to hear from you. Come say hi on Slack, and learn more about Malloy at malloydata.dev and in the documentation.

How it works

   MCP client (claude.ai)            Browser (you)
           │ OAuth 2.1                     │ Google sign-in
   ┌───────▼──────────┐          ┌─────────▼─────────┐
   │  MCP server /mcp │          │      Web UI       │
   │ analytical tools │          │  datasets · ltool │
   └───────┬──────────┘          └─────────┬─────────┘
           └──────────────┬────────────────┘
                          │  compile · run             ┌────────────────────┐
                          │                            │  Authoring Models  │
                          │                            │   • Malloyyo CLI   │
                          │                            │   • Claude         │
                          │                            └─────┬───────────┬──┘
                          │                          push    │           │ develop
   ┌──────────────────────▼───────────────────────┐  (deploy)|           │
   │                   Malloyyo                   │◀────────┘           │
   │        load · compile · store · serve        │                      │
   └──────┬─────────────────────────────┬─────────┘                      │
          │                             │                                │
   ┌──────▼──────┐             ┌────────▼────────┐                       │
   │    Neon     │             │  Analytical DB  │◀─────────────────────┘
   │  Postgres   │             │  • BigQuery     │  direct (dev)
   │  metadata   │             │  • MotherDuck   │
   │             │             │  • Snowflake    │
   │  datasets   │             │  • Databricks   │
   │  malloy_    │             │   (or S3/GCS)   │
   │    models   │             │                 │
   │  users      │             │  your data      │
   └─────────────┘             └─────────────────┘

Adding a dataset

Develop your model in a repo with an index.malloy at its root, then publish it with the malloyyo CLI:

malloyyo login <target>     # one-time browser sign-in
malloyyo publish <target>   # bundle *.malloy + malloy-config.json and push

The CLI records the git commit it published from; Malloyyo compiles and introspects the model and stores a new version. If it doesn't compile, the push is rejected and the live model is left unchanged.

Alternatively, point Malloyyo at a GitHub repo and it pulls index.malloy (and any imports) directly — a webhook endpoint (/api/datasets/<id>/webhook/github) refreshes it on every push.

The two databases

MCP tools served at /mcp

The MCP endpoint speaks OAuth 2.1, so claude.ai's remote MCP integration can connect after a one-time authorization flow.

Developing Malloy models

You don't have to know Malloy — or wire up the database by hand. Install the malloyyo CLI, register it with Claude, then ask Claude to do the rest: connect to your data, turn your existing SQL / dbt / Looker definitions into a Malloy model, and test it against real data before you publish. One tool for the whole loop.

1. Install and register the local test window.

npm install -g @malloydata/malloyyo        # one tool for the whole loop
claude mcp add malloyyo -- malloyyo mcp    # register it with Claude Code

malloyyo mcp runs a local stdio MCP server over the Malloy model in the current directory — the same explore surface (list_sources, describe_source, query) the hosted instance serves, so what you test locally is exactly what consumers get. It also registers the writing-malloy-with-mcp skill and yo_help, which teach Claude how to author Malloy, set up connections, and recover from compiler errors. (For Claude Desktop or another client, add the same malloyyo mcp command to its MCP config instead.)

2. Build your model with Claude. Start claude in your model directory and just describe what you have:

claude

> connect to my Postgres warehouse and build a Malloy model from these dbt sources
> add a "net revenue" measure and verify it against last quarter's numbers

Claude sets up the connection and writes the .malloy for you. It follows a compiler-in-the-loop discipline — validate against the compiler, run real queries through the test window, confirm the numbers — and you steer from the results. You rarely write Malloy by hand.

The connection lives in malloy-config.json at the model root, and Claude knows how to write it. Two things worth knowing:

• DuckDB needs no config at all — with no malloy-config.json, an in-memory duckdb connection just works, so Claude can read your local CSV / Parquet files immediately. Other backends get a small malloy-config.json entry.
• Secrets stay out of the committed file — any value can be { "env": "VAR_NAME" } (e.g. ANALYTICAL_DATABASE_SECRET), resolved from the environment when the connection opens.

3. Publish. Once the model queries cleanly, sign in once and push:

malloyyo login <your-instance>     # one-time browser sign-in
malloyyo publish <your-instance>   # bundle *.malloy + malloy-config.json and push

Stack

• Next.js 16 App Router
• Your Analytical Database
• Neon Postgres + DrizzleORM
• Malloy
• NextAuth v5 + Google OAuth
• OAuth 2.1 provider
• malloyyo CLI

Deploy your own

The button forks the repo into your GitHub and creates a Vercel project. The schema self-initializes on first boot (RUN_MIGRATIONS_ON_BOOT=1), so you never run a migration.

Your model's malloy-config.json references your analytical database's secret from an env var (e.g. ANALYTICAL_DATABASE_SECRET — see Developing Malloy models). Set that var on the project so the server can connect. GITHUB_TOKEN is optional (private-repo model pulls).

Self-host with Docker

Prefer containers to Vercel? The repo ships a production Dockerfile. See Self-hosting with Docker for build and run instructions.

Running locally

Copy .env.local.example to local/<instance> and fill in the blanks:

DATABASE_URL=postgresql://...          # Neon (or any Postgres)
APP_BASE_URL=http://localhost:3000
APP_ADMIN_EMAILS=you@example.com
AUTH_SECRET=...                        # openssl rand -base64 32
AUTH_GOOGLE_ID=...                     # Google OAuth client ID
AUTH_GOOGLE_SECRET=...                 # Google OAuth client secret

Google sign-in needs a Google OAuth app. Add this Authorized redirect URI:

http://localhost:3000/api/auth/callback/google

npm install
npx dotenv-cli -e local/main -- npx drizzle-kit push   # first run only
npx dotenv-cli -e local/main -- npm run dev

Code map

1. packages/cli + src/app/api/datasets/[id]/model/push — the malloyyo publish path.
2. src/lib/github.ts + src/lib/github-refresh.ts — the GitHub pull path.
3. src/lib/malloy.ts — single-file and multi-file Malloy compilation and execution via InMemoryURLReader.
4. src/lib/mcp-tools.ts + src/app/mcp/route.ts — the MCP server.
5. src/db/schema.ts — Drizzle schema for all Postgres tables.

This is the first in an ongoing ad-hoc series of posts on Apache Kafka performance. I have no general direction, I’ll just share interesting insights based on the performance testing I do on Apache Kafka.

Recently I was curious to see if there was any general performance improvement since Kafka 3.X. So I ran a suite of benchmarks against 3.7.2 and 4.3.0. I saw two common patterns:

Pattern 1: Low load benchmarks showed that end-to-end latency was higher with Kafka 4.3 compared to 3.7.2. The following is a 45 minute no-record-key workload of 5000 record/s, 20 topics (120 partitions), fan-out 2 (240 consumers), full TLS, on 3 brokers each allocated 8 SMT CPUs in k8s (on my Threadripper 9980X).

Pattern 2: On more stressful loads, 3.7.2 would show much more spiky end-to-end latency compared to 4.3. The following is for the same workload at 100K records/s (200K out).

It seemed that somewhere between 3.7.2 and now, big performance gains had occurred. Then my subconscious kicked in and reminded me that at some point in that period, the default linger.ms had been changed from 0 to 5 ms. This would correlate with the low-load end-to-end latency result.

Doing the math on linger.ms

The linger.ms producer config controls how long the producer is willing to wait before sending a non-full batch (controlled by batch.size). If a batch reaches batch.size first, it can be sent earlier. The point of linger.ms is simple: give more records a chance to accumulate into the same batch, because larger batches are more efficient than many tiny batches.

The important quantity is the rate “per producer, per partition” (rather than the aggregate rate). Kafka producers build batches per partition, so a producer sending 1,000 records/s to one partition has very different batching behavior from a producer sending 1,000 records/s evenly across 100 partitions.

A rough way to reason about it is:

expected records per batch ≈ 1 (first record) + linger_seconds × records_per_second_per_producer_per_partition

For example, linger.ms=50 with a per-producer-per-partition rate of 100, we might expect 6 records per batch.

This is only an approximation as it ignores arrival jitter, partition skew, batch.size config (default 16KB), compression, in-flight request limits, and broker backpressure. But it is good enough to build intuition.

The math and the 5K workload

In the 5K records/s workload, each producer was sending about 41 records/s:

5000 records/s / 120 producers = ~41 records/s per producer

That is one record every:

1000 ms / 41 = ~24 ms

This was also a no-record-key workload. With the default partitioning behavior, records from a producer tend to stick to one partition for a while before moving to another sticky partition. So, for batching purposes, the producer was usually sending roughly one record every 24 ms to its current sticky partition.

That makes linger.ms=5 unlikely to help. A 5 ms linger is much shorter than the ~24 ms average gap between records, so most batches still contain a single record. To reliably get more than one record into a batch, the linger would need to be on the order of the inter-arrival time (tens of milliseconds), not 5 ms.

So the low-load result made sense: Kafka 4.3’s default linger.ms=5 added a little extra waiting causing a higher end-to-end latency, but did not create meaningfully larger batches and its load was so low that larger batching wouldn't have helped anyway.

The math and the 100K workload

The 100K records/s workload was different. There, each producer was sending about 833 records/s:

100000 records/s / 120 producers = ~833 records/s per producer

That is one record every:

1000 ms / 833 = ~1.2 ms

At that rate, linger.ms=5 can make a real difference. A producer has time to collect several records before sending a batch. In this workload, I saw the average batch size reach about 5 KB, or roughly five 1 KB records per batch.

That reduced the number of small produce batches the cluster had to process. It also improved downstream efficiency for the brokers and consumers. The result was a large reduction in tail latency:

• the 3.7.2 run, with the old default linger.ms=0, had periodic p99.9 spikes around 700 ms,
• while the 4.3.0 run, with the new default linger.ms=5, had a much lower and more stable p99.9 around 8 ms.

So the benchmark was not necessarily showing a deep Kafka 3.7.2 versus 4.3.0 performance difference. A large part of the effect could be explained by one client-side default changing: linger.ms moved from 0 to 5 ms in Kafka 4.0.

Running a linger.ms experiment on 3.7.2 and 4.3.0

I decided to run a similar benchmark again, explicitly setting linger rather than using defaults. This time I used half the producers (better for batching) but with record keys (much worse for batching).

I ran Dimster on Kafka 3.7.2 (broker and clients) and 4.3.0 (broker and clients), with six test points across two scenarios:

runConfig:
  testDurationMinutes: 60
  warmupDurationMinutes: 5

baseWorkload:
  topics: 20
  partitionsPerTopic: 6
  messageSize: 1000
  producersPerTopic: 3
  messageDistributor: "KEY_ROUND_ROBIN"
  consumerGroupsPerTopic: 2
  consumersPerGroup: 6
  consumerType: CONSUMER_GROUP
  kafkaConfig:
    replicationFactor: 3
    topicConfig:
      min.insync.replicas: 2
    producerConfig:
      acks: all
      enable.idempotence: true
    classicConsumerConfig:
      auto.offset.reset: earliest
      enable.auto.commit: false
      dimster.commit.mode: CommitAsync

scenarios:
  - name: 5K/s
    label: ["linger-0", "linger-5", "linger-20"]
    producerRate: 5000
    runConfig:
      warmupDurationMinutes: 5
    kafkaConfig:
      producerConfig:
        linger.ms: [0, 5, 20]

  - name: 100K/s
    label: ["linger-0", "linger-5", "linger-20"]
    producerRate: 100000
    runConfig:
      warmupDurationMinutes: 30
    kafkaConfig:
      producerConfig:
        linger.ms: [0, 5, 20]

The 5K scenario

If we look purely at the batching behavior, none of the linger values helped in the 5K records/s tests as the per-producer rate coupled with record keys meant that linger was ineffective at creating larger batches due to the low per-producer-per-partition rate. The chart below shows Kafka 4.3.0 over the three test points with linger of 0, 5 and 20. Only a linger of 20 slightly moved the needle.

The exact same pattern occurred with 3.7.2.

This workload did not need larger batches: the latency distribution for linger.ms=0 was already good. There was no difference in performance between 3.7.2 and 4.3.0.

The 100K scenario

The place where linger mattered was the 100K records/s keyed test. In that workload, linger.ms=20 showed a massive improvement over a linger of 0 and 5.

linger.ms=5 did not help much at all and we can understand why by doing the math:

100000 / 60 producers = 1666 records/s per producer.

Due to record keys, 1666 / 6 partitions = 277 records/s per partition, per producer

Send interval per partition = 1000 / 277 = 3.6 ms

A simple estimate would predict about two records per batch at linger.ms=5 and about six at linger.ms=20, which lines up with the observed producer batch-size metrics.

The batching improvement with linger.ms=20 was reflected in the end-to-end latencies, with p99.9 of only 23 ms, compared to over 700 ms for a linger of linger.ms=5.

Noteworthy is that the results for 3.7.2 and 4.3.0 with linger.ms=20 were essentially identical. 4.3.0 pulled ahead in the lower lingers, but there is often huge variance in the higher latencies, so from one run, this is inconclusive.

Recommendations

Don’t over-index on this one set of benchmarks. No benchmark is fully generalizable, and the right linger.ms value depends heavily on the workload. The main takeaway is simply this: pay attention to producer batch sizes.

When producers are sending batches with only one record, Kafka can hit performance limits much sooner than you might expect. The broker has to process more produce requests, more record batches, more replication work, and more fetch-side batch metadata for the same logical throughput. A small amount of batching can make a large difference.

The most important number to understand, with regard to likely batch sizes, is the per-producer-per-partition send rate. Total cluster throughput can be misleading. A workload doing 100K records/s may still produce tiny batches if each producer is spreading records across many partitions. Keyed workloads are especially prone to this, because the key determines the destination partition. If each producer writes to many keyed partitions, the effective rate into each producer-partition pair may be low.

Under enough load, Kafka producers will often start batching more even with a low linger.ms, simply because the sender thread cannot drain records immediately. Broker latency, network saturation, throttling, or in-flight request limits can all cause records to accumulate in the producer. But relying on backpressure to create batching is not ideal. In some workloads, setting a higher linger.ms lets you get the batching benefit before the system is already under stress.

The default linger.ms changed from 0 to 5 in Apache Kafka 4.0. That means some Kafka 4.x client upgrades may show performance improvements simply because the producer is now batching more by default. Conversely, if you are using Kafka 3.x clients, explicitly testing linger.ms=5 is a low-risk experiment.

As for Kafka 3.7.2 versus 4.3.0, anecdotally, I’ve seen improvements in Kafka 4.x, and I may do more benchmarking to isolate those changes.

RayTracer — a ray tracer in ClickHouse SQL

A path tracer written entirely as ClickHouse SQL queries, rendering straight to a PNG via ClickHouse's PNG output format. No UDFs and no external code — a single SELECT computes every pixel.

It renders the word ClickHouse as glassy, chrome lettering — in the spirit of Andrew Kensler's famous Pixar business-card ray tracer — and, in the scene above, sets it floating over a procedurally generated landscape, reflecting the terrain and casting shadows on the hills.

See also:

• NoiSQL — Generating Music With SQL Queries
• SQL Mandelbrot Benchmark
• Click-V: A RISC-V emulator built with ClickHouse SQL
• DOOMHouse is a "Doom-like" game engine that renders the 3D graphics entirely in ClickHouse SQL

How it works

The whole renderer lives in one query:

• Pixels are rows. numbers_mt(width * height * samples) produces one row per (pixel, sample); samples are averaged with GROUP BY pixel, and the output columns r, g, b (in [0, 1]) plus explicit x, y coordinate columns (pixel % width, intDiv(pixel, width)) are written to the PNG output format. The explicit coordinates let the writer place each pixel by its position, so the rows need no ORDER BY and the heavy per-pixel work stays parallel across all cores.
• 3D math on tuples. Vectors are Tuple(Float64, Float64, Float64); dotProduct, L2Normalize, tuplePlus, tupleMultiplyByNumber, … do the linear algebra, wrapped in short lambda aliases (va, vs, vm, vd, vn, vc, vref).
• The bounce loop is arrayFold. Every ray is advanced exactly one mirror bounce per fold step over range(maxDepth) — a loop inside each row, so rows stay independent and the render parallelizes across all CPU cores. (The first version used a WITH RECURSIVE CTE instead — see the queries below.)
• let-bindings via arrayMap. ClickHouse WITH lambdas are call-by-name, so passing a value as a parameter re-expands the expression and blows up the query tree. Intermediates are therefore bound by value with arrayMap(x -> body, [expr])[1], a one-element-array "let".

Geometry (Constructive Solid Geometry)

• Cylinders — round rods with flat caps for the straight strokes (l, i, k, H, u, the bar of e).
• Tori — the round letters (C, c, o, u, s, e), ray-marched through their signed-distance field; the openings of C/c/s/e are a box subtracted from the ring.
• Spheres — the dot on i, plus a chrome "planet" that is a sphere minus a sphere.
• Oriented boxes / parallelepipeds — available for flat-faced strokes.

So the scene exercises boxes, cylinders, tori and spheres, with CSG union, difference (the planet and the letter openings) and a ray-marched distance field (the tori).

Terrain

A height field z = amp · fBm(x, y), where fBm sums several octaves of lattice value-noise. Camera rays are ray-marched against it — the march skips the empty air (it starts where the ray first drops to the terrain's maximum height) and linearly interpolates the surface crossing, so it is both fast and free of step-banding. It is shaded with a height color ramp (water → sand → grass → rock → snow), a warm sun plus cool sky-ambient model, marched shadows (terrain self-shadow and the letters' cast shadows), and distance fog into the sky.

Gallery

ClickBulb — a desk lamp built entirely from spheres hops in, leaps behind the banner, rakes its light through the letter gaps, then pokes its head through to peer at you. Every frame is the same ClickHouse SQL query, run once per frame.

Running

Every file is complete and self-contained, and parameterized: the image size comes from ClickHouse's image-output settings (read in SQL with getSetting) and the samples per pixel from a {SAMPLES:UInt32} query parameter — one query renders any resolution. Render one with:

clickhouse local --output_format_image_width 2560 --output_format_image_height 1200 \
                 --param_SAMPLES 8 --queries-file queries/clickhouse_terrain.sql > out.png

The queries are emitted by the Python generators; the scene and bounce depth are baked at generation time, while image size and samples per pixel stay runtime parameters of the emitted query. For example, to re-create the hero image:

python3 generators/gen_combined.py 2560 1200 8 2 > scene.sql   # depth 2; W/H/samples are runtime
clickhouse local --output_format_image_width 2560 --output_format_image_height 1200 \
                 --param_SAMPLES 8 --queries-file scene.sql > scene.png

Rendering arbitrary text

The sphere-banner generators take an optional 5th argument: the text to render. It is laid out from the 7-row bitmap font (uppercase A–Z, digits, and common punctuation; the original mixed-case "ClickHouse" glyphs are preserved, so the default render is unchanged), and the camera, light, and chrome sphere are recentered on the banner automatically.

python3 generators/gen_fold.py 640 256 16 4 "HELLO SQL" > hello.sql
clickhouse local --output_format_image_width 640 --output_format_image_height 256 \
                 --param_SAMPLES 16 --queries-file hello.sql > hello.png

Preview a banner as ASCII art without rendering: python3 generators/font.py "HELLO SQL".

Trying with other databases

• 🐛 CedarDB: works, but 33 times slower and with bugs.
• ☠️ DuckDB: does not work, tried multiple ways - with arrays and recursive CTE, but it can't process even the smallest resolution image.

Credits

Inspired by Andrew Kensler's business-card ray tracer and Paul Heckbert's minimal ray tracers — re-imagined as pure ClickHouse SQL.

License

Creative Commons Attribution-NonCommercial-ShareAlike 4.0, the same license as ClickBench.

Tripo AI, a global artificial intelligence company building AI 3D foundation models and world models, said it raised a new round of $150 million in funding.

The new funding comes just a month after the company said it completed funding rounds that raised $200 million in total. As you can tell from the numbers here, it’s still a frothy time for AI fundings, including some that have to do with gaming.

The aim for the generative AI technology for use in gamers is to lower the barrier to entry for artists in creative work, not replacement, the company’s chief scientist, Yanpei Cao, told me a month ago in an interview. I also did a written interview with him that you can see below for this funding, and he reiterated the pledge to work with artists.

At GDC, CEO Simon Song showed me the company’s 3D generation models and vibe coding tools, with results showing in seconds. As befits a company with a lot of people in China, they weren’t that concerned about whether the AI would replace human game developers.

The latest investors in the Series A3 financing come from the automotive, automotive, gaming, internet, and technology sectors. Automotive backers included Geely Capital among other strategic investors. Gaming companies 4399 Network, Tanwan, and Giant Network also took part, alongside strategic investors Fosun Capital and Orinno Capital.

Financial investors CoStone Capital, Addor Capital, T-Capital, and Muhua Tech Ventures joined as well. Existing shareholders INCE Capital and Genesis Capital further increased their investments, underscoring their continued confidence in Tripo AI’s strategic direction and long-term growth prospects. The breadth of the investor base reflects growing cross-sector recognition of Tripo AI’s leadership in 3D foundation models and world model technologies. These technologies are expected to play an increasingly important role in intelligent manufacturing, interactive entertainment, internet applications, and embodied intelligence.

Over the past six months, Tripo AI has released a series of advanced 3D foundation models, including Tripo H3.1 and Tripo P1.0, while introducing major algorithmic breakthroughs such as 8KTexture Generation and Segmentation V2. In independent blind evaluations and community voting, Tripo AI has consistently ranked among the world’s top AI 3D models. The company also introduced Project Eden, its world model research preview, pioneering native architecture that decouples underlying state simulation from visual rendering and establishing a new paradigm for world model development.Following the completion of this round, Tripo AI will further increase investment in 3D foundation models and world models, with a focus on core algorithm development, data infrastructure, and the recruitment of top global talent.

The company will also accelerate its global commercialization and strengthen its industry ecosystem, reinforcing its technology and product leadership while delivering more powerful AI solutions to creators and enterprise customers worldwide.

Tripo AI said is advancing AI toward true spatial understanding, physical simulation, and real-world production.

Written Q&A with chief scientist Yanpei Cao

What was the most convincing demo or tech that you showed investors that helped them decide to invest in this big round?

To be completely frank, investors in 2026 are well past being dazzled by a rotating video of a 3D object. What ultimately convinced them wasn’t a single “magic demo,” but rather the hard proof that we have crossed the threshold from experimental AI to industrial-grade production, alongside a sound roadmap for the future.

We demonstrated this across two parallel tracks:

First, on the 3D Generation side, we proved that AI 3D is no longer a visual toy; it is an active pipeline multiplier. We showed them the combined power of our native architectures: TripoH3.1 for extreme geometric fidelity, P1.0 for structured, engine-ready topology, and our latest capabilities like Segmentation V2 (native semantic part-understanding) and 8K Textures.

But more importantly, we showed them exactly how this tech is being used in the wild right now, backed by strong month-over-month revenue growth. For example:

• In Professional Gaming: While I can’t name specific IPs for confidentiality reasons, we demonstrated how major mobile and AAA RPG studios are already integrating Tripo into their level-design workflows. Instead of outsourcing background props and waiting weeks, their technical artists use our pipeline to generate assets with clean edge flows and 8K textures, dropping them straight into game engine with zero manual retopology.
• In the Automotive Sector: We showed how automotive design teams are utilizing Segmentation V2. They can generate a highly detailed vehicle concept and natively isolate specific components (like headlights or chassis parts) for rapid prototyping and aerodynamic simulation environments, completely bypassing early-stage CAD bottlenecks.
• In the UGC and Creator Ecosystem: We highlighted stories from grassroots creators. During a recent ‘Vibe Jam’ event, we saw solo indie developers and hobbyists use Tripo to populate entire playable game levels in a single weekend. They used our AI to instantly generate everything from mechanical props to environment structures, turning what used to require an entire art department into a seamless, solo creative process. Second, on the World Model side, we showed them the underlying framework of Project Eden, where the underlying structured physical state is completely separated from the visual rendering.
• This specific architectural choice resonated deeply, not just in the boardroom, but among top-tier researchers globally. When you decouple the state from the rendering, you natively solve object permanence, spatial consistency, and multi-agent concurrency. People looked at it and realized this isn’t just an engine for next-generation interactive entertainment. We are building the exact logically consistent, physics-bound simulation environments required to train robotics and Embodied AI (EAI). They saw the foundational architecture applied to real scenarios, and they believed in that future.

What will you do with the money?

The continuous rounds give us the computational firepower and runway to aggressively accelerate our R&D, but we are allocating it with extreme precision across three main pillars:

Scaling our 3D Foundation Models and World Models. We are pushing our native 3D architectures to their absolute limits. For our 3D generation models (the H and P series), we are scaling our parameter counts and proprietary data curation pipelines to handle unprecedented geometric complexity. The goal is to consistently deliver assets with flawless, production-ready topology that require zero human cleanup.

For our World Model, a massive portion of our R&D is dedicated to solving ‘State Transitions.’ We are investing heavily in multimodal training (combining 2D Video with native 3D data) so our models can learn ‘latent general actions’, essentially teaching the AI how physical structures should bend, break, and collide. This compute will allow us to move from generating static environments to simulating concurrent, multi-agent interactions in real time.

User Experience and Workflow Integration. Great models are useless if they don’t fit into how people actually work. For our enterprise clients and professional developers, we are using this funding to build deeper, more robust API infrastructure and native plugins. We want Tripo to act as an invisible, seamless utility embedded directly within traditional DCCs (like Maya and Blender) and game engines (like Unreal and Unity), completely removing the friction of asset transfer.

For independent creators and UGC platforms, we are investing in the interactive runtime experience. We are building the interface layer where a user can simply use natural language to instantly instantiate and orchestrate a playable, physics-based environment, lowering the barrier to game creation to near zero.

Real-World Applications and Functional AI. Finally, we are expanding our R&D beyond just visual assets into functional assets to unlock new industrial use cases. We are actively researching native kinematics and physical parameter generation. This means an AI-generated car won’t just look like a car; its wheels will be topologically separated with defined rotational axes.

By generating assets that natively understand their own physical structure and function, we are actively expanding our footprint beyond interactive entertainment. This is the exact technological leap required to provide the highly physically accurate simulation environments needed for Embodied AI, robotics training, and advanced industrial design.

How many employees do you have?

We currently have several hundred team members including full-time employees and research interns. Given the pace at which we’re scaling, any specific number would likely be outdated within weeks.

Where are most of the investors from?

Rather than looking at our cap table through the lens of geography, we look at it through the lens of strategic ecosystem alignment. Our backers are a combination of top-tier financial institutions and major strategic investors spanning the automotive, gaming, entertainment, and broader technology sectors.

To give you an idea, our strategic investors include Geely Capital on the automotive and industrial side, major gaming companies like 4399, Tanwan, and Giant Network, as well as the strategic investment arm of a leading internet conglomerate.

When you have automotive, gaming, internet, and heavy industrial capital all converging on the exact same company, it sends a definitive message to the market. It signals that native AI 3D and World Models are no longer being viewed as a niche vertical tool just for making video game props. They are being universally recognized as horizontal infrastructure: the foundational spatial computing engine required to power everything from next-generation UGC gaming platforms to autonomous driving simulations and Embodied AI.

Who uses Tripo? How will this help with game development?

If you look at our user base today, it spans the entire spectrum of interactive 3D and digital content creation. We are being heavily utilized by professional AAA game studios, indie developers, AR/VR teams, and animation houses. But because our AI understands native 3D geometry and doesn’t just generate visual illusions, we also have massive adoption in heavy industries like automotive design, architectural pre-visualization, and 3D printing, where physical accuracy is mandatory.

When it comes to game development specifically, Tripo is fundamentally rewiring the production pipeline.

In a traditional workflow, every single character, background building, and environmental prop has to be modeled, retopologized, UV-unwrapped, and textured by hand. It is an incredibly expensive, time-consuming bottleneck. What Tripo does is collapse that entire process. Using our native architectures (like P1.0 and H3.1) developers can use text or images to instantly generate high-fidelity meshes.

But here is the critical difference: we aren’t just generating a “concept.” Because we solve for strict geometric details and logical edge flows, we are outputting pipeline-ready assets. And they export natively in industry-standard formats like FBX, OBJ, and GLB, complete with semantic structures, PBR textures, and skeletal rigging.

This means a technical artist can drop our generated assets directly into mainstream DCC tools like Blender or Maya for immediate editing, or pull them straight into Unreal Engine and Unity for gameplay integration, with zero manual cleanup required.

Ultimately, this changes how games are built. We are moving from a paradigm where assetsare pre-fabricated in isolated silos, to a workflow where creators can summon assets on demand during the level-design process. It elevates AI from being just a “cost-reduction tool” into the foundational engine for a real-time, UGC world-building ecosystem.

Do you compete with General Intuition? How can you beat them?

The short answer is no, we don’t compete with them. In fact, if you look at the broader AI ecosystem, we are highly complementary. We are solving two different halves of the same equation.

To understand the difference, you have to look at the end goals.

General Intuition is fundamentally focused on Action Models and Policy Learning. They are using a clever dataset: hundreds of millions of hours of 2D gameplay video paired with explicit button presses, to train an AI agent to understand causality and take actions. For them, their world model (which generates environments frame-by-frame from video) is not their final product; it is just an internal gym used to train their agentic models. They want to build the brain that navigates the world.

Tripo, on the other hand, is not an embodied AI or agent company. We have zero intention of building the “brain.” We are building the universe it operates in.

Our technical focus is entirely on architecting the foundational physical simulator. As we discussed earlier, generating an environment frame-by-frame from 2D video is essentially an optical simulator. While it can learn basic visual correlations like shadows or walls, it lacks a true state. If you want to train an agent to precisely grasp a complex mechanical part, or if you want to support hundreds of agents interacting concurrently in the exact same room, a pixel-based video simulator will hit a hard mathematical ceiling regarding object permanence and compute costs.

This is where Tripo’s decoupled architecture comes in. Because our World Model is built on a persistent, structured state rather than guessing pixels frame-by-frame, we provide an environment with absolute physical boundaries, native multi-agent concurrency, and long-horizon consistency.

So, rather than framing other world model companies as direct competitors, we view them as vital parts of the ecosystem. In the future, companies building advanced AI agents (like General Intuition) will inevitably need structural, highly diverse, and persistent simulation environments to train their models beyond what 2D video can offer. Tripo is building the exact foundational infrastructure to provide that environment.

How do you feel about the resistance from gamers and game developers to generative AI?

We consistently view AI as a tool to augment the capabilities of creators, rather than a tool to replace them. The value of AI lies in helping teams reduce highly repetitive production cycles, allowing artists and developers to dedicate more time to pure creativity. From the perspective of industry evolution, technological progress typically reshapes job structures rather than simply eliminating roles. For example, as tools have evolved within the gaming industry, new roles have emerged:—such as Tools Engineers and, more recently, AI Tools Artists.

I believe AI will bring about a similar transformation. In the future, we will see more roles centered around AI workflow design, data management, and creative control. Of course, we also attach great importance to issues such as data usage, copyright, and creator rights.

Establishing a transparent and responsible technological ecosystem is vital; this requires the joint participation and standardization efforts of regulatory bodies, content creators, and industry organizations.

Overall, I do not believe AI will diminish the importance of human creativity. On the contrary, it will lower the barrier to entry, allowing more people to participate in the creation of 3D content and interactive worlds. The true value lies in how we combine technology with creativity, rather than allowing one to replace the other.

The new capital will be used to expand Tripo AI’s AI 3D and world model research teams, accelerate core algorithm development, strengthen data and infrastructure systems, and broaden the company’s global product and ecosystem presence.

Alongside the financing, Tripo AI introduced Project Eden, a world model research initiative designed to support persistent, reusable, and multiplayer interactive environments. Project Eden represents Tripo AI’s next step toward enabling creators, developers, and researchers to create, modify, and enter interactive worlds that can persist over time.

Full article content is not available for inline reading.

Read the original article →

shader magic for modern frontends

The component library for creative WebGPU effects in the browser

Ship interactive backgrounds, logo shaders & image effects

Elevate your designs in seconds with our WebGPU components and presets

Find your perfect shader with 0 Pro presets.

Explore 0 hand-crafted collections — ready to drop into any project, available with Shaders Pro.

Customize visually. Copy & paste code.

Create effects in the design editor, then export clean, production-ready code for your framework of choice.

Connect your agent. Watch it do magic.

Search presets, install with one command, and customize props — all from your AI agent, without leaving your terminal. Connects to any MCP-compatible agent in minutes.

Mix and match from 120 base components.

Stack, nest, mask and blend to create endlessly customizable effects in familiar component structure.

Free for personal & commercial use.

Framework Quickstart Guides

• Vue: Quickstart guide
• React: Quickstart guide
• Svelte: Quickstart guide
• Solid: Quickstart guide
• JS: Quickstart guide

When anyone can crank out a decent looking page, shaders is what will take your designs to the next level. This is a fantastic tool to help you turn your creativity into those experiences.

— Wes Bos, Syntax

Shaders significantly lowers the barrier to creating web shader animations, making it possible to design fancy, interactive visuals in seconds. Highly recommended for all frontend developers and design engineers.

— SerKo, Vue Language Tools

The project is incredibly smooth and easy to work with! It's just amazing how, with very little effort, you can add things that have a stunning visual effect!

— Hugo Richard, Nuxt / Vercel

Frequently asked questions

Everything you need to know before getting started with Shaders. For more detailed guides, check out our full documentation.

• What is Shaders?
• Is Shaders free?
• What does the Pro license include?
• Do I need to know shader programming or GLSL?
• Can I use Shaders in production?
• What frameworks does Shaders support?
• How do I install and use Shaders?
• What do I actually export from the editor?
• What can I do with the Shaders MCP?
• Which AI agents support the Shaders MCP?
• Are the effects animated or interactive?
• Can I use my own media in my shaders?
• Does Shaders use WebGPU or WebGL?
• Is there documentation available?
• Is there a community or place to get help?

AI coding agents can write functional code. But getting them to produce something that also looks good is hard. The default output tends toward purple gradients, rounded corners, and Inter font everywhere, and layouts that check boxes without conviction. This isn’t the agent’s fault. It has no sense of taste.

Enter AI skills. These are markdown files you install into Claude Code, Codex, Cursor, or any agent that supports them.

Below I’ve collected some design-focused AI skills and skill directories that I think are worth adding to your AI agent. Once installed, it will teach your agent a specific design language from Swiss grid systems to OKLCH color science.

So let’s take a look at each skill and what it could bring to your agent’s design vocabulary.

1. OKLCH Color Skill

OKLCH Color Skill teaches your agent to work with the OKLCH color space, a perceptually uniform alternative to HSL that eliminates hue drift and makes palette generation predictable.

Instead of guessing hex values or producing muddy HSL combinations, the agent learns to convert between formats, generate uniform palette scales from 50 to 950, derive dark mode colors through lightness manipulation, check contrast against WCAG 2 and APCA standards, and build Tailwind v4 themes with proper OKLCH tokens.

Install: npx skills add jakubkrehel/oklch-skill

Use: /oklch-skill or just describe a color task. The skill activates automatically when the agent encounters color-related work.

If your project uses custom design tokens or you’re tired of agents producing color combinations that look fine in isolation but fall apart on dark mode, this skill is a solid fix.

2. Swiss Design System Skill

Swiss Design System brings the Swiss International Style to AI agents. The skill teaches grotesque sans-serif typography such as Helvetica, Univers, Neue Haas Grotesk, along with strict grid systems, generous whitespace, and restrained color.

It expresses these principles through Tailwind CSS so the agent can apply these rules using standard utility classes. The skill includes a complete font specimen ranking by fidelity to the original style, grid system references from Josef Mueller-Brockmann, and data on both the Zurich and Basel schools of thought.

Install: npx skills add zeke/swiss-design-skill

Use: Ask your agent to build a layout in Swiss International Style. It will apply grid discipline, select appropriate grotesque typefaces, and maintain breathing room around content automatically.

Great for landing pages, editorial layouts, and any project where clarity and visual hierarchy matter more than decorative flourish.

3. Transitions.dev

Transitions.dev is a collection of nine essential UI transitions packaged as a copy-paste showcase and an installable agent skill. The transitions cover card resize, number pop-in with digit flip and blur, notification badge diagonal slide, and a lot more – each comes with ready-to-use CSS that agents can apply directly in your project.

The skill is generated from the index.html on the showcase site, so the code your agent produces always matches what you see demonstrated. If you’ve ever had an agent produce a modal that pops in from nowhere with no transition, this skill fixes that.

Install: Clone the repo or reference the skill from github.com/Jakubantalik/transitions.dev

Use: Mention any of these transition patterns and the agent applies the pre-optimized CSS.

4. DESIGN.md Directory

A directory of DESIGN.md files built specifically for AI coding agents. Each file describes a complete design system including color palettes, typography rules, spacing systems, component specs, and layout principles. Pick a style, copy the markdown, and add it to your project as a DESIGN.md file. Your agent reads it and applies that design language to everything it builds from buttons to full page layouts.

Use: Browse the directory, find a design system that matches your aesthetic, download or copy the DESIGN.md, and place it in your project root. The agent picks it up automatically without additional configuration.

This is the lowest-friction option on the list: no CLI, no npm, no install command. Just a markdown file in your project.

5. Agents with Taste

Agents with Taste is an installable design engineering skill by Emil Kowalski that covers animation principles, component design, easing curve selection, duration guidelines, typography rules, and practical tips drawn from his open source projects.

The skill includes a table of practical tips, an easing decision flowchart that teaches agents exactly which curve to use based on context, duration guidelines from micro-interactions (100-150ms) to modals (200-300ms), and strict typography rules like capping body text at 65 characters.

Install: npx skills add emilkowalski/skill

Use: Ask the agent to improve an animation or design a component. The skill applies Emil’s personal design philosophy automatically.

6. Hue

The Hue app is an open-source skill by Dominik Martin that learns any brand from a URL, name, or screenshot and generates a complete design system from it. You point it at Cursor.com or Raycast or any brand, and it produces a design system with color tokens, typography, spacing, components, dark mode support, and icon recommendations.

Hue comes with 17 example brands built in, from Atlas to Velvet. Each includes a design-model.yaml and a landing-page.html showing the system rendered.

Install: git clone https://github.com/dominikmartn/hue ~/.claude/skills/hue

Use: “Make a design skill from cursor.com” or “Create a design language inspired by Raycast.” The agent walks through the analysis and generates the skill.

7. Refero Styles

Refero Styles is a visual search engine for design references and an installable agent skill. Search by brand, mood, color, typography, or paste a URL. For each result, Refero provides the full style breakdown: colors, type, spacing, components, and a DESIGN.md file your agent can use directly. Under the hood, the skill gives your agent access to 150,000+ real app screens and 6,000+ user flows from products like Stripe, Linear, Notion, and Figma.

Use: Search for a style that matches what you’re building. Open any result to copy its DESIGN.md and place it in your project. Your agent instantly adopts that design language.

8. TypeUI Design Skills

TypeUI Design Skills is a collection of design skills made for Claude Design, Google Stitch, Claude Code, Codex, Cursor, and other tools. Each skill is an optimized SKILL.md or DESIGN.md file that gives your agent a specific design style. Browse the collection, find a look you like, and either copy-paste the file, download it, or use their CLI.

Install: npx typeui.sh pull [name]

Use: Browse the gallery, pick a design, pull it into your project. Your agent builds everything from that point using the chosen aesthetic.

9. UI/UX Pro Max Skill

UI/UX Pro Max is a design skill with 161 reasoning rules and 67 UI styles. The main feature is the Design System Generator – you describe your project and it builds a design system with layout patterns, colors, typography, effects, and things to avoid.

It supports multiple frameworks and stacks including HTML + Tailwind, React, and Next.js, and works with VS Code agents like Claude Code, Cursor, Windsurf, Copilot and Codex.

Install: npx ui-pro init inside any project or npm install -g uipro-cli for global access

Use: Describe your project, for example a premium tour booking app, and the generator recommends a complete design system with pattern, style, colors, typography, and UX rules. The skill also includes a search script for finding design references by query and domain.

10. AI UX Playground

AI UX Playground is a library of AI-specific UX design patterns for products like ChatGPT, Claude, Perplexity, and Cursor. Each pattern is documented with examples, use cases, and platform-specific instructions for how to apply it in ChatGPT, Claude, Google Gemini, Cursor, or other tools.

Use: Browse the pattern library, find the UX pattern you need, and follow the platform-specific instructions to apply it. Each pattern includes examples from real AI products and tips for implementing it in your own tooling.

What’s next?

We’ve just scratched the surface of what AI skills can do for designers. The tools here are just a starting point, and AI skills for design are still early – expect rough edges.

But the direction is clear. Design knowledge is shifting from static documents that humans read to executable rules that agents run. Designers who learn to write those rules will shape what agents build. This article gives you a starting point. Pick one skill from the list, install it, and see what your agent produces differently.

A Taxonomy of Self-evolving Agents

Self-evolving agents are becoming popular. Hermes Agent enables automatic reusable skills. RSI Lab tries to discover new algorithms recursively.

AI is unlocking valuable datasets from the physical world by combining cheaper sensors, robotics, and multimodal models. New data flywheels around infrastructure, healthcare, and industrial automation could create the next generation of enduring AI businesses.

SpaceXAI has released its latest model, Grok 4.5 — the first since the company went public several weeks ago.

In a blog post published Wednesday, SpaceXAI characterized its new release as a workhorse that can tackle all of the typical tasks that the AI industry has sought to automate: coding and app-building, office and clerical work, research, writing, and other forms of routine knowledge work.

Grok can supposedly do all this for less spend, too, as SpaceXAI says that its model has “twice greater token efficiency” than other leading models. If it carries through to real-world use cases, that efficiency would be a big advantage for SpaceXAI, since the cost of tokens has been a growing concern for AI consumers.

The company released benchmark metrics Wednesday that appeared to show Grok’s competitiveness with other top models from SpaceXAI competitors, although just short of best-in-class:

In a post on his social media platform X (which is a subsidiary of SpaceXAI), founder Elon Musk compared the model to Opus, Anthropic’s LLM designed for intensive and complex tasks.

“Based on strong positive feedback from customers in our beta test program, @SpaceXAI will make Grok 4.5 available to the public tomorrow. It is an Opus-class model, but faster, more token-efficient and lower cost,” wrote Musk in his post on X.

Musk later added: “Our internal assessment is that Grok 4.5 is roughly comparable to Opus 4.7, but much faster. The combination of capability, faster speed and lower cost is what makes it competitive.”

SpaceXAI says that its new model costs $2 per million input tokens and $6 per million output tokens. That’s quite competitive, if Grok’s capabilities match SpaceXAI’s rhetoric.

Opus 4.7, by comparison, costs $5 per million input tokens and $25 per million output tokens. OpenAI has tiered costs for different model versions: Sol, its most expensive, costs $5 for 1 million input tokens and $30 for 1 million output tokens, while its least expensive, Luna, costs $1 for 1 million input and $6 for 1 million output tokens.

It’s a big week for AI model releases. OpenAI is planning to release GPT 5.6, its latest, most powerful model, on Thursday. The release of that model had previously been limited by the Trump administration, due to concerns about its security implications. OpenAI has called it its “strongest model yet.”

The proliferation of nuclear power in space got a little more real Tuesday with the launch of a small satellite developed by a Florida-based company specializing in nuclear micro-power technology.

It’s a long way from launching a bona fide nuclear reactor, a breakthrough that could help power a permanent Moon base and efficiently drive rockets throughout the Solar System. But you have to start somewhere.

The satellite from Miami-based City Labs is named BOHR, short for Betavoltaic Orbital High-Reliability, and it launched on a SpaceX rideshare mission Tuesday alongside 80 other payloads. SpaceX’s Falcon 9 rocket released the BOHR satellite into an orbit between 350 and 400 miles (nearly 600 km) in altitude.

Starting small

City Labs bills the BOHR mission as “the world’s first commercial nuclear-powered satellite and first nuclear CubeSat.” CubeSats are modest in scale, and images released by City Labs suggest BOHR is built on a “1U” CubeSat platform, a cubical design measuring about the same size as a softball. BOHR’s power source is a nuclear betavoltaic battery that generates electricity from the decay of tritium, a radioactive isotope of hydrogen.

“This is a historic step for commercial nuclear power in space,” said Peter Cabauy, CEO of City Labs, in a statement. “BOHR demonstrates that safe, compact, and regulatory-approved nuclear power systems are ready for routine commercial deployment. This capability enables persistent, always-on payload operations that are not constrained by sunlight or battery life.”

City Labs will use its experimental NanoTritium power generator in demonstration mode to supply electricity to a payload onboard the BOHR CubeSat. The spacecraft itself uses conventional solar power for regular operations, the company said. Betavoltaic batteries are best suited for low-power applications that require a reliable, long-duration source of electricity. These use cases include remote terrestrial sensors—such as in undersea or polar locations—and instrumentation for secure communications. City Labs is also studying the use of its NanoTritium technology to power implantable medical devices.

The space industry is the other near-term market for City Labs. NASA has worked with City Labs to look at using nuclear tritium power sources to support a network of small sensors that could be deployed into permanently shadowed craters on the Moon to scout for resources like water ice. The US Air Force and Space Force have given City Labs several research contracts, funding the development of an experimental tritium AA battery for cryptographic devices and a self-powered wireless autonomous imaging sensor. City Labs says its betavoltaic systems could also power heaters for microelectronics in harsh environments.

It’s important to remember that the company’s betavoltaic power systems are small—in the nanowatt to microwatt range—far short of the electricity required to power a smartphone, much less a large spacecraft or a Moon base. Still, the BOHR mission is a step in the right direction for proponents of nuclear power in space. Until now, nuclear-powered spacecraft have been solely owned by government agencies like NASA and the US military.

Commercial nuclear-powered space missions face regulatory hurdles, and BOHR was the first commercial nuclear mission to pass through the Federal Aviation Administration’s new nuclear launch approval process. The FAA authorized City Labs to launch the BOHR mission last September.

It helped that the BOHR satellite carries just a tiny amount of radioactive material, and the tritium isotope decays more quickly than plutonium or uranium. It’s also less toxic than other well-known nuclear fuels. “Tritium emits a weak form of radiation, a low-energy beta particle similar to an electron. The tritium radiation does not travel very far in air and cannot penetrate the skin,” the Nuclear Regulatory Commission says on its website.

Future missions will have to launch with far more nuclear material than City Labs’ BOHR mission, but this week’s launch served as a first step.

“The BOHR mission serves as a pathfinder for future nuclear-powered spacecraft supporting both civil and national security missions,” City Labs said in a statement.

Own the Outer Loop

In the past year, the conversation around agentic engineering has moved to harnesses and loops, fleets and software factories. My 2c is engineers need to own the outer loop - the accountability for...

Meta is testing out a smart glasses prototype with always-on recording to help users remember things about their day. The prototype will not have a light-up LED indicator for when the glasses are recording. There are divergent views within the company about whether the data collected by the glasses should be stored on Meta's servers and used to train AI. Creating the technology will be challenging due to the battery and other issues.

FTC reaches settlement that brings right-to-repair to John Deere farm equipment

The debate over right-to-repair doesn't only apply to electronics. Today, the US Federal Trade Commission announced that, alongside five states, it has reached a settlement with Deere & Company, the manufacturer of John Deere farm equipment. The regulator sued Deere last year on allegations that the company engaged in unfair practices that limited farmers' ability to repair their own equipment. Those policies required equipment owners and independent repair providers to pay higher prices for any needed services.

Under the terms of the settlement, for the next 10 years, Deere will be required "to provide farmers and independent repair providers with the same equipment repair resources, including applicable software capabilities, that it currently provides to authorized Deere dealers." The company will also be subject to reporting and oversight requirements, and the initial decade-long agreement could be extended if it breaks the terms.

US PIRG Senior Right to Repair Campaign Director Nathan Proctor issued a statement about the result. "We should be able to fix our own stuff. This settlement from the FTC gives farmers more and better options to repair their equipment. It is a win for farmers and all of us who want a more fixable world," he said. "Our goal from the start of our campaign was to ensure that farmers and independent mechanics get everything they need to fix equipment. We will continue to monitor the situation and advocate to ensure that goal is a reality."

In light of the skyrocketing costs of build new tech, with prices unlikely to drop any time soon, the vision of a more fixable world sure sounds like a good one.

DuckDuckGo Browser can now block video ads, including YouTube's

An unsubtle shot at Chrome.

DuckDuckGo announced that it can now block most video ads, particularly those on YouTube, when a video is playing in its browser.

In a blog post about the new feature, the company explained that its YouTube ad detection and blocking is based on the open-source community filter lists from uBlockOrigin. DuckDuckGo noted that it may also apply its own rules for better compatibility, but viewers may see longer buffering times when using the blocker and there may still be some unexpected hiccups.

The blocking feature will be on by default for most DuckDuckGo users on iPhone, Windows and Mac. It will be automatically enabled soon on Android but can be manually activated in the browser settings menu in the meantime. All platforms can also have YouTube ad blocking turned on or off from the settings menu. And just to state the obvious, remember you'll actually have to be watching the video in the DuckDuckGo browser rather than the YouTube app in order to take advantage of the blocker.

A framework for building reliable agents across three layers: the model, the agent harness that runs tool loops, context management, guardrails, memory, observability, and retries, and the harness configuration that gives the agent the right use-case context, tools, permissions, and human-review thresholds.

Apache Ossie, formerly the Open Semantic Interchange effort, is now an Apache Incubating project. The spec models datasets, fields, metrics, dimensions, and relationships so teams can keep definitions consistent across tools while giving agents governed business context.

The Grammar of Data: Define Once, Run Anywhere with Cross-Engine Expressions

Grammars for languages or any other field are a beautiful thing. They compress complex systems into a language with a couple of rules. For the spoken language example, we know when to capitalize a letter or how to start a sentence. There are clear rules. Grammars also help us remember, as we do not need to recall every little rule, but apply them in a structured way.

For text editing, we have Vim motions that help us navigate a text document with 1000s of shortcuts, but because there is a grammar, we do not need to remember them all, but learn the structure of the grammar and combine them. But what if you work in data? What if we could have the same for data, a grammar for data engineering, or a language that defines it?

Expressing our needs declaratively and decisively? Also, expressing it in a way that leads to reproducible outcomes, or works with multiple parts and execution engines already out there. This is what we will discuss in this article. How existing tooling, such as Ibis, provides some capabilities, and how xorq extends them by adding full lineage and transparency for humans, with included executable memory for useful tabular data, all manifested in a single git repository.

Expressions for Data Engineering Workloads

Having a grammar for data engineering means we can express the workloads in a declarative manner, and then be sure we can deterministically reproduce and apply that exact definition.

It’s similar to the concept of a Declarative Data Stack I introduced a while back, but it gives the stack not only configurations but also a language with in-built manifestation and execution engines.

In the above image, we see: 1. How to express (write) our transformations and business logic. It’s the context of every ML or DE pipeline. 2. We can build the expression into a manifest that has a unique hash, runs input validations, tracks lineage, creates a deterministic cache, and produces a human-readable expr.yaml you can diff and review in a PR. 3. Lastly, we can execute it in any execution engine with the same manifest.

This is hugely powerful and separates the concerns of defining logic, verification in the manifest step, and execution as a composable data stack, as Wes McKinney called it, with multi-compute engine possibilities.

How the DE Language Works: Different Expression Types

Every grammar starts with nouns, and here the noun is the source, a node that holds data but carries no transformation yet. It might be an in-memory table, a registered connection to a warehouse, or just a lazy pointer to a file on disk that hasn’t been read. They’re simply referenced, the way a noun refers to a thing before any verb acts on it.

The verbs in our language are transforms such as filter, select, mutate, aggregate, join, order, limit. Each one takes a source (or another transformed expression) and returns a new, immutable expression. You do not mutate anything before it, only describe what should happen next.

Looking at a definition such as .filter(...).aggregate(...).mutate(...), we can see this as a sentence. The moment a verb is applied, the expression stops being a plain noun and becomes a statement, a description of “data plus what should happen to it.” But the sentence isn’t spoken yet, it stays inert, fully composed but unexecuted, until something finally asks it to run. That’s the deferred part of the grammar: writing the sentence and saying it out loud are two different acts.

There’s a third part of speech worth naming: the template. Instead of writing a sentence about a specific noun, you can write one about a noun’s shape, a schema with no rows behind it. A template says “given something with a column of this type, here is what I’ll do to it,” and only later gets bound to an actual source, at which point the placeholder resolves and it becomes an ordinary statement again.

And we have modifiers that ride alongside a statement without changing what it computes. They’re small tags of metadata that say “this expression also represents a fitted model” or “this is a saved reference to something else.” It’s like a footnote with additional metadata that doesn’t change the surface meaning, but adds context for later use.

This analogy makes the grammar compose the same way regardless of which engine eventually executes it. There are more parts, but with just these four, noun, verb, template, modifier, you can read (and write) arbitrarily complex data pipelines the same way learning a handful of verb-and-object combinations in a text editor lets you compose arbitrarily complex edits.

With this grammar, we can avoid repeatedly implementing the same logic we already have, but manifest and express our logic once, and reuse it with different execution engines, exactly what Ibis and xorq allow.

Why a Grammar is Really Good for LLMs

Having a grammar is really good for LLMs, too. It helps them first to declare data artifacts and second to execute them reproducibly.

On top, expressions can be LLM-agnostic, and we can interchange the LLMs we use just with an expression. Also, the chart is just an expression, or the data catalog and the metrics.

Model Once, Represent Everywhere: Expressing the Full Data Stack with a Single Expression

Like UDA (Unified Data Architecture) from Netflix, we define our expressions once and represent them everywhere. Netflix built UDA to solve duplicated models, inconsistent terminology, and siloed systems, where the same concept like ‘actor’ or ‘movie’ gets modeled differently across teams, with no shared foundation. Their answer was a full knowledge graph with a metamodel, making the conceptual model part of the actual control plane.

Not everyone needs Netflix-scale tooling, though. For a code-first approach, xorq gives you the same core principle: define once, execute anywhere by writing a declarative Ibis expression, serializing them as content-addressed YAML artifacts, and running against any supported engine, fully reproducible.

The difference worth noting: UDA is a semantic layer defining what data means across systems. Xorq is a computational layer defining what transformations do across engines. Both reject the same anti-pattern of re-implementing the same logic for every system.

Entering Xorq: The Horizontal Data Architecture

Xorq is an executable memory system for tabular data that works horizontally across your data stack, supporting everything from discovery with a catalog to defining transformation logic to modeling.

It has declarative transformation (Pandas style), and you can build ML pipelines and prepare data with its semantics in a single stack that is not vertically integrated, but horizontally integrated, giving your agents a catalog of executable pipelines and turning short-lived agent work such as wrangling scripts, sklearn pipelines, ad-hoc tables into durable, composable, executable artifacts that any future agent or human can discover, reproduce, and reuse.

The horizontal data stack shows what Xorq brings to the table. Xorq’s origins started from a git-native semantic layer, for data analysts out of college, to build semantic models for a living, to make their lives easier.

From point-and-click tools, dragging tables and drawing joins manually, only to add more reporting tools on top to create pixel-perfect reports. Also performance-wise, it didn’t scale, meaning we needed cubes to make it faster, adding another layer of complexity.

And there was no lineage that shows from source to dashboard. The question asked was: “what if we could do this end-to-end data engineering workflow locally?”. This is what the horizontal data stack and xorq are providing.

To add semantic layer capabilities, Julien Hurault and Hussain built the Boring Semantic Layer + the Xorq catalog, providing a semantic model you define in Python, check into git, and query from the CLI.

Compressing Logic into a Single Executable

Compression of a full data stack into a single executable is hard, but xorq tries exactly this with the help of Ibis, git, uv, and DataFusion.

The design choices of xorq showcase even better what it is, and what they enable:

• Ibis as expression layer (v9.5.0+, partial): Declarative dataframe expressions compiled to multiple backends (xorq supports a subset of the Ibis API, not the full surface)
• Git for state and storage: The catalog is a git repo of entries with git-annex support for large files
• uv for reproducible environments: Each entry ships with a wheel and pinned requirements.txt.
• DataFusion for embedded compute: Pipelines execute in-process with SQL and UDFs

Composable Data Engines

Another big advantage of expressions and having a grammar for data engineering is easily switching between backends, with no change to the transformation or business logic. It’s just defining the backend from Apache Arrow Flight to DuckDB or any other engine.

We write the definitions and express our tabular data and computations. The engine, in this case xorq, can build it into a manifest file that is deterministic and hashed.

Xorq uses Ibis as the expression layer for single-backend logic, then builds the cross-engine expression tree into a serialized YAML artifact. When moving data between backends, xorq transfers Apache Arrow RecordBatch streams between them—each backend acts as a RecordBatch transducer. No CSV serialization, no JSON encoding needed. This makes backend switching fast and memory-efficient. Write declarative Ibis expressions that run like a tool—xorq extends Ibis with caching, multi-engine execution, and UDFs.

Here’s an example of using DuckDB and Postgres in conjunction:

import xorq.api as xo

# Connect to engines
pg = xo.postgres.connect_env()
db = xo.duckdb.connect()

# Load data from different sources
batting = pg.table("batting")
awards = xo.examples.awards_players.fetch(backend=db)

# Filter in respective engines
recent = batting.filter(batting.yearID == 2015)
nl_awards = awards.filter(awards.lgID == "NL")

# Move data to postgres for join
result = recent.join(
    nl_awards.into_backend(pg),
    ["playerID"]
)

result.execute()

You can see how easily you choose your most optimized execution engine, whether in the above example choosing DuckDB for filtering recent batting and using Postgres to filter NL (National League) awards, and joining the two with the Postgres engine.

Engines supported by xorq as of now, with the ability to move data between them, are:

• Embedded: DataFusion, DuckDB, SQLite, Pandas
• Warehouses: Snowflake, Databricks, Trino, Postgres
• Lakehouse: PyIceberg
• Arrow Flight: GizmoSQL (DuckDB over Arrow Flight SQL)

Cross-Engine Expression Tree

With different engines supported, we can use the compressed single executable logic across engines. We can build expression graphs before executing them, which works like this, with one expression, many engines:

expr = penguins.into_backend(xo.sqlite.connect())
expr.ls.backends

The output of building a cross-engine expression is a directory containing your serialized pipeline with a unique hash identifying each build and its artifacts and expressions. When executed, the output is the resulting object or data.

And the expressions are tools, Arrow is the pipe. E.g., a Unix pipe streams text between small programs. Xorq pipes Arrow streams between expressions: unix : programs :: xorq : arrow-transforms

A Shared Language for Data

This article introduces a new way of describing data transformations for machine learning or data engineering pipelines in a direct and simple way that works locally with any execution engine, without changing the code itself.

It’s a good place if you need a trusted harness for a data engineering persona. We can define once and use it with the engine that works best for your workload and data engineering environment.

We had a look at how we write -> manifest -> execute with xorq, its advantages, and why you might use it for modeling once and representing everywhere. By adding AI agents to the mix, which help us pull the right lever, instead of bigger, more expensive models or more tokens, we improve accuracy with more semantic understanding, with a grammar the model can learn and apply, even pre-manifest before execution, and run them deterministically every time. This is a huge addition to working just with agentic Skills files that are free-form Markdown and pull data all over, or are not defined precisely enough. It’s all about having high-quality context in the right format, with a clear definition where humans and AI agents can interchange and help each other.

There’s a lot more to come, with showcasing the horizontal data stack and the use cases it supports, how we build expressions versus running computations, and how data catalogs are integrated into the picture, too.

Meta’s new Muse Image model can pull other Instagram users into AI photos

Users can ’@ mention’ accounts in their prompts.

Meta is launching the first AI image generation model made by its Superintelligence Labs division. The Muse Image model now powers the image-making tools across the Meta AI app, Instagram, and WhatsApp, and it’s coming soon to Facebook and Messenger, according to an announcement on Tuesday.

It’s part of the growing Muse family of AI models that replace Meta’s Llama lineup. Alexandr Wang, who Meta hired to head up its Superintelligence Labs last year, says on Threads that Muse Image is “agentic,” meaning it works with its Muse Spark large language model “to reason through your prompt, search the web, and plan before it generates.” Meta is also planning to launch a Muse Video model, which Wang teased, saying it’s “competitive on prompt adherence, visual fidelity, temporal consistency.”

As noted by Meta, users can @ mention other Instagram accounts in Muse Image prompts, allowing the AI model to incorporate their likeness into its output. Meta says “tagging a username lets Meta AI use public photos to build a visual,” but notes that users can control how people reuse their content for AI.

Users can also transform images using suggested prompts and create designs for things like invitations and postcards. Additionally, Muse Image can redesign rooms based on an image pulled from Facebook Marketplace (or elsewhere on the web) and allows users to make changes directly to photos by drawing right on top of them, which they can then share to their feed, story, or chat. The Muse Image model will power the 30 new AI effects coming to Instagram Stories in the US before rolling out to other countries and in more areas of Meta’s apps soon.

Figma’s Loredana Crisan talks taste, last-minute decisions and the art of prioritisation

In House is a new column from It’s Nice That where we go inside the workings of major brands, agencies and companies shaping the future of the creative industry.

In late June, thousands of designers descended on San Francisco for Figma’s annual Config conference. As ever, the company announced several new features, including Figma Motion, Code Layers that let users code on the canvas, and the ability to build custom plug-ins with an AI agent.

But it’s also a chance for the company to communicate a bigger vision for the future, at a time when new AI tools seem to be trying to eat its lunch.

We spoke with chief design officer Loredana Crisan, who joined Figma in September after nearly a decade leading design and product teams at Meta (including Messenger, Instagram DMs and GenAI Products).

We discussed Config’s last-minute charm, Figma’s relentless focus on keeping human creativity front and centre, and building a culture to help ideas reach “escape velocity”.

Rob Alderson: You’re fresh out of Config, what’s the biggest challenge bringing it all together?

Loredana Crisan: There are teams that are thinking about Config pretty much all year long. Each Config has a different brand identity and we start thinking about that early, to allow lots of ideas to simmer.

But we don’t decide on the product line-up until very close to the end, so the speakers, the demos, everything else takes shape in its final form the weekend before.

That’s the challenge of Config, but I think it’s also the charm, because it doesn’t become so rehearsed. You see the Figma team in its raw form.

“Prioritisation is always an art that we pretend is a science.”

RA: Is it easier or harder to be chief design officer at a company whose target audience is designers?

LC: On the one hand, there’s a lot of responsibility. Designers are a very discerning group and I take that very seriously.

But honestly, it’s so much easier because I get to work on a product that I’ve spent a decade creating in myself, for an audience that I understand.

RA: But when you’re walking through Config, do you get a lot of people coming up to you with ideas and feedback?

LC: I solicit that. I’m like, “I take requests.” I want to understand where we’re falling short because if you learn anything in design, it’s that your mind doesn’t see all of the possibilities.

RA: How do you decide which feedback gets addressed?

LC: Prioritisation is always an art that we pretend is a science.

Sometimes you get feedback that deeply resonates with something that you’ve already felt in the tool. Other times, you’re forced to hold off on implementing a good idea because something else has to come first.

That leads to what I call ‘finally features’ as in, “Finally Figma has this.” In some ways, Motion was a ‘finally feature’.

RA: What’s the biggest thing you’ve learned in your ten months in the job?

LC: Coming into Figma, I knew very clearly that AI was going to be part of the remit and we needed to understand how it figures in the creative process. My initial understanding was that we had to make AI a better designer, in order to become a better collaborator.

I still think that AI being fluent in the vocabulary of design is important, but that’s only half the story. The other half is keeping the human in the creative process – not having AI overtake the creative imagination.

Oftentimes we talk about the human like an editor – as the judgment, and the taste layer. But in reality, a lot of the taste comes from doing, and you don’t discover an idea until you’ve gone into the rabbit hole looking for it.

It’s an oversimplification that humans can just come and pick one of the variants that AI gives them.

RA: How do you decide where AI is added into the design process?

LC: We think it’s important that AI always scaffolds a human action.

So our plug-ins and gen effects that we announced at Config are created by AI, but they’re tools that you manipulate directly, and you see the effect on the canvas as you’re doing it.

Keeping that embodied sensation, with your brain and your hands working together, is really important.

“In reality, a lot of the taste comes from doing, and you don’t discover an idea until you’ve gone into the rabbit hole looking for it.”

RA: Figma’s view seems to be that creative people shouldn’t feel threatened by gen AI…

LC: Creative people have something that AI is missing, which is a genuine point of view. That comes from experience, from their unique way of seeing, from their values. That’s very hard to replicate.

We’re seeing a lot more software being produced but we’re seeing that it’s largely ignored, because human attention is limited. For something to stand out, it really has to grab you emotionally. It has to make us feel something. That’s not just poetic – it’s literally neuroscience.

That’s why we believe the demand for creative thinkers will only increase.

RA: When Claude Design launched, there was a lot of talk about what it meant for Figma. Do you think tools like this challenge your place in the industry?

LC: We certainly believe that AI raises the floor. And we’re adding AI to Figma, but we’re adding it in a way that has a different point of view than many of these tools.

Our goal is not to generate more work, but to create better work. AI becomes a tool in the human arsenal, but it’s one of the tools they can use.

I actually think Figma becomes even more important in this scenario, because you have a multiplayer canvas, you’re there with your team, you have an infinite space for exploration and direct control over what you’re producing, or what AI produces.

And if you look at all of these tools, their first action is to take whatever AI generates into Figma, because they realise that’s what creative people actually want.

RA: How do you build a creative culture?

LC: I think a lot about helping ideas reach escape velocity. Ideas are fragile, and they might not become something, depending on the environment and how you cultivate them.

I’ll give you Motion as an example. We knew we wanted to work on motion, but it was really easy to overthink what motion would be at Figma. Will it be a mode? Will it be a tab?

We had a few rounds of conversations where instead of making, we were just debating things. The idea did not have momentum. In the end, the right move was to get the product in staging. The team could pick any direction – just get it to be real so we can play with it.

And honestly the moment it was in staging, the team accelerated on their own. There were very few decisions where we got stuck.

“Ideas are fragile, and they might not become something, depending on the environment and how you cultivate them.”

RA: What about encouraging people to bring new ideas?

LC: Our plug-in product was dreamed up by one of our growth designers, who put a prototype in one of our Slack channels. It was clear that it was a really good idea and a lot of other people mobilised around it. As a leader, I could have been like, “It’s not on the road map. We have all these other things that we need to do for Config.”

But we recognised it had momentum, and so we created a project. If you don’t treat new ideas as important, people won’t come forward with them.

But none of this works unless you have safety in the team, where it’s ok for unfinished ideas to show up. I obviously have plenty of creative conviction – things that I love and I don’t love. But giving the team ownership and agency is critical.

RA: Is that easier said than done?

LC: Absolutely. Creative control is the name of the game as we build our careers. It takes maturity and trust to let things go in a different direction.

RA: What other skills did you have to develop when you first stepped into design leadership roles?

LC: On the one hand, leadership gives you more influence over what gets made. But at the same time you have far less control, as the making gets abstracted away from you, and you start talking about metrics and org charts and frameworks and systems.

It’s easy to get lost in that abstraction. But I found it helpful to stay grounded in what’s real. For me, that means the joy of making. I want to be in the tools. I don’t create the most important projects for Figma, but I’m constantly playing and making demos.

Then it’s about growing people. That’s not a name on an org chart, that’s a person, and you need to really care about them.

The third thing that’s always guided me is the value we put out into the world. If you keep the people you serve top of mind, I think you’ll be fine.

“Here is a new craft developing around these non-deterministic systems that I truly want to invite designers into, because this is what will create the experiences of the future.”

RA: What is a key skill designers will need in the future?

LC: The biggest change they need to embrace is designing for non-determinism.

With AI, all of the products that we build will have to be used either manually or through agents. And so you want to make sure that the agent experience fits what you believe is a great experience.

There is a new craft developing around these non-deterministic systems that I truly want to invite designers into, because this is what will create the experiences of the future.

RA: What doesn’t change – what skills will continue to be important?

LC: Empathy. Sorry, this is absolutely cliche, but that will never change. AI is part of the process, but we are people building for other people, and it’s really important for us to keep that top of mind.

The other thing that always guides me as a leader is cultivating the talent that will succeed you. If you think about many of the products that we’re building, they’re digital. They’re meant to only exist for a little bit, and then evolve into something very different. But the people that you build up and pull into the industry will really be your legacy.

Edit entire videos with a single prompt

Drop in your clips and say what you want. The cuts, captions, and music land in minutes, ready to post.

Turn my 6 vlog clips into a short Behind the Scenes montage.

Done. Cut your 6 clips into a 28-second montage, trimmed the dead air, and added music.

Loved by creators who post daily.

“The edit used to cost me my evening workout. Now I say what I want and go to the gym.”

Nick Tarmossin, Creator (150K followers)

“I post a build vlog every single day. Stanley cuts it while I'm still pushing commits. That's the whole reason the streak is alive.”

Omar Zeineddine, Creator (138K followers)

“I'll film five spots in one Toronto day and the clips used to sit for a week. Now they're all posted before Monday.”

Winnie Chen, Creator (20K followers)

Just talk. Stanley does the edit.

Say it like you’d say it to an editor. Rambles, umms and all. Stanley turns it into clean, structured edits and applies them to your footage.

Restyled your captions: TikTok Sans, bold pop, timed word by word.

Added a punch-in at “this is the one”, a hard cut zoom held for a beat.

Added a lofi bed under the dialog, ducked while you speak.

No timeline, no dragging

Describe the edit you want. This is the real prompt behind Daniel's reel.

The prompt used:

make video 9:16. Trim silences. Target 30-45s -- use your judgment. Open with a ~3s text hook: white text, black background wrapped tight around it. make captions tiktok sans and no animations Add harsh cut zooms for emphasis. Keep everything within safe zones. add light music in background and add image overlays where appropriate.

Stanley edits everything for you.

Each of these is one sentence in Studio Chat.

Cut silences

Dead air, umms, and false starts come out on their own. Stanley reads the audio track and trims every gap. You never scrub for them.

Captions

Word-timed captions in your style: clean, bold pop, or highlight. Ask for “the TikTok ones” and they land styled, synced, and inside safe zones.

Punch-ins

Harsh cut zooms for emphasis, exactly where you ask. “When I say ‘this is the one’, punch in.” That sentence is the whole edit.

Music

A bed that fits the mood, tucked under your dialog. Stanley picks the track and keeps it out of the way of your voice.

Text hooks

A ~3-second hook up top: white text, black wrap, tight around the words.

Autofill B-roll

Reads your script and drops the right footage where it belongs. Images and clips land on the beat they're mentioned, not where you drag them.

Reframe 9:16

Vertical crops that keep you centred and inside platform safe zones.

Transcript edits

Delete a sentence from the transcript and it's gone from the video. The words are the timeline. Edit text, and the cut follows.

Best takes

Said the line five times? Stanley keeps the strongest one. The other takes stay one click away as alternates.

Sound effects

Whooshes, risers, and hits matched to the moment you describe.

Less editing. More posting. Drop your footage and see for yourself.

128 questions to level up your web accessibility game. Practice WCAG 2.2, ARIA, semantic HTML, keyboard, and contrast, with plain-English answers.

NVIDIA underscores the importance of open and synthetic data for developing robust AI agents, highlighting its use in the Nemotron datasets for enhancing capabilities like reasoning and tool use. Open datasets contribute to AI system transparency and reproducibility, allowing developers to inspect and understand agent behavior.

Claude delivers the most leverage when used for high-impact work rather than routine automation.

Google Photos has introduced a Gemini Omni-powered tool that can quickly relight clips, replace backgrounds, and apply artistic styles such as watercolor, sketchbook, and oil painting.

Blue Origin is raising its first outside funding round. Jeff Bezos is set to contribute $2 billion to the round. The company suffered a setback in late May when one of its New Glenn rockets exploded on a launch pad. It has set an aggressive goal to return New Glenn to flight by the end of the year.

AI companies are distorting San Francisco's housing market. Some homeowners are accepting offers for shares of OpenAI or Anthropic as payments for their homes. Property prices are surging as buyers bet that whoever they overpay today will look cheap tomorrow. Landlords are putting out tenants to sell into the hotter market. Many properties are closing at prices at least $1 million more than their asking price last month.

Apple is introducing RAW 9 in iOS 27, iPadOS 27, and macOS 27, its biggest upgrade yet to the system-level RAW image processing engine. Powered by on-device machine learning and the Apple Neural Engine, RAW 9 combines demosaicing and noise reduction to deliver sharper details, more accurate colors, and significantly better low-light performance—even when reprocessing older RAW photos from nearly 800 supported camera models.

Rigid brand guidelines often prioritize visual polish over web usability, quietly hurting conversion through poor contrast, unreadable type, and missing hierarchy. Testing a brand-compliant homepage against a tweaked version through an AI attention tool showed clarity and focus up ~20% and predicted engagement with the main call to action up over 80%. Rather than treating brands as sacred, businesses should test whether their branding actually communicates intended values and works on screen, fixing readability conflicts before considering a full digital-first rebrand.

Strong brand systems make visual consistency easier than ever, but many brands are losing their distinctiveness because their behaviour—personality, tone, pacing, and confidence—is harder to define and maintain than logos, colors, or typography. As content production scales across more channels and teams, consistent brand behaviour becomes a key differentiator, with motion design playing a particularly important role in expressing a brand's character and making it instantly recognizable.

AI can do more than speed up UX work.

Women with two decades of high-level creative experience are quietly leaving the industry once they reach the top, as rigid, presence-based work structures fail to accommodate their lives outside work.

LLM, RAG, and eval skills are becoming table stakes, much like SQL.