Back to Devoured - May 14, 2026
DEVOURED
Agentic search models

Agentic search models

Data aillmagentssearch softwaredoug.com
Agentic search models are emerging to orchestrate entire retrieval workflows, replacing traditional brittle stacks with LLMs specifically trained for domain-specific search, prioritizing smaller size and lower latency over general-purpose frontier models.
What: Doug Turnbull discusses the rise of "agentic search models" like SID-1 from SID and Waldo from Glean, which are LLMs specifically trained to control the search task. Unlike monolithic search stacks using embeddings, rerankers, and query classifiers, these agents orchestrate simpler backend primitives, seeing the "whole" search process. They aim to capture the "last 20%" of domain-specific search nuances that frontier models like GPT-5 often miss, offering smaller size and lower latency for applications like e-commerce or job search.
Why it matters: This signals a paradigm shift in retrieval system design, moving from manually orchestrated, component-based pipelines to a more holistic, intelligent agent-driven approach. It suggests a future where specialized, smaller LLMs are common for specific enterprise search domains, akin to the current landscape of specialized embedding models.
Takeaway: If you are building or maintaining a search system, start evaluating the potential of agentic search models to simplify your architecture and improve domain-specific relevance, especially for I/O-bound processes.
Deep dive
  • Traditional search systems are built from a "thick monolith" of disparate components like embeddings, rerankers, query classifiers, and BM25, each focusing on a specific part of the problem.
  • Agentic search models are LLMs specifically trained to orchestrate the entire search task, using simpler backend retrieval primitives (e.g., basic keyword search, embedding models with filters).
  • These models aim to address the "last 20%" of search relevance, which includes domain-specific nuances (e.g., "bistro tables" meaning "small outdoor tables" in a furniture store) that general-purpose frontier LLMs (like GPT-5) often miss.
  • Early examples include SID-1 by SID and Waldo by Glean, which emphasize smaller size and lower latency compared to larger frontier models.
  • The benefit is a more intelligent, holistic approach to search, where a single model manages the entire process from query understanding to hybrid search, reducing the need for complex, manual query and reranking pipelines.
  • This approach is expected to lead to a family of domain-tuned agentic search models, similar to the current specialization in embedding models for finance, legal, or e-commerce.
Decoder
  • Agentic search models: Large Language Models (LLMs) specifically trained to understand and orchestrate the entire search and retrieval process within a given domain, rather than just generating text or embeddings.
  • Frontier LLMs: Refers to the most advanced and largest general-purpose LLMs from leading AI labs (e.g., OpenAI's GPT-5, Anthropic's Sonnet).
  • BM25: A ranking function used by search engines to estimate the relevance of documents to a given search query.
  • Embeddings: Numerical representations of text, images, or other data that capture their semantic meaning, used for similarity search.
  • Rerankers: Models or algorithms that reorder initial search results to improve relevance based on additional criteria or deeper analysis.
  • Query classifiers: Components that categorize incoming search queries to apply specific business rules or route them to specialized retrieval backends.
  • RAG (Retrieval-Augmented Generation): An AI framework that retrieves facts from an external knowledge base to ground large language models on the most accurate, up-to-date information and cite sources.
Original article

In retrieval, we know the lego pieces well. Embeddings. Rerankers. Query understanding. BM25. All the parts we put together in the standard stack.

But a new power is rising: agentic search models. LLMs trained specifically on controlling the search task. Compared to GPT-5 and pals, these models might focus on tasks / domains. They can be smaller, faster, and easier to deploy in our search infrastructure.

Let’s set the stage.

With the traditional lego pieces, we build something like this thick search monolith:

That’s what we’ve built for 1-2 decades. Queries flow into the system. We apply business rules and classify queries. We search one or more retrieval backends. We post-process and rerank what comes back.

It’s all very manual, programmatic, and bespoke. Each piece doesn’t see the whole. Every piece focuses on its part of the problem (rerankers, query classifiers, etc) ignorant of the whole.

Agentic search orchestrates the pieces

Agentic search unbundles the pieces to see the whole. An agent built on GPT-5 or Sonnet knows it has tools. It’s given knowledge / context to solve the user’s query.

The agent helps the user: driving a few simple retrieval primitives, exploring candidates to return what’s relevant. Instead of thick, monolithic search, the underlying search tools become thin wrappers on our backend indices.

Unlike the traditional stack, the agent sees the whole process. It’s not a series of reductive steps. Instead the model orchestrates a solution for the user using what’s available - retrieval tools, other subagents, or a knowledge base to guide the user.

In other words, agentic search unbundles the retrieval stack. The parts still matter. But the whole can be managed by a single, intelligent model.

GPT-5 doesn’t really know search

Frontier lab models (GPT-5, Sonnet and pals) do well in the 80% case. They understand the queries with general knowledge. They surface defensible search results.

But it’s the last 20% that moves the needle.

What’s in the last 20%? It’s the non-obvious stuff in our users / domain. GPT-5 doesn’t know that in our furniture store a search for “bistro tables” actually means “small outdoor tables” not restaurant equipment. GPT-5 doesn’t know our fashion search users click on dark or plain patterns over complex ones.

Why not? Isn’t GPT-5 itself also trained on search?

Yes, of course.

GPT-5 and pals think of ‘search’ as web search. Mainline models expect search tools that work near flawlessly. To give them answers they don’t have. Unfortunately, our search isn’t Google. We work on smaller, focused teams: building simpler search backends for our narrow domains. We need agents trained to reason and orchestrate simpler retrieval systems.

Currently, as I teach in Cheat at Search with Agents, we build constraints + checks around the model. But this intense level of context engineering eats up tokens and costs $$$.

Agentic search models for the last 20%

What if we could train an LLM to focus on search specifically? Even better: search in our domain, for our users?

We’re beginning to see models in this direction: SID was first with their SID-1 model. We’ve since seen Glean release Waldo. Startups like Charcoal tailor to your corpus.

Teams advertise these replacements by training on document search specifically - they focus on that last 20% needed to find useful results. Sid, for example, points to smaller size and lower latency compared to GPT-5 for agentic search.

Think about the implications. I’ve described a future where we don’t build complex query + reranking pipeline. Instead agents orchestrate simpler retrieval primitives. A basic keyword search or embedding model with a few filters. Basic, scalable, and simple retrieval tools.

Then we deploy an agentic search model like SID-1. It knows what’s relevant in our domain. It gets our users. Now it can orchestrate the tools to find relevant search results. Currently, sure, the focus begins with RAG and classic passage retrieval (ie chunks). That will surely expand to a family of models, targeting different domains from e-commerce to job search to ??.

Today we have scores of embedding models on huggingface, targeting financial, legal, e-commerce. Why wouldn’t we also have many agentic search models? Tuned to domains?

An embedding only solves a narrow portion of the problem (similarity). An agentic search model could encompass the entire process from query understanding to hybrid search. Yes they’re too slow today to drive site search. But that will surely change in the years ahead.

It’s a future I’m excited about. Search will look radically different.

If you want to learn about these topics, check out my Cheat at Search w/ Agents class starting Monday.

Explore all 40 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi