Back to Devoured - November 20, 2026
DEVOURED
Introducing Tool Retrieval in the Gemini API

Introducing Tool Retrieval in the Gemini API

AI Google
Google’s new Tool Retrieval API for Gemini optimizes performance by loading function schemas dynamically instead of forcing agents to hold entire libraries in context.
What: The feature, available in preview in the Gemini Interactions API, allows developers to set defer_loading: true for tool functions. The model accesses only necessary schemas during execution, reducing input token counts and latency while supporting Gemini 3.6+ models.
Why it matters: This shift addresses the efficiency wall for agentic workflows where increasingly large tool libraries threaten to exceed context limits and degrade performance.
Takeaway: If you are managing large tool sets in Gemini, add a tool_search tool and flag deferred functions to reduce your input token usage.
Deep dive
  • Implements on-demand schema loading to keep prompt context focused.
  • Supports both server-side retrieval and custom client-side registries via the tool_search tool.
  • Integrates with MCP (Model Context Protocol) servers for remote tool access.
  • Leverages prefix caching by keeping stable schemas out of the variable function-call context.
  • No additional cost beyond standard model pricing for tokens used.
Decoder
  • Tool Retrieval: An optimization technique where an AI model dynamically fetches function definitions as needed rather than having them pre-loaded in the context window.
  • MCP (Model Context Protocol): An open standard for connecting AI models to external data and tool sources.
  • Prefix Caching: A feature where the system saves the computation of common, repeated parts of a prompt to speed up future requests.
Original article

Introducing Tool Retrieval in the Gemini API

Today we're launching Tool Retrieval in the Gemini API, available in preview in the Interactions API.

When an agent connects to dozens or even hundreds of tools, sending every full parameter schema on every request adds up fast. It inflates input token counts, adds latency, and lowers tool-selection accuracy as the candidate list grows. Tool Retrieval lets your agent keep its full tool library on standby and load parameter schemas only when the current task needs them.

How it works

To turn on Tool Retrieval, add a tool_search tool to your request and set defer_loading: true on the tools you want to hold back. The model still sees the name and description of every deferred tool, so it knows what's available, but the parameter schemas stay out of context. When the model needs a tool, it asks for it by name, the API loads the full schema, and the model calls the function, all inside the same request.

from google import genai

client = genai.Client()

tools = [
   {"type": "tool_search"},
   {
       "type": "function",
       "name": "issue_refund",
       "description": "Refund an order to the original payment method.",
       "defer_loading": True,
       "parameters": {
           "type": "object",
           "properties": {
               "order_id": {"type": "string"},
               "amount": {"type": "number"},
           },
           "required": ["order_id", "amount"],
       },
   },
   # ... 50+ other deferred functions (leave frequent tools undeferred)
]

interaction = client.interactions.create(
   model="gemini-4-hydrogen",
   input="Order A-1042 arrived damaged. Please refund 40 dollars.",
   tools=tools,
)

Inside that single API call, the model searches for issue_refund, the API loads its schema, and the model emits the function_call. All three steps appear in interaction.steps, making it easy to inspect which tools the model considered when debugging tool names or descriptions:

"steps": [
 {
   "type": "tool_search_call",
   "id": "tsc_7f2a",
   "arguments": {"function_names": ["issue_refund"]},
   "signature": "..."
 },
 {
   "type": "tool_search_result",
   "call_id": "tsc_7f2a",
   "result": [{"type": "function", "name": "issue_refund", "parameters": {...}}],
   "signature": "..."
 },
 {
   "type": "function_call",
   "id": "fc_91c3",
   "name": "issue_refund",
   "arguments": {"order_id": "A-1042", "amount": 40}
 }
]

From here it's normal function calling: your code runs the refund and sends back a function_result.

There's no index to build or host. Gemini runs the search on the server, so the only changes to your code are the new tool and a flag on the tools you want to defer. You can keep your most-used tools loaded up front and defer the long tail.

Tool Retrieval also plays well with prefix caching. Deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn.

Tool Retrieval also works with remote MCP servers in the Gemini API. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand, so you can connect a large server without listing its tools one by one.

How client-side retrieval works

If you maintain your own tool registry, embedding index, or per-user permission filter, declare the tool_search with "execution": "client" and define your search parameters. Client-side retrieval adds one round trip per search, because your app runs the lookup. Tools do not need to be declared upfront, so you can also expose new tools mid-conversation when a user connects an integration:

tools = [{
    "type": "tool_search",
    "execution": "client",
    "name": "search_tools",
    "description": "Search the internal tool registry for functions.",
    "parameters": {
        "type": "object",
        "properties": {"query": {"type": "string"}},
        "required": ["query"],
    },
}]

# When the model emits a function_call for "search_tools",
# return matching {"type": "function", ...} schemas in function_result:
followup = client.interactions.create(
    model="gemini-4-hydrogen",
    previous_interaction_id=interaction.id,
    tools=tools,
    input=[{
        "type": "function_result",
        "name": "search_tools",
        "call_id": step.id,
        "result": my_registry.search(**step.arguments),
    }],
)

How pricing and supported models work

Tool Retrieval has no separate charge. Input and output tokens are billed at the standard rates for the model you call. Turning on tool_search adds a small, fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation, the same as a tool declared up front. You save tokens when the schemas you never load outweigh the ones you do, which is the usual case for large libraries. Measure it against your own traffic.

Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, in the Interactions API.

How to get started

Explore all 56 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi