AI & Development

Fifty Tools, One Context: Loading Tools On Demand

Fifty tools are resent in full on every call, and the model picks worse for it. The catalog pattern in plain TypeScript, then the Claude API tool search.

Published on September 14, 202613 min read

"Move my Thursday sync and let Marc know."

Eight words. Before the model reads them, the application has sent it something else: the full description of every one of its tools. The name, what it does, the exact shape of its parameters. For an assistant that manages a calendar, a mailbox, files, a CRM and notes, that is around fifty tools. The Agent SDK documentation1 counts 10,000 to 20,000 tokens for fifty tools, about twenty pages of text, shipped before every reply.

Of those fifty, the request uses two.

Anthropic gives an order of magnitude in its post2 on advanced tool use: five MCP servers, fifty-eight tools, roughly 55,000 tokens consumed before the conversation starts. The setup is ordinary, GitHub, Slack, Sentry, Grafana and Splunk wired together. Add Jira, which alone weighs 17,000 tokens, and you are closing in on 100,000.

This article lays out the problem, then the pattern that fixes it: a catalog instead of a list, and a tool's full definition loaded at the moment the model needs it. First in plain TypeScript, to see the mechanism. Then as Anthropic built it into its API. This is the kind of tradeoff worth settling early on an agents and workflows powered by language models engagement, before the tool catalog grows out of control.

Three words before we start

  • a tool definition is the instruction sheet for a button the model can press: a name, one sentence about what it does, and a JSON schema describing its parameters. A calendar_move tool with its id, start, end and notify fields fits in a few lines. A CRM search tool with twelve filters takes far more;
  • the context window is everything the model reads before answering: the instructions, the conversation, and the tool definitions. It is finite, and it is billed per token;
  • a call is one turn of the loop. The model keeps nothing between two calls: everything it needs to know is resent in full every time, definitions included.

That last point is the one people forget. We weigh our tools once, at startup, when the weight has to be multiplied by the number of calls a conversation makes.

What fifty tools cost

The first cost is the one on the invoice. In Anthropic's example2, the definitions of fifty-odd MCP tools weigh about 72,000 tokens, and the context reaches 77,000 tokens before any work begins. A ten-turn conversation resends that block ten times.

The second cost does not show on the invoice, and it is worse: the model chooses less well.

The Claude API documentation3 states that the model's ability to pick the right tool degrades once you exceed thirty to fifty available tools. It cites no study, but published research points the same way. A paper4 from May 2026 measures, on the BFCL benchmark and its 370 tools, that an adaptive selection showing seven tools on average reaches the same coverage as showing fifty, 90.3% versus 90.8%. Another, RAG-MCP5, raises selection accuracy from 13.6% to 43.1% by filtering tools before the call.

The more tools there are, the more of them look alike. Anthropic cites notification-send-user and notification-send-channel2. Our assistant has mail_send, notes_share and crm_contact, all three of which read as "let Marc know" from a distance. And while the model hesitates between them, it is reading twenty pages of descriptions, nineteen of which are useless for the request at hand.

Count before you defer

If the fifty tools are really five families of ten variants of the same gesture, notes_add_heading, notes_add_list, notes_add_table and so on, the problem lies elsewhere. I covered it in the article on multi-agent architecture: a family collapses into a single tool with a parameter carrying the type. Anthropic's documentation on defining tools6 says the same, fewer and more capable tools reduce selection ambiguity.

Assume the consolidation is done. Fifty genuinely distinct tools remain, because the assistant covers five domains with nothing in common. The calendar shares no parameter with the CRM. That is the case on-demand loading addresses.

The pattern: a catalog instead of a list

The model starts with a list of what exists, and a way to fetch the full instruction sheet when it needs one. Three parts:

  1. A compact index. For each tool, the name and one line. Fifty lines, a few hundred tokens. That is all that enters the context at the start.
  2. A search tool. The model gives it a query, it returns the names of matching tools. It is a tool like any other, except that it does nothing in the world.
  3. Deferred loading. When a tool is found, its full definition joins the conversation and stays there until the end. On the next turn, the model has it at hand without searching again.

Before and after, for our assistant:

BEFORE                                   AFTER
  50 full definitions                      1 search tool
  on every call                            3 tools always loaded
                                             (the most frequent)
                                           1 index: 50 names, one line each
  the model picks among 50                 + the tools found, loaded once
                                           the model picks among 5 or 6

Anthropic puts numbers on the same before and after2 for its fifty-tool MCP case: about 8,700 tokens of context instead of 77,000, an 85% reduction. Anthropic gives that order of magnitude as an example and does not present it as a benchmark. It holds whenever most tools are irrelevant to a given request.

In TypeScript, without a framework, the mechanism fits in about forty lines:

type ToolDef = { name: string; description: string; input_schema: object }

const catalog: ToolDef[] = [
  /* the 50 tools, full definitions */
]
const alwaysOn = new Set(['calendar_list', 'mail_read', 'notes_search'])

// 1. The index: one line per tool, injected into the system prompt
const index = catalog
  .map((t) => `- ${t.name}: ${t.description.split('.')[0]}`)
  .join('\n')

// 2. The search tool: an ordinary tool, with no side effect
const searchTool: ToolDef = {
  name: 'search_tool',
  description: 'Searches for a tool by keywords. Returns up to five names.',
  input_schema: {
    type: 'object',
    properties: { query: { type: 'string' } },
    required: ['query'],
  },
}

function search(query: string): string[] {
  const words = query.toLowerCase().split(/\s+/)
  return catalog
    .map((t) => {
      const text = `${t.name} ${t.description}`.toLowerCase()
      return { t, score: words.filter((w) => text.includes(w)).length }
    })
    .filter((x) => x.score > 0)
    .sort((a, b) => b.score - a.score)
    .slice(0, 5)
    .map((x) => x.t.name)
}

// 3. Deferred loading: the visible set grows over the conversation
const loaded = new Set(alwaysOn)

function visibleTools(): ToolDef[] {
  return [searchTool, ...catalog.filter((t) => loaded.has(t.name))]
}

// In the loop, when the model calls search_tool:
//   const found = search(args.query)
//   found.forEach((n) => loaded.add(n))
//   then return found as the tool result.
//   On the next turn, visibleTools() includes the tools found.

The search here counts words. A real system uses BM25, embeddings, or a small model that reads the index and names the useful tools. The MCP client best practices documentation7 lists those four strategies and recommends switching to this mode once definitions exceed 1 to 5% of the context window.

Two consequences to keep in mind. A tool found stays loaded: the visible list only grows during the conversation, which avoids a new search for every message. And the search costs a round trip: the model calls search_tool, receives names, then calls the tool on the following turn. On a conversation that uses ten tools, that is one extra call. On an assistant with only eight tools in total, it is one call too many, and loading everything is the better choice.

The native implementation in the Claude API

Anthropic built this pattern into its API under the name tool search tool. Announced in beta on November 24, 2025, it has been available without a beta header since February 17, 2026, according to the release notes8, on every model from Claude 4.5 onward per the documentation3.

The principle is the one from the previous section, with two differences. The search runs server side, and deferred loading is a single flag on each tool.

import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic()

const response = await client.messages.create({
  model: 'claude-sonnet-5-5',
  max_tokens: 1024,
  tools: [
    // the search tool, provided by the API
    { type: 'tool_search_tool_bm25_20251119', name: 'tool_search_tool_bm25' },

    // the three most frequent tools, loaded upfront
    { name: 'calendar_list', description: '...', input_schema: { /* ... */ } },
    { name: 'mail_read', description: '...', input_schema: { /* ... */ } },
    { name: 'notes_search', description: '...', input_schema: { /* ... */ } },

    // the other forty-seven, deferred
    {
      name: 'calendar_move',
      description: 'Moves an event and notifies attendees if asked.',
      input_schema: { /* ... */ },
      defer_loading: true,
    },
    // ...
  ],
  messages: [
    { role: 'user', content: 'Move my Thursday sync and let Marc know.' },
  ],
})

What to know before using it, as the documentation3 describes it:

  • You still send everything. The defer_loading flag does not change what the request contains, it changes what enters the model's context. All fifty definitions go out on every call; only the ones found by search are shown to the model.
  • Two search variants. tool_search_tool_regex: the model writes a regular expression pattern, 200 characters at most. tool_search_tool_bm25: it writes a natural language query, 500 characters at most. Both search names, descriptions, and the names and descriptions of parameters. Neither replaces the other.
  • At least one tool stays loaded. Deferring everything returns a 400 error. Anthropic recommends keeping the three to five most used tools loaded.
  • Five results by default, adjustable from 1 to 10,000. The ceiling is 10,000 deferred tools per request.
  • The response carries references. The model calls the search, the API returns a tool_search_tool_result block holding tool_reference blocks, and expands them itself into full definitions before the next turn. You have nothing to send back for that call.
  • No separate billing. The search is not metered as a server tool. The definitions it loads are billed as input tokens, like any definition.

The most useful point for the invoice is elsewhere: prompt caching is preserved9. Deferred tools are not part of the cached prefix. When the model discovers one, its definition is appended to the conversation history, and the prefix does not move. That is what the homemade implementation from the previous section cannot do: every time your tools array changes, the prefix changes, and the cache is lost.

On accuracy, Anthropic publishes two internal measurements2 on its large-catalog MCP evaluations: Opus 4 goes from 49% to 74% with tool search, Opus 4.5 from 79.5% to 88.1%.

For tools coming from an MCP server connected through the API's connector10, the same flag is set on the whole server, or tool by tool.

The same pattern, elsewhere

Once you see it, you find it everywhere, under different names.

Claude Code11 enables it by default for MCP tools: only names enter at session start. The ENABLE_TOOL_SEARCH=auto variable loads everything while definitions weigh less than 10% of the window, and switches to deferred beyond that.

Mastra ships a ToolSearchProcessor12 since version 1.2.0, released in February13, with two meta-tools, search_tools and load_tool, and BM25 ranking14. The changelog13 claims a context reduction of about 94% on large catalogs, with no published measurement. The framework is covered in the Mastra 1.0 article.

The Vercel AI SDK15 has no search engine of its own, but activeTools and prepareStep restrict the visible tools at each step of the loop, and its Anthropic provider exposes the native tool search through a per-tool deferLoading option16.

LangChain17 offers two middlewares: one has a secondary model pick the relevant tools before the main call, the other delegates the search to the provider.

On the protocol side, the MCP specification18 still has no search primitive: tools/list paginates, a server can signal that its list changed, and since July 2026 it provides cache hints. A proposal for a filter on tools/list was closed without follow-up19 in June 2026. The roadmap20 of August 22, 2026 opens a "progressive discovery" effort so a server can offer a small entry point and reveal its catalog as the conversation narrows. Until then, it is up to the host, or the model provider. The role of MCP next to A2A is detailed in the article on the two protocols.

There is finally a more radical variant, described by Anthropic in "Code execution with MCP"21: tools are presented to the model as files in a directory tree, one folder per server, one TypeScript file per tool, and the model reads the ones it needs. On their example, a Google Drive to Salesforce flow drops from 150,000 to 2,000 tokens. It is the same principle, with the filesystem as the index.

What it does not fix

A badly named tool stays unfindable. The search reads names and descriptions, and nothing else. A process tool with a one-line description will never surface for "let Marc know". Anthropic recommends22 prefixing names by domain, calendar_, mail_, crm_, and notes that the choice between prefix and suffix has a measurable effect on their evaluations. For a SaaS vendor, this has become an acquisition question, covered in the article on the SaaS funnel.

The model has to know a tool might exist in order to search for it. If nothing in the prompt says the assistant can touch the CRM, it will answer that it cannot notify Marc, without ever running a search. The one-line-per-tool index, or at minimum one sentence per family, is there for that.

The pattern adds a round trip, and it consolidates nothing. A catalog of fifty variants of the same gesture remains a bad catalog, with or without search. The first step is still to count.

What to take away

The model has to know the tools exist and have a way to fetch their instruction sheet: an index, a search, a load that happens only once. Tool calling and the other base patterns are in the guide to designing an agentic module.

Sources

  1. Anthropic, Claude Agent SDK, Tool search, accessed 2026-10-29.
  2. Anthropic, Introducing advanced tool use, 2025-11-24.
  3. Anthropic, Claude API, Tool search tool, accessed 2026-10-29.
  4. Repantis et al., How Many Tools Should an LLM Agent See? A Chance-Corrected Answer, arXiv, 2026-05-23.
  5. Gan and Sun, RAG-MCP, arXiv, 2025-05-06.
  6. Anthropic, Claude API, Define tools, accessed 2026-10-29.
  7. Model Context Protocol, Client best practices, accessed 2026-10-29.
  8. Anthropic, Claude Developer Platform release notes, accessed 2026-10-29.
  9. Anthropic, Claude API, Tool use with prompt caching, accessed 2026-10-29.
  10. Anthropic, Claude API, MCP connector, accessed 2026-10-29.
  11. Anthropic, Claude Code, Connect Claude Code to tools via MCP, accessed 2026-10-29.
  12. Mastra, ToolSearchProcessor reference, accessed 2026-10-29.
  13. Mastra, Changelog 2026-02-04, 2026-02-04.
  14. Mastra, pull request 12290, merged 2026-01-31.
  15. Vercel, AI SDK Core, Tools and tool calling, accessed 2026-10-29.
  16. Vercel, AI SDK, Anthropic provider, accessed 2026-10-29.
  17. LangChain, Built-in middleware, accessed 2026-10-29.
  18. Model Context Protocol, Specification changelog 2026-07-28, 2026-07-28.
  19. Model Context Protocol, issue 1821, Dynamic Tool Discovery, closed 2026-06-24.
  20. Model Context Protocol, The New MCP Roadmap, 2026-08-22.
  21. Anthropic, Code execution with MCP, 2025-11-04.
  22. Anthropic, Writing tools for agents, 2025-09-11.