🧰Study Finds Plan-then-Retrieve Beats Rival Tool Discovery on 46,984 APIs

Your agent's tool picker might be costing you double

Stop running one tool discovery strategy for everything. A new study shows the right architecture depends on how many tools a query needs, and getting it wrong either wastes money or misses the answer. If you're building agents that call APIs, this is your problem. Plan-then-Retrieve (PtR) has the model decompose the request into functional sub-needs, then embeds each one and pulls matching tools with approximate nearest-neighbor search. Retrieve-then-Plan (RtP) goes the other way. The catch: PtR's extra discovery call roughly doubles inference cost, so on simple single-tool lookups you're paying for nothing. The study ran on ToolBench, which has 46,984 unique API endpoints across 50 categories. The crossover held at every retrieval budget from k=1 to k=20 and across registry sizes from 1k to 47k APIs. PtR also hallucinates stubs more often, a failure mode the authors call double-edged, because the same abstraction that causes misses helps with multi-faceted intent. RtP matches proper nouns directly, which PtR's stubs throw away. Worth watching: the authors argue for complexity-adaptive routing, classifying queries up front and choosing the pipeline per request. That's the practical takeaway. Nobody has shipped it at scale yet.