Cheap, efficient AI architecture: compose tools, don't build a monolith

In Syntha, the AI chat has access to data from Helios ERP, our own competitor-tracking scrapers, and analytics from GA4 and Microsoft Clarity. The first version we tried gave the model access to all three sources at once — one big system prompt with three database schemas, one model expected to handle everything.

It worked, but expensively and unreliably. Every query — even the simplest one, like "how many units did we sell last week" — paid the token cost for all three schemas, even though it only needed one. And the more tools and context the model got at once, the more often it picked the wrong tool or hallucinated a column that didn't exist.

We rebuilt it as a set of small, narrowly-scoped tools — one for Helios, one for scraper data, one for analytics — each with its own small schema and a clearly defined job. A lightweight router sits above them and only decides which tool a query needs; only then does the model get called with that one source's schema.

Crucially, not every source gets connected at once. The router first breaks the query down and builds its own plan — a roadmap of what needs to be asked of which tool. Based on specific words in the query (like "top 10", "yesterday", "only category XYZ") it decides the filters and limits to send to each tool in advance, so only what's actually needed for the answer ever passes through memory and context. The decision follows a hierarchical tree — first which area the query touches, then within that which specific tool and with what parameters, and only then does the call happen.

The result: a typical query is significantly cheaper, because you only pay for the context you actually need. It's also more reliable — a small tool with one clear job is much easier to test, log and fix than one model with access to everything at once.