Not Every Question Needs a Tool
You ask your AI assistant "what's a foreign key in SQL?" while you have a database MCP server connected. Should the assistant query your database to answer this? Obviously not. It's a knowledge question that the model can answer directly. But the decision boundary isn't always this clear.
"How many tables are in our database?" requires a tool. "What's the difference between INNER JOIN and LEFT JOIN?" doesn't. "Show me the schema of the users table" requires a tool. "What data types does PostgreSQL support?" probably doesn't. The assistant needs to make this distinction dozens of times per conversation.
When Tools Add Value
Tools add value when the answer depends on your specific data, your specific environment, or real-time information. Anything the model can't know from training, it needs a tool to find out. Your database schema, your file contents, your API responses, current web search results: these all require tool access.
Tools don't add value (and actually slow things down) when the question is about general knowledge, conceptual explanation, or opinion. Making a tool call for these wastes tokens, adds latency, and sometimes produces worse answers than the model's direct response.
Optimizing the Balance
If your assistant overuses tools (calling the database to answer general SQL questions), you can adjust through prompting. "Only use tools when you need information about our specific data or systems. For general knowledge questions, answer directly." This simple instruction significantly reduces unnecessary tool calls.
If your assistant underuses tools (trying to answer data-specific questions from memory instead of checking), the opposite instruction helps: "When the user asks about our data, always verify by querying rather than guessing from context."
Related Reading
- How AI Assistants Choose Which Tool to Use
- The Cost Economics of Running AI Agents
- Why Your AI Assistant Ignores Perfectly Good Tools
Browse MCP servers on Skillful.sh. Search 137,000+ AI tools.