Daniel Habibi
Sharing my latest approach: SIMPLE Agents with COMPLEX tools
First, let me state the obvious - not all models are good at the same thing. Sonnet sucks at math, big time. But Sonnet is amazing at creating a multi-step plan. The more I play with different models - the more models I like for specific things.
Further, to get the OPTIMAL result from a specific model - you MUST prompt differently. So … when we do A/B tests with same prompt … it’s NOT actually an ideal model comparison. A better test harness is prompt A,B,C preformed on model A,B,C.
This leads me to my latest approach.
If you are building a NON-CHAT agent - consider the following (full disclosure - ALL agents I have been building are non-chat, I am about to embark on my first chat agent soon and I’m excited to learn explore with that - but for the purpose of this post - consider my opinions for a non-chat agent as I have yet to go doe the chat agent rabbit hole):
Put all your complexity into the TOOLS. Make the Agent just a simple “tool chooser”.
If you take this approach - you can use a simple, fast & cheap model for the Agent itself - Gemini Flash, even GPT 4.1. And your custom tools can use ANY model you want - optimized for the task at hand with as complex and deep tool system prompts as needed - optimized for the model used in the specific too.
In your Agents system prompt - remember that you have full control over the ORDER of tools chosen - not just which tool is chosen, for example - you can make the Agent only run TOOL C after TOOL B has been run, etc.
You can go down a rabbit hole making your Agent's system prompt perfect if you are putting more complexity into the agent vs the TOOLS the agent calls - and I find it will typically compromise something.
Consider making your Agent dead simple - and put all complexity into tools.