Designing a Decision Layer for AI Agents
An agent-first product that helps AI agents, developers, and AI teams choose the best tools and models for specific tasks using recommendations, rankings, telemetry, cost, reliability, and outcome feedback.
- Role
- Founder, Product Designer, Systems Architect
- Stage
- Live product
ToolRoute dashboard — tool and model ranking, telemetry signals, task-fit scoring, and routing recommendations with rationale.
The Problem
Agents fail because they choose the wrong tools
AI agents do not fail only because the model is weak. They often fail because they choose the wrong tool, use the wrong model for the task, lack a clear escalation path, or have no feedback loop to learn from outcomes. As the number of models, MCP servers, APIs, and agent tools grows, the decision problem becomes harder. ToolRoute was created to solve that problem.
Design Challenge
Designing for both humans and autonomous software
Most product interfaces are designed for human users. ToolRoute required a different frame: what does it mean to design for both humans and agents? Humans need clear navigation, trustworthy comparisons, and understandable recommendations. Agents need structured signals, routing logic, fallback paths, and telemetry they can use programmatically. That dual-user model shaped the entire product direction.
Key Insights
Principles that shaped the product
Recommendations need reasons
A recommendation without explanation is not trustworthy. ToolRoute explains why a tool or model is recommended and what trade-offs come with it.
Ranking is multidimensional
The best model depends on the task, budget, latency tolerance, reliability needs, and acceptable risk. No single score captures this.
Telemetry creates compounding value
Every routing decision generates feedback that makes future recommendations better. Telemetry turns a directory into a living decision system.
Trust must be visible and specific
Trust signals need to be explainable and tied to the specific task. Verified integration, active maintenance, high task success, low failure rate.
Recommendation card mockup showing task-fit score, latency/cost/reliability badges, and a short rationale linking back to telemetry.
Impact
A live product turning infrastructure into product clarity
ToolRoute shows how I approach emerging product spaces: identify the underlying system problem, turn technical ambiguity into a clear product model, define the right user layers, and design an experience that can evolve from MVP into infrastructure.
- Live product at toolroute.io with early telemetry
- Agent-first adoption strategy with human developer layer
- Searchable directory, leaderboards, and recommendation-first routing
- Routing API direction for direct agent integration
Next case study
Johnson & Johnson