Improve agent selection for search tools

Problem

Our agent system has accumulated many tools, creating two interconnected problems:

1. Tool selection confusion: Agents struggle to pick the right tool when multiple options could potentially serve the same purpose. This is particularly bad for code search, where agents can choose between semantic search, grep search, and other alternatives.
2. Discoverability friction: Users and agents need to explicitly request semantic search by name, which creates friction and likely leads to underutilization by external customers. The tool's capabilities and appropriate use cases aren't surfaced naturally.

The combination of these issues means semantic search is both hard to discover AND hard for agents to select appropriately even when they're aware of it.

Proposed Solution

Update the system prompt to explicitly highlight semantic search availability and provide clear guidance on when to use it versus alternatives. This would help both users and agents understand:

• That semantic search exists and is available
• When semantic search is the right choice vs. grep or other search tools
• Concrete examples of queries that benefit from semantic search

Potential Broader Impact

This pattern could benefit other tools with similar discoverability and selection challenges, such as the knowledge graph tool.

Goals

• Get engineering input on feasibility and implementation approach
• Understand potential side effects on other tools and overall system performance
• Determine if this pattern should be applied more broadly to clarify tool selection
• Create a proof of concept if the approach seems viable

Related

We are currently working on Context Building 2.0: Specialist Agents for Bet... (&19353) - which is another approach to solving this problem. However, that is more of a long-term approach that will likely not be ready for a few milestones.