Repository navigation
Reduce total number of tools, introduce "Meta Mode" for the MCP server #894
Closed
Daniel Gonzalez (dggsax)
started this conversation in
Ideas
Replies: 1 comment
|
Is there are a reason you can't just use domains to only bring in the areas you are using? |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I started writing this idea manually and then figured "let me use agent mode to make this go a little faster" in VS Code :) This proposal was drafted (and reviewed) with the assistance of Claude Opus 4.6. 😄 I'm not sure if this idea is even possible, but I do think that something like a
--meta-modeshould be considered to reduce the bloat of the MCP server and give Agentic tools a better change of success, especially in non-interactive environments where the MCP is configured with all tools.Gonzo
Proposal: Dynamic Tool Discovery for the Azure DevOps MCP Server
🔥 Problem
The ADO MCP server currently exposes all tools at once via
tools/list. This creates several issues for LLM-based agents:💡 Proposed Solution: Meta-Mode
Behind the
--meta-modeflag, the flat tool list would be replaced with a progressive discovery surface of 4 meta-tools:list_domainslist_domain_toolsget_tool_schemaexecute_toolThese meta-tools would be a thin layer over the existing tool infrastructure.
execute_toolwould delegate directly to the underlying tool implementation — no new permission model, no new client instantiation, no duplicated logic.🛠️ Proposed Tool Definitions
1.
list_domainsWould return the set of available capability domains, scoped to the server's configured
enabledDomains.{ "name": "list_domains", "description": "Lists available Azure DevOps capability domains. Call this to understand what areas of functionality are available before discovering specific tools.", "inputSchema": { "type": "object", "properties": {}, "required": [] } }Example response:
{ "domains": [ { "name": "repositories", "description": "Git repositories, pull requests, branches, commits, and code reviews" }, { "name": "pipelines", "description": "Build and release pipelines, runs, approvals, and pipeline permissions" }, { "name": "work-items", "description": "Work items, queries, tags, and work item tracking" }, { "name": "work", "description": "Boards, sprints, iterations, team capacity, and backlogs" }, { "name": "search", "description": "Code search, work item search, and wiki search across the organization" }, { "name": "wiki", "description": "Wiki pages, creation, and updates" }, { "name": "core", "description": "Projects, teams, and organization-level operations" }, { "name": "advanced-security", "description": "Security alerts, code scanning results, and dependency vulnerabilities" }, { "name": "test-plans", "description": "Test plans, test suites, test cases, and test results" } ] }2.
list_domain_toolsWould return summaries only (name + description + action tag) for tools within a domain. No full schemas — keeping the response compact so the agent can scan and decide which tool to use.
{ "name": "list_domain_tools", "description": "Lists available tools within a specific ADO domain as summaries (name, description, and action type). Optionally filter by action type. Available domains: repositories (git, PRs, branches), pipelines (builds, releases, approvals), work-items (tracking, queries, tags), work (boards, sprints, backlogs), search (code, work items, wiki), wiki (pages, updates), core (projects, teams), advanced-security (alerts, scanning), test-plans (test cases, results). Use list_domains for full descriptions.", "inputSchema": { "type": "object", "properties": { "domain": { "type": "string", "description": "The domain to list tools for.", "enum": [] }, "action": { "type": "string", "description": "Optional filter for the type of action. Omit to return all tools in the domain.", "enum": ["read", "create", "update", "delete"] } }, "required": ["domain"] } }Example request:
{ "domain": "repositories", "action": "create" }Example response:
{ "domain": "repositories", "action": "create", "tools": [ { "name": "create_pull_request", "description": "Creates a new pull request in a Git repository", "action": "create" }, { "name": "create_repository", "description": "Creates a new Git repository in a project", "action": "create" }, { "name": "create_branch", "description": "Creates a new branch from a specified commit or ref", "action": "create" }, { "name": "create_pull_request_comment", "description": "Adds a comment to a pull request thread", "action": "create" } ] }Without the
actionfilter:{ "domain": "repositories" }{ "domain": "repositories", "tools": [ { "name": "list_repositories", "description": "Lists all Git repositories in a project", "action": "read" }, { "name": "get_pull_request", "description": "Gets details of a specific pull request by ID", "action": "read" }, { "name": "list_pull_requests", "description": "Lists pull requests matching filter criteria", "action": "read" }, { "name": "create_pull_request", "description": "Creates a new pull request in a Git repository", "action": "create" }, { "name": "update_pull_request", "description": "Updates properties of an existing pull request", "action": "update" }, { "name": "create_pull_request_comment", "description": "Adds a comment to a pull request thread", "action": "create" } ] }The
actiontag on each tool summary would help the LLM reason about intent even when no filter is applied.3.
get_tool_schemaWould return the full
inputSchemafor a specific tool. This would allow the agent to inspect required and optional parameters before callingexecute_tool, without needing to fail first.{ "name": "get_tool_schema", "description": "Returns the full input schema for a specific tool, including all required and optional parameters with descriptions. Use this to understand how to call a tool before executing it.", "inputSchema": { "type": "object", "properties": { "tool_name": { "type": "string", "description": "The name of the tool to get the schema for (e.g., 'create_pull_request')" } }, "required": ["tool_name"] } }Example request:
{ "tool_name": "create_pull_request" }Example response:
{ "tool_name": "create_pull_request", "description": "Creates a new pull request in a Git repository", "domain": "repositories", "action": "create", "inputSchema": { "type": "object", "properties": { "project": { "type": "string", "description": "The project name" }, "repositoryId": { "type": "string", "description": "The repository name or ID" }, "sourceRefName": { "type": "string", "description": "Source branch (e.g., refs/heads/feature)" }, "targetRefName": { "type": "string", "description": "Target branch (e.g., refs/heads/main)" }, "title": { "type": "string", "description": "Pull request title" }, "description": { "type": "string", "description": "Pull request description" } }, "required": ["project", "repositoryId", "sourceRefName", "targetRefName", "title"] } }This would keep
execute_toolresponses lean (result only on success, schema only on validation failure), while giving agents an explicit way to fetch schemas proactively when they need them.Consideration: Example values in schema responses
Including
examplevalues alongside parameter descriptions could significantly reduce hallucination for format-sensitive fields (e.g.,refs/heads/feature/authforsourceRefName). JSON Schema supports anexampleskeyword, and some MCP implementations may pass it through — but it's worth verifying whether the MCP protocol and common MCP hosts preserveexamplesorexamplefields in tool schemas. If supported,get_tool_schemaresponses could include:This would be especially valuable for ADO-specific conventions like ref name formatting, WIQL query syntax, and reviewer identity formats.
4.
execute_toolWould execute a specific tool by delegating to the existing tool implementation. The meta-mode layer would add no new permission logic — the underlying tool would handle auth, client instantiation, and domain scoping exactly as it does today.
{ "name": "execute_tool", "description": "Executes a specific ADO tool by name with the given arguments. If you already know the tool name and parameters (from a previous interaction or from your instructions), you can call this directly without calling list_domains or list_domain_tools first.", "inputSchema": { "type": "object", "properties": { "tool_name": { "type": "string", "description": "The name of the tool to execute (e.g., 'create_pull_request')" }, "arguments": { "type": "object", "description": "The arguments to pass to the tool" } }, "required": ["tool_name", "arguments"] } }On validation failure (wrong/missing params), the error response includes the full schema so the agent can self-correct:
{ "error": "Missing required parameter: sourceRefName", "tool_name": "create_pull_request", "inputSchema": { "type": "object", "properties": { "project": { "type": "string", "description": "The project name" }, "repositoryId": { "type": "string", "description": "The repository name or ID" }, "sourceRefName": { "type": "string", "description": "Source branch (e.g., refs/heads/feature)" }, "targetRefName": { "type": "string", "description": "Target branch (e.g., refs/heads/main)" }, "title": { "type": "string", "description": "Pull request title" }, "description": { "type": "string", "description": "Pull request description" } }, "required": ["project", "repositoryId", "sourceRefName", "targetRefName", "title"] } }🎓 Skipping Discovery: SKILL.md with Auto-Generated Schemas
Agents that already know which tools they need could call
execute_tooldirectly without the discovery round-trips. To enable this, a SKILL.md file could be auto-generated from the tool source definitions.How it could work
tools/listresponses).Generation config
The operator could specify which tools to surface in the skill file via a configuration:
{ "skillTools": [ "create_pull_request", "list_pull_requests", "get_pull_request", "run_work_item_query", "list_repositories" ] }Auto-generated output example
The build step could produce a SKILL.md like:
Benefits
execute_toolon turn 1.🏗️ Architecture: How Meta-Mode Could Work Under the Hood
The meta-mode layer would:
tools/listand return only the 4 meta-tools.list_domain_toolscall).execute_tooldirectly to the underlyingtools/call— zero changes to tool implementations.🔄 Expected Interaction Flow Comparison
Current (flat tools/list)
Meta-mode: Discovery path (typical)
Because
list_domain_toolsembeds domain names and short descriptions in its tool description, agents can typically skiplist_domainsand go straight to the domain they need:Meta-mode: Discovery path (unfamiliar agent)
An agent that doesn't know which domain to use can call
list_domainsfirst:Meta-mode: Experienced agent with SKILL.md
Estimated context savings
⚖️ Tradeoffs & Concerns
Round-trip cost
The meta-mode discovery path introduces additional round-trips compared to the current flat
tools/listapproach. In the best case (experienced agent with SKILL.md), it's a singleexecute_toolcall. In the typical discovery case, it's 4 turns. In the worst case (unfamiliar agent, cross-domain task), it could be more.This is an intentional tradeoff: fewer tokens upfront in exchange for more turns. For most LLM hosting environments, turns are cheap relative to the context window cost of 20K+ tokens of tool schemas that the agent has to carry for the entire conversation.
Cross-domain tasks
If a user asks something like "Create a PR and link it to work item #1234", the agent needs tools from both the
repositoriesandwork-itemsdomains. This would require callinglist_domain_toolstwice (once per domain), and possiblyget_tool_schematwice — adding up to several discovery round-trips for a common composite task.Mitigations:
list_domain_toolsdescription already embeds all domain names, so the agent can identify both relevant domains from the tool description and issue bothlist_domain_toolscalls in parallel (if the host supports parallel tool calls).search_toolsmeta-tool that searches across all domains by keyword (e.g.,{"query": "pull request work item link"}) could address this more directly, but adds complexity to the initial proposal.❓ Open Questions
Action taxonomy: The
actionenum should align with the action types already used by the Azure DevOps REST API rather than inventing a new taxonomy. This needs investigation — what action categorization does ADO use across its resource areas, and can it be derived from the API metadata?Server-level defaults in tool summaries: When asked to evaluate the proposal, Claude Opus 4.6 suggested:
This may not be feasible depending on how the underlying MCP server exposes (or doesn't expose) configured defaults, but it's worth investigating.
✅ Decisions
--meta-modeis opt-in. The default behavior remains the current flattools/list. The team can decide later whether to make meta-mode the default or remove the flat mode entirely.execute_toolreturns only the result on success, and includes the fullinputSchemaon validation failure. Agents that want the schema proactively useget_tool_schema.list_domainsresponse and theenumonlist_domain_toolsalready advertise the exact set of valid domain names.All reactions