my-pi/pi-tool-search/README.md

pi-tool-search

Bounded, model-assisted tool-group loading for Pi. This source is maintained directly in the my-pi bundle from upstream v0.3.6; see UPSTREAM.md.

Why

A name plus a truncated first sentence can misrepresent a tool, while an add-only active set eventually restores every full schema in a long session. This maintained version instead builds validated retrieval metadata from complete tool definitions, presents compact group cards, activates whole workflow groups, and caps the dynamic active set with LRU eviction.

Generated metadata never replaces executable schemas. Exact names, parameter types, required fields, and source definitions remain authoritative in Pi. The repository provides curated metadata for its own tools; a model may generate only summaries, usage guidance, keywords, and assignments for additional unrecognized tools.

Catalog lifecycle

1. session_start keeps tool_search, Pi core tools, and configured alwaysEnabled names active. PowerShell is filtered out on non-Windows hosts; on Windows it is pinned beside Bash as the same local-shell capability class.
2. Hidden tool definitions are hashed with the grouping constraints.
3. Every tool exposed by the standard my-pi bundle is matched against the checked-in catalog in extensions/bundle-groups.ts. If all tools are recognized, no model call and no user cache are needed.
4. User groupOverrides take priority over checked-in assignments. Optional or unavailable bundle tools are simply filtered out of their predefined groups.
5. Additional user or third-party tools receive deterministic prefix/source groups immediately. A matching private model-enriched cache at <agent-dir>/tool-search/catalog-v1.json is reused when present.
6. Only when unrecognized tools exist and the call does not name an exact existing group may tool_search ask the current authenticated model to enrich the complete catalog. Exact group activation always uses the current deterministic catalog immediately. Validation requires all checked-in bundle tools to remain in their predefined groups before a 0600 cache is accepted.
7. Missing authentication, invalid JSON, timeout, cancellation, changed bundle assignments, or cache errors leave the checked-in plus deterministic hybrid catalog usable.

A standard bundle installation therefore sends no tool definitions to a model during catalog setup. Complete definitions are sent to the selected provider only when extra unrecognized tools require enrichment during an unknown-group or query lookup; exact existing group loads never wait for that nested call. Any nested-call usage is attached to the tool_search result.

Run /tool-search-status to inspect the current catalog source, pinned count, dynamic capacity, LRU order, and latest eviction reason. Run /tool-search-rebuild to remove model enrichment; standard bundle tools immediately return to the checked-in catalog and extra tools can be enriched lazily on the next search.

Precomputed my-pi groups

Group	Bundle tools
filesystem-navigation	ls, multi_grep
ssh-connection	ssh_connect
ssh-remote-files	ssh_read, ssh_write, ssh_edit
ssh-remote-search	ssh_find, ssh_grep
ssh-remote-shell	ssh_cd, ssh_bash
code-intelligence	CodeGraph and all LSP tools
web-tavily	Tavily search/fetch
web-exa	Exa search/advanced/fetch
web-keenable	Keenable search/fetch
context-execution	ctx_execute, ctx_execute_file, ctx_batch_execute
context-knowledge	Context indexing, searching, and web-doc ingestion
context-observability	Context stats, doctor, and Insight
context-administration	Context upgrade and destructive purge
memory-recall	Official Hippo recall/context/status and condensed-output recovery
memory-management	Official Hippo remember/outcome tools
subagents	Launch agents, collect background results, and steer running agents
user-interaction	ask_user_question model-initiated TUI questions
chrome-navigation	Chrome bridge setup, tabs, navigation, snapshots/find/inspect, waits, and screenshots
chrome-interaction	Chrome click/type/fill/key/hover/drag/tap/scroll input
chrome-debugging	Chrome page evaluation, console/network diagnostics, response retrieval, and file upload
mcp-management	Shared MCP adapter management

Pinned tools are removed from these dynamic groups at runtime. The defaults pin codegraph_explore, lsp_diagnostics, and the three high-frequency Context Mode execution tools, so none of them consume dynamic group or tool capacity.

Loading and eviction

Call tool_search with an exact catalog group id when possible:

{ "group": "web-tavily" }

A workflow can atomically activate up to five exact groups in one call. The complete request must fit the configured group and tool limits; otherwise no active state is changed:

{
  "groups": ["ssh-connection", "ssh-remote-shell", "ssh-remote-files", "ssh-remote-search"]
}

A natural-language query uses weighted group, alias, and exact-tool-name metadata when the group is unclear. Weak or closely tied matches return ranked candidates without activating an arbitrary group. Loading exposes the original full schemas on Pi's next model request.

Pinned tools do not consume the dynamic budget. Dynamic groups update their recency when loaded or when a member starts executing. Loading beyond maxActiveGroups or maxDynamicTools atomically evicts least-recently-used non-requested groups before activation.

The first loads are purely additive and retain Pi's native deferred-loading propagation. An eviction necessarily removes and adds schemas in one transition, so Pi uses its safe fallback rather than treating that transition as purely additive.

Configuration

Pi reads this configuration from <agent-dir>/settings.json (normally ~/.pi/agent/settings.json). The bundle adds fields only when missing and never replaces explicit user choices:

{
  "toolSearch": {
    "alwaysEnabled": [
      "codegraph_explore",
      "lsp_diagnostics",
      "ctx_execute",
      "ctx_execute_file",
      "ctx_batch_execute"
    ],
    "showToolSearchFooterStatus": false,
    "maxActiveGroups": 5,
    "maxToolsPerGroup": 8,
    "maxDynamicTools": 28,
    "groupOverrides": {},
    "bundleDefaultsVersion": 2
  }
}

Key	Bundle default	Description
alwaysEnabled	CodeGraph explore, LSP diagnostics, Context execution trio	Exact names pinned alongside Pi core tools
showToolSearchFooterStatus	false	Show active tool/group counts
maxActiveGroups	5	Maximum simultaneously active dynamic groups
maxToolsPerGroup	8	Maximum size accepted from model-generated groups
maxDynamicTools	28	Hard cap across active dynamic group tools
groupOverrides	{}	Exact tool lists that override checked-in, generated, or fallback assignments
bundleDefaultsVersion	2 on a newly created default section	Bundle provenance for controlled default migrations; ignored by runtime routing

Example override:

{
  "toolSearch": {
    "groupOverrides": {
      "web-tavily": ["tavily_web_search", "tavily_web_fetch"],
      "lsp": ["lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"]
    }
  }
}

An explicit override larger than maxDynamicTools remains visible but cannot be activated until the limit or override is corrected. Normal extension loading preserves existing values and writes owner-only settings through a unique same-directory temporary file. When the user explicitly runs the bundle's updated update.sh, its post-update migration helper changes only the complete former 3/20 default snapshot: it creates an owner-only timestamped backup, atomically writes the current defaults, and records bundleDefaultsVersion: 2. Customized or invalid settings are preserved, and a content change detected immediately before commit aborts the migration.

Current host integration

This version directly targets @earendil-works/pi-coding-agent, current typebox, ModelRegistry.complete(), and ExtensionAPI.setActiveTools(). It does not load or alias @mariozechner/pi-coding-agent, install another Pi runtime, rewrite provider payloads, proxy tool execution, or enqueue hidden sendMessage loops.

Repository layout

• extensions/index.ts — Pi lifecycle, model call, activation, and group LRU
• extensions/catalog.ts — hashing, strict catalog validation, fallback grouping, cache, and ranking
• extensions/bundle-groups.ts — checked-in authoritative groups for tools exposed by my-pi
• extensions/config.ts — safe bundle-default deployment and config parsing
• test/ — catalog, config, model-generation, and active-set tests
• docs/dynamic-tool-loading.md — current Pi propagation and replacement behavior
• UPSTREAM.md — imported snapshot and local divergence

Development

npm install
npm run typecheck
npm run test
npm run build

The root bundle loads ./pi-tool-search/extensions/index.ts directly and references this package through file:./pi-tool-search.