my-pi/pi-tool-search/README.md

# pi-tool-search

Bounded, model-assisted tool-group loading for Pi. This source is maintained directly in the `my-pi` bundle from upstream `v0.3.6`; see [UPSTREAM.md](./UPSTREAM.md).

## Why

A name plus a truncated first sentence can misrepresent a tool, while an add-only active set eventually restores every full schema in a long session. This maintained version instead builds validated retrieval metadata from complete tool definitions, presents compact group cards, activates whole workflow groups, and caps the dynamic active set with LRU eviction.

Generated metadata never replaces executable schemas. Exact names, parameter types, required fields, and source definitions remain authoritative in Pi. The repository provides curated metadata for its own tools; a model may generate only summaries, usage guidance, keywords, and assignments for additional unrecognized tools.

## Catalog lifecycle

1. `session_start` keeps `tool_search`, Pi core tools, and configured `alwaysEnabled` names active. PowerShell is filtered out on non-Windows hosts; on Windows it is pinned beside Bash as the same local-shell capability class.
2. Hidden tool definitions are hashed with the grouping constraints.
3. Every tool exposed by the standard `my-pi` bundle is matched against the checked-in catalog in `extensions/bundle-groups.ts`. If all tools are recognized, no model call and no user cache are needed.
4. User `groupOverrides` take priority over checked-in assignments. Optional or unavailable bundle tools are simply filtered out of their predefined groups.
5. Additional user or third-party tools receive deterministic prefix/source groups immediately. A matching private model-enriched cache at `<agent-dir>/tool-search/catalog-v1.json` is reused when present.
6. Only when unrecognized tools exist and the call does not name an exact existing group may `tool_search` ask the current authenticated model to enrich the complete catalog. Exact group activation always uses the current deterministic catalog immediately. Validation requires all checked-in bundle tools to remain in their predefined groups before a `0600` cache is accepted.
7. Missing authentication, invalid JSON, timeout, cancellation, changed bundle assignments, or cache errors leave the checked-in plus deterministic hybrid catalog usable.

A standard bundle installation therefore sends no tool definitions to a model during catalog setup. Complete definitions are sent to the selected provider only when extra unrecognized tools require enrichment during an unknown-group or query lookup; exact existing group loads never wait for that nested call. Any nested-call usage is attached to the `tool_search` result.

Run `/tool-search-status` to inspect the current catalog source, pinned count, dynamic capacity, LRU order, and latest eviction reason. Run `/tool-search-rebuild` to remove model enrichment; standard bundle tools immediately return to the checked-in catalog and extra tools can be enriched lazily on the next search.

## Precomputed my-pi groups

| Group | Bundle tools |
|---|---|
| `filesystem-navigation` | `ls`, `multi_grep` |
| `ssh-connection` | `ssh_connect` |
| `ssh-remote-files` | `ssh_read`, `ssh_write`, `ssh_edit` |
| `ssh-remote-search` | `ssh_find`, `ssh_grep` |
| `ssh-remote-shell` | `ssh_cd`, `ssh_bash` |
| `code-intelligence` | CodeGraph and all LSP tools |
| `web-tavily` | Tavily search/fetch |
| `web-exa` | Exa search/advanced/fetch |
| `web-keenable` | Keenable search/fetch |
| `context-execution` | `ctx_execute`, `ctx_execute_file`, `ctx_batch_execute` |
| `context-knowledge` | Context indexing, searching, and web-doc ingestion |
| `context-observability` | Context stats, doctor, and Insight |
| `context-administration` | Context upgrade and destructive purge |
| `memory-recall` | Hermes memory/session search and condensed-output recovery |
| `memory-management` | Hermes memory writes and procedural skill management |
| `subagents` | Launch agents, collect background results, and steer running agents |
| `user-interaction` | `ask_user_question` model-initiated TUI/RPC questions |
| `chrome-navigation` | Chrome bridge setup, tabs, navigation, snapshots/find/inspect, waits, and screenshots |
| `chrome-interaction` | Chrome click/type/fill/key/hover/drag/tap/scroll input |
| `chrome-debugging` | Chrome page evaluation, console/network diagnostics, response retrieval, and file upload |
| `mcp-management` | Shared MCP adapter management |

Pinned tools are removed from these dynamic groups at runtime. The defaults pin `codegraph_explore`, `lsp_diagnostics`, the three high-frequency Context Mode execution tools, and all six Hermes memory tools, so none of them consume dynamic group or tool capacity.

## Loading and eviction

Call `tool_search` with an exact catalog group id when possible:

```json
{ "group": "web-tavily" }
```

A workflow can atomically activate up to five exact groups in one call. The complete request must fit the configured group and tool limits; otherwise no active state is changed:

```json
{
  "groups": ["ssh-connection", "ssh-remote-shell", "ssh-remote-files", "ssh-remote-search"]
}
```

A natural-language `query` uses weighted group, alias, and exact-tool-name metadata when the group is unclear. Weak or closely tied matches return ranked candidates without activating an arbitrary group. Loading exposes the original full schemas on Pi's next model request.

Pinned tools do not consume the dynamic budget. Dynamic groups update their recency when loaded or when a member starts executing. Loading beyond `maxActiveGroups` or `maxDynamicTools` atomically evicts least-recently-used non-requested groups before activation.

The first loads are purely additive and retain Pi's native deferred-loading propagation. An eviction necessarily removes and adds schemas in one transition, so Pi uses its safe fallback rather than treating that transition as purely additive.

## Configuration

Pi reads this configuration from `<agent-dir>/settings.json` (normally `~/.pi/agent/settings.json`). The bundle adds fields only when missing and never replaces explicit user choices:

```json
{
  "toolSearch": {
    "alwaysEnabled": [
      "codegraph_explore",
      "lsp_diagnostics",
      "ctx_execute",
      "ctx_execute_file",
      "ctx_batch_execute",
      "memory_search",
      "session_search",
      "memory_add",
      "memory_replace",
      "memory_remove",
      "skill_manage"
    ],
    "showToolSearchFooterStatus": false,
    "maxActiveGroups": 5,
    "maxToolsPerGroup": 8,
    "maxDynamicTools": 28,
    "groupOverrides": {},
    "bundleDefaultsVersion": 3
  }
}
```

| Key | Bundle default | Description |
|---|---|---|
| `alwaysEnabled` | CodeGraph explore, LSP diagnostics, Context execution trio, all six Hermes memory tools | Exact names pinned alongside Pi core tools |
| `showToolSearchFooterStatus` | `false` | Show active tool/group counts |
| `maxActiveGroups` | `5` | Maximum simultaneously active dynamic groups |
| `maxToolsPerGroup` | `8` | Maximum size accepted from model-generated groups |
| `maxDynamicTools` | `28` | Hard cap across active dynamic group tools |
| `groupOverrides` | `{}` | Exact tool lists that override checked-in, generated, or fallback assignments |
| `bundleDefaultsVersion` | `2` on a newly created default section | Bundle provenance for controlled default migrations; ignored by runtime routing |

Example override:

```json
{
  "toolSearch": {
    "groupOverrides": {
      "web-tavily": ["tavily_web_search", "tavily_web_fetch"],
      "lsp": ["lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"]
    }
  }
}
```

An explicit override larger than `maxDynamicTools` remains visible but cannot be activated until the limit or override is corrected. Normal extension loading preserves existing values and writes owner-only settings through a unique same-directory temporary file. When the user explicitly runs the bundle's updated `update.sh`, its post-update migration helper changes only the complete former `3`/`20` default snapshot: it creates an owner-only timestamped backup, atomically writes the current defaults, and records `bundleDefaultsVersion: 2`. Customized or invalid settings are preserved, and a content change detected immediately before commit aborts the migration.

## Current host integration

This version directly targets `@earendil-works/pi-coding-agent`, current `typebox`, `ModelRegistry.complete()`, and `ExtensionAPI.setActiveTools()`. It does not load or alias `@mariozechner/pi-coding-agent`, install another Pi runtime, rewrite provider payloads, proxy tool execution, or enqueue hidden `sendMessage` loops.

## Repository layout

- `extensions/index.ts` — Pi lifecycle, model call, activation, and group LRU
- `extensions/catalog.ts` — hashing, strict catalog validation, fallback grouping, cache, and ranking
- `extensions/bundle-groups.ts` — checked-in authoritative groups for tools exposed by `my-pi`
- `extensions/config.ts` — safe bundle-default deployment and config parsing
- `test/` — catalog, config, model-generation, and active-set tests
- `docs/dynamic-tool-loading.md` — current Pi propagation and replacement behavior
- `UPSTREAM.md` — imported snapshot and local divergence

## Development

```bash
npm install
npm run typecheck
npm run test
npm run build
```

The root bundle loads `./pi-tool-search/extensions/index.ts` directly and references this package through `file:./pi-tool-search`.