From 5a85660b5d1df19263b680b2ad82163a5ca5620d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E6=9C=8D=E5=8A=A1=E9=83=A8-=E5=8F=B6=E6=9E=97?= =?UTF-8?q?=E7=AB=8B?= <1361666059@qq.com> Date: Wed, 19 Aug 2026 11:04:03 +0800 Subject: [PATCH] feat: add routed online search providers --- AGENTS.md | 8 +- README.md | 16 ++- extensions/exa-tool-prefix.ts | 57 ++++++++ extensions/exa.ts | 32 +++++ extensions/tavily-override.ts | 21 +++ extensions/tavily-tool-prefix.ts | 25 ++++ extensions/tool-routing.ts | 28 ++++ package-lock.json | 227 +++++++++++++++++++++++++++++++ package.json | 8 +- tests/exa-override.test.ts | 42 ++++++ tests/tavily-override.test.ts | 22 +++ tests/tool-routing.test.ts | 19 +++ 12 files changed, 500 insertions(+), 5 deletions(-) create mode 100644 extensions/exa-tool-prefix.ts create mode 100644 extensions/exa.ts create mode 100644 extensions/tavily-override.ts create mode 100644 extensions/tavily-tool-prefix.ts create mode 100644 tests/exa-override.test.ts create mode 100644 tests/tavily-override.test.ts diff --git a/AGENTS.md b/AGENTS.md index ccdbeb0..12b3eb5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,6 +13,9 @@ - 初始导入快照：`8d196e4ef0884cac8326c366191dad3f585d470a`（`@mzwing/pi-permission-auto-review@0.2.0`）。 - 该目录从上游源码仓库导入并由本仓库直接维护，不使用 npm 预编译产物，不是 submodule，也不保留嵌套 `.git`。 - `extensions/fff-override.ts`：强制使用 FFF 官方 `override` 模式，统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全；显式 CLI flag 仍遵循 FFF 官方优先级。 +- `extensions/tavily-override.ts`：加载 `@tavily/pi-extension`，把上游工具重命名为 `tavily_web_search` / `tavily_web_fetch`，并同步改写 label 与工具提示中的内部名称。 +- `@keenable/pi-search`：加载 `keenable_search` / `keenable_fetch` 及其 skill，默认 keyless，`KEENABLE_API_KEY` 仅用于提高速率限制。 +- `extensions/exa.ts`：通过隔离的 `pi-mcp-adapter` 配置连接 Exa 托管 MCP，把原始工具名映射为统一的来源前缀形式 `exa_web_search`、`exa_web_fetch` 和 `exa_web_search_advanced`。 - `context-mode`：加载上游 Pi adapter 与 skills，提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。 - `extensions/hashline.ts`：部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`，以行哈希锚点覆盖内置 `read` / `edit`；Hashline 自带 `grep` 强制关闭。 - `extensions/codegraph.ts`：通过隔离的 `pi-mcp-adapter` 配置连接机器现有的 `codegraph serve --mcp`，只暴露 `codegraph_explore`。 @@ -22,7 +25,7 @@ - `pi-lsp@0.1.7`：提供声明式 LSP 接入；组合包内置 `typescript-language-server@5.3.0` + `typescript@6.0.3`，并配置机器级 `kotlin-lsp --stdio` 与 `jdtls`。 - `pi-hermes-memory@0.9.6`：提供持久记忆、会话搜索、后台学习和 secret scanning，默认使用 policy-only 模式。 - `@ogulcancelik/pi-codex-compaction@0.1.3`：为 `openai-codex` 提供原生远程 compaction，默认阈值为 90%。 -- 根包还固定安装 `pi-hashline-edit@0.8.3`、`typescript-language-server@5.3.0`、`typescript@6.0.3`、`pi-context-view`、`@firstpick/pi-extension-codex-fast-mode` 和 `@gotgenes/pi-permission-system`。 +- 根包还固定安装 `@tavily/pi-extension@0.1.2`、`@keenable/pi-search@0.1.2`、`pi-hashline-edit@0.8.3`、`typescript-language-server@5.3.0`、`typescript@6.0.3`、`pi-context-view`、`@firstpick/pi-extension-codex-fast-mode` 和 `@gotgenes/pi-permission-system`。 - `install.sh`：先安装根组合包，再交互检查 Kitty/Solarized Dark、Oh My Zsh/Powerlevel10k/Zsh 插件、CodeGraph、Kotlin LSP、Java 21+ 和 JDT LS；缺失项目只在用户明确选择 `Y` 后安装或配置。Powerlevel10k 默认配置来自仓库内置的 `config/p10k.zsh`（当前 Rainbow/ASCII 单行紧凑主题）。 - `update.sh`：先通过 `pi update` 升级根组合包，再只升级当前已安装的终端环境和机器级依赖；未安装项直接跳过。升级前先查询并比较本地与远端版本，只有版本不同时才下载或替换；Powerlevel10k 配置优先从 `pi update` 后的已安装组合包读取，并与 `.zshrc` 受管块一起按内容比较后增量同步。 - `uninstall.sh`：只移除根组合包，不卸载或还原可能被其他项目共享的终端环境、CodeGraph、Kotlin LSP、JDT LS 或 Java。 @@ -36,10 +39,11 @@ - 若以后开启 read 压缩，优先考虑 `readCompaction + smartTruncate`，源码过滤仍独立评估；必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。 - Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文；FFF 仍负责精确字面搜索，RTK 仍处理未走 Context Mode 的普通输出。 - Hashline 默认以 2 字符行哈希覆盖内置 `read` / `edit`，`grep` 和 `replaceText` 均关闭；它保证锚定编辑而不负责压缩，大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。 -- 工具路由规则不得替换 Pi 默认系统提示词：代码结构、调用关系和待修改 symbol 优先 CodeGraph，定义/引用/类型/诊断优先 LSP；工作区内大文件探索、分析和总结优先 `ctx_execute_file`，日志、构建与不可预测命令输出优先 Context Mode；字面搜索先用 FFF `find` 收敛文件位置，再在收敛路径内用 `grep`/`multi_grep` 获取行号；真正修改前才用小范围 Hashline `read` 获取新鲜锚点并用锚定 `edit`。遇到大量结果、截断或上限时继续缩小 path/glob/pattern，不得靠提高 limit 或倾倒全部结果解决。 +- 工具路由规则不得替换 Pi 默认系统提示词：代码结构、调用关系和待修改 symbol 优先 CodeGraph，定义/引用/类型/诊断优先 LSP；工作区内大文件探索、分析和总结优先 `ctx_execute_file`，日志、构建与不可预测命令输出优先 Context Mode；字面搜索先用 FFF `find` 收敛文件位置，再在收敛路径内用 `grep`/`multi_grep` 获取行号；真正修改前才用小范围 Hashline `read` 获取新鲜锚点并用锚定 `edit`。在线搜索按场景只先选一个服务：广泛发现、新闻和候选来源用 Tavily，官方/技术/论文和高确定性来源用 Exa，中文、站点限定和日期筛选用 Keenable；只有必要时才跨服务复核。遇到大量结果、截断或上限时继续缩小范围，不得靠提高 limit 或倾倒全部结果解决。 - `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录，不提交到仓库，也不作为组合包运行时配置源。 - CodeGraph 扩展只配置 Pi 到外部 `codegraph` 命令的 MCP 连接。根包安装本身不安装 CodeGraph；便捷脚本仅在组合包安装完成且用户明确选择 `Y` 后调用官方安装器。仓库不执行 `codegraph init`，不创建或管理 `.codegraph/`，也不改动索引、更新或遥测设置。 - CodeGraph MCP 使用 `keep-alive` 并只直接暴露 `codegraph_explore`；命令缺失、项目未初始化或连接失败时不得阻止其他扩展加载。 +- Exa MCP 使用托管 Streamable HTTP 端点和 `eager` lifecycle；所有在线搜索工具统一采用“来源名 + 原始语义工具名”的命名形式，例如 `tavily_web_search`、`exa_web_search`、`keenable_search`。Exa 高级搜索必须显式约束结果数量和文本长度；Tavily 输出默认控制 `max_results` 且非必要不请求 raw content；Keenable 优先利用中文、站点和日期筛选能力。 - `pi-context-view` 只观察上下文占用，不参与压缩策略。 - Codex fast mode 只为符合条件的 `openai-codex-responses` 请求设置 priority service tier，由 `/fast-mode` 在会话内控制。 - 权限策略默认允许常规工具，允许 FFF 工具；拒绝 Bash 直搜和敏感凭据路径；Git 非只读操作、包管理、外部目录、文件/系统/网络高风险操作与普通 MCP 调用先由 `pi-permission-system` 判为 `ask`。 diff --git a/README.md b/README.md index bd42531..86026a7 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,9 @@ - `@ff-labs/pi-fff@0.10.5`：使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。 - 本仓库维护的 `pi-rtk-optimizer`：只处理 Bash/read 等输出压缩，默认不改写命令，也不处理搜索工具结果。 - `context-mode@1.0.169`：通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。 -- `pi-mcp-adapter@2.26.0`：只用于把机器现有的 CodeGraph MCP Server 接入为 `codegraph_explore`。 +- `@tavily/pi-extension@0.1.2`：提供 Tavily 广泛搜索与页面提取；通过本地 override 暴露为 `tavily_web_search` / `tavily_web_fetch`。 +- `@keenable/pi-search@0.1.2`：提供紧凑的 `keenable_search` / `keenable_fetch`，并加载其搜索 skill；API key 可选。 +- `pi-mcp-adapter@2.26.0`：分别用隔离配置接入机器现有的 CodeGraph 和 Exa 托管 MCP。 - `pi-context-view@0.4.2`：查看上下文占用。 - `@firstpick/pi-extension-codex-fast-mode@0.1.1`：为 Codex provider 提供会话级 `/fast-mode`。 - `pi-lsp@0.1.7`：为 TypeScript/JavaScript、Kotlin 和 Java 提供 LSP 诊断、跳转与符号工具；TypeScript 后端由组合包内置。 @@ -115,7 +117,9 @@ CodeGraph 本体不由组合包安装。需要使用 CodeGraph 的机器应自 ### 工具与搜索路由 -`extensions/tool-routing.ts` 不替换 Pi 默认系统提示词，而是在每轮开始前根据当前激活工具追加简短规则：代码结构、调用关系和待修改 symbol 优先使用 CodeGraph，定义、引用、类型和修改后诊断优先使用 LSP；字面搜索先用 FFF `find` 缩小文件或目录范围，再在已收敛的路径中用 `grep`/`multi_grep` 获取行号。宽泛搜索命中上百或上千结果、发生截断或达到上限时，应继续缩小路径、glob 或 pattern，而不是提高 limit 或输出全部结果。 +`extensions/tool-routing.ts` 不替换 Pi 默认系统提示词，而是在每轮开始前根据当前激活工具追加简短规则。代码结构、调用关系和待修改 symbol 优先使用 CodeGraph，定义、引用、类型和修改后诊断优先使用 LSP；仓库字面搜索先用 FFF `find` 缩小文件或目录范围，再在已收敛的路径中用 `grep`/`multi_grep` 获取行号。宽泛搜索命中大量结果、发生截断或达到上限时，应继续缩小路径、glob 或 pattern，而不是提高 limit 或输出全部结果。 + +在线搜索默认按任务只选一个服务：关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily；官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa；中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核，避免默认把三套结果全部塞进上下文。 日志、测试/构建输出、工作区内大文件的探索/分析/总结以及不可预测的大输出优先交给 Context Mode。只有确实需要精确源码或准备修改时，才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点，随后用锚定 `edit` 修改；成功编辑返回的新锚点可继续复用，只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索，避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。 @@ -149,6 +153,14 @@ Context Mode 直接加载 npm 包内置的 Pi adapter 和 skills，不需要额 FFF 仍负责精确字面搜索；RTK 继续压缩未走 Context Mode 的 Bash、build、test、lint 和 Git 输出。Context Mode 不替换这两个扩展的现有配置。 +### 在线搜索服务 + +Tavily 通过 `extensions/tavily-override.ts` 加载官方扩展。override 只改工具身份，不改官方执行逻辑：上游 `web_search` / `web_fetch` 被注册为 `tavily_web_search` / `tavily_web_fetch`，label 和提示词中的内部工具引用也同步带上 Tavily 标识。使用前需设置 `TAVILY_API_KEY`。Tavily 适合广泛发现，但来源可能混杂且输出较长，应从较小的 `max_results` 开始，非必要不返回 raw content。 + +Keenable 直接加载 npm 包的扩展和 skill，默认无需 key；设置 `KEENABLE_API_KEY` 可提高速率限制。它适合中文内容、`site` 限定和发布日期筛选。搜索结果保持紧凑，只对选中的 URL 调用 `keenable_fetch`。 + +`extensions/exa.ts` 通过已有的 `pi-mcp-adapter` 连接 `https://mcp.exa.ai/mcp`，把 MCP 原始工具映射为统一的来源前缀形式：`exa_web_search`、`exa_web_fetch` 和 `exa_web_search_advanced`。Exa 适合官方文档、技术资料、版本信息和论文；高级搜索必须显式限制结果数量与返回文本长度，避免响应过大。 + ### CodeGraph 只负责连接 `extensions/codegraph.ts` 使用隔离的 `pi-mcp-adapter` 配置启动 diff --git a/extensions/exa-tool-prefix.ts b/extensions/exa-tool-prefix.ts new file mode 100644 index 0000000..c8dae6c --- /dev/null +++ b/extensions/exa-tool-prefix.ts @@ -0,0 +1,57 @@ +const EXA_TOOL_ALIASES: Record = { + web_search_exa: "exa_web_search", + web_fetch_exa: "exa_web_fetch", + web_search_advanced_exa: "exa_web_search_advanced", +}; + +export type ExaToolDefinition = { + name: string; + label: string; + [key: string]: unknown; +}; + +export function mapExaToolName(name: string): string { + return EXA_TOOL_ALIASES[name] ?? name; +} + +export function prefixExaTool(definition: T): T { + const name = mapExaToolName(definition.name); + if (name === definition.name) return definition; + + const displayName = name + .slice("exa_".length) + .split("_") + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); + return { + ...definition, + name, + label: `Exa ${displayName}`, + } as T; +} + +type ToolApi = { + registerTool: (tool: unknown) => unknown; + setActiveTools: (names: string[]) => unknown; + unregisterTool?: (name: string) => boolean; +}; + +export function createExaToolApiOverride(pi: T): T { + const api = pi as T & ToolApi; + return new Proxy(pi, { + get(target, property) { + if (property === "registerTool") { + return (definition: ExaToolDefinition) => api.registerTool(prefixExaTool(definition)); + } + if (property === "setActiveTools") { + return (names: string[]) => api.setActiveTools([...new Set(names.map(mapExaToolName))]); + } + if (property === "unregisterTool") { + return api.unregisterTool ? (name: string) => api.unregisterTool?.(mapExaToolName(name)) : undefined; + } + + const value = Reflect.get(target, property, target); + return typeof value === "function" ? value.bind(target) : value; + }, + }) as T; +} diff --git a/extensions/exa.ts b/extensions/exa.ts new file mode 100644 index 0000000..1ed1e58 --- /dev/null +++ b/extensions/exa.ts @@ -0,0 +1,32 @@ +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import { createMcpAdapter } from "pi-mcp-adapter"; +import { createExaToolApiOverride } from "./exa-tool-prefix.ts"; + +const EXA_TOOLS = ["web_search_exa", "web_fetch_exa", "web_search_advanced_exa"]; +const EXA_MCP_URL = `https://mcp.exa.ai/mcp?tools=${EXA_TOOLS.join(",")}`; + +const exaAdapter = createMcpAdapter({ + config: { + settings: { + disableProxyTool: true, + scriptMode: false, + }, + mcpServers: { + exa: { + url: EXA_MCP_URL, + protocolVersion: "auto", + lifecycle: "eager", + directTools: EXA_TOOLS, + includeTools: EXA_TOOLS, + toolPrefix: "none", + approveTools: false, + exposeResources: false, + }, + }, + }, +}); + +/** Connect Pi to Exa while exposing every direct tool as `exa_`. */ +export default function exaExtension(pi: ExtensionAPI): void { + exaAdapter(createExaToolApiOverride(pi)); +} diff --git a/extensions/tavily-override.ts b/extensions/tavily-override.ts new file mode 100644 index 0000000..a9011b8 --- /dev/null +++ b/extensions/tavily-override.ts @@ -0,0 +1,21 @@ +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import tavilyExtension from "../node_modules/@tavily/pi-extension/index.ts"; +import { prefixTavilyTool, type TavilyToolDefinition } from "./tavily-tool-prefix.ts"; + + +/** Load Tavily while making the provider explicit in every registered tool name. */ +export default function tavilyOverrideExtension(pi: ExtensionAPI): void { + const prefixedPi = new Proxy(pi, { + get(target, property) { + if (property === "registerTool") { + return (definition: TavilyToolDefinition) => + (target.registerTool as (tool: unknown) => void)(prefixTavilyTool(definition)); + } + + const value = Reflect.get(target, property, target); + return typeof value === "function" ? value.bind(target) : value; + }, + }); + + tavilyExtension(prefixedPi); +} diff --git a/extensions/tavily-tool-prefix.ts b/extensions/tavily-tool-prefix.ts new file mode 100644 index 0000000..e3dfea7 --- /dev/null +++ b/extensions/tavily-tool-prefix.ts @@ -0,0 +1,25 @@ +const TAVILY_TOOL_PREFIX = "tavily_"; + +export type TavilyToolDefinition = { + name: string; + label: string; + description: string; + promptSnippet?: string; + promptGuidelines?: string[]; + [key: string]: unknown; +}; + +function rewriteToolReferences(text: string): string { + return text.replace(/\bweb_search\b/g, "tavily_web_search").replace(/\bweb_fetch\b/g, "tavily_web_fetch"); +} + +export function prefixTavilyTool(definition: T): T { + return { + ...definition, + name: `${TAVILY_TOOL_PREFIX}${definition.name}`, + label: definition.label.startsWith("Tavily ") ? definition.label : `Tavily ${definition.label}`, + description: rewriteToolReferences(definition.description), + promptSnippet: definition.promptSnippet ? rewriteToolReferences(definition.promptSnippet) : undefined, + promptGuidelines: definition.promptGuidelines?.map(rewriteToolReferences), + } as T; +} diff --git a/extensions/tool-routing.ts b/extensions/tool-routing.ts index 50d4835..c9e8a81 100644 --- a/extensions/tool-routing.ts +++ b/extensions/tool-routing.ts @@ -11,6 +11,10 @@ function hasTool(selectedTools: SelectedTools, name: string): boolean { return selectedTools === undefined || selectedTools.includes(name); } +function hasAnyTool(selectedTools: SelectedTools, names: string[]): boolean { + return names.some((name) => hasTool(selectedTools, name)); +} + export function buildToolRoutingSection(selectedTools: SelectedTools): string { const rules: string[] = []; @@ -40,6 +44,30 @@ export function buildToolRoutingSection(selectedTools: SelectedTools): string { ); } + const tavilyTools = ["tavily_web_search", "tavily_web_fetch"]; + const exaTools = ["exa_web_search", "exa_web_fetch", "exa_web_search_advanced"]; + const keenableTools = ["keenable_search", "keenable_fetch"]; + if (hasAnyTool(selectedTools, [...tavilyTools, ...exaTools, ...keenableTools])) { + rules.push( + "- Choose one online search provider based on the task instead of querying Tavily, Exa, and Keenable by default; cross-check with another provider only when source quality or uncertainty warrants it.", + ); + } + if (hasAnyTool(selectedTools, tavilyTools)) { + rules.push( + "- Use Tavily tools for broad discovery when keywords are uncertain, for news/recent/trending topics, or when many candidate sources are useful. Tavily results can mix official pages, blogs, forums, and secondary sources, so keep max_results modest, avoid raw content unless needed, and fetch only selected pages when depth matters.", + ); + } + if (hasAnyTool(selectedTools, exaTools)) { + rules.push( + "- Use Exa tools when source precision matters: official or API documentation, release/version information, academic papers, research material, and high-quality fact confirmation. Use exa_web_fetch to continue reading selected pages; for exa_web_search_advanced, explicitly bound the result count and returned text length to prevent oversized responses.", + ); + } + if (hasAnyTool(selectedTools, keenableTools)) { + rules.push( + "- Use Keenable tools for Chinese-language pages, policies and government notices, site-restricted search, and publication-date filtering. Prefer its compact search results and fetch only selected pages to keep returned content controlled; KEENABLE_API_KEY is optional but raises rate limits.", + ); + } + if (hasTool(selectedTools, "ctx_execute") || hasTool(selectedTools, "ctx_execute_file")) { rules.push( "- Use Context Mode for logs, test/build output, generated data, large-file exploration/analysis/summarization, and any command whose output may be large or unpredictable. Prefer ctx_execute_file over read when deriving an answer from a large file inside the workspace, and print only the derived answer needed for the task.", diff --git a/package-lock.json b/package-lock.json index 9254cee..3211f2a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -11,7 +11,9 @@ "@ff-labs/pi-fff": "0.10.5", "@firstpick/pi-extension-codex-fast-mode": "0.1.1", "@gotgenes/pi-permission-system": "26.2.1", + "@keenable/pi-search": "0.1.2", "@ogulcancelik/pi-codex-compaction": "0.1.3", + "@tavily/pi-extension": "0.1.2", "context-mode": "1.0.169", "pi-context-view": "0.4.2", "pi-hashline-edit": "0.8.3", @@ -334,6 +336,17 @@ "hono": "^4" } }, + "node_modules/@keenable/pi-search": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/@keenable/pi-search/-/pi-search-0.1.2.tgz", + "integrity": "sha512-3akVIB+0vgaSdsqwFZ46nTHOnjEywgKEkho/l6ixVkJS07T8A8W8gLsg26L526DlaZo+EMW+44EbZASTfKZU6A==", + "license": "MIT", + "peerDependencies": { + "@earendil-works/pi-ai": "*", + "@earendil-works/pi-coding-agent": "*", + "typebox": "*" + } + }, "node_modules/@mixmark-io/domino": { "version": "2.2.0", "resolved": "https://registry.npmjs.org/@mixmark-io/domino/-/domino-2.2.0.tgz", @@ -690,6 +703,29 @@ "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==", "license": "MIT" }, + "node_modules/@tavily/core": { + "version": "0.7.7", + "resolved": "https://registry.npmjs.org/@tavily/core/-/core-0.7.7.tgz", + "integrity": "sha512-6Sc+u4nPEdzLB2D7Ubut/sLI2wR3Wc81P1VCJNTsdFtvgLOpVGGGEYAG9VUCLOzVCeEUT5SzHRIMSnsqJ4Pdow==", + "license": "MIT", + "dependencies": { + "axios": "^1.7.7", + "https-proxy-agent": "^7.0.6", + "js-tiktoken": "^1.0.14" + } + }, + "node_modules/@tavily/pi-extension": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/@tavily/pi-extension/-/pi-extension-0.1.2.tgz", + "integrity": "sha512-12RFWh1ni7rvr+jIHOu1uKpidWeBIVgAMeOvWayUxejab5CQIRchz+83o0f1Fw0wIOH55cziCynO1nOSX2s95Q==", + "dependencies": { + "@tavily/core": "^0.7.3" + }, + "peerDependencies": { + "@earendil-works/pi-ai": "*", + "@earendil-works/pi-coding-agent": "*" + } + }, "node_modules/@tokenizer/inflate": { "version": "0.4.1", "resolved": "https://registry.npmjs.org/@tokenizer/inflate/-/inflate-0.4.1.tgz", @@ -903,6 +939,15 @@ "node": ">= 0.6" } }, + "node_modules/agent-base": { + "version": "7.1.4", + "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz", + "integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==", + "license": "MIT", + "engines": { + "node": ">= 14" + } + }, "node_modules/ajv": { "version": "8.20.0", "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz", @@ -948,6 +993,49 @@ "url": "https://github.com/chalk/ansi-regex?sponsor=1" } }, + "node_modules/asynckit": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz", + "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", + "license": "MIT" + }, + "node_modules/axios": { + "version": "1.19.0", + "resolved": "https://registry.npmjs.org/axios/-/axios-1.19.0.tgz", + "integrity": "sha512-ht/iuYZXEjFxLH/Hkezgd7m6JKlHHXEUSneaDz8uZe1Gj5QZtCnpyDsckvAiEnT89OEbCLmnte4R4sn7P0EKFw==", + "license": "MIT", + "dependencies": { + "follow-redirects": "^1.16.0", + "form-data": "^4.0.6", + "https-proxy-agent": "^5.0.1", + "proxy-from-env": "^2.1.0" + } + }, + "node_modules/axios/node_modules/agent-base": { + "version": "6.0.2", + "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz", + "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==", + "license": "MIT", + "dependencies": { + "debug": "4" + }, + "engines": { + "node": ">= 6.0.0" + } + }, + "node_modules/axios/node_modules/https-proxy-agent": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.1.tgz", + "integrity": "sha512-dFcAjpTQFgoLMzC2VwU+C/CbS7uRL0lWmxDITmqm7C+7F0Odmj6s9l6alZc6AELXhrnggM2CeWSXHGOdX2YtwA==", + "license": "MIT", + "dependencies": { + "agent-base": "6", + "debug": "4" + }, + "engines": { + "node": ">= 6" + } + }, "node_modules/base64-js": { "version": "1.5.1", "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", @@ -1122,6 +1210,18 @@ "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==", "license": "ISC" }, + "node_modules/combined-stream": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz", + "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", + "license": "MIT", + "dependencies": { + "delayed-stream": "~1.0.0" + }, + "engines": { + "node": ">= 0.8" + } + }, "node_modules/content-disposition": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz", @@ -1312,6 +1412,15 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/delayed-stream": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz", + "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", + "license": "MIT", + "engines": { + "node": ">=0.4.0" + } + }, "node_modules/depd": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz", @@ -1407,6 +1516,21 @@ "node": ">= 0.4" } }, + "node_modules/es-set-tostringtag": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz", + "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==", + "license": "MIT", + "dependencies": { + "es-errors": "^1.3.0", + "get-intrinsic": "^1.2.6", + "has-tostringtag": "^1.0.2", + "hasown": "^2.0.2" + }, + "engines": { + "node": ">= 0.4" + } + }, "node_modules/escape-html": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", @@ -1624,6 +1748,63 @@ "url": "https://opencollective.com/express" } }, + "node_modules/follow-redirects": { + "version": "1.16.0", + "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.0.tgz", + "integrity": "sha512-y5rN/uOsadFT/JfYwhxRS5R7Qce+g3zG97+JrtFZlC9klX/W5hD7iiLzScI4nZqUS7DNUdhPgw4xI8W2LuXlUw==", + "funding": [ + { + "type": "individual", + "url": "https://github.com/sponsors/RubenVerborgh" + } + ], + "license": "MIT", + "engines": { + "node": ">=4.0" + }, + "peerDependenciesMeta": { + "debug": { + "optional": true + } + } + }, + "node_modules/form-data": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz", + "integrity": "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==", + "license": "MIT", + "dependencies": { + "asynckit": "^0.4.0", + "combined-stream": "^1.0.8", + "es-set-tostringtag": "^2.1.0", + "hasown": "^2.0.4", + "mime-types": "^2.1.35" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/form-data/node_modules/mime-db": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz", + "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", + "license": "MIT", + "engines": { + "node": ">= 0.6" + } + }, + "node_modules/form-data/node_modules/mime-types": { + "version": "2.1.35", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz", + "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", + "license": "MIT", + "dependencies": { + "mime-db": "1.52.0" + }, + "engines": { + "node": ">= 0.6" + } + }, "node_modules/forwarded": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz", @@ -1736,6 +1917,21 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/has-tostringtag": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz", + "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==", + "license": "MIT", + "dependencies": { + "has-symbols": "^1.0.3" + }, + "engines": { + "node": ">= 0.4" + }, + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, "node_modules/hasown": { "version": "2.0.4", "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.4.tgz", @@ -1777,6 +1973,19 @@ "url": "https://opencollective.com/express" } }, + "node_modules/https-proxy-agent": { + "version": "7.0.6", + "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz", + "integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==", + "license": "MIT", + "dependencies": { + "agent-base": "^7.1.2", + "debug": "4" + }, + "engines": { + "node": ">= 14" + } + }, "node_modules/iconv-lite": { "version": "0.7.3", "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.3.tgz", @@ -1912,6 +2121,15 @@ "url": "https://github.com/sponsors/panva" } }, + "node_modules/js-tiktoken": { + "version": "1.0.21", + "resolved": "https://registry.npmjs.org/js-tiktoken/-/js-tiktoken-1.0.21.tgz", + "integrity": "sha512-biOj/6M5qdgx5TKjDnFT1ymSpM5tbd3ylwDtrQvFQSu0Z7bBYko2dF+W/aUkXUPuk6IVpRxk/3Q2sHOzGlS36g==", + "license": "MIT", + "dependencies": { + "base64-js": "^1.5.1" + } + }, "node_modules/json-schema-traverse": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz", @@ -2324,6 +2542,15 @@ "node": ">= 0.10" } }, + "node_modules/proxy-from-env": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-2.1.0.tgz", + "integrity": "sha512-cJ+oHTW1VAEa8cJslgmUZrc+sjRKgAKl3Zyse6+PV38hZe/V6Z14TbCuXcan9F9ghlz4QrFr2c92TNF82UkYHA==", + "license": "MIT", + "engines": { + "node": ">=10" + } + }, "node_modules/pump": { "version": "3.0.4", "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz", diff --git a/package.json b/package.json index eab6c44..695e695 100644 --- a/package.json +++ b/package.json @@ -23,6 +23,9 @@ "pi": { "extensions": [ "./extensions/fff-override.ts", + "./extensions/tavily-override.ts", + "./node_modules/@keenable/pi-search/src/index.ts", + "./extensions/exa.ts", "./pi-rtk-optimizer/index.ts", "./extensions/hashline.ts", "./node_modules/context-mode/build/adapters/pi/extension.js", @@ -37,14 +40,17 @@ "./extensions/tool-routing.ts" ], "skills": [ - "./node_modules/context-mode/skills" + "./node_modules/context-mode/skills", + "./node_modules/@keenable/pi-search/skills" ] }, "dependencies": { "@ff-labs/pi-fff": "0.10.5", "@firstpick/pi-extension-codex-fast-mode": "0.1.1", "@gotgenes/pi-permission-system": "26.2.1", + "@keenable/pi-search": "0.1.2", "@ogulcancelik/pi-codex-compaction": "0.1.3", + "@tavily/pi-extension": "0.1.2", "context-mode": "1.0.169", "pi-context-view": "0.4.2", "pi-hashline-edit": "0.8.3", diff --git a/tests/exa-override.test.ts b/tests/exa-override.test.ts new file mode 100644 index 0000000..f656037 --- /dev/null +++ b/tests/exa-override.test.ts @@ -0,0 +1,42 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { createExaToolApiOverride, mapExaToolName, prefixExaTool } from "../extensions/exa-tool-prefix.ts"; + +test("Exa tools use a provider prefix followed by the original semantic name", () => { + assert.equal(mapExaToolName("web_search_exa"), "exa_web_search"); + assert.equal(mapExaToolName("web_fetch_exa"), "exa_web_fetch"); + assert.equal(mapExaToolName("web_search_advanced_exa"), "exa_web_search_advanced"); + assert.equal(mapExaToolName("unrelated_tool"), "unrelated_tool"); + + const tool = prefixExaTool({ name: "web_search_exa", label: "MCP: web_search_exa" }); + assert.equal(tool.name, "exa_web_search"); + assert.equal(tool.label, "Exa Web Search"); +}); + +test("Exa API override keeps registered and active tool names aligned", () => { + const registered: Array<{ name: string; label: string }> = []; + let active: string[] = []; + const unregistered: string[] = []; + const api = createExaToolApiOverride({ + registerTool(tool: unknown) { + registered.push(tool as { name: string; label: string }); + }, + setActiveTools(names: string[]) { + active = names; + }, + unregisterTool(name: string) { + unregistered.push(name); + return true; + }, + }); + + api.registerTool({ name: "web_fetch_exa", label: "MCP: web_fetch_exa" }); + api.setActiveTools(["read", "web_fetch_exa", "web_fetch_exa"]); + api.unregisterTool("web_fetch_exa"); + + assert.equal(registered[0]?.name, "exa_web_fetch"); + assert.equal(registered[0]?.label, "Exa Web Fetch"); + assert.deepEqual(active, ["read", "exa_web_fetch"]); + assert.deepEqual(unregistered, ["exa_web_fetch"]); +}); diff --git a/tests/tavily-override.test.ts b/tests/tavily-override.test.ts new file mode 100644 index 0000000..ee795d6 --- /dev/null +++ b/tests/tavily-override.test.ts @@ -0,0 +1,22 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { prefixTavilyTool } from "../extensions/tavily-tool-prefix.ts"; + +test("Tavily override prefixes tool identity and internal tool references", () => { + const tool = prefixTavilyTool({ + name: "web_search", + label: "Web Search", + description: "Use web_search, then web_fetch with Tavily.", + promptSnippet: "Call web_search.", + promptGuidelines: ["Use web_fetch after web_search."], + parameters: {} as never, + execute: async () => ({ content: [{ type: "text" as const, text: "ok" }] }), + }); + + assert.equal(tool.name, "tavily_web_search"); + assert.equal(tool.label, "Tavily Web Search"); + assert.equal(tool.description, "Use tavily_web_search, then tavily_web_fetch with Tavily."); + assert.equal(tool.promptSnippet, "Call tavily_web_search."); + assert.deepEqual(tool.promptGuidelines, ["Use tavily_web_fetch after tavily_web_search."]); +}); diff --git a/tests/tool-routing.test.ts b/tests/tool-routing.test.ts index 4ebe976..a14992e 100644 --- a/tests/tool-routing.test.ts +++ b/tests/tool-routing.test.ts @@ -44,6 +44,23 @@ test("LSP routing covers navigation and post-edit diagnostics", () => { assert.match(section, /post-edit diagnostics/); }); +test("online search routing selects providers by task", () => { + const tavily = buildToolRoutingSection(["tavily_web_search", "tavily_web_fetch"]); + assert.match(tavily, /broad discovery when keywords are uncertain/); + assert.match(tavily, /news\/recent\/trending topics/); + assert.doesNotMatch(tavily, /official or API documentation/); + + const exa = buildToolRoutingSection(["exa_web_search", "exa_web_fetch", "exa_web_search_advanced"]); + assert.match(exa, /official or API documentation/); + assert.match(exa, /explicitly bound the result count and returned text length/); + assert.doesNotMatch(exa, /Chinese-language pages/); + + const keenable = buildToolRoutingSection(["keenable_search", "keenable_fetch"]); + assert.match(keenable, /Chinese-language pages, policies and government notices/); + assert.match(keenable, /site-restricted search/); + assert.match(keenable, /publication-date filtering/); +}); + test("routing includes only guidance for active optional tools", () => { const section = buildToolRoutingSection(["read"]); @@ -52,6 +69,8 @@ test("routing includes only guidance for active optional tools", () => { assert.doesNotMatch(section, /Use Context Mode/); assert.doesNotMatch(section, /ctx_execute_file is confined/); assert.doesNotMatch(section, /fresh LINE#HASH anchors/); + assert.doesNotMatch(section, /Choose one online search provider/); + assert.doesNotMatch(section, /broad discovery when keywords are uncertain/); assert.match(section, /Use read only when exact source is needed/); });