mirror of
https://bitbucket.org/siakitem/my-pi.git
synced 2026-10-26 10:45:22 +00:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c9823ef5c | ||
|
|
ee1e6999d7 | ||
|
|
6fe6cb29ae |
@@ -21,11 +21,10 @@
|
|||||||
- 初始导入快照:`3f9d35cd078d18a141eb5a6d8f4fc5010d756280`(`v0.18.0`)。
|
- 初始导入快照:`3f9d35cd078d18a141eb5a6d8f4fc5010d756280`(`v0.18.0`)。
|
||||||
- 该目录从明确 tag 导入并由本仓库直接维护,不使用 npm 预编译产物,不是 submodule,也不保留上游 `.pi` 状态;源码、测试、wrapper 与权限集成实现继续保留,但当前不属于根组合包运行时依赖,`extensions/subagents.ts` 也不在默认扩展加载列表中。
|
- 该目录从明确 tag 导入并由本仓库直接维护,不使用 npm 预编译产物,不是 submodule,也不保留上游 `.pi` 状态;源码、测试、wrapper 与权限集成实现继续保留,但当前不属于根组合包运行时依赖,`extensions/subagents.ts` 也不在默认扩展加载列表中。
|
||||||
- `extensions/fff-override.ts`:强制使用 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全;显式 CLI flag 仍遵循 FFF 官方优先级。
|
- `extensions/fff-override.ts`:强制使用 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全;显式 CLI flag 仍遵循 FFF 官方优先级。
|
||||||
- `extensions/tavily-override.ts`:加载 `@tavily/pi-extension`,把上游工具重命名为 `tavily_web_search` / `tavily_web_fetch`,并同步改写 label 与工具提示中的内部名称。
|
- `pi-web-search/`:完全自维护的统一在线搜索扩展。对模型只暴露 `web_search` / `web_fetch`,schema 与提示词不含厂商名;`web_search` 由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`)按内置路由规则在 Tavily / Exa / Keenable 间选择并经统一参数翻译调用,`details` 记录实际 provider 与路由方式,决策按会话内 LRU 缓存,路由失败或被选厂商报错按可用性顺序降级;`web_fetch` 固定走 Exa `web_fetch_exa`,失败时静默降级 Keenable → Tavily。`clients/` 为纯 `fetch` 直连:Tavily 走 `api.tavily.com`(必须有 key),Keenable 复刻上游 client 保留 keyless `/public` 端点与 SSRF 防护,Exa 直接对 `mcp.exa.ai/mcp` 发 JSON-RPC(缓存 `mcp-session-id`,404 自动重建会话,key 只走 `x-api-key` 头),不使用 `pi-mcp-adapter` 或任何上游扩展包。
|
||||||
- `@keenable/pi-search`:加载 `keenable_search` / `keenable_fetch` 及其 skill,默认 keyless,`KEENABLE_API_KEY` 仅用于提高速率限制。
|
- `search_config.sh` 与 `scripts/search-auth.mjs`:以交互或参数方式把 `tavily` / `exa` / `keenable` 三条 `api_key` 凭据写入 Pi 的 `auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`);加锁后原子写、权限 `600`、保留全部其他条目,支持 `--migrate` 从旧 `search.env` 导入。`pi-web-search/auth.ts` 每次调用读取并复刻上游解析语义(`!command` 执行、`$VAR`/`${VAR}` 插值、credential `env` 优先于进程 env、`$$`/`$!` 转义);显式进程环境变量不再直接读取,env 注入通过 `"key": "$VAR"` 表达。
|
||||||
- `search_config.sh` 与 `extensions/search-config.ts`:以交互式或参数方式把三家搜索 key 写入用户级 `search.env`(权限 `600`),并在搜索扩展初始化前加载;显式进程环境变量优先。
|
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,以 `execFile` argv 形式调用机器级 `codegraph` CLI(每次调用为短命进程,无常驻 server/daemon);未初始化项目先经 `ctx.ui.confirm` 确认再 `codegraph init` 并自动重试,拒绝或失败则把结果打回让模型改用 `find`/`grep`/`read`/LSP;结果按 `codegraph.db` mtime 做有界缓存,索引一变即失效。
|
||||||
- `extensions/mcp.ts`:通过单个共享 `pi-mcp-adapter` 实例同时连接 Exa 托管 MCP 与机器现有的 `codegraph serve --mcp`,避免重复注册 Pi 的全局 MCP flag 与命令;Exa key 只通过 `x-api-key` 请求头发送,原始工具映射为统一的来源前缀形式,CodeGraph 只暴露 `codegraph_explore`。
|
- `extensions/plugin-init.ts`:已移除;原 `/plugin_init` 的 CodeGraph 初始化职责由 `extensions/codegraph.ts` 在首次 `codegraph_explore` 调用内按需确认完成。
|
||||||
- `extensions/plugin-init.ts`:注册用户显式触发的 `/plugin_init`,在确认当前项目后只初始化缺失的 CodeGraph 状态,并在全部成功后调用 Pi 官方热重载。
|
|
||||||
- `context-mode`:加载上游 Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。
|
- `context-mode`:加载上游 Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。
|
||||||
- `extensions/hashline.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`,以行哈希锚点覆盖内置 `read` / `edit`;Hashline 自带 `grep` 强制关闭。
|
- `extensions/hashline.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`,以行哈希锚点覆盖内置 `read` / `edit`;Hashline 自带 `grep` 强制关闭。
|
||||||
- `extensions/permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置。`config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc` 由 `install.sh` / `update.sh` 经 `scripts/rebuild-bundle-config.mjs` 部署到 `<agentDir>/my-pi/permission-system/`(policy.jsonc 与 hard-guards.jsonc 且权限 owner-only);旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理。项目级 policy 不读取。
|
- `extensions/permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置。`config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc` 由 `install.sh` / `update.sh` 经 `scripts/rebuild-bundle-config.mjs` 部署到 `<agentDir>/my-pi/permission-system/`(policy.jsonc 与 hard-guards.jsonc 且权限 owner-only);旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理。项目级 policy 不读取。
|
||||||
@@ -65,7 +64,7 @@
|
|||||||
- 上游参考一:<https://github.com/jjuraszek/pi-condense>,快照 `59c95110aacd8131db71e21296765dd80b72fae8`(`v2.9.1`)。
|
- 上游参考一:<https://github.com/jjuraszek/pi-condense>,快照 `59c95110aacd8131db71e21296765dd80b72fae8`(`v2.9.1`)。
|
||||||
- 上游来源二:<https://github.com/ogulcancelik/pi-extensions/tree/main/packages/pi-codex-compaction>,导入快照 `c4d6ca8fb484ba51684296ba8b4715f8c7b3ac91`(`0.1.4`)。
|
- 上游来源二:<https://github.com/ogulcancelik/pi-extensions/tree/main/packages/pi-codex-compaction>,导入快照 `c4d6ca8fb484ba51684296ba8b4715f8c7b3ac91`(`0.1.4`)。
|
||||||
- `pi-compaction` 采用全新的 `pi-compaction` version 1 Session 记录和 `settings.json.piCompaction` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune` 或 `openai-codex-native-compaction` 状态;切换后应使用新 Session。两份上游 MIT 许可证及来源差异保留在包内。
|
- `pi-compaction` 采用全新的 `pi-compaction` version 1 Session 记录和 `settings.json.piCompaction` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune` 或 `openai-codex-native-compaction` 状态;切换后应使用新 Session。两份上游 MIT 许可证及来源差异保留在包内。
|
||||||
- 根包还固定安装 `@tavily/pi-extension@0.1.2`、`@keenable/pi-search@0.1.2`、`pi-hashline-edit@0.8.3`、`typescript-language-server@5.3.0` 和 `typescript@6.0.3`;`@gotgenes/pi-permission-system`、`@smoose/pi-notify`、`pi-ask-user`、`pi-chrome`、`pi-compaction`、`pi-hermes-memory`、`pi-minimal-footer`、`pi-ssh`、`pi-tool-search` 与 `pi-worktree` 分别指向仓库内对应目录的本地 `file:` 依赖;`pi-subagents/` 仅保留源码,不作为根运行时依赖。
|
- 根包还固定安装 `pi-hashline-edit@0.8.3`、`typescript-language-server@5.3.0` 和 `typescript@6.0.3`;`@gotgenes/pi-permission-system`、`@smoose/pi-notify`、`pi-ask-user`、`pi-chrome`、`pi-compaction`、`pi-hermes-memory`、`pi-minimal-footer`、`pi-ssh`、`pi-tool-search`、`pi-web-search` 与 `pi-worktree` 分别指向仓库内对应目录的本地 `file:` 依赖;`pi-subagents/` 仅保留源码,不作为根运行时依赖。
|
||||||
- `install.sh`:先安装根组合包,再通过 `scripts/rebuild-bundle-config.mjs` 整体重建 `<agentDir>/my-pi/` 受管配置目录,然后交互检查 Kitty/Solarized Dark、Oh My Zsh/Powerlevel10k/Zsh 插件、CodeGraph、Kotlin LSP、Java 21+ 和 JDT LS;缺失项目只在用户明确选择 `Y` 后安装或配置。Powerlevel10k 默认配置来自仓库内置的 `config/p10k.zsh`(当前 Rainbow/ASCII 单行紧凑主题)。
|
- `install.sh`:先安装根组合包,再通过 `scripts/rebuild-bundle-config.mjs` 整体重建 `<agentDir>/my-pi/` 受管配置目录,然后交互检查 Kitty/Solarized Dark、Oh My Zsh/Powerlevel10k/Zsh 插件、CodeGraph、Kotlin LSP、Java 21+ 和 JDT LS;缺失项目只在用户明确选择 `Y` 后安装或配置。Powerlevel10k 默认配置来自仓库内置的 `config/p10k.zsh`(当前 Rainbow/ASCII 单行紧凑主题)。
|
||||||
- `update.sh`:先通过 `pi update` 升级根组合包,再只升级当前已安装的终端环境和机器级依赖;未安装项直接跳过。不得在 `pi update` 前预清理安装缓存;`pi update` 失败时 fail-fast 停止——Pi 1.3.0 会在下一次 `pi update` 时依据其 `.pi-update-incomplete` 标记自行重跑 `git clean -fdx` 与依赖安装完成自愈,本脚本不做人工恢复。升级成功后调用可定位的更新后组合包 Node helper `scripts/rebuild-bundle-config.mjs` 整体重建 `<agentDir>/my-pi/`:先按 `config/bundle-manifest.json` 校验全部来源与目标,在 agent 目录内构建临时树,再删除旧 `my-pi/`(符号链接只删链接不跟随)并原子替换,不备份、不迁移、不合并不属于组合包的内容。无法从 `PI_PACKAGE_SOURCE` 定位更新后 helper 时必须报告失败,不能回退到可能陈旧的脚本目录 helper。其他组件升级前先查询并比较本地与远端版本,只有版本不同时才下载或替换;Powerlevel10k 配置优先从 `pi update` 后的已安装组合包读取,并与 `.zshrc` 受管块一起按内容比较后增量同步。
|
- `update.sh`:先通过 `pi update` 升级根组合包,再只升级当前已安装的终端环境和机器级依赖;未安装项直接跳过。不得在 `pi update` 前预清理安装缓存;`pi update` 失败时 fail-fast 停止——Pi 1.3.0 会在下一次 `pi update` 时依据其 `.pi-update-incomplete` 标记自行重跑 `git clean -fdx` 与依赖安装完成自愈,本脚本不做人工恢复。升级成功后调用可定位的更新后组合包 Node helper `scripts/rebuild-bundle-config.mjs` 整体重建 `<agentDir>/my-pi/`:先按 `config/bundle-manifest.json` 校验全部来源与目标,在 agent 目录内构建临时树,再删除旧 `my-pi/`(符号链接只删链接不跟随)并原子替换,不备份、不迁移、不合并不属于组合包的内容。无法从 `PI_PACKAGE_SOURCE` 定位更新后 helper 时必须报告失败,不能回退到可能陈旧的脚本目录 helper。其他组件升级前先查询并比较本地与远端版本,只有版本不同时才下载或替换;Powerlevel10k 配置优先从 `pi update` 后的已安装组合包读取,并与 `.zshrc` 受管块一起按内容比较后增量同步。
|
||||||
- `uninstall.sh`:先移除根组合包,其他共享终端环境和机器工具不卸载。不得删除 `.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/` 或任何用户记忆数据。
|
- `uninstall.sh`:先移除根组合包,其他共享终端环境和机器工具不卸载。不得删除 `.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/` 或任何用户记忆数据。
|
||||||
@@ -80,7 +79,7 @@
|
|||||||
- Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文;FFF 仍负责精确字面搜索,RTK 仍处理未走 Context Mode 的普通输出。
|
- Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文;FFF 仍负责精确字面搜索,RTK 仍处理未走 Context Mode 的普通输出。
|
||||||
- `chrome_snapshot`、`chrome_find` 以及带 `includeSnapshot=true` 的交互工具必须把完整结构化快照写入当前项目 `.pi/chrome-context/` 下的 Session 临时文件,工具结果只返回相对路径与有界摘要;模型必须立即用 `ctx_execute_file` 提取下一步所需信息,不得用 `read` / `cat` 把原始 JSON 重新塞入上下文。每个 Session/目标只保留最新快照,页面交互后旧快照视为失效,Session shutdown、撤销或授权过期时清理。
|
- `chrome_snapshot`、`chrome_find` 以及带 `includeSnapshot=true` 的交互工具必须把完整结构化快照写入当前项目 `.pi/chrome-context/` 下的 Session 临时文件,工具结果只返回相对路径与有界摘要;模型必须立即用 `ctx_execute_file` 提取下一步所需信息,不得用 `read` / `cat` 把原始 JSON 重新塞入上下文。每个 Session/目标只保留最新快照,页面交互后旧快照视为失效,Session shutdown、撤销或授权过期时清理。
|
||||||
- Hashline 默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 和 `replaceText` 均关闭;它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
|
- Hashline 默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 和 `replaceText` 均关闭;它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
|
||||||
- 工具路由规则不得替换 Pi 默认系统提示词:代码结构、调用关系和待修改 symbol 优先 CodeGraph,定义/引用/类型/诊断优先 LSP;工作区内大文件探索、分析和总结优先 `ctx_execute_file`,日志、构建与不可预测命令输出优先 Context Mode;字面搜索先用 FFF `find` 收敛文件位置,再在收敛路径内用 `grep`/`multi_grep` 获取行号;真正修改前才用小范围 Hashline `read` 获取新鲜锚点并用锚定 `edit`。在线搜索按场景只先选一个服务:广泛发现、新闻和候选来源用 Tavily,官方/技术/论文和高确定性来源用 Exa,中文、站点限定和日期筛选用 Keenable;只有必要时才跨服务复核。遇到大量结果、截断或上限时继续缩小范围,不得靠提高 limit 或倾倒全部结果解决。
|
- 工具路由规则不得替换 Pi 默认系统提示词:代码结构、调用关系和待修改 symbol 优先 CodeGraph,定义/引用/类型/诊断优先 LSP;工作区内大文件探索、分析和总结优先 `ctx_execute_file`,日志、构建与不可预测命令输出优先 Context Mode;字面搜索先用 FFF `find` 收敛文件位置,再在收敛路径内用 `grep`/`multi_grep` 获取行号;真正修改前才用小范围 Hashline `read` 获取新鲜锚点并用锚定 `edit`。在线搜索由 `pi-web-search` 内部路由,模型不感知厂商:路由规则为广泛发现、新闻和候选来源偏 Tavily,官方/技术/论文和高确定性来源偏 Exa,中文、站点限定和日期筛选偏 Keenable;路由决策由当前主模型做出,确定性逻辑只处理可用性与失败兜底,不得改写成启发式断路。遇到大量结果、截断或上限时继续缩小范围,不得靠提高 limit 或倾倒全部结果解决。
|
||||||
- `ask_user_question` 只用于模型继续任务所必需的用户决定、偏好、确认或澄清;不得重复询问直接用户消息中已有的信息。工具支持交互式 TUI 与具备 Pi extension UI bridge 的 RPC 客户端、顺序执行和最多 8 题;RPC 必须使用标准 `select` / `input` 对话框逐题收集,JSON/print 等无 UI 模式保持拒绝。取消或 abort 必须返回 `cancelled: true`,不得伪造答案,且 `/qna` 不属于该扩展。其成功 `details.answers` 必须保留 `question` / `answer` 字段,以匹配 AutoReview 已有的可信结构化用户交互边界。
|
- `ask_user_question` 只用于模型继续任务所必需的用户决定、偏好、确认或澄清;不得重复询问直接用户消息中已有的信息。工具支持交互式 TUI 与具备 Pi extension UI bridge 的 RPC 客户端、顺序执行和最多 8 题;RPC 必须使用标准 `select` / `input` 对话框逐题收集,JSON/print 等无 UI 模式保持拒绝。取消或 abort 必须返回 `cancelled: true`,不得伪造答案,且 `/qna` 不属于该扩展。其成功 `details.answers` 必须保留 `question` / `answer` 字段,以匹配 AutoReview 已有的可信结构化用户交互边界。
|
||||||
- Tool Search 默认常驻核心 `read` / `write` / `edit` / `bash` / `grep` / `find`、`codegraph_explore`、`lsp_diagnostics`、`ctx_execute` / `ctx_execute_file` / `ctx_batch_execute`、六个记忆工具与 `context_tree_query`,以及 `tool_search`;Windows 上内置 `powershell` 必须与 `bash` 同属固定本地 Shell 核心工具,非 Windows 平台必须从目录中过滤。配置从 `<agentDir>/my-pi/tool-search/config.json` 读取(由 rebuild 脚本部署 `config/tool-search.json`),文件缺失或无效时回退 `BUNDLE_TOOL_SEARCH_DEFAULTS`;扩展自身不播种、不迁移,也不再读写 `settings.json` 的 `toolSearch` 段。其余组合包工具必须优先使用 `pi-tool-search/extensions/bundle-groups.ts` 中提交的权威工作流分组;精确已有分组必须直接激活,不得等待无关未知工具的模型增强。默认最多 5 个动态组、模型组最多 8 个工具、动态工具总数最多 28;`groups` 可原子激活最多 5 个精确组,完整请求超限时不得改变当前状态;加载或组内调用更新 LRU,超限先移除最久未用的非请求组。自然语言查询必须优先精确工具名和已提交 workflow alias,弱匹配或近似同分只能返回候选,不得任意激活;`/tool-search-status` 必须显示当前容量、LRU 顺序与最近淘汰原因。
|
- Tool Search 默认常驻核心 `read` / `write` / `edit` / `bash` / `grep` / `find`、`codegraph_explore`、`lsp_diagnostics`、`ctx_execute` / `ctx_execute_file` / `ctx_batch_execute`、六个记忆工具与 `context_tree_query`,以及 `tool_search`;Windows 上内置 `powershell` 必须与 `bash` 同属固定本地 Shell 核心工具,非 Windows 平台必须从目录中过滤。配置从 `<agentDir>/my-pi/tool-search/config.json` 读取(由 rebuild 脚本部署 `config/tool-search.json`),文件缺失或无效时回退 `BUNDLE_TOOL_SEARCH_DEFAULTS`;扩展自身不播种、不迁移,也不再读写 `settings.json` 的 `toolSearch` 段。其余组合包工具必须优先使用 `pi-tool-search/extensions/bundle-groups.ts` 中提交的权威工作流分组;精确已有分组必须直接激活,不得等待无关未知工具的模型增强。默认最多 5 个动态组、模型组最多 8 个工具、动态工具总数最多 28;`groups` 可原子激活最多 5 个精确组,完整请求超限时不得改变当前状态;加载或组内调用更新 LRU,超限先移除最久未用的非请求组。自然语言查询必须优先精确工具名和已提交 workflow alias,弱匹配或近似同分只能返回候选,不得任意激活;`/tool-search-status` 必须显示当前容量、LRU 顺序与最近淘汰原因。
|
||||||
- `pi-chrome` 在未授权时不注册 Chrome 工具;显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
|
- `pi-chrome` 在未授权时不注册 Chrome 工具;显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
|
||||||
@@ -88,11 +87,11 @@
|
|||||||
- Tool Search 必须直接使用当前 `@earendil-works/pi-coding-agent`、`typebox`、`ModelRegistry.complete()` 和 `setActiveTools()`;首次纯增加载使用当前 Pi 的增量结果传播,LRU 替换允许走宿主安全 fallback。不得安装或映射旧 `@mariozechner` runtime,不得恢复 provider payload 改写、代理分发或隐藏 `sendMessage` steer/retry 兼容循环。Tool Search 是唯一可调用 `getActiveTools()` / `setActiveTools()` 管理工具显隐的组合包扩展;`pi-permission-system` 只能在调用时裁决,不能读写 active tool 列表。
|
- Tool Search 必须直接使用当前 `@earendil-works/pi-coding-agent`、`typebox`、`ModelRegistry.complete()` 和 `setActiveTools()`;首次纯增加载使用当前 Pi 的增量结果传播,LRU 替换允许走宿主安全 fallback。不得安装或映射旧 `@mariozechner` runtime,不得恢复 provider payload 改写、代理分发或隐藏 `sendMessage` steer/retry 兼容循环。Tool Search 是唯一可调用 `getActiveTools()` / `setActiveTools()` 管理工具显隐的组合包扩展;`pi-permission-system` 只能在调用时裁决,不能读写 active tool 列表。
|
||||||
- `pi-worktree` 只注册 `/wt` / `/worktree` 斜杠命令,不暴露任何模型可调用工具;其 git 操作由用户显式触发,经 `execFile` argv 直接调用(带超时、不经 shell),按设计绕过 Bash 工具权限策略,不新增权限审批层。
|
- `pi-worktree` 只注册 `/wt` / `/worktree` 斜杠命令,不暴露任何模型可调用工具;其 git 操作由用户显式触发,经 `execFile` argv 直接调用(带超时、不经 shell),按设计绕过 Bash 工具权限策略,不新增权限审批层。
|
||||||
- `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录,不提交到仓库,也不作为组合包运行时配置源。
|
- `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录,不提交到仓库,也不作为组合包运行时配置源。
|
||||||
- CodeGraph MCP 扩展只配置 Pi 到外部 `codegraph` 命令的连接。根包安装本身不安装 CodeGraph;便捷脚本仅在组合包安装完成且用户明确选择 `Y` 后调用官方安装器。安装、升级和普通启动流程不得执行 `codegraph init`;只有用户在目标项目显式调用 `/plugin_init` 并确认后,命令才可以 `.codegraph/codegraph.db` 为权威标志初始化缺失状态。命令必须先预检 CLI,失败不得热重载,也不得隐瞒此前已完成的部分初始化;不得改动 CodeGraph 更新或遥测设置。
|
- CodeGraph 是机器级依赖:`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后,扩展才可以 `codegraph init` 当前目录。init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。
|
||||||
- CodeGraph MCP 使用 `keep-alive` 并只直接暴露 `codegraph_explore`;命令缺失、项目未初始化或连接失败时不得阻止其他扩展加载。
|
- `codegraph_explore` 为原生 Pi 工具而非 MCP 工具:每次调用通过 `execFile`/`pi.exec` 以 argv 形式运行 `codegraph explore`(带超时、不经 shell),结果按 `.codegraph/codegraph.db` mtime 缓存;CLI 缺失、init 被拒绝或执行失败都不得抛出或阻止其他工具,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。
|
||||||
- 所有 MCP 服务必须合并到 `extensions/mcp.ts` 创建的唯一 `pi-mcp-adapter` 实例;不得新增独立 adapter 扩展入口,否则会重复注册 `--mcp-config`、`/mcp` 等全局接口。根测试必须扫描全部本地扩展入口并强制这一不变量。
|
- 组合包不加载任何 MCP adapter;Exa 由 `pi-web-search/clients/exa.ts` 以纯 `fetch` JSON-RPC 直连托管端点 `https://mcp.exa.ai/mcp`:`initialize` 一次并缓存 `mcp-session-id`,随后 `tools/call`,HTTP 404 视为会话失效自动重建并重试一次。`exa` 凭据通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。
|
||||||
- Exa MCP 使用托管 Streamable HTTP 端点和 `eager` lifecycle;`EXA_API_KEY` 通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。所有在线搜索工具统一采用“来源名 + 原始语义工具名”的命名形式,例如 `tavily_web_search`、`exa_web_search`、`keenable_search`。Exa 高级搜索必须显式约束结果数量和文本长度;Tavily 输出默认控制 `max_results` 且非必要不请求 raw content;Keenable 优先利用中文、站点和日期筛选能力。
|
- 在线搜索对模型只暴露 `web_search` / `web_fetch` 两个统一工具名,不得重新引入按厂商命名的工具;`web_search` 必须经 `router.ts` 的主模型分类选择 provider,`web_fetch` 固定优先 Exa。Exa 搜索统一走 `web_search_advanced_exa` 并显式约束 `numResults` 与 `textMaxCharacters`;Tavily 默认 `search_depth: basic` 且必须在 auth.json 存在 `tavily` api_key 凭据才参与路由;Keenable keyless 可用,`keenable` 凭据仅提高速率限制。
|
||||||
- 搜索 key 默认保存在 `${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env`,文件必须为 `600` 且不得提交;`search_config.sh` 只输出配置状态,禁止回显 key。
|
- 搜索 key 统一保存在 Pi 的 `auth.json`(`tavily`/`exa`/`keenable` 各一条 `api_key` 凭据,文件权限 `600` 且不得提交);key 支持 `!command` 与 `$VAR` 间接寻址,`search_config.sh` / `search-auth.mjs` 只输出配置状态,禁止回显 key。旧的 `search.env` 不再被读取,仅由 `--migrate` 一次性导入。
|
||||||
- `extensions/context-view.ts` 是只读的本地 effective-context observer,只读取 `pi-compaction/src/service.ts` 发布的 projection metrics,不注册生产 `context` writer;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
|
- `extensions/context-view.ts` 是只读的本地 effective-context observer,只读取 `pi-compaction/src/service.ts` 发布的 projection metrics,不注册生产 `context` writer;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
|
||||||
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
||||||
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。
|
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。
|
||||||
|
|||||||
@@ -5,9 +5,8 @@
|
|||||||
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。
|
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。
|
||||||
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
|
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
|
||||||
- `context-mode@1.0.169`:通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。
|
- `context-mode@1.0.169`:通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。
|
||||||
- `@tavily/pi-extension@0.1.2`:提供 Tavily 广泛搜索与页面提取;通过本地 override 暴露为 `tavily_web_search` / `tavily_web_fetch`。
|
- 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。
|
||||||
- `@keenable/pi-search@0.1.2`:提供紧凑的 `keenable_search` / `keenable_fetch`,并加载其搜索 skill;API key 可选。
|
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,每次调用以短命 `codegraph explore` CLI 进程完成,无常驻 server/daemon;未初始化项目经确认后才 `codegraph init`,结果按索引库 mtime 缓存。
|
||||||
- `pi-mcp-adapter@2.26.0`:通过单个共享 adapter 实例接入机器现有的 CodeGraph 和 Exa 托管 MCP,避免重复注册 Pi 的全局 MCP flag 与命令。
|
|
||||||
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
|
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
|
||||||
- `pi-lsp@0.1.7`:为 TypeScript/JavaScript、Kotlin 和 Java 提供 LSP 诊断、跳转与符号工具;TypeScript 后端由组合包内置。
|
- `pi-lsp@0.1.7`:为 TypeScript/JavaScript、Kotlin 和 Java 提供 LSP 诊断、跳转与符号工具;TypeScript 后端由组合包内置。
|
||||||
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
|
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
|
||||||
@@ -52,7 +51,7 @@ pi install git:git@bitbucket.org:siakitem/my-pi.git
|
|||||||
`.zshrc` 的 `plugins` 数组中启用。后两个插件缺少源码时会先克隆到 Oh My Zsh 的
|
`.zshrc` 的 `plugins` 数组中启用。后两个插件缺少源码时会先克隆到 Oh My Zsh 的
|
||||||
custom plugins 目录;语法高亮按最后一个插件处理。
|
custom plugins 目录;语法高亮按最后一个插件处理。
|
||||||
- 最后继续检查 `codegraph`、`kotlin-lsp`、Java 21+ 和 `jdtls`。Kotlin、Java 和 JDT LS
|
- 最后继续检查 `codegraph`、`kotlin-lsp`、Java 21+ 和 `jdtls`。Kotlin、Java 和 JDT LS
|
||||||
的自动安装使用 Homebrew。
|
的自动安装使用 Homebrew,CodeGraph 使用官方安装脚本。
|
||||||
|
|
||||||
脚本首次实际改动 `.zshrc` 前会在同目录创建带时间戳的备份。Oh My Zsh loader、
|
脚本首次实际改动 `.zshrc` 前会在同目录创建带时间戳的备份。Oh My Zsh loader、
|
||||||
Powerlevel10k 主题/配置和各 Zsh 插件使用 `# >>> my-pi:<id> >>>` 到
|
Powerlevel10k 主题/配置和各 Zsh 插件使用 `# >>> my-pi:<id> >>>` 到
|
||||||
@@ -120,15 +119,15 @@ Tool Search 常驻,保证 policy 指示与模型可见工具一致。
|
|||||||
(项目级),可选配置为 `~/.pi/agent/hermes-memory-config.json`。这些是用户数据,安装、升级与卸载脚本
|
(项目级),可选配置为 `~/.pi/agent/hermes-memory-config.json`。这些是用户数据,安装、升级与卸载脚本
|
||||||
不会创建、迁移或删除。
|
不会创建、迁移或删除。
|
||||||
|
|
||||||
进入目标项目后可在 Pi 中执行 `/plugin_init`:命令会显示当前目录并要求确认,以
|
CodeGraph 采用惰性初始化:`codegraph_explore` 在未建立索引的项目上被调用时会先向用户确认,
|
||||||
`.codegraph/codegraph.db` 为初始化标志,只为缺失状态运行 `codegraph init`;全部成功后通过 Pi 官方热重载
|
确认后才运行 `codegraph init` 并自动重试本次探索;不需要手动执行任何初始化命令。
|
||||||
重新连接 CodeGraph MCP。也可以继续在终端手动执行 `codegraph init`。
|
|
||||||
|
|
||||||
默认配置不需要系统 `rtk` CLI。只有以后在 `/rtk` 中主动开启 RTK command rewriting 时,才需要另外安装 `rtk` 可执行文件。
|
默认配置不需要系统 `rtk` CLI。只有以后在 `/rtk` 中主动开启 RTK command rewriting 时,才需要另外安装 `rtk` 可执行文件。
|
||||||
|
|
||||||
CodeGraph 本体不由组合包安装。需要使用 CodeGraph 的机器应自行确保 `codegraph` 在
|
CodeGraph 本体不由组合包安装。需要使用 CodeGraph 的机器应自行确保 `codegraph` 在
|
||||||
启动 Pi 的 `PATH` 中。组合包只有在用户显式调用 `/plugin_init` 并确认当前项目后才会创建
|
启动 Pi 的 `PATH` 中(`install.sh` 会交互式提供官方安装)。扩展只有在用户于确认对话框中
|
||||||
`.codegraph/`;安装、升级和普通启动流程不会创建或维护索引,也不会修改 CodeGraph 的更新或遥测设置。
|
明确同意后才会创建 `.codegraph/`;安装、升级和普通启动流程不会创建或维护索引,也不会修改
|
||||||
|
CodeGraph 的更新或遥测设置。
|
||||||
|
|
||||||
## 组合行为
|
## 组合行为
|
||||||
|
|
||||||
@@ -256,42 +255,40 @@ FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash
|
|||||||
./search_config.sh --tavily '...' --exa '...' --keenable '...'
|
./search_config.sh --tavily '...' --exa '...' --keenable '...'
|
||||||
```
|
```
|
||||||
|
|
||||||
只传部分参数时保留其他服务的已有值。脚本把 key 原子写入
|
只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的
|
||||||
`${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env` 并设置权限为 `600`;
|
`auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`,权限 `600`):
|
||||||
`MY_PI_SEARCH_CONFIG` 或 `--config` 可覆盖路径。组合包在搜索扩展初始化前加载该文件,
|
每个 provider 一条 `{ "type": "api_key", "key": "..." }` 条目,原子加锁写入,
|
||||||
但调用 Pi 时显式传入的同名环境变量优先。配置文件不属于仓库,不会随安装包提交。
|
其他条目(如 OAuth 凭据)原样保留。`--list` 只显示配置状态、不回显 key;
|
||||||
|
`--migrate` 可从旧的 `search.env` 导入缺失条目;`--auth FILE` 可覆盖文件路径。
|
||||||
|
|
||||||
Tavily 通过 `extensions/tavily-override.ts` 加载官方扩展。override 只改工具身份,不改官方执行逻辑:上游 `web_search` / `web_fetch` 被注册为 `tavily_web_search` / `tavily_web_fetch`,label 和提示词中的内部工具引用也同步带上 Tavily 标识。使用前需设置 `TAVILY_API_KEY`。Tavily 适合广泛发现,但来源可能混杂且输出较长,应从较小的 `max_results` 开始,非必要不返回 raw content。
|
key 值支持 auth.json 原生解析语义:`$VAR` / `${VAR}` 环境变量插值、`!command`
|
||||||
|
执行命令取 stdout(如 `"!op read op://Vault/Tavily/key"`)、`$$` / `$!` 转义。
|
||||||
|
显式进程环境变量因此不再被直接读取——需要 env 注入时把 key 写成
|
||||||
|
`"$TAVILY_API_KEY"` 即可。`KEENABLE_API_URL`(非 secret 的端点覆盖)仍走
|
||||||
|
进程环境变量,也可放在 keenable 凭据的 `env` 字段中。
|
||||||
|
|
||||||
Keenable 直接加载 npm 包的扩展和 skill,默认无需 key;设置 `KEENABLE_API_KEY` 可提高速率限制。它适合中文内容、`site` 限定和发布日期筛选。搜索结果保持紧凑,只对选中的 URL 调用 `keenable_fetch`。
|
三个服务由本仓库自维护的 `pi-web-search/` 统一封装:对模型只暴露 `web_search` / `web_fetch` 两个工具,schema 与提示词中不出现任何厂商名;每次 `web_search` 先由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`,零额外配置)按内置路由规则在候选厂商中选一个,再按统一参数翻译后调用,`details` 记录实际 provider 与路由方式;选择结果按会话内 LRU 缓存,路由失败或被选厂商报错时按可用性顺序自动降级。`web_fetch` 固定调用 Exa 的 `web_fetch_exa` 提取正文(默认每页 3000 字符上限),失败时静默降级到 Keenable、再到 Tavily。
|
||||||
|
|
||||||
`extensions/mcp.ts` 通过单个 `pi-mcp-adapter` 实例同时连接 Exa 与 CodeGraph,避免两个 adapter 重复注册 `--mcp-config`、`/mcp` 和 `/mcp-auth`。其中 Exa 连接 `https://mcp.exa.ai/mcp`,并把 MCP 原始工具映射为统一的来源前缀形式:`exa_web_search`、`exa_web_fetch` 和 `exa_web_search_advanced`。配置 `EXA_API_KEY` 后通过 `x-api-key` 请求头认证,不把 key 放进 URL。Exa 适合官方文档、技术资料、版本信息和论文;高级搜索必须显式限制结果数量与返回文本长度,避免响应过大。
|
`pi-web-search/clients/` 全部是纯 `fetch` 实现,三个 key 统一由 `pi-web-search/auth.ts` 从 auth.json 解析(Tavily 直连 `api.tavily.com/search|extract`,必须有 `tavily` 凭据,无 key 时不参与路由;Keenable 复刻上游 client,默认走 keyless `/public` 端点,`keenable` 凭据只提高速率限制并保留 SSRF host 防护;Exa 直接对 `https://mcp.exa.ai/mcp` 发 JSON-RPC——initialize 一次并缓存 `mcp-session-id`、随后 `tools/call`,会话失效(HTTP 404)自动重建,`exa` 凭据只通过 `x-api-key` 请求头发送,不再依赖 `pi-mcp-adapter`)。Exa 搜索统一走 `web_search_advanced_exa` 并显式限制结果数量与 `textMaxCharacters`,避免响应过大。
|
||||||
|
|
||||||
根测试会扫描所有本地扩展入口,强制只有 `extensions/mcp.ts` 可以创建或导入 `pi-mcp-adapter`。新增 MCP 服务必须合并进这个共享实例,不能再增加独立 adapter 入口;隔离 Pi 加载仍作为发布前的运行时验证。
|
### CodeGraph 是原生工具
|
||||||
|
|
||||||
### CodeGraph 只负责连接
|
`extensions/codegraph.ts` 注册原生 `codegraph_explore` 工具,不走 MCP:每次调用以 argv 形式
|
||||||
|
直接执行机器级 `codegraph explore`(上游保证其输出与 `codegraph_explore` MCP 工具一致),
|
||||||
|
进程随调用结束,不维护常驻 server 或 daemon。`projectPath` 缺省为会话 cwd,沿目录向上寻找
|
||||||
|
最近的 `.codegraph/codegraph.db`;`maxFiles` 透传 `--max-files`。
|
||||||
|
|
||||||
同一个 `extensions/mcp.ts` adapter 配置启动
|
结果按 `(索引根, query, maxFiles)` 缓存在扩展内,并以索引库 mtime 作为有效性判据:任何
|
||||||
`codegraph serve --mcp`,不会读取或覆盖用户已有的 MCP 配置。连接采用
|
重建或同步都会更新 mtime 使旧缓存自动失效,模型对同一区域反复探索时重复调用立即返回。
|
||||||
`keep-alive`,并只把官方当前主工具 `codegraph_explore` 作为同名 Pi 工具暴露;
|
|
||||||
CodeGraph 的文件 watcher 由该 MCP 进程自身维护。
|
|
||||||
|
|
||||||
若机器没有 `codegraph`,或当前项目尚未建立索引,CodeGraph 会报告连接/初始化错误,
|
若目标目录没有索引,工具先弹出确认;用户同意后运行 `codegraph init` 并自动重试本次调用,
|
||||||
其他扩展仍可正常工作。安装与索引状态可在机器上自行检查:
|
拒绝或 CLI 不可用/执行失败时把结果打回并提示改用 `find`/`grep`/`read`/LSP,不影响其他扩展。
|
||||||
|
索引状态可在项目内自行检查:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
codegraph --version
|
|
||||||
codegraph status
|
codegraph status
|
||||||
```
|
```
|
||||||
|
|
||||||
在当前项目需要初始化 CodeGraph 时,可直接执行:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/plugin_init
|
|
||||||
```
|
|
||||||
|
|
||||||
该命令拒绝文件系统根目录和用户主目录,先确认 CLI 可用,再初始化缺失数据库状态。失败时不会热重载,也不会回滚此前已成功完成的初始化;已初始化时仍可用该命令快速热重载扩展。
|
|
||||||
|
|
||||||
### 统一权限系统、硬防御与 AutoReview
|
### 统一权限系统、硬防御与 AutoReview
|
||||||
|
|
||||||
`pi-permission-system/` 从上游 `pi-permission-system-v26.2.1` tag 导入,初始源码快照为 `ec4fdb11343dc94f7185b113e559a4cf9f8dc035`。原 `pi-permission-auto-review` 的 reviewer、可信证据边界和 Codex Guardian 风格策略已整合进同一个扩展;其 MIT 许可和来源记录保留在 `pi-permission-system/LICENSES/` 与 `docs/auto-review/`。根组合包不再加载第二个 AutoReview 扩展。
|
`pi-permission-system/` 从上游 `pi-permission-system-v26.2.1` tag 导入,初始源码快照为 `ec4fdb11343dc94f7185b113e559a4cf9f8dc035`。原 `pi-permission-auto-review` 的 reviewer、可信证据边界和 Codex Guardian 风格策略已整合进同一个扩展;其 MIT 许可和来源记录保留在 `pi-permission-system/LICENSES/` 与 `docs/auto-review/`。根组合包不再加载第二个 AutoReview 扩展。
|
||||||
|
|||||||
@@ -136,11 +136,6 @@
|
|||||||
"wget *": "ask",
|
"wget *": "ask",
|
||||||
"printenv*": "ask"
|
"printenv*": "ask"
|
||||||
},
|
},
|
||||||
"mcp": {
|
|
||||||
"*": "ask",
|
|
||||||
"mcp_status": "allow",
|
|
||||||
"mcp_list": "allow"
|
|
||||||
},
|
|
||||||
"skill": {
|
"skill": {
|
||||||
"*": "allow"
|
"*": "allow"
|
||||||
}
|
}
|
||||||
|
|||||||
Binary file not shown.
@@ -1,57 +0,0 @@
|
|||||||
const EXA_TOOL_ALIASES: Record<string, string> = {
|
|
||||||
web_search_exa: "exa_web_search",
|
|
||||||
web_fetch_exa: "exa_web_fetch",
|
|
||||||
web_search_advanced_exa: "exa_web_search_advanced",
|
|
||||||
};
|
|
||||||
|
|
||||||
export type ExaToolDefinition = {
|
|
||||||
name: string;
|
|
||||||
label: string;
|
|
||||||
[key: string]: unknown;
|
|
||||||
};
|
|
||||||
|
|
||||||
export function mapExaToolName(name: string): string {
|
|
||||||
return EXA_TOOL_ALIASES[name] ?? name;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function prefixExaTool<T extends ExaToolDefinition>(definition: T): T {
|
|
||||||
const name = mapExaToolName(definition.name);
|
|
||||||
if (name === definition.name) return definition;
|
|
||||||
|
|
||||||
const displayName = name
|
|
||||||
.slice("exa_".length)
|
|
||||||
.split("_")
|
|
||||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
|
|
||||||
.join(" ");
|
|
||||||
return {
|
|
||||||
...definition,
|
|
||||||
name,
|
|
||||||
label: `Exa ${displayName}`,
|
|
||||||
} as T;
|
|
||||||
}
|
|
||||||
|
|
||||||
type ToolApi = {
|
|
||||||
registerTool: (tool: unknown) => unknown;
|
|
||||||
setActiveTools: (names: string[]) => unknown;
|
|
||||||
unregisterTool?: (name: string) => boolean;
|
|
||||||
};
|
|
||||||
|
|
||||||
export function createExaToolApiOverride<T extends object>(pi: T): T {
|
|
||||||
const api = pi as T & ToolApi;
|
|
||||||
return new Proxy(pi, {
|
|
||||||
get(target, property) {
|
|
||||||
if (property === "registerTool") {
|
|
||||||
return (definition: ExaToolDefinition) => api.registerTool(prefixExaTool(definition));
|
|
||||||
}
|
|
||||||
if (property === "setActiveTools") {
|
|
||||||
return (names: string[]) => api.setActiveTools([...new Set(names.map(mapExaToolName))]);
|
|
||||||
}
|
|
||||||
if (property === "unregisterTool") {
|
|
||||||
return api.unregisterTool ? (name: string) => api.unregisterTool?.(mapExaToolName(name)) : undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
const value = Reflect.get(target, property, target);
|
|
||||||
return typeof value === "function" ? value.bind(target) : value;
|
|
||||||
},
|
|
||||||
}) as T;
|
|
||||||
}
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
|
|
||||||
export const EXA_TOOLS = ["web_search_exa", "web_fetch_exa", "web_search_advanced_exa"];
|
|
||||||
const EXA_MCP_URL = `https://mcp.exa.ai/mcp?tools=${EXA_TOOLS.join(",")}`;
|
|
||||||
|
|
||||||
export function createMcpConfig(env: Record<string, string | undefined>) {
|
|
||||||
const hasExaApiKey = Boolean(env.EXA_API_KEY?.trim());
|
|
||||||
return {
|
|
||||||
settings: {
|
|
||||||
disableProxyTool: true,
|
|
||||||
scriptMode: false,
|
|
||||||
},
|
|
||||||
mcpServers: {
|
|
||||||
exa: {
|
|
||||||
url: EXA_MCP_URL,
|
|
||||||
...(hasExaApiKey ? { headers: { "x-api-key": "$env:EXA_API_KEY" } } : {}),
|
|
||||||
protocolVersion: "auto" as const,
|
|
||||||
lifecycle: "eager" as const,
|
|
||||||
directTools: EXA_TOOLS,
|
|
||||||
includeTools: EXA_TOOLS,
|
|
||||||
toolPrefix: "none" as const,
|
|
||||||
approveTools: false,
|
|
||||||
exposeResources: false,
|
|
||||||
},
|
|
||||||
codegraph: {
|
|
||||||
command: "codegraph",
|
|
||||||
args: ["serve", "--mcp"],
|
|
||||||
lifecycle: "keep-alive" as const,
|
|
||||||
directTools: ["codegraph_explore"],
|
|
||||||
includeTools: ["codegraph_explore"],
|
|
||||||
toolPrefix: "none" as const,
|
|
||||||
approveTools: false,
|
|
||||||
exposeResources: false,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
||||||
import { createMcpAdapter } from "pi-mcp-adapter";
|
|
||||||
import { createExaToolApiOverride } from "./exa-tool-prefix.ts";
|
|
||||||
import { createMcpConfig } from "./mcp-config.ts";
|
|
||||||
import { loadSearchConfigEnv } from "./search-config.ts";
|
|
||||||
|
|
||||||
/** Connect Pi to Exa and CodeGraph through one shared MCP adapter instance. */
|
|
||||||
export default function mcpExtension(pi: ExtensionAPI): void {
|
|
||||||
const searchEnvironment = loadSearchConfigEnv();
|
|
||||||
const mcpAdapter = createMcpAdapter({ config: createMcpConfig(searchEnvironment) });
|
|
||||||
mcpAdapter(createExaToolApiOverride(pi));
|
|
||||||
}
|
|
||||||
@@ -1,158 +0,0 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
||||||
import { stat } from "node:fs/promises";
|
|
||||||
import { homedir } from "node:os";
|
|
||||||
import { join, parse, resolve } from "node:path";
|
|
||||||
|
|
||||||
interface ExecResult {
|
|
||||||
code: number;
|
|
||||||
stdout: string;
|
|
||||||
stderr: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export type PluginInitExecutor = (
|
|
||||||
command: string,
|
|
||||||
args: string[],
|
|
||||||
options: { cwd: string },
|
|
||||||
) => Promise<ExecResult>;
|
|
||||||
|
|
||||||
interface InitTarget {
|
|
||||||
name: "CodeGraph";
|
|
||||||
command: "codegraph";
|
|
||||||
database: string;
|
|
||||||
initArgs(cwd: string): string[];
|
|
||||||
}
|
|
||||||
|
|
||||||
const INIT_TARGETS: InitTarget[] = [
|
|
||||||
{
|
|
||||||
name: "CodeGraph",
|
|
||||||
command: "codegraph",
|
|
||||||
database: join(".codegraph", "codegraph.db"),
|
|
||||||
initArgs: (cwd) => ["init", cwd],
|
|
||||||
},
|
|
||||||
];
|
|
||||||
|
|
||||||
async function isFile(path: string): Promise<boolean> {
|
|
||||||
try {
|
|
||||||
return (await stat(path)).isFile();
|
|
||||||
} catch (error) {
|
|
||||||
if (typeof error === "object" && error !== null && "code" in error && error.code === "ENOENT") return false;
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export function validatePluginInitCwd(cwd: string): string {
|
|
||||||
const projectRoot = resolve(cwd);
|
|
||||||
if (projectRoot === parse(projectRoot).root) {
|
|
||||||
throw new Error("拒绝在文件系统根目录初始化项目插件。");
|
|
||||||
}
|
|
||||||
if (projectRoot === resolve(homedir())) {
|
|
||||||
throw new Error("拒绝直接在用户主目录初始化项目插件;请先进入具体项目目录。");
|
|
||||||
}
|
|
||||||
return projectRoot;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function findMissingPluginInitializers(cwd: string): Promise<InitTarget[]> {
|
|
||||||
const projectRoot = validatePluginInitCwd(cwd);
|
|
||||||
const states = await Promise.all(
|
|
||||||
INIT_TARGETS.map(async (target) => ({
|
|
||||||
target,
|
|
||||||
initialized: await isFile(join(projectRoot, target.database)),
|
|
||||||
})),
|
|
||||||
);
|
|
||||||
return states.filter(({ initialized }) => !initialized).map(({ target }) => target);
|
|
||||||
}
|
|
||||||
|
|
||||||
function compactFailure(result: ExecResult): string {
|
|
||||||
const output = (result.stderr.trim() || result.stdout.trim()).replace(/\s+/gu, " ");
|
|
||||||
return output ? output.slice(0, 400) : `退出码 ${result.code}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function executeChecked(
|
|
||||||
exec: PluginInitExecutor,
|
|
||||||
target: InitTarget,
|
|
||||||
args: string[],
|
|
||||||
cwd: string,
|
|
||||||
phase: "检测" | "初始化",
|
|
||||||
): Promise<void> {
|
|
||||||
let result: ExecResult;
|
|
||||||
try {
|
|
||||||
result = await exec(target.command, args, { cwd });
|
|
||||||
} catch (error) {
|
|
||||||
const detail = error instanceof Error ? error.message : String(error);
|
|
||||||
throw new Error(`${target.name} ${phase}失败:${detail}`, { cause: error });
|
|
||||||
}
|
|
||||||
if (result.code !== 0) {
|
|
||||||
throw new Error(`${target.name} ${phase}失败:${compactFailure(result)}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function initializeMissingPlugins(
|
|
||||||
cwd: string,
|
|
||||||
targets: InitTarget[],
|
|
||||||
exec: PluginInitExecutor,
|
|
||||||
onStart?: (target: InitTarget) => void,
|
|
||||||
): Promise<void> {
|
|
||||||
const projectRoot = validatePluginInitCwd(cwd);
|
|
||||||
|
|
||||||
// Preflight every CLI before creating project state.
|
|
||||||
for (const target of INIT_TARGETS) {
|
|
||||||
await executeChecked(exec, target, ["--version"], projectRoot, "检测");
|
|
||||||
}
|
|
||||||
const completed: InitTarget[] = [];
|
|
||||||
for (const target of targets) {
|
|
||||||
onStart?.(target);
|
|
||||||
try {
|
|
||||||
await executeChecked(exec, target, target.initArgs(projectRoot), projectRoot, "初始化");
|
|
||||||
completed.push(target);
|
|
||||||
} catch (error) {
|
|
||||||
if (completed.length === 0) throw error;
|
|
||||||
const detail = error instanceof Error ? error.message : String(error);
|
|
||||||
throw new Error(
|
|
||||||
`${detail};已完成的 ${completed.map(({ name }) => name).join("、")} 初始化会保留,不会自动回滚。`,
|
|
||||||
{ cause: error },
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export default function pluginInitExtension(pi: ExtensionAPI): void {
|
|
||||||
pi.registerCommand("plugin_init", {
|
|
||||||
description: "Initialize CodeGraph for the current project, then hot-reload Pi",
|
|
||||||
handler: async (_args, ctx) => {
|
|
||||||
if (!ctx.hasUI) return;
|
|
||||||
|
|
||||||
let projectRoot: string;
|
|
||||||
let missing: InitTarget[];
|
|
||||||
try {
|
|
||||||
projectRoot = validatePluginInitCwd(ctx.cwd);
|
|
||||||
missing = await findMissingPluginInitializers(projectRoot);
|
|
||||||
} catch (error) {
|
|
||||||
ctx.ui.notify(error instanceof Error ? error.message : String(error), "error");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const pending = missing.length > 0 ? missing.map(({ name }) => name).join("、") : "无(均已初始化)";
|
|
||||||
const confirmed = await ctx.ui.confirm(
|
|
||||||
"初始化项目插件",
|
|
||||||
`项目:${projectRoot}\n待初始化:${pending}\n\n完成后将热重载 Pi 扩展。是否继续?`,
|
|
||||||
);
|
|
||||||
if (!confirmed) return;
|
|
||||||
|
|
||||||
try {
|
|
||||||
await initializeMissingPlugins(
|
|
||||||
projectRoot,
|
|
||||||
missing,
|
|
||||||
(command, args, options) => pi.exec(command, args, options),
|
|
||||||
(target) => ctx.ui.notify(`正在初始化 ${target.name}…`, "info"),
|
|
||||||
);
|
|
||||||
ctx.ui.notify(
|
|
||||||
missing.length > 0 ? "项目插件初始化完成,正在热重载 Pi…" : "项目插件均已初始化,正在热重载 Pi…",
|
|
||||||
"info",
|
|
||||||
);
|
|
||||||
await ctx.reload();
|
|
||||||
} catch (error) {
|
|
||||||
ctx.ui.notify(error instanceof Error ? error.message : String(error), "error");
|
|
||||||
}
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
import { readFileSync } from "node:fs";
|
|
||||||
import { homedir } from "node:os";
|
|
||||||
import { join } from "node:path";
|
|
||||||
|
|
||||||
export const SEARCH_API_KEY_NAMES = ["TAVILY_API_KEY", "EXA_API_KEY", "KEENABLE_API_KEY"] as const;
|
|
||||||
|
|
||||||
type SearchApiKeyName = (typeof SEARCH_API_KEY_NAMES)[number];
|
|
||||||
type SearchEnvironment = Record<string, string | undefined>;
|
|
||||||
|
|
||||||
const SAFE_VALUE = /^[A-Za-z0-9._-]+$/;
|
|
||||||
|
|
||||||
export function getSearchConfigPath(env: SearchEnvironment = process.env): string {
|
|
||||||
if (env.MY_PI_SEARCH_CONFIG?.trim()) {
|
|
||||||
return env.MY_PI_SEARCH_CONFIG.trim();
|
|
||||||
}
|
|
||||||
const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config");
|
|
||||||
return join(configHome, "my-pi", "search.env");
|
|
||||||
}
|
|
||||||
|
|
||||||
export function parseSearchConfig(text: string): Partial<Record<SearchApiKeyName, string>> {
|
|
||||||
const parsed: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
for (const rawLine of text.split(/\r?\n/u)) {
|
|
||||||
const line = rawLine.trim().replace(/^export[ \t]+/u, "");
|
|
||||||
if (!line || line.startsWith("#")) continue;
|
|
||||||
const separator = line.indexOf("=");
|
|
||||||
if (separator < 1) continue;
|
|
||||||
const name = line.slice(0, separator).trim() as SearchApiKeyName;
|
|
||||||
const value = line.slice(separator + 1).trim();
|
|
||||||
if (!SEARCH_API_KEY_NAMES.includes(name) || !SAFE_VALUE.test(value)) continue;
|
|
||||||
parsed[name] = value;
|
|
||||||
}
|
|
||||||
return parsed;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Load package-managed search credentials without overriding explicit process environment values. */
|
|
||||||
export function loadSearchConfigEnv(
|
|
||||||
env: SearchEnvironment = process.env,
|
|
||||||
configPath = getSearchConfigPath(env),
|
|
||||||
): Partial<Record<SearchApiKeyName, string>> {
|
|
||||||
let parsed: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
try {
|
|
||||||
parsed = parseSearchConfig(readFileSync(configPath, "utf8"));
|
|
||||||
} catch {
|
|
||||||
// Missing or unreadable config is equivalent to an empty package-managed config.
|
|
||||||
}
|
|
||||||
const effective: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
for (const name of SEARCH_API_KEY_NAMES) {
|
|
||||||
if (!env[name]?.trim() && parsed[name]) env[name] = parsed[name];
|
|
||||||
if (env[name]?.trim()) effective[name] = env[name]?.trim();
|
|
||||||
}
|
|
||||||
return effective;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Load search API keys before provider extensions initialize. */
|
|
||||||
export default function searchConfigExtension(): void {
|
|
||||||
loadSearchConfigEnv();
|
|
||||||
}
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
||||||
import tavilyExtension from "../node_modules/@tavily/pi-extension/index.ts";
|
|
||||||
import { prefixTavilyTool, type TavilyToolDefinition } from "./tavily-tool-prefix.ts";
|
|
||||||
|
|
||||||
|
|
||||||
/** Load Tavily while making the provider explicit in every registered tool name. */
|
|
||||||
export default function tavilyOverrideExtension(pi: ExtensionAPI): void {
|
|
||||||
const prefixedPi = new Proxy(pi, {
|
|
||||||
get(target, property) {
|
|
||||||
if (property === "registerTool") {
|
|
||||||
return (definition: TavilyToolDefinition) =>
|
|
||||||
(target.registerTool as (tool: unknown) => void)(prefixTavilyTool(definition));
|
|
||||||
}
|
|
||||||
|
|
||||||
const value = Reflect.get(target, property, target);
|
|
||||||
return typeof value === "function" ? value.bind(target) : value;
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
tavilyExtension(prefixedPi);
|
|
||||||
}
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
const TAVILY_TOOL_PREFIX = "tavily_";
|
|
||||||
|
|
||||||
export type TavilyToolDefinition = {
|
|
||||||
name: string;
|
|
||||||
label: string;
|
|
||||||
description: string;
|
|
||||||
promptSnippet?: string;
|
|
||||||
promptGuidelines?: string[];
|
|
||||||
[key: string]: unknown;
|
|
||||||
};
|
|
||||||
|
|
||||||
function rewriteToolReferences(text: string): string {
|
|
||||||
return text.replace(/\bweb_search\b/g, "tavily_web_search").replace(/\bweb_fetch\b/g, "tavily_web_fetch");
|
|
||||||
}
|
|
||||||
|
|
||||||
export function prefixTavilyTool<T extends TavilyToolDefinition>(definition: T): T {
|
|
||||||
return {
|
|
||||||
...definition,
|
|
||||||
name: `${TAVILY_TOOL_PREFIX}${definition.name}`,
|
|
||||||
label: definition.label.startsWith("Tavily ") ? definition.label : `Tavily ${definition.label}`,
|
|
||||||
description: rewriteToolReferences(definition.description),
|
|
||||||
promptSnippet: definition.promptSnippet ? rewriteToolReferences(definition.promptSnippet) : undefined,
|
|
||||||
promptGuidelines: definition.promptGuidelines?.map(rewriteToolReferences),
|
|
||||||
} as T;
|
|
||||||
}
|
|
||||||
@@ -62,27 +62,9 @@ export function buildToolRoutingSection(selectedTools: SelectedTools): string {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const tavilyTools = ["tavily_web_search", "tavily_web_fetch"];
|
if (hasAnyTool(selectedTools, ["web_search", "web_fetch"])) {
|
||||||
const exaTools = ["exa_web_search", "exa_web_fetch", "exa_web_search_advanced"];
|
|
||||||
const keenableTools = ["keenable_search", "keenable_fetch"];
|
|
||||||
if (hasAnyTool(selectedTools, [...tavilyTools, ...exaTools, ...keenableTools])) {
|
|
||||||
rules.push(
|
rules.push(
|
||||||
"- Choose one online search provider based on the task instead of querying Tavily, Exa, and Keenable by default; cross-check with another provider only when source quality or uncertainty warrants it.",
|
"- Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Then read selected pages with web_fetch instead of repeating similar searches.",
|
||||||
);
|
|
||||||
}
|
|
||||||
if (hasAnyTool(selectedTools, tavilyTools)) {
|
|
||||||
rules.push(
|
|
||||||
"- Use Tavily tools for broad discovery when keywords are uncertain, for news/recent/trending topics, or when many candidate sources are useful. Tavily results can mix official pages, blogs, forums, and secondary sources, so keep max_results modest, avoid raw content unless needed, and fetch only selected pages when depth matters.",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if (hasAnyTool(selectedTools, exaTools)) {
|
|
||||||
rules.push(
|
|
||||||
"- Use Exa tools when source precision matters: official or API documentation, release/version information, academic papers, research material, and high-quality fact confirmation. Use exa_web_fetch to continue reading selected pages; for exa_web_search_advanced, explicitly bound the result count and returned text length to prevent oversized responses.",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if (hasAnyTool(selectedTools, keenableTools)) {
|
|
||||||
rules.push(
|
|
||||||
"- Use Keenable tools for Chinese-language pages, policies and government notices, site-restricted search, and publication-date filtering. Prefer its compact search results and fetch only selected pages to keep returned content controlled; KEENABLE_API_KEY is optional but raises rate limits.",
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Generated
+28
-805
File diff suppressed because it is too large
Load Diff
+5
-10
@@ -26,6 +26,7 @@
|
|||||||
"pi-tool-search/CHANGELOG.md",
|
"pi-tool-search/CHANGELOG.md",
|
||||||
"pi-tool-search/LICENSE",
|
"pi-tool-search/LICENSE",
|
||||||
"pi-tool-search/UPSTREAM.md",
|
"pi-tool-search/UPSTREAM.md",
|
||||||
|
"pi-web-search",
|
||||||
"pi-worktree",
|
"pi-worktree",
|
||||||
"pi-permission-system/LICENSES",
|
"pi-permission-system/LICENSES",
|
||||||
"pi-permission-system/docs/auto-review",
|
"pi-permission-system/docs/auto-review",
|
||||||
@@ -48,12 +49,9 @@
|
|||||||
],
|
],
|
||||||
"pi": {
|
"pi": {
|
||||||
"extensions": [
|
"extensions": [
|
||||||
"./extensions/search-config.ts",
|
|
||||||
"./extensions/fff-override.ts",
|
"./extensions/fff-override.ts",
|
||||||
"./extensions/tavily-override.ts",
|
"./pi-web-search/index.ts",
|
||||||
"./node_modules/@keenable/pi-search/src/index.ts",
|
"./extensions/codegraph.ts",
|
||||||
"./extensions/mcp.ts",
|
|
||||||
"./extensions/plugin-init.ts",
|
|
||||||
"./pi-rtk-optimizer/index.ts",
|
"./pi-rtk-optimizer/index.ts",
|
||||||
"./extensions/hashline.ts",
|
"./extensions/hashline.ts",
|
||||||
"./node_modules/context-mode/build/adapters/pi/extension.js",
|
"./node_modules/context-mode/build/adapters/pi/extension.js",
|
||||||
@@ -73,17 +71,14 @@
|
|||||||
"./pi-tool-search/extensions/index.ts"
|
"./pi-tool-search/extensions/index.ts"
|
||||||
],
|
],
|
||||||
"skills": [
|
"skills": [
|
||||||
"./node_modules/context-mode/skills",
|
"./node_modules/context-mode/skills"
|
||||||
"./node_modules/@keenable/pi-search/skills"
|
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@ff-labs/pi-fff": "0.10.5",
|
"@ff-labs/pi-fff": "0.10.5",
|
||||||
"@gotgenes/pi-permission-system": "file:./pi-permission-system",
|
"@gotgenes/pi-permission-system": "file:./pi-permission-system",
|
||||||
"@keenable/pi-search": "0.1.2",
|
|
||||||
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
|
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
|
||||||
"@smoose/pi-notify": "file:./pi-notify",
|
"@smoose/pi-notify": "file:./pi-notify",
|
||||||
"@tavily/pi-extension": "0.1.2",
|
|
||||||
"context-mode": "1.0.169",
|
"context-mode": "1.0.169",
|
||||||
"jiti": "2.7.0",
|
"jiti": "2.7.0",
|
||||||
"pi-ask-user": "file:./pi-ask-user",
|
"pi-ask-user": "file:./pi-ask-user",
|
||||||
@@ -92,9 +87,9 @@
|
|||||||
"pi-hashline-edit": "0.8.3",
|
"pi-hashline-edit": "0.8.3",
|
||||||
"pi-hermes-memory": "file:./pi-hermes-memory",
|
"pi-hermes-memory": "file:./pi-hermes-memory",
|
||||||
"pi-lsp": "0.1.7",
|
"pi-lsp": "0.1.7",
|
||||||
"pi-mcp-adapter": "2.26.0",
|
|
||||||
"pi-ssh": "file:./pi-ssh",
|
"pi-ssh": "file:./pi-ssh",
|
||||||
"pi-tool-search": "file:./pi-tool-search",
|
"pi-tool-search": "file:./pi-tool-search",
|
||||||
|
"pi-web-search": "file:./pi-web-search",
|
||||||
"pi-worktree": "file:./pi-worktree",
|
"pi-worktree": "file:./pi-worktree",
|
||||||
"ssh2": "1.17.0",
|
"ssh2": "1.17.0",
|
||||||
"typescript": "6.0.3",
|
"typescript": "6.0.3",
|
||||||
|
|||||||
@@ -32,9 +32,7 @@ Run `/tool-search-status` to inspect the current catalog source, pinned count, d
|
|||||||
| `ssh-remote-search` | `ssh_find`, `ssh_grep` |
|
| `ssh-remote-search` | `ssh_find`, `ssh_grep` |
|
||||||
| `ssh-remote-shell` | `ssh_cd`, `ssh_bash` |
|
| `ssh-remote-shell` | `ssh_cd`, `ssh_bash` |
|
||||||
| `code-intelligence` | CodeGraph and all LSP tools |
|
| `code-intelligence` | CodeGraph and all LSP tools |
|
||||||
| `web-tavily` | Tavily search/fetch |
|
| `web-search` | `web_search`, `web_fetch` (pi-web-search unified tools) |
|
||||||
| `web-exa` | Exa search/advanced/fetch |
|
|
||||||
| `web-keenable` | Keenable search/fetch |
|
|
||||||
| `context-execution` | `ctx_execute`, `ctx_execute_file`, `ctx_batch_execute` |
|
| `context-execution` | `ctx_execute`, `ctx_execute_file`, `ctx_batch_execute` |
|
||||||
| `context-knowledge` | Context indexing, searching, and web-doc ingestion |
|
| `context-knowledge` | Context indexing, searching, and web-doc ingestion |
|
||||||
| `context-observability` | Context stats, doctor, and Insight |
|
| `context-observability` | Context stats, doctor, and Insight |
|
||||||
@@ -55,7 +53,7 @@ Pinned tools are removed from these dynamic groups at runtime. The defaults pin
|
|||||||
Call `tool_search` with an exact catalog group id when possible:
|
Call `tool_search` with an exact catalog group id when possible:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "group": "web-tavily" }
|
{ "group": "web-search" }
|
||||||
```
|
```
|
||||||
|
|
||||||
A workflow can atomically activate up to five exact groups in one call. The complete request must fit the configured group and tool limits; otherwise no active state is changed:
|
A workflow can atomically activate up to five exact groups in one call. The complete request must fit the configured group and tool limits; otherwise no active state is changed:
|
||||||
@@ -114,7 +112,7 @@ Example override:
|
|||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"groupOverrides": {
|
"groupOverrides": {
|
||||||
"web-tavily": ["tavily_web_search", "tavily_web_fetch"],
|
"web-search": ["web_search", "web_fetch"],
|
||||||
"lsp": ["lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"]
|
"lsp": ["lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -77,31 +77,13 @@ export const BUNDLE_GROUP_DEFINITIONS: BundleGroupDefinition[] = [
|
|||||||
tools: ["codegraph_explore", "lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"],
|
tools: ["codegraph_explore", "lsp_definition", "lsp_references", "lsp_hover", "lsp_symbols", "lsp_diagnostics"],
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "web-tavily",
|
id: "web-search",
|
||||||
title: "Tavily web discovery",
|
title: "Web search and fetch",
|
||||||
summary: "Discover broad, current, or news-oriented web sources with Tavily and fetch selected pages.",
|
summary: "Search the web for current or citable information and read selected pages as clean markdown.",
|
||||||
useWhen: ["Keywords are uncertain, many candidate sources help, or current/news coverage is needed"],
|
useWhen: ["Current events, external facts, documentation, papers, or page content are needed"],
|
||||||
avoidWhen: ["Official technical sources or Chinese site/date constraints are more important"],
|
avoidWhen: ["The answer is already in the workspace or local code tools suffice"],
|
||||||
keywords: ["tavily", "web", "news", "discovery", "网页", "新闻", "广泛搜索"],
|
keywords: ["web", "search", "fetch", "news", "docs", "网页", "搜索", "新闻", "文档"],
|
||||||
tools: ["tavily_web_search", "tavily_web_fetch"],
|
tools: ["web_search", "web_fetch"],
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "web-exa",
|
|
||||||
title: "Exa precision research",
|
|
||||||
summary: "Find precise official, technical, academic, company, or API sources with Exa and read selected pages.",
|
|
||||||
useWhen: ["Source precision, official documentation, papers, releases, or advanced filters matter"],
|
|
||||||
avoidWhen: ["Broad news discovery or Chinese policy search is the primary need"],
|
|
||||||
keywords: ["exa", "official docs", "paper", "api", "research", "官方文档", "论文", "精确搜索"],
|
|
||||||
tools: ["exa_web_search", "exa_web_search_advanced", "exa_web_fetch"],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "web-keenable",
|
|
||||||
title: "Keenable focused search",
|
|
||||||
summary: "Search Chinese-language, site-restricted, policy, government, or date-filtered sources and fetch selected pages.",
|
|
||||||
useWhen: ["Chinese pages, policies, government notices, domain restrictions, or publication dates matter"],
|
|
||||||
avoidWhen: ["Broad international discovery or academic semantic search is a better fit"],
|
|
||||||
keywords: ["keenable", "Chinese", "policy", "site", "date", "中文", "政策", "站点", "日期"],
|
|
||||||
tools: ["keenable_search", "keenable_fetch"],
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "context-execution",
|
id: "context-execution",
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ test("bundle seed names are unique and cover the declared my-pi tools", () => {
|
|||||||
assert.equal(result.coveredNames.size, names.length);
|
assert.equal(result.coveredNames.size, names.length);
|
||||||
assert.equal(result.catalog.tools.length, names.length);
|
assert.equal(result.catalog.tools.length, names.length);
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "ctx_purge")?.primaryGroup, "context-administration");
|
assert.equal(result.catalog.tools.find((card) => card.name === "ctx_purge")?.primaryGroup, "context-administration");
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "tavily_web_fetch")?.primaryGroup, "web-tavily");
|
assert.equal(result.catalog.tools.find((card) => card.name === "web_fetch")?.primaryGroup, "web-search");
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "memory_search")?.primaryGroup, "memory-recall");
|
assert.equal(result.catalog.tools.find((card) => card.name === "memory_search")?.primaryGroup, "memory-recall");
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "session_search")?.primaryGroup, "memory-recall");
|
assert.equal(result.catalog.tools.find((card) => card.name === "session_search")?.primaryGroup, "memory-recall");
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "memory_add")?.primaryGroup, "memory-management");
|
assert.equal(result.catalog.tools.find((card) => card.name === "memory_add")?.primaryGroup, "memory-management");
|
||||||
@@ -51,11 +51,11 @@ test("bundle seed names are unique and cover the declared my-pi tools", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("user overrides take priority and unknown tools retain deterministic fallback groups", () => {
|
test("user overrides take priority and unknown tools retain deterministic fallback groups", () => {
|
||||||
const tools = [tool("tavily_web_search"), tool("custom_analyze")];
|
const tools = [tool("web_search"), tool("custom_analyze")];
|
||||||
const configured = { maxToolsPerGroup: 8, groupOverrides: { preferred: ["tavily_web_search"] } };
|
const configured = { maxToolsPerGroup: 8, groupOverrides: { preferred: ["web_search"] } };
|
||||||
const hash = buildManifestHash(tools, configured);
|
const hash = buildManifestHash(tools, configured);
|
||||||
const result = createBundleCatalog(tools, hash, configured);
|
const result = createBundleCatalog(tools, hash, configured);
|
||||||
assert.deepEqual(result.unknownTools.map((item) => item.name), ["custom_analyze"]);
|
assert.deepEqual(result.unknownTools.map((item) => item.name), ["custom_analyze"]);
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "tavily_web_search")?.primaryGroup, "preferred");
|
assert.equal(result.catalog.tools.find((card) => card.name === "web_search")?.primaryGroup, "preferred");
|
||||||
assert.equal(result.catalog.tools.find((card) => card.name === "custom_analyze")?.primaryGroup, "custom");
|
assert.equal(result.catalog.tools.find((card) => card.name === "custom_analyze")?.primaryGroup, "custom");
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -174,8 +174,8 @@ test("builds and caches a validated catalog with the current model on first sear
|
|||||||
const prompts: string[] = [];
|
const prompts: string[] = [];
|
||||||
const sourceTools = [
|
const sourceTools = [
|
||||||
sourceTool("read", "Read a file"),
|
sourceTool("read", "Read a file"),
|
||||||
sourceTool("web_search", "Search the public web with filters and return sources"),
|
sourceTool("research_search", "Search the public web with filters and return sources"),
|
||||||
sourceTool("web_fetch", "Fetch the complete content of a selected web page"),
|
sourceTool("research_fetch", "Fetch the complete content of a selected web page"),
|
||||||
];
|
];
|
||||||
const api = {
|
const api = {
|
||||||
getAllTools: () => sourceTools,
|
getAllTools: () => sourceTools,
|
||||||
@@ -197,12 +197,12 @@ test("builds and caches a validated catalog with the current model on first sear
|
|||||||
summary: "Search and fetch current web sources.",
|
summary: "Search and fetch current web sources.",
|
||||||
useWhen: ["Current external facts are needed"],
|
useWhen: ["Current external facts are needed"],
|
||||||
avoidWhen: ["The answer is entirely local"],
|
avoidWhen: ["The answer is entirely local"],
|
||||||
tools: ["web_search", "web_fetch"],
|
tools: ["research_search", "research_fetch"],
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
tools: [
|
tools: [
|
||||||
{ name: "web_search", summary: "Find web sources.", useWhen: ["Discover sources"], avoidWhen: [], keywords: ["search", "搜索"], primaryGroup: "web-research" },
|
{ name: "research_search", summary: "Find web sources.", useWhen: ["Discover sources"], avoidWhen: [], keywords: ["search", "搜索"], primaryGroup: "web-research" },
|
||||||
{ name: "web_fetch", summary: "Read a selected source.", useWhen: ["A URL is known"], avoidWhen: [], keywords: ["fetch", "抓取"], primaryGroup: "web-research" },
|
{ name: "research_fetch", summary: "Read a selected source.", useWhen: ["A URL is known"], avoidWhen: [], keywords: ["fetch", "抓取"], primaryGroup: "web-research" },
|
||||||
],
|
],
|
||||||
};
|
};
|
||||||
const usage = { input: 1, output: 1, cacheRead: 0, cacheWrite: 0, totalTokens: 2, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } };
|
const usage = { input: 1, output: 1, cacheRead: 0, cacheWrite: 0, totalTokens: 2, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } };
|
||||||
@@ -250,8 +250,8 @@ test("uses the precomputed bundle groups without calling a model", async () => {
|
|||||||
let modelCalls = 0;
|
let modelCalls = 0;
|
||||||
const sourceTools = [
|
const sourceTools = [
|
||||||
sourceTool("read", "Read a file"),
|
sourceTool("read", "Read a file"),
|
||||||
sourceTool("tavily_web_search", "Search broadly for current web sources"),
|
sourceTool("web_search", "Search broadly for current web sources"),
|
||||||
sourceTool("tavily_web_fetch", "Fetch a selected Tavily source"),
|
sourceTool("web_fetch", "Fetch a selected web source"),
|
||||||
sourceTool("ctx_execute", "Run commands in a context sandbox"),
|
sourceTool("ctx_execute", "Run commands in a context sandbox"),
|
||||||
sourceTool("ctx_execute_file", "Analyze a file in a context sandbox"),
|
sourceTool("ctx_execute_file", "Analyze a file in a context sandbox"),
|
||||||
sourceTool("ctx_batch_execute", "Run a bounded command batch"),
|
sourceTool("ctx_batch_execute", "Run a bounded command batch"),
|
||||||
@@ -292,12 +292,12 @@ test("uses the precomputed bundle groups without calling a model", async () => {
|
|||||||
|
|
||||||
const result = await registered.get("tool_search")?.execute(
|
const result = await registered.get("tool_search")?.execute(
|
||||||
"call-2",
|
"call-2",
|
||||||
{ query: "tavily_web_search" },
|
{ query: "web_search" },
|
||||||
undefined,
|
undefined,
|
||||||
undefined,
|
undefined,
|
||||||
ctx,
|
ctx,
|
||||||
);
|
);
|
||||||
assert.equal(result?.details.loadedGroup, "web-tavily");
|
assert.equal(result?.details.loadedGroup, "web-search");
|
||||||
assert.equal(result?.details.catalogSource, "bundle");
|
assert.equal(result?.details.catalogSource, "bundle");
|
||||||
assert.equal(modelCalls, 0);
|
assert.equal(modelCalls, 0);
|
||||||
await assert.rejects(readFile(join(agentDir, "my-pi", "tool-search", "catalog-v1.json"), "utf8"), /ENOENT/);
|
await assert.rejects(readFile(join(agentDir, "my-pi", "tool-search", "catalog-v1.json"), "utf8"), /ENOENT/);
|
||||||
@@ -337,8 +337,8 @@ test("loads an exact known group without waiting for unknown-tool model enrichme
|
|||||||
let modelCalls = 0;
|
let modelCalls = 0;
|
||||||
const sourceTools = [
|
const sourceTools = [
|
||||||
sourceTool("read", "Read a file"),
|
sourceTool("read", "Read a file"),
|
||||||
sourceTool("tavily_web_search", "Search broadly for current web sources"),
|
sourceTool("web_search", "Search broadly for current web sources"),
|
||||||
sourceTool("tavily_web_fetch", "Fetch a selected Tavily source"),
|
sourceTool("web_fetch", "Fetch a selected web source"),
|
||||||
sourceTool("unrecognized_extra", "An unrecognized third-party capability"),
|
sourceTool("unrecognized_extra", "An unrecognized third-party capability"),
|
||||||
];
|
];
|
||||||
const api = {
|
const api = {
|
||||||
@@ -366,12 +366,12 @@ test("loads an exact known group without waiting for unknown-tool model enrichme
|
|||||||
|
|
||||||
const result = await registered.get("tool_search")?.execute(
|
const result = await registered.get("tool_search")?.execute(
|
||||||
"call-1",
|
"call-1",
|
||||||
{ group: "web-tavily" },
|
{ group: "web-search" },
|
||||||
undefined,
|
undefined,
|
||||||
undefined,
|
undefined,
|
||||||
ctx,
|
ctx,
|
||||||
);
|
);
|
||||||
assert.equal(result?.details.loadedGroup, "web-tavily");
|
assert.equal(result?.details.loadedGroup, "web-search");
|
||||||
assert.equal(result?.details.catalogSource, "hybrid");
|
assert.equal(result?.details.catalogSource, "hybrid");
|
||||||
assert.equal(modelCalls, 0);
|
assert.equal(modelCalls, 0);
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 sia
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# pi-web-search
|
||||||
|
|
||||||
|
Unified web search and fetch for Pi. The main model only ever sees two tools —
|
||||||
|
`web_search` and `web_fetch` — provider selection happens inside the extension.
|
||||||
|
|
||||||
|
- `web_search(query, max_results?, site?, published_after?, published_before?, depth?)`
|
||||||
|
routes each query to Tavily, Exa, or Keenable using the session's current main
|
||||||
|
model (`ctx.model` + `ctx.modelRegistry.complete`), guided by per-provider
|
||||||
|
rules. Results include `details.provider` / `details.routedBy` for debugging.
|
||||||
|
- `web_fetch(urls, max_characters?)` always reads pages via Exa
|
||||||
|
(`web_fetch_exa`), degrading silently to Keenable then Tavily.
|
||||||
|
|
||||||
|
## Providers (`clients/`)
|
||||||
|
|
||||||
|
Plain `fetch` calls — no MCP adapter, no upstream extension wrappers:
|
||||||
|
|
||||||
|
- `clients/tavily.ts` — `POST api.tavily.com/search|extract`, Bearer key.
|
||||||
|
Requires a `tavily` credential.
|
||||||
|
- `clients/keenable.ts` — `api.keenable.ai`, keyless `/public` endpoints by
|
||||||
|
default, a `keenable` credential raises rate limits. Adapted from
|
||||||
|
`@keenable/pi-search`.
|
||||||
|
- `clients/exa.ts` — JSON-RPC over `mcp.exa.ai/mcp` (Streamable HTTP):
|
||||||
|
`initialize` once, cache `mcp-session-id`, `tools/call` per request, re-init
|
||||||
|
on session expiry. An `exa` credential is optional (`x-api-key` header).
|
||||||
|
|
||||||
|
Keys live in Pi's `auth.json` as one `api_key` credential per provider
|
||||||
|
(`tavily`, `exa`, `keenable`), read by `auth.ts` with upstream resolution
|
||||||
|
semantics — literal values, `$VAR`/`${VAR}` interpolation, and `!command`
|
||||||
|
indirection (e.g. `"!op read op://Vault/Tavily/key"`). `search_config.sh`
|
||||||
|
writes them (atomically, mode 600, preserving all other entries); process
|
||||||
|
environment variables are not read directly — use `"key": "$TAVILY_API_KEY"`
|
||||||
|
to indirect through env. `KEENABLE_API_URL` stays env-only, or can be stored
|
||||||
|
in the keenable credential's `env` map.
|
||||||
|
|
||||||
|
## Routing
|
||||||
|
|
||||||
|
`router.ts` asks the current model to answer with exactly one provider name,
|
||||||
|
given a prompt containing the per-provider strengths and the query's requested
|
||||||
|
filters. Decisions are cached per session (LRU, 32 entries). Failure paths are
|
||||||
|
deterministic: an unroutable answer or a dead provider falls back across the
|
||||||
|
remaining available providers — never a heuristic choice over the model.
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
/**
|
||||||
|
* Search provider API keys live in Pi's own auth.json — one `api_key`
|
||||||
|
* credential per provider id (`tavily`, `exa`, `keenable`) next to Pi's
|
||||||
|
* model credentials. Key values follow upstream resolve-config-value
|
||||||
|
* semantics (replicated here because the upstream resolver is not exported):
|
||||||
|
*
|
||||||
|
* - `!command` executes the rest of the string and uses trimmed stdout,
|
||||||
|
* cached for the process lifetime (same as upstream's command cache)
|
||||||
|
* - `$NAME` / `${NAME}` interpolate environment variables; a credential-level
|
||||||
|
* `env` map wins over process env; a missing variable resolves to undefined
|
||||||
|
* - `$$` and `$!` escape to literal `$` and `!`
|
||||||
|
* - anything else is a literal key
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { execSync } from "node:child_process";
|
||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
|
||||||
|
export type SearchAuthProvider = "tavily" | "exa" | "keenable";
|
||||||
|
export const SEARCH_AUTH_PROVIDERS: readonly SearchAuthProvider[] = ["tavily", "exa", "keenable"];
|
||||||
|
|
||||||
|
/** Resolves a provider's API key; injectable so tests never touch real auth.json. */
|
||||||
|
export type SearchKeyResolver = (provider: SearchAuthProvider) => string | undefined;
|
||||||
|
|
||||||
|
export type SearchAuthEnvironment = Record<string, string | undefined>;
|
||||||
|
|
||||||
|
export interface SearchAuthOptions {
|
||||||
|
/** auth.json location; defaults to `<agent dir>/auth.json`. */
|
||||||
|
authPath?: string;
|
||||||
|
/**
|
||||||
|
* Environment used for the agent dir and `$VAR` interpolation. When
|
||||||
|
* provided it fully replaces `process.env` so tests stay isolated.
|
||||||
|
*/
|
||||||
|
env?: SearchAuthEnvironment;
|
||||||
|
/** Executes `!command` key references; defaults to a cached execSync runner. */
|
||||||
|
runCommand?: (command: string) => string | undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SearchApiKeyCredential {
|
||||||
|
type: "api_key";
|
||||||
|
key?: string;
|
||||||
|
env?: Record<string, string>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COMMAND_TIMEOUT_MS = 10_000;
|
||||||
|
const ENV_VAR_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/;
|
||||||
|
const ENV_VAR_NAME_PREFIX = /^[A-Za-z_][A-Za-z0-9_]*/;
|
||||||
|
const commandResultCache = new Map<string, string | undefined>();
|
||||||
|
|
||||||
|
/** auth.json lives in Pi's agent dir: $PI_CODING_AGENT_DIR or ~/.pi/agent. */
|
||||||
|
export function getSearchAuthPath(env: SearchAuthEnvironment = process.env): string {
|
||||||
|
const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent");
|
||||||
|
return join(agentDir, "auth.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Read the raw `api_key` credential for a provider; undefined when absent or malformed. */
|
||||||
|
export function readSearchCredential(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): SearchApiKeyCredential | undefined {
|
||||||
|
let data: unknown;
|
||||||
|
try {
|
||||||
|
data = JSON.parse(readFileSync(options.authPath ?? getSearchAuthPath(options.env), "utf8"));
|
||||||
|
} catch {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (!data || typeof data !== "object" || Array.isArray(data)) return undefined;
|
||||||
|
const entry = (data as Record<string, unknown>)[provider];
|
||||||
|
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return undefined;
|
||||||
|
const credential = entry as Record<string, unknown>;
|
||||||
|
if (credential.type !== "api_key") return undefined;
|
||||||
|
if (credential.key !== undefined && typeof credential.key !== "string") return undefined;
|
||||||
|
const env = credential.env;
|
||||||
|
const scopedEnv =
|
||||||
|
env && typeof env === "object" && !Array.isArray(env)
|
||||||
|
? Object.fromEntries(
|
||||||
|
Object.entries(env as Record<string, unknown>).filter(
|
||||||
|
(pair): pair is [string, string] => typeof pair[1] === "string",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
: undefined;
|
||||||
|
return { type: "api_key", key: credential.key as string | undefined, env: scopedEnv };
|
||||||
|
}
|
||||||
|
|
||||||
|
type TemplatePart = { type: "literal"; value: string } | { type: "env"; name: string };
|
||||||
|
|
||||||
|
/** Mirrors upstream parseConfigValueTemplate: `$$`/`$!` escape, `${A}`/`$A` interpolate. */
|
||||||
|
export function parseAuthKeyTemplate(config: string): TemplatePart[] {
|
||||||
|
const parts: TemplatePart[] = [];
|
||||||
|
const appendLiteral = (value: string) => {
|
||||||
|
if (!value) return;
|
||||||
|
const previous = parts[parts.length - 1];
|
||||||
|
if (previous?.type === "literal") {
|
||||||
|
previous.value += value;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
parts.push({ type: "literal", value });
|
||||||
|
};
|
||||||
|
let index = 0;
|
||||||
|
while (index < config.length) {
|
||||||
|
const dollarIndex = config.indexOf("$", index);
|
||||||
|
if (dollarIndex < 0) {
|
||||||
|
appendLiteral(config.slice(index));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
appendLiteral(config.slice(index, dollarIndex));
|
||||||
|
const nextChar = config[dollarIndex + 1];
|
||||||
|
if (nextChar === "$" || nextChar === "!") {
|
||||||
|
appendLiteral(nextChar);
|
||||||
|
index = dollarIndex + 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (nextChar === "{") {
|
||||||
|
const endIndex = config.indexOf("}", dollarIndex + 2);
|
||||||
|
if (endIndex < 0) {
|
||||||
|
appendLiteral("$");
|
||||||
|
index = dollarIndex + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const name = config.slice(dollarIndex + 2, endIndex);
|
||||||
|
if (ENV_VAR_NAME.test(name)) {
|
||||||
|
parts.push({ type: "env", name });
|
||||||
|
} else {
|
||||||
|
appendLiteral(config.slice(dollarIndex, endIndex + 1));
|
||||||
|
}
|
||||||
|
index = endIndex + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const match = config.slice(dollarIndex + 1).match(ENV_VAR_NAME_PREFIX);
|
||||||
|
if (match) {
|
||||||
|
parts.push({ type: "env", name: match[0] });
|
||||||
|
index = dollarIndex + 1 + match[0].length;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
appendLiteral("$");
|
||||||
|
index = dollarIndex + 1;
|
||||||
|
}
|
||||||
|
return parts;
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultRunCommand(command: string): string | undefined {
|
||||||
|
if (commandResultCache.has(command)) return commandResultCache.get(command);
|
||||||
|
let value: string | undefined;
|
||||||
|
try {
|
||||||
|
const output = execSync(command, {
|
||||||
|
encoding: "utf-8",
|
||||||
|
timeout: COMMAND_TIMEOUT_MS,
|
||||||
|
stdio: ["ignore", "pipe", "ignore"],
|
||||||
|
});
|
||||||
|
value = output.trim() || undefined;
|
||||||
|
} catch {
|
||||||
|
value = undefined;
|
||||||
|
}
|
||||||
|
commandResultCache.set(command, value);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Mirrors upstream resolveConfigValue, including credential-scoped env precedence. */
|
||||||
|
export function resolveAuthKeyValue(
|
||||||
|
config: string,
|
||||||
|
credentialEnv: Record<string, string> | undefined,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
if (config.startsWith("!")) {
|
||||||
|
return (options.runCommand ?? defaultRunCommand)(config.slice(1));
|
||||||
|
}
|
||||||
|
const ambient = options.env ?? process.env;
|
||||||
|
let resolved = "";
|
||||||
|
for (const part of parseAuthKeyTemplate(config)) {
|
||||||
|
if (part.type === "literal") {
|
||||||
|
resolved += part.value;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const value = credentialEnv?.[part.name] || ambient[part.name];
|
||||||
|
if (value === undefined) return undefined;
|
||||||
|
resolved += value;
|
||||||
|
}
|
||||||
|
return resolved;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Provider API key from auth.json, or undefined when not configured. */
|
||||||
|
export function getSearchApiKey(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
const credential = readSearchCredential(provider, options);
|
||||||
|
if (!credential?.key) return undefined;
|
||||||
|
const resolved = resolveAuthKeyValue(credential.key, credential.env, options);
|
||||||
|
return resolved?.trim() || undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A credential-scoped config value (e.g. KEENABLE_API_URL), or undefined. */
|
||||||
|
export function getSearchCredentialEnv(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
name: string,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
const value = readSearchCredential(provider, options)?.env?.[name]?.trim();
|
||||||
|
return value || undefined;
|
||||||
|
}
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
/**
|
||||||
|
* Exa client: direct JSON-RPC over the hosted endpoint https://mcp.exa.ai/mcp.
|
||||||
|
* This is a plain HTTPS POST client (no MCP adapter, no SDK): initialize once,
|
||||||
|
* cache the `mcp-session-id` response header, then POST `tools/call` per call.
|
||||||
|
* Keyless works; the `exa` api_key credential from auth.json is sent as the
|
||||||
|
* `x-api-key` header when present.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey } from "../auth.ts";
|
||||||
|
|
||||||
|
export class ExaError extends Error {}
|
||||||
|
|
||||||
|
const EXA_MCP_URL = "https://mcp.exa.ai/mcp?tools=web_search_exa,web_search_advanced_exa,web_fetch_exa";
|
||||||
|
const PROTOCOL_VERSION = "2025-03-26";
|
||||||
|
const SESSION_TIMEOUT_MS = 15_000;
|
||||||
|
const CALL_TIMEOUT_MS = 60_000;
|
||||||
|
|
||||||
|
/** Cap page text per result so advanced search responses stay bounded. */
|
||||||
|
export const EXA_SEARCH_TEXT_MAX_CHARACTERS = 2000;
|
||||||
|
export const EXA_FETCH_MAX_CHARACTERS = 3000;
|
||||||
|
|
||||||
|
export type FetchLike = (input: string | URL, init?: RequestInit) => Promise<Response>;
|
||||||
|
|
||||||
|
export interface ExaSearchParams {
|
||||||
|
query: string;
|
||||||
|
numResults?: number;
|
||||||
|
includeDomains?: string[];
|
||||||
|
startPublishedDate?: string;
|
||||||
|
endPublishedDate?: string;
|
||||||
|
textMaxCharacters?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
let cachedSessionId: string | undefined;
|
||||||
|
let pendingSession: Promise<string> | undefined;
|
||||||
|
let nextRequestId = 1;
|
||||||
|
|
||||||
|
/** Clear the cached MCP session (used on expiry and by tests). */
|
||||||
|
export function resetExaSession(): void {
|
||||||
|
cachedSessionId = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function baseHeaders(): Record<string, string> {
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
"content-type": "application/json",
|
||||||
|
accept: "application/json, text/event-stream",
|
||||||
|
};
|
||||||
|
const key = getSearchApiKey("exa");
|
||||||
|
if (key) headers["x-api-key"] = key;
|
||||||
|
return headers;
|
||||||
|
}
|
||||||
|
|
||||||
|
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||||
|
const timeout = AbortSignal.timeout(timeoutMs);
|
||||||
|
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Extract the JSON-RPC message for `id` from a raw JSON body or an SSE stream
|
||||||
|
* (one or more `data: {...}` lines).
|
||||||
|
*/
|
||||||
|
export function parseRpcPayload(bodyText: string, id: number): Record<string, unknown> {
|
||||||
|
const candidates: string[] = [];
|
||||||
|
const trimmed = bodyText.trim();
|
||||||
|
if (trimmed.startsWith("{")) candidates.push(trimmed);
|
||||||
|
for (const line of trimmed.split(/\r?\n/u)) {
|
||||||
|
if (line.startsWith("data:")) candidates.push(line.slice(5).trim());
|
||||||
|
}
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(candidate) as Record<string, unknown>;
|
||||||
|
if (parsed && typeof parsed === "object" && !Array.isArray(parsed) && parsed.id === id) {
|
||||||
|
return parsed;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Not JSON — try the next candidate line.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new ExaError("Exa endpoint returned an unreadable response.");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function post(
|
||||||
|
body: Record<string, unknown>,
|
||||||
|
sessionId: string | undefined,
|
||||||
|
timeoutMs: number,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
fetchImpl: FetchLike,
|
||||||
|
): Promise<{ status: number; sessionId?: string; text: string }> {
|
||||||
|
const headers = baseHeaders();
|
||||||
|
if (sessionId) headers["mcp-session-id"] = sessionId;
|
||||||
|
let response: Response;
|
||||||
|
try {
|
||||||
|
response = await fetchImpl(EXA_MCP_URL, {
|
||||||
|
method: "POST",
|
||||||
|
headers,
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
signal: withTimeout(signal, timeoutMs),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
throw new ExaError(`Could not reach the Exa endpoint: ${(error as Error).message}`);
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
status: response.status,
|
||||||
|
sessionId: response.headers.get("mcp-session-id") ?? undefined,
|
||||||
|
text: await response.text(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function initialize(signal: AbortSignal | undefined, fetchImpl: FetchLike): Promise<string> {
|
||||||
|
const id = nextRequestId++;
|
||||||
|
const res = await post(
|
||||||
|
{
|
||||||
|
jsonrpc: "2.0",
|
||||||
|
id,
|
||||||
|
method: "initialize",
|
||||||
|
params: {
|
||||||
|
protocolVersion: PROTOCOL_VERSION,
|
||||||
|
capabilities: {},
|
||||||
|
clientInfo: { name: "my-pi-web-search", version: "0.1.0" },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
undefined,
|
||||||
|
SESSION_TIMEOUT_MS,
|
||||||
|
signal,
|
||||||
|
fetchImpl,
|
||||||
|
);
|
||||||
|
if (res.status !== 200) throw new ExaError(`Exa initialize failed (HTTP ${res.status}).`);
|
||||||
|
const sessionId = res.sessionId;
|
||||||
|
if (!sessionId) throw new ExaError("Exa endpoint did not return an MCP session id.");
|
||||||
|
const payload = parseRpcPayload(res.text, id);
|
||||||
|
if (payload.error) throw new ExaError(`Exa initialize error: ${(payload.error as { message?: string }).message ?? "unknown"}`);
|
||||||
|
|
||||||
|
const notice = await post(
|
||||||
|
{ jsonrpc: "2.0", method: "notifications/initialized" },
|
||||||
|
sessionId,
|
||||||
|
SESSION_TIMEOUT_MS,
|
||||||
|
signal,
|
||||||
|
fetchImpl,
|
||||||
|
);
|
||||||
|
if (notice.status >= 400) {
|
||||||
|
throw new ExaError(`Exa initialized notification failed (HTTP ${notice.status}).`);
|
||||||
|
}
|
||||||
|
return sessionId;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sessionId(signal: AbortSignal | undefined, fetchImpl: FetchLike): Promise<string> {
|
||||||
|
if (cachedSessionId) return Promise.resolve(cachedSessionId);
|
||||||
|
pendingSession ??= initialize(signal, fetchImpl)
|
||||||
|
.then((id) => {
|
||||||
|
cachedSessionId = id;
|
||||||
|
return id;
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
pendingSession = undefined;
|
||||||
|
});
|
||||||
|
return pendingSession;
|
||||||
|
}
|
||||||
|
|
||||||
|
function contentText(result: Record<string, unknown>): string {
|
||||||
|
const content = Array.isArray(result.content) ? (result.content as Record<string, unknown>[]) : [];
|
||||||
|
return content
|
||||||
|
.filter((part) => part?.type === "text" && typeof part.text === "string")
|
||||||
|
.map((part) => part.text as string)
|
||||||
|
.join("\n")
|
||||||
|
.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function callTool(
|
||||||
|
toolName: string,
|
||||||
|
args: Record<string, unknown>,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
fetchImpl: FetchLike,
|
||||||
|
allowRetry: boolean,
|
||||||
|
): Promise<Record<string, unknown>> {
|
||||||
|
const sid = await sessionId(signal, fetchImpl);
|
||||||
|
const id = nextRequestId++;
|
||||||
|
const res = await post(
|
||||||
|
{ jsonrpc: "2.0", id, method: "tools/call", params: { name: toolName, arguments: args } },
|
||||||
|
sid,
|
||||||
|
CALL_TIMEOUT_MS,
|
||||||
|
signal,
|
||||||
|
fetchImpl,
|
||||||
|
);
|
||||||
|
if (res.status === 404 && allowRetry) {
|
||||||
|
// Session expired server-side: drop it, re-initialize, retry once.
|
||||||
|
cachedSessionId = undefined;
|
||||||
|
return callTool(toolName, args, signal, fetchImpl, false);
|
||||||
|
}
|
||||||
|
if (res.status !== 200) {
|
||||||
|
throw new ExaError(`Exa ${toolName} failed (HTTP ${res.status}).`);
|
||||||
|
}
|
||||||
|
const payload = parseRpcPayload(res.text, id);
|
||||||
|
if (payload.error) {
|
||||||
|
const message = (payload.error as { message?: string }).message ?? "unknown";
|
||||||
|
if (allowRetry && /session/iu.test(message)) {
|
||||||
|
cachedSessionId = undefined;
|
||||||
|
return callTool(toolName, args, signal, fetchImpl, false);
|
||||||
|
}
|
||||||
|
throw new ExaError(`Exa ${toolName} error: ${message}`);
|
||||||
|
}
|
||||||
|
const result = (payload.result ?? {}) as Record<string, unknown>;
|
||||||
|
if (result.isError) {
|
||||||
|
throw new ExaError(`Exa ${toolName} failed: ${contentText(result) || "unknown error"}`);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** tools/call web_search_advanced_exa — precise search with bounded text per result. */
|
||||||
|
export async function exaSearch(
|
||||||
|
params: ExaSearchParams,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<string> {
|
||||||
|
const result = await callTool(
|
||||||
|
"web_search_advanced_exa",
|
||||||
|
{
|
||||||
|
query: params.query,
|
||||||
|
type: "auto",
|
||||||
|
numResults: params.numResults ?? 8,
|
||||||
|
textMaxCharacters: params.textMaxCharacters ?? EXA_SEARCH_TEXT_MAX_CHARACTERS,
|
||||||
|
...(params.includeDomains?.length ? { includeDomains: params.includeDomains } : {}),
|
||||||
|
...(params.startPublishedDate ? { startPublishedDate: params.startPublishedDate } : {}),
|
||||||
|
...(params.endPublishedDate ? { endPublishedDate: params.endPublishedDate } : {}),
|
||||||
|
},
|
||||||
|
signal,
|
||||||
|
fetchImpl,
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
return contentText(result) || "Exa returned no content.";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** tools/call web_fetch_exa — full page content as clean markdown for a batch of URLs. */
|
||||||
|
export async function exaFetchPages(
|
||||||
|
urls: string[],
|
||||||
|
maxCharacters?: number,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<string> {
|
||||||
|
const result = await callTool(
|
||||||
|
"web_fetch_exa",
|
||||||
|
{ urls, maxCharacters: maxCharacters ?? EXA_FETCH_MAX_CHARACTERS },
|
||||||
|
signal,
|
||||||
|
fetchImpl,
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
return contentText(result) || "Exa returned no content.";
|
||||||
|
}
|
||||||
@@ -0,0 +1,229 @@
|
|||||||
|
/**
|
||||||
|
* Keenable REST client, adapted from @keenable/pi-search (keenable-client.ts).
|
||||||
|
* Keyless by default (`/v1/*\/public`), authenticated when the `keenable`
|
||||||
|
* api_key credential exists in auth.json (`/v1/*`), HTTPS-only base URL,
|
||||||
|
* `X-Keenable-Title` attribution.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { isIP } from "node:net";
|
||||||
|
|
||||||
|
import { getSearchApiKey, getSearchCredentialEnv } from "../auth.ts";
|
||||||
|
|
||||||
|
export class KeenableError extends Error {}
|
||||||
|
|
||||||
|
const CLIENT_VERSION = "0.1.0";
|
||||||
|
const DEFAULT_BASE_URL = "https://api.keenable.ai";
|
||||||
|
const BASE_URL_ENV = "KEENABLE_API_URL";
|
||||||
|
const ATTRIBUTION_TITLE = "Pi";
|
||||||
|
const USER_AGENT = `pi-web-search/${CLIENT_VERSION}`;
|
||||||
|
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||||
|
|
||||||
|
export type FetchLike = (input: string | URL, init?: RequestInit) => Promise<Response>;
|
||||||
|
|
||||||
|
export interface KeenableSearchParams {
|
||||||
|
query: string;
|
||||||
|
mode?: "pro";
|
||||||
|
site?: string;
|
||||||
|
published_after?: string;
|
||||||
|
published_before?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface KeenableSearchResult {
|
||||||
|
title?: string;
|
||||||
|
url?: string;
|
||||||
|
description?: string;
|
||||||
|
published_at?: string;
|
||||||
|
acquired_at?: string;
|
||||||
|
[key: string]: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface KeenableFetchResult {
|
||||||
|
url?: string;
|
||||||
|
title?: string;
|
||||||
|
content?: string;
|
||||||
|
description?: string;
|
||||||
|
author?: string;
|
||||||
|
published_at?: string;
|
||||||
|
[key: string]: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolve the API base URL from `KEENABLE_API_URL` (process env first, then
|
||||||
|
* the keenable credential's `env` map), enforcing HTTPS (http only for loopback).
|
||||||
|
*/
|
||||||
|
function resolveBaseUrl(): string {
|
||||||
|
const base = (process.env[BASE_URL_ENV] || getSearchCredentialEnv("keenable", BASE_URL_ENV) || DEFAULT_BASE_URL).replace(
|
||||||
|
/\/+$/u,
|
||||||
|
"",
|
||||||
|
);
|
||||||
|
let parsed: URL;
|
||||||
|
try {
|
||||||
|
parsed = new URL(base);
|
||||||
|
} catch {
|
||||||
|
throw new KeenableError(`${BASE_URL_ENV} must be a valid URL with a host, got ${base}`);
|
||||||
|
}
|
||||||
|
if (parsed.hostname) {
|
||||||
|
if (parsed.protocol === "https:") return base;
|
||||||
|
if (parsed.protocol === "http:" && ["localhost", "127.0.0.1", "::1"].includes(parsed.hostname)) {
|
||||||
|
return base;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw new KeenableError(`${BASE_URL_ENV} must be an https:// URL with a host, got ${base}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** API key from auth.json, or undefined to use the keyless public tier. */
|
||||||
|
function resolveApiKey(): string | undefined {
|
||||||
|
return getSearchApiKey("keenable");
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildHeaders(apiKey: string | undefined): Record<string, string> {
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
Accept: "application/json",
|
||||||
|
"User-Agent": USER_AGENT,
|
||||||
|
"X-Keenable-Title": ATTRIBUTION_TITLE,
|
||||||
|
};
|
||||||
|
if (apiKey) headers["X-API-Key"] = apiKey;
|
||||||
|
return headers;
|
||||||
|
}
|
||||||
|
|
||||||
|
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||||
|
const timeout = AbortSignal.timeout(timeoutMs);
|
||||||
|
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isPrivateIp(host: string): boolean {
|
||||||
|
const v = isIP(host);
|
||||||
|
if (v === 4) {
|
||||||
|
const p = host.split(".").map(Number);
|
||||||
|
return (
|
||||||
|
p[0] === 10 ||
|
||||||
|
p[0] === 127 ||
|
||||||
|
p[0] === 0 ||
|
||||||
|
(p[0] === 169 && p[1] === 254) ||
|
||||||
|
(p[0] === 172 && p[1] >= 16 && p[1] <= 31) ||
|
||||||
|
(p[0] === 192 && p[1] === 168) ||
|
||||||
|
(p[0] === 100 && p[1] >= 64 && p[1] <= 127) ||
|
||||||
|
(p[0] === 198 && (p[1] === 18 || p[1] === 19)) ||
|
||||||
|
p[0] >= 224
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (v === 6) {
|
||||||
|
const h = host.toLowerCase();
|
||||||
|
return h === "::1" || h === "::" || h.startsWith("fe80") || h.startsWith("fc") || h.startsWith("fd");
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Refuse private/internal fetch targets before sending (SSRF). */
|
||||||
|
function assertPublicFetchTarget(rawUrl: string): void {
|
||||||
|
let parsed: URL;
|
||||||
|
try {
|
||||||
|
parsed = new URL(rawUrl);
|
||||||
|
} catch {
|
||||||
|
throw new KeenableError(`Not a valid URL: ${rawUrl}`);
|
||||||
|
}
|
||||||
|
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
|
||||||
|
throw new KeenableError(`Refusing to fetch a non-http(s) URL: ${rawUrl}`);
|
||||||
|
}
|
||||||
|
const host = (parsed.hostname || "").toLowerCase();
|
||||||
|
if (!host) throw new KeenableError(`Refusing to fetch a URL with no host: ${rawUrl}`);
|
||||||
|
if (host === "localhost" || host === "metadata.google.internal" || isPrivateIp(host)) {
|
||||||
|
throw new KeenableError(`Refusing to fetch a private/internal host: ${host}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function raiseForStatus(response: Response, keyed: boolean): Promise<void> {
|
||||||
|
if (response.ok) return;
|
||||||
|
if (!keyed && response.status === 429) {
|
||||||
|
throw new KeenableError(
|
||||||
|
'Keenable keyless requests hit their rate limit. Add a "keenable" api_key credential to auth.json to raise the limits.',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
let detail = "";
|
||||||
|
try {
|
||||||
|
const body = (await response.json()) as Record<string, unknown>;
|
||||||
|
detail = String(body.message ?? body.error ?? body.detail ?? "");
|
||||||
|
} catch {
|
||||||
|
detail = (await response.text().catch(() => "")).trim();
|
||||||
|
}
|
||||||
|
const label =
|
||||||
|
{
|
||||||
|
401: "Keenable authentication failed (401)",
|
||||||
|
402: "Keenable: insufficient credits (402)",
|
||||||
|
429: "Keenable rate limit exceeded (429)",
|
||||||
|
}[response.status] ?? `Keenable API error (${response.status})`;
|
||||||
|
throw new KeenableError(detail ? `${label}: ${detail}` : label);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readJsonObject(response: Response): Promise<Record<string, unknown>> {
|
||||||
|
let data: unknown;
|
||||||
|
try {
|
||||||
|
data = await response.json();
|
||||||
|
} catch {
|
||||||
|
throw new KeenableError("Keenable API returned a non-JSON response.");
|
||||||
|
}
|
||||||
|
if (!data || typeof data !== "object" || Array.isArray(data)) {
|
||||||
|
throw new KeenableError("Unexpected (non-object) response from the Keenable API.");
|
||||||
|
}
|
||||||
|
return data as Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** POST /v1/search (keyed) or /v1/search/public (keyless). */
|
||||||
|
export async function keenableSearch(
|
||||||
|
params: KeenableSearchParams,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<KeenableSearchResult[]> {
|
||||||
|
const apiKey = resolveApiKey();
|
||||||
|
const path = apiKey ? "/v1/search" : "/v1/search/public";
|
||||||
|
const payload: Record<string, unknown> = { query: params.query, mode: params.mode ?? "pro" };
|
||||||
|
for (const field of ["site", "published_after", "published_before"] as const) {
|
||||||
|
const value = params[field];
|
||||||
|
if (value) payload[field] = value;
|
||||||
|
}
|
||||||
|
|
||||||
|
let response: Response;
|
||||||
|
try {
|
||||||
|
response = await fetchImpl(`${resolveBaseUrl()}${path}`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { ...buildHeaders(apiKey), "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify(payload),
|
||||||
|
signal: withTimeout(signal, DEFAULT_TIMEOUT_MS),
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
throw new KeenableError(`Could not reach the Keenable API: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
await raiseForStatus(response, Boolean(apiKey));
|
||||||
|
const data = await readJsonObject(response);
|
||||||
|
const results = data.results;
|
||||||
|
if (!Array.isArray(results)) {
|
||||||
|
throw new KeenableError("Unexpected response from the Keenable search API (no results array).");
|
||||||
|
}
|
||||||
|
return results as KeenableSearchResult[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** GET /v1/fetch?url= (keyed) or /v1/fetch/public?url= (keyless). */
|
||||||
|
export async function keenableFetch(
|
||||||
|
url: string,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<KeenableFetchResult> {
|
||||||
|
assertPublicFetchTarget(url);
|
||||||
|
const apiKey = resolveApiKey();
|
||||||
|
const path = apiKey ? "/v1/fetch" : "/v1/fetch/public";
|
||||||
|
const endpoint = new URL(`${resolveBaseUrl()}${path}`);
|
||||||
|
endpoint.searchParams.set("url", url);
|
||||||
|
|
||||||
|
let response: Response;
|
||||||
|
try {
|
||||||
|
response = await fetchImpl(endpoint, {
|
||||||
|
method: "GET",
|
||||||
|
headers: buildHeaders(apiKey),
|
||||||
|
signal: withTimeout(signal, DEFAULT_TIMEOUT_MS),
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
throw new KeenableError(`Could not reach the Keenable API: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
await raiseForStatus(response, Boolean(apiKey));
|
||||||
|
return (await readJsonObject(response)) as KeenableFetchResult;
|
||||||
|
}
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
/**
|
||||||
|
* Tavily REST client: POST https://api.tavily.com/search|extract with a Bearer
|
||||||
|
* key from auth.json (`tavily` api_key credential) — there is no keyless tier.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey } from "../auth.ts";
|
||||||
|
|
||||||
|
export class TavilyError extends Error {}
|
||||||
|
|
||||||
|
const TAVILY_BASE_URL = "https://api.tavily.com";
|
||||||
|
const SEARCH_TIMEOUT_MS = 45_000;
|
||||||
|
const EXTRACT_TIMEOUT_MS = 60_000;
|
||||||
|
const CLIENT_SOURCE = "my-pi-web-search";
|
||||||
|
|
||||||
|
export type FetchLike = (input: string | URL, init?: RequestInit) => Promise<Response>;
|
||||||
|
|
||||||
|
export interface TavilySearchParams {
|
||||||
|
query: string;
|
||||||
|
max_results?: number;
|
||||||
|
search_depth?: "basic" | "advanced";
|
||||||
|
topic?: "general" | "news" | "finance";
|
||||||
|
include_domains?: string[];
|
||||||
|
start_date?: string;
|
||||||
|
end_date?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TavilySearchResult {
|
||||||
|
title?: string;
|
||||||
|
url?: string;
|
||||||
|
content?: string;
|
||||||
|
published_date?: string;
|
||||||
|
[key: string]: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function tavilyAvailable(resolve: (provider: "tavily") => string | undefined = getSearchApiKey): boolean {
|
||||||
|
return Boolean(resolve("tavily"));
|
||||||
|
}
|
||||||
|
|
||||||
|
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||||
|
const timeout = AbortSignal.timeout(timeoutMs);
|
||||||
|
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function post(
|
||||||
|
endpoint: string,
|
||||||
|
body: Record<string, unknown>,
|
||||||
|
timeoutMs: number,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
fetchImpl: FetchLike,
|
||||||
|
): Promise<Record<string, unknown>> {
|
||||||
|
const apiKey = getSearchApiKey("tavily");
|
||||||
|
if (!apiKey) {
|
||||||
|
throw new TavilyError('Tavily is not configured: no "tavily" api_key credential in auth.json.');
|
||||||
|
}
|
||||||
|
let response: Response;
|
||||||
|
try {
|
||||||
|
response = await fetchImpl(`${TAVILY_BASE_URL}/${endpoint}`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
Authorization: `Bearer ${apiKey}`,
|
||||||
|
"X-Client-Source": CLIENT_SOURCE,
|
||||||
|
},
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
signal: withTimeout(signal, timeoutMs),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
throw new TavilyError(`Could not reach the Tavily API: ${(error as Error).message}`);
|
||||||
|
}
|
||||||
|
if (!response.ok) {
|
||||||
|
let detail = "";
|
||||||
|
try {
|
||||||
|
const data = (await response.json()) as Record<string, unknown>;
|
||||||
|
detail = String(data.error ?? data.message ?? "");
|
||||||
|
} catch {
|
||||||
|
detail = "";
|
||||||
|
}
|
||||||
|
throw new TavilyError(`Tavily API error (${response.status})${detail ? `: ${detail}` : ""}`);
|
||||||
|
}
|
||||||
|
return (await response.json()) as Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** POST /search — broad discovery; returns the answer plus a compact result list. */
|
||||||
|
export async function tavilySearch(
|
||||||
|
params: TavilySearchParams,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<{ answer?: string; results: TavilySearchResult[] }> {
|
||||||
|
const payload: Record<string, unknown> = {
|
||||||
|
query: params.query,
|
||||||
|
max_results: params.max_results,
|
||||||
|
search_depth: params.search_depth,
|
||||||
|
topic: params.topic,
|
||||||
|
include_domains: params.include_domains,
|
||||||
|
start_date: params.start_date,
|
||||||
|
end_date: params.end_date,
|
||||||
|
};
|
||||||
|
for (const key of Object.keys(payload)) {
|
||||||
|
if (payload[key] === undefined) delete payload[key];
|
||||||
|
}
|
||||||
|
const data = await post("search", payload, SEARCH_TIMEOUT_MS, signal, fetchImpl);
|
||||||
|
const rawResults = Array.isArray(data.results) ? (data.results as Record<string, unknown>[]) : [];
|
||||||
|
return {
|
||||||
|
answer: typeof data.answer === "string" ? data.answer : undefined,
|
||||||
|
results: rawResults.map((result) => ({
|
||||||
|
title: result.title,
|
||||||
|
url: result.url,
|
||||||
|
content: result.content,
|
||||||
|
published_date: result.publishedDate ?? result.published_date,
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** POST /extract — full page content as markdown for a batch of URLs. */
|
||||||
|
export async function tavilyExtract(
|
||||||
|
urls: string[],
|
||||||
|
signal?: AbortSignal,
|
||||||
|
fetchImpl: FetchLike = fetch,
|
||||||
|
): Promise<{ results: Array<{ url?: string; content?: string }>; failed: string[] }> {
|
||||||
|
const data = await post("extract", { urls, format: "markdown" }, EXTRACT_TIMEOUT_MS, signal, fetchImpl);
|
||||||
|
const rawResults = Array.isArray(data.results) ? (data.results as Record<string, unknown>[]) : [];
|
||||||
|
const rawFailed = Array.isArray(data.failed_results) ? (data.failed_results as Record<string, unknown>[]) : [];
|
||||||
|
return {
|
||||||
|
results: rawResults.map((result) => ({ url: result.url as string | undefined, content: result.raw_content as string | undefined })),
|
||||||
|
failed: rawFailed.map((result) => String(result.url ?? "unknown")),
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,281 @@
|
|||||||
|
/**
|
||||||
|
* pi-web-search: one unified web_search + web_fetch surface.
|
||||||
|
*
|
||||||
|
* The main model never sees provider names — web_search routes internally via
|
||||||
|
* the session's current model (see router.ts), and web_fetch always reads
|
||||||
|
* pages through Exa with graceful degradation. Provider clients are plain
|
||||||
|
* fetch calls (clients/*) — no MCP adapter, no upstream extension wrappers.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { AgentToolResult, ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
|
||||||
|
import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts";
|
||||||
|
import { exaFetchPages, exaSearch } from "./clients/exa.ts";
|
||||||
|
import { keenableFetch, keenableSearch } from "./clients/keenable.ts";
|
||||||
|
import { tavilyAvailable, tavilyExtract, tavilySearch } from "./clients/tavily.ts";
|
||||||
|
import {
|
||||||
|
availableProviders,
|
||||||
|
routeSearch,
|
||||||
|
type SearchProvider,
|
||||||
|
type RouterContext,
|
||||||
|
} from "./router.ts";
|
||||||
|
|
||||||
|
const DEFAULT_MAX_RESULTS = 8;
|
||||||
|
const MAX_FETCH_URLS = 10;
|
||||||
|
const DEFAULT_FETCH_MAX_CHARACTERS = 3000;
|
||||||
|
|
||||||
|
export interface WebSearchParams {
|
||||||
|
query: string;
|
||||||
|
max_results?: number;
|
||||||
|
site?: string;
|
||||||
|
published_after?: string;
|
||||||
|
published_before?: string;
|
||||||
|
depth?: "basic" | "advanced";
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface WebFetchParams {
|
||||||
|
urls: string | string[];
|
||||||
|
max_characters?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ProviderRunner = (params: WebSearchParams, signal?: AbortSignal) => Promise<string>;
|
||||||
|
export type FetchRunner = (urls: string[], maxCharacters: number, signal?: AbortSignal) => Promise<string>;
|
||||||
|
|
||||||
|
function formatSearchPayload(payload: unknown): string {
|
||||||
|
return JSON.stringify(payload, null, 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toTavilyParams(params: WebSearchParams) {
|
||||||
|
return {
|
||||||
|
query: params.query,
|
||||||
|
max_results: params.max_results ?? DEFAULT_MAX_RESULTS,
|
||||||
|
search_depth: (params.depth ?? "basic") as "basic" | "advanced",
|
||||||
|
include_domains: params.site ? [params.site] : undefined,
|
||||||
|
start_date: params.published_after,
|
||||||
|
end_date: params.published_before,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toExaParams(params: WebSearchParams) {
|
||||||
|
return {
|
||||||
|
query: params.query,
|
||||||
|
numResults: params.max_results ?? DEFAULT_MAX_RESULTS,
|
||||||
|
includeDomains: params.site ? [params.site] : undefined,
|
||||||
|
startPublishedDate: params.published_after,
|
||||||
|
endPublishedDate: params.published_before,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toKeenableParams(params: WebSearchParams) {
|
||||||
|
return {
|
||||||
|
query: params.query,
|
||||||
|
site: params.site,
|
||||||
|
published_after: params.published_after,
|
||||||
|
published_before: params.published_before,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Default per-provider parameter mapping from the unified schema. */
|
||||||
|
export function createProviderRunners(
|
||||||
|
overrides: Partial<Record<SearchProvider, ProviderRunner>> = {},
|
||||||
|
): Record<SearchProvider, ProviderRunner> {
|
||||||
|
return {
|
||||||
|
tavily:
|
||||||
|
overrides.tavily ??
|
||||||
|
(async (params, signal) => formatSearchPayload(await tavilySearch(toTavilyParams(params), signal))),
|
||||||
|
exa: overrides.exa ?? ((params, signal) => exaSearch(toExaParams(params), signal)),
|
||||||
|
keenable:
|
||||||
|
overrides.keenable ??
|
||||||
|
(async (params, signal) => formatSearchPayload(await keenableSearch(toKeenableParams(params), signal))),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function errorText(error: unknown): string {
|
||||||
|
return error instanceof Error ? error.message : String(error);
|
||||||
|
}
|
||||||
|
|
||||||
|
function toResult(text: string, details: Record<string, unknown>): AgentToolResult {
|
||||||
|
return { content: [{ type: "text", text }], details };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Route once via the main model, then try the picked provider first and fall
|
||||||
|
* back across the remaining available providers on transport errors.
|
||||||
|
*/
|
||||||
|
export async function dispatchSearch(
|
||||||
|
params: WebSearchParams,
|
||||||
|
ctx: RouterContext,
|
||||||
|
routeCache: Map<string, SearchProvider>,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
runners: Record<SearchProvider, ProviderRunner> = createProviderRunners(),
|
||||||
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
|
): Promise<AgentToolResult> {
|
||||||
|
const decision = await routeSearch(params, ctx, routeCache, signal, resolve);
|
||||||
|
const order = [decision.provider, ...availableProviders(resolve).filter((p) => p !== decision.provider)];
|
||||||
|
let lastError: unknown;
|
||||||
|
for (const provider of order) {
|
||||||
|
try {
|
||||||
|
const text = await runners[provider](params, signal);
|
||||||
|
return toResult(text, {
|
||||||
|
provider,
|
||||||
|
routedBy: provider === decision.provider ? decision.routedBy : "provider-fallback",
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return toResult(`web_search failed on every available provider: ${errorText(lastError)}`, { error: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function normalizeUrls(urls: string | string[]): string[] {
|
||||||
|
const list = (Array.isArray(urls) ? urls : [urls]).map((url) => url.trim()).filter(Boolean);
|
||||||
|
return [...new Set(list)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** web_fetch always prefers Exa; Keenable and Tavily are silent degradation paths. */
|
||||||
|
export function createFetchRunners(
|
||||||
|
overrides: Partial<Record<SearchProvider, FetchRunner>> = {},
|
||||||
|
): Record<SearchProvider, FetchRunner> {
|
||||||
|
return {
|
||||||
|
exa: overrides.exa ?? ((urls, maxCharacters, signal) => exaFetchPages(urls, maxCharacters, signal)),
|
||||||
|
keenable:
|
||||||
|
overrides.keenable ??
|
||||||
|
(async (urls, _maxCharacters, signal) => {
|
||||||
|
const pages = await Promise.all(urls.map((url) => keenableFetch(url, signal)));
|
||||||
|
return pages
|
||||||
|
.map((page) => `# ${page.title ?? page.url ?? "page"}\n${page.content ?? "(no content)"}`)
|
||||||
|
.join("\n\n---\n\n");
|
||||||
|
}),
|
||||||
|
tavily:
|
||||||
|
overrides.tavily ??
|
||||||
|
(async (urls, _maxCharacters, signal) => {
|
||||||
|
const { results, failed } = await tavilyExtract(urls, signal);
|
||||||
|
const body = results
|
||||||
|
.map((page) => `# ${page.url ?? "page"}\n${page.content ?? "(no content)"}`)
|
||||||
|
.join("\n\n---\n\n");
|
||||||
|
return failed.length ? `${body}\n\nFailed URLs: ${failed.join(", ")}` : body;
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function dispatchFetch(
|
||||||
|
params: WebFetchParams,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
fetchers: Record<SearchProvider, FetchRunner> = createFetchRunners(),
|
||||||
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
|
): Promise<AgentToolResult> {
|
||||||
|
const urls = normalizeUrls(params.urls);
|
||||||
|
if (!urls.length) {
|
||||||
|
return toResult("web_fetch needs at least one http(s) URL.", { error: true });
|
||||||
|
}
|
||||||
|
if (urls.length > MAX_FETCH_URLS) {
|
||||||
|
return toResult(`web_fetch accepts at most ${MAX_FETCH_URLS} URLs per call; received ${urls.length}.`, { error: true });
|
||||||
|
}
|
||||||
|
const maxCharacters = params.max_characters ?? DEFAULT_FETCH_MAX_CHARACTERS;
|
||||||
|
const order: SearchProvider[] = ["exa", "keenable"];
|
||||||
|
if (tavilyAvailable(resolve)) order.push("tavily");
|
||||||
|
|
||||||
|
let lastError: unknown;
|
||||||
|
for (const provider of order) {
|
||||||
|
try {
|
||||||
|
const text = await fetchers[provider](urls, maxCharacters, signal);
|
||||||
|
return toResult(text, { provider });
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return toResult(`web_fetch failed on every available provider: ${errorText(lastError)}`, { error: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function webSearchExtension(pi: ExtensionAPI): void {
|
||||||
|
const routeCache = new Map<string, SearchProvider>();
|
||||||
|
|
||||||
|
pi.registerTool({
|
||||||
|
name: "web_search",
|
||||||
|
label: "Web Search",
|
||||||
|
description:
|
||||||
|
"Search the web for current or citable information and return a compact list of candidate " +
|
||||||
|
"pages (title, url, summary). Provider selection is automatic. Follow up with web_fetch to " +
|
||||||
|
"read the most relevant pages.",
|
||||||
|
promptSnippet: "Search the web for current or citable information.",
|
||||||
|
promptGuidelines: [
|
||||||
|
"Use web_search when the task needs current, real-world, or citable web information.",
|
||||||
|
"Keep queries focused; split multi-topic requests into separate web_search calls.",
|
||||||
|
"Use site and published_* filters only when they materially narrow the results.",
|
||||||
|
"After web_search, read the most relevant pages with web_fetch before relying on a claim.",
|
||||||
|
],
|
||||||
|
parameters: {
|
||||||
|
type: "object",
|
||||||
|
properties: {
|
||||||
|
query: {
|
||||||
|
type: "string",
|
||||||
|
description: "The search query — a natural-language description of the ideal page, not just keywords.",
|
||||||
|
},
|
||||||
|
max_results: {
|
||||||
|
type: "number",
|
||||||
|
description: `Maximum number of results to return (default ${DEFAULT_MAX_RESULTS}).`,
|
||||||
|
},
|
||||||
|
site: {
|
||||||
|
type: "string",
|
||||||
|
description: "Restrict results to a single domain, e.g. 'arxiv.org'.",
|
||||||
|
},
|
||||||
|
published_after: {
|
||||||
|
type: "string",
|
||||||
|
description: "Only pages published on/after this date (YYYY-MM-DD).",
|
||||||
|
},
|
||||||
|
published_before: {
|
||||||
|
type: "string",
|
||||||
|
description: "Only pages published on/before this date (YYYY-MM-DD).",
|
||||||
|
},
|
||||||
|
depth: {
|
||||||
|
type: "string",
|
||||||
|
enum: ["basic", "advanced"],
|
||||||
|
description: "'basic' (default) balanced search; 'advanced' for higher precision at higher cost.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ["query"],
|
||||||
|
additionalProperties: false,
|
||||||
|
},
|
||||||
|
async execute(_toolCallId, params, signal, _onUpdate, ctx) {
|
||||||
|
return dispatchSearch(params as WebSearchParams, ctx as unknown as RouterContext, routeCache, signal);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
pi.registerTool({
|
||||||
|
name: "web_fetch",
|
||||||
|
label: "Web Fetch",
|
||||||
|
description:
|
||||||
|
"Fetch the full content of one or more http(s) URLs as clean markdown. Use after web_search " +
|
||||||
|
"to read selected pages, or directly when the user provides a page to read.",
|
||||||
|
promptSnippet: "Read the full content of known URLs.",
|
||||||
|
promptGuidelines: [
|
||||||
|
"Use web_fetch to read the full content of a URL found via web_search or provided by the user.",
|
||||||
|
"Batch multiple URLs into a single web_fetch call when possible.",
|
||||||
|
],
|
||||||
|
parameters: {
|
||||||
|
type: "object",
|
||||||
|
properties: {
|
||||||
|
urls: {
|
||||||
|
anyOf: [
|
||||||
|
{ type: "string", description: "A single http(s) URL to read." },
|
||||||
|
{
|
||||||
|
type: "array",
|
||||||
|
items: { type: "string" },
|
||||||
|
maxItems: MAX_FETCH_URLS,
|
||||||
|
description: `A list of http(s) URLs to read (max ${MAX_FETCH_URLS}).`,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
max_characters: {
|
||||||
|
type: "number",
|
||||||
|
description: `Maximum characters to extract per page (default ${DEFAULT_FETCH_MAX_CHARACTERS}).`,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ["urls"],
|
||||||
|
additionalProperties: false,
|
||||||
|
},
|
||||||
|
async execute(_toolCallId, params, signal) {
|
||||||
|
return dispatchFetch(params as WebFetchParams, signal);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
{
|
||||||
|
"name": "pi-web-search",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"description": "Unified web search and fetch tools with main-model provider routing (Tavily, Exa, Keenable).",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"license": "MIT",
|
||||||
|
"main": "./index.ts",
|
||||||
|
"exports": {
|
||||||
|
".": "./index.ts"
|
||||||
|
},
|
||||||
|
"files": [
|
||||||
|
"index.ts",
|
||||||
|
"auth.ts",
|
||||||
|
"router.ts",
|
||||||
|
"clients",
|
||||||
|
"README.md",
|
||||||
|
"LICENSE"
|
||||||
|
],
|
||||||
|
"pi": {
|
||||||
|
"extensions": [
|
||||||
|
"./index.ts"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"test": "node --test test/*.test.ts"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"@earendil-works/pi-coding-agent": "*",
|
||||||
|
"typebox": "*"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"@earendil-works/pi-coding-agent": {
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"typebox": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=22.19.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
/**
|
||||||
|
* Provider router for web_search. The decision is delegated to the session's
|
||||||
|
* current main model (ctx.model via ctx.modelRegistry.complete) so no extra
|
||||||
|
* model configuration is needed; deterministic rules only handle availability
|
||||||
|
* and failure fallbacks, never the routing itself.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts";
|
||||||
|
|
||||||
|
export type SearchProvider = "tavily" | "exa" | "keenable";
|
||||||
|
|
||||||
|
export interface SearchRequest {
|
||||||
|
query: string;
|
||||||
|
site?: string;
|
||||||
|
published_after?: string;
|
||||||
|
published_before?: string;
|
||||||
|
depth?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type RoutedBy = "model" | "cache" | "fallback";
|
||||||
|
|
||||||
|
export interface RouteDecision {
|
||||||
|
provider: SearchProvider;
|
||||||
|
routedBy: RoutedBy;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ROUTER_TIMEOUT_MS = 15_000;
|
||||||
|
export const ROUTE_CACHE_LIMIT = 32;
|
||||||
|
|
||||||
|
/** Order used only when the model cannot decide or no candidates remain. */
|
||||||
|
export const PROVIDER_FALLBACK_ORDER: SearchProvider[] = ["tavily", "exa", "keenable"];
|
||||||
|
|
||||||
|
const PROVIDER_RULES: Record<SearchProvider, string> = {
|
||||||
|
tavily:
|
||||||
|
"Broad discovery. Best when keywords are uncertain, for news and recent or trending topics, " +
|
||||||
|
"and when many candidate sources are useful. Results can mix official pages, blogs, forums, " +
|
||||||
|
"and secondary articles; returned content is naturally long.",
|
||||||
|
exa:
|
||||||
|
"Precision sources. Best for official or API documentation, release notes and version " +
|
||||||
|
"information, academic papers and research material, and fact confirmation that needs " +
|
||||||
|
"high-quality sources.",
|
||||||
|
keenable:
|
||||||
|
"Focused and compact. Best for Chinese-language pages (policies, government notices), " +
|
||||||
|
"site-restricted search, publication-date filtering, and tight control over returned " +
|
||||||
|
"content length.",
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Tavily requires a key in auth.json; Exa and Keenable both work keyless. */
|
||||||
|
export function availableProviders(resolve: SearchKeyResolver = getSearchApiKey): SearchProvider[] {
|
||||||
|
const providers: SearchProvider[] = [];
|
||||||
|
if (resolve("tavily")) providers.push("tavily");
|
||||||
|
providers.push("exa", "keenable");
|
||||||
|
return providers;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildRouterPrompt(request: SearchRequest, candidates: SearchProvider[]): string {
|
||||||
|
const rules = candidates.map((provider) => `- ${provider}: ${PROVIDER_RULES[provider]}`).join("\n");
|
||||||
|
const lines = [
|
||||||
|
"Pick the single best web search provider for this query.",
|
||||||
|
`Answer with exactly one word: ${candidates.join(", ")}.`,
|
||||||
|
"",
|
||||||
|
rules,
|
||||||
|
"",
|
||||||
|
`Query: ${request.query}`,
|
||||||
|
];
|
||||||
|
if (request.site) lines.push(`The caller asked to restrict results to the site: ${request.site}`);
|
||||||
|
if (request.published_after || request.published_before) {
|
||||||
|
lines.push(
|
||||||
|
`The caller asked for a publication-date filter: ${request.published_after ?? "…"} to ${request.published_before ?? "…"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (request.depth === "advanced") lines.push("The caller asked for a deep, high-precision search.");
|
||||||
|
lines.push("Provider:");
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
/** First candidate-matching provider token in the model's reply, else undefined. */
|
||||||
|
export function parseProviderChoice(text: string, candidates: SearchProvider[]): SearchProvider | undefined {
|
||||||
|
const matches = text.toLowerCase().match(/\b(tavily|exa|keenable)\b/gu) ?? [];
|
||||||
|
for (const match of matches) {
|
||||||
|
const pick = match as SearchProvider;
|
||||||
|
if (candidates.includes(pick)) return pick;
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function routeCacheKey(request: SearchRequest): string {
|
||||||
|
return [
|
||||||
|
request.query.trim().toLowerCase(),
|
||||||
|
request.site ?? "",
|
||||||
|
request.published_after ?? "",
|
||||||
|
request.published_before ?? "",
|
||||||
|
request.depth ?? "",
|
||||||
|
].join(" ");
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ModelMessage {
|
||||||
|
stopReason?: string;
|
||||||
|
errorMessage?: string;
|
||||||
|
content?: Array<{ type?: string; text?: string }>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Minimal structural view of the extension context needed for routing. */
|
||||||
|
export interface RouterContext {
|
||||||
|
model?: unknown;
|
||||||
|
modelRegistry?: {
|
||||||
|
complete?: (model: unknown, request: unknown, options: unknown) => Promise<ModelMessage>;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function routeSearch(
|
||||||
|
request: SearchRequest,
|
||||||
|
ctx: RouterContext,
|
||||||
|
cache: Map<string, SearchProvider>,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
|
): Promise<RouteDecision> {
|
||||||
|
const candidates = availableProviders(resolve);
|
||||||
|
const fallback = PROVIDER_FALLBACK_ORDER.find((provider) => candidates.includes(provider)) ?? candidates[0];
|
||||||
|
if (!fallback || candidates.length <= 1) {
|
||||||
|
return { provider: fallback ?? "keenable", routedBy: "fallback" };
|
||||||
|
}
|
||||||
|
|
||||||
|
const key = routeCacheKey(request);
|
||||||
|
const cached = cache.get(key);
|
||||||
|
if (cached !== undefined && candidates.includes(cached)) {
|
||||||
|
return { provider: cached, routedBy: "cache" };
|
||||||
|
}
|
||||||
|
|
||||||
|
const complete = ctx.modelRegistry?.complete;
|
||||||
|
if (!ctx.model || typeof complete !== "function") {
|
||||||
|
return { provider: fallback, routedBy: "fallback" };
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const timeout = AbortSignal.timeout(ROUTER_TIMEOUT_MS);
|
||||||
|
const message = await complete.call(ctx.modelRegistry, ctx.model, {
|
||||||
|
messages: [
|
||||||
|
{
|
||||||
|
role: "user",
|
||||||
|
content: [{ type: "text", text: buildRouterPrompt(request, candidates) }],
|
||||||
|
timestamp: Date.now(),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}, {
|
||||||
|
signal: signal ? AbortSignal.any([signal, timeout]) : timeout,
|
||||||
|
});
|
||||||
|
if (message.stopReason === "error" || message.stopReason === "aborted") {
|
||||||
|
throw new Error(message.errorMessage ?? `routing stopped: ${message.stopReason}`);
|
||||||
|
}
|
||||||
|
const text = (message.content ?? [])
|
||||||
|
.flatMap((part) => (part.type === "text" && typeof part.text === "string" ? [part.text] : []))
|
||||||
|
.join("\n");
|
||||||
|
const pick = parseProviderChoice(text, candidates);
|
||||||
|
if (!pick) return { provider: fallback, routedBy: "fallback" };
|
||||||
|
if (!cache.has(key) && cache.size >= ROUTE_CACHE_LIMIT) {
|
||||||
|
const oldest = cache.keys().next().value;
|
||||||
|
if (oldest !== undefined) cache.delete(oldest);
|
||||||
|
}
|
||||||
|
cache.set(key, pick);
|
||||||
|
return { provider: pick, routedBy: "model" };
|
||||||
|
} catch {
|
||||||
|
return { provider: fallback, routedBy: "fallback" };
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,154 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
|
||||||
|
import webSearchExtension, {
|
||||||
|
createFetchRunners,
|
||||||
|
createProviderRunners,
|
||||||
|
dispatchFetch,
|
||||||
|
dispatchSearch,
|
||||||
|
normalizeUrls,
|
||||||
|
toExaParams,
|
||||||
|
toKeenableParams,
|
||||||
|
toTavilyParams,
|
||||||
|
type WebSearchParams,
|
||||||
|
} from "../index.ts";
|
||||||
|
|
||||||
|
const PARAMS: WebSearchParams = { query: "release notes" };
|
||||||
|
|
||||||
|
function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }) {
|
||||||
|
return {
|
||||||
|
model: { provider: "test", id: "main" },
|
||||||
|
modelRegistry: {
|
||||||
|
complete: async () => {
|
||||||
|
calls.count += 1;
|
||||||
|
return { content: [{ type: "text", text: answer }] };
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const WITH_TAVILY = () => "k";
|
||||||
|
const KEYLESS = () => undefined;
|
||||||
|
|
||||||
|
test("extension registers only web_search and web_fetch, with no provider in the schema", () => {
|
||||||
|
const tools = new Map<string, { parameters?: { properties?: Record<string, unknown> }; description?: string }>();
|
||||||
|
const pi = {
|
||||||
|
registerTool(definition: { name: string }) {
|
||||||
|
tools.set(definition.name, definition as never);
|
||||||
|
},
|
||||||
|
on() {},
|
||||||
|
} as unknown as ExtensionAPI;
|
||||||
|
|
||||||
|
webSearchExtension(pi);
|
||||||
|
assert.deepEqual([...tools.keys()].sort(), ["web_fetch", "web_search"]);
|
||||||
|
const schemaText = JSON.stringify([...tools.values()].map((t) => t.parameters));
|
||||||
|
assert.doesNotMatch(schemaText, /tavily|exa|keenable|"provider"/iu);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("dispatchSearch runs the model-picked provider first and records details", async () => {
|
||||||
|
const ran: string[] = [];
|
||||||
|
const runners = createProviderRunners({
|
||||||
|
exa: async () => {
|
||||||
|
ran.push("exa");
|
||||||
|
return "exa results";
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
|
assert.deepEqual(ran, ["exa"]);
|
||||||
|
assert.equal(result.content[0]!.type === "text" && result.content[0]!.text, "exa results");
|
||||||
|
assert.deepEqual(result.details, { provider: "exa", routedBy: "model" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("dispatchSearch falls back across remaining providers when the pick fails", async () => {
|
||||||
|
const ran: string[] = [];
|
||||||
|
const runners = createProviderRunners({
|
||||||
|
tavily: async () => {
|
||||||
|
ran.push("tavily");
|
||||||
|
throw new Error("tavily down");
|
||||||
|
},
|
||||||
|
exa: async () => {
|
||||||
|
ran.push("exa");
|
||||||
|
return "exa results";
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const result = await dispatchSearch(PARAMS, ctxAnswering("tavily"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
|
assert.deepEqual(ran, ["tavily", "exa"]);
|
||||||
|
assert.equal((result.details as Record<string, unknown>).provider, "exa");
|
||||||
|
assert.equal((result.details as Record<string, unknown>).routedBy, "provider-fallback");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("dispatchSearch reports an error result when every provider fails", async () => {
|
||||||
|
const runners = createProviderRunners({
|
||||||
|
tavily: async () => { throw new Error("t"); },
|
||||||
|
exa: async () => { throw new Error("e"); },
|
||||||
|
keenable: async () => { throw new Error("k"); },
|
||||||
|
});
|
||||||
|
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
|
assert.equal((result.details as Record<string, unknown>).error, true);
|
||||||
|
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /failed on every available provider/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("normalizeUrls accepts a string or array, trims, and deduplicates", () => {
|
||||||
|
assert.deepEqual(normalizeUrls(" https://a.example "), ["https://a.example"]);
|
||||||
|
assert.deepEqual(normalizeUrls(["https://a", " https://a ", "https://b"]), ["https://a", "https://b"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("dispatchFetch prefers exa and degrades to keenable without a model call", async () => {
|
||||||
|
const fetchers = createFetchRunners({
|
||||||
|
exa: async (urls) => {
|
||||||
|
throw new Error(`exa down for ${urls.length}`);
|
||||||
|
},
|
||||||
|
keenable: async (urls) => `keenable pages: ${urls.join(",")}`,
|
||||||
|
});
|
||||||
|
const result = await dispatchFetch({ urls: ["https://a.example", "https://b.example"] }, undefined, fetchers, KEYLESS);
|
||||||
|
assert.equal((result.details as Record<string, unknown>).provider, "keenable");
|
||||||
|
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /keenable pages: https:\/\/a\.example,https:\/\/b\.example/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("dispatchFetch validates input and caps the URL count", async () => {
|
||||||
|
const empty = await dispatchFetch({ urls: [] }, undefined);
|
||||||
|
assert.equal((empty.details as Record<string, unknown>).error, true);
|
||||||
|
|
||||||
|
const tooMany = await dispatchFetch({ urls: Array.from({ length: 11 }, (_, i) => `https://e${i}.example`) }, undefined);
|
||||||
|
assert.equal((tooMany.details as Record<string, unknown>).error, true);
|
||||||
|
assert.match(tooMany.content[0]!.type === "text" ? tooMany.content[0]!.text : "", /at most 10/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("unified params map onto each provider's native arguments", () => {
|
||||||
|
const params: WebSearchParams = {
|
||||||
|
query: "q",
|
||||||
|
max_results: 4,
|
||||||
|
site: "a.dev",
|
||||||
|
published_after: "2025-01-01",
|
||||||
|
published_before: "2025-06-01",
|
||||||
|
depth: "advanced",
|
||||||
|
};
|
||||||
|
assert.deepEqual(toTavilyParams(params), {
|
||||||
|
query: "q",
|
||||||
|
max_results: 4,
|
||||||
|
search_depth: "advanced",
|
||||||
|
include_domains: ["a.dev"],
|
||||||
|
start_date: "2025-01-01",
|
||||||
|
end_date: "2025-06-01",
|
||||||
|
});
|
||||||
|
assert.deepEqual(toExaParams(params), {
|
||||||
|
query: "q",
|
||||||
|
numResults: 4,
|
||||||
|
includeDomains: ["a.dev"],
|
||||||
|
startPublishedDate: "2025-01-01",
|
||||||
|
endPublishedDate: "2025-06-01",
|
||||||
|
});
|
||||||
|
assert.deepEqual(toKeenableParams(params), {
|
||||||
|
query: "q",
|
||||||
|
site: "a.dev",
|
||||||
|
published_after: "2025-01-01",
|
||||||
|
published_before: "2025-06-01",
|
||||||
|
});
|
||||||
|
|
||||||
|
const defaults = toTavilyParams({ query: "q" });
|
||||||
|
assert.equal(defaults.max_results, 8);
|
||||||
|
assert.equal(defaults.search_depth, "basic");
|
||||||
|
assert.equal(defaults.include_domains, undefined);
|
||||||
|
});
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import { ExaError, exaFetchPages, exaSearch, parseRpcPayload, resetExaSession } from "../clients/exa.ts";
|
||||||
|
|
||||||
|
function sse(payload: Record<string, unknown>): string {
|
||||||
|
return `event: message\ndata: ${JSON.stringify(payload)}\n\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface RecordedCall {
|
||||||
|
method?: string;
|
||||||
|
params?: Record<string, unknown>;
|
||||||
|
headers: Record<string, string>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A fetch stub that speaks the Exa endpoint protocol: initialize answers with
|
||||||
|
* the scripted session id and the request's own id, notifications get a bare
|
||||||
|
* 202, and tools/call delegates to the supplied handler (default 404).
|
||||||
|
*/
|
||||||
|
function exaStub(options: {
|
||||||
|
sessionIds?: string[];
|
||||||
|
onToolCall?: (args: Record<string, unknown>, id: number, session: string | undefined) => Response;
|
||||||
|
} = {}) {
|
||||||
|
const calls: RecordedCall[] = [];
|
||||||
|
let sessionIndex = 0;
|
||||||
|
const fetchImpl = async (_url: string | URL, init?: RequestInit): Promise<Response> => {
|
||||||
|
const headers = (init?.headers ?? {}) as Record<string, string>;
|
||||||
|
const body = JSON.parse(String(init?.body)) as { id?: number; method?: string; params?: Record<string, unknown> };
|
||||||
|
calls.push({ method: body.method, params: body.params, headers });
|
||||||
|
if (body.method === "initialize") {
|
||||||
|
const sid = options.sessionIds?.[Math.min(sessionIndex++, (options.sessionIds?.length ?? 1) - 1)] ?? `sid-${sessionIndex}`;
|
||||||
|
return new Response(
|
||||||
|
sse({ jsonrpc: "2.0", id: body.id, result: { protocolVersion: "2025-03-26", capabilities: {}, serverInfo: { name: "exa" } } }),
|
||||||
|
{ status: 200, headers: { "mcp-session-id": sid, "content-type": "text/event-stream" } },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (body.method === "notifications/initialized") return new Response("", { status: 202 });
|
||||||
|
if (body.method === "tools/call") {
|
||||||
|
return options.onToolCall?.(body.params ?? {}, body.id ?? 0, headers["mcp-session-id"]) ?? new Response("Not Found", { status: 404 });
|
||||||
|
}
|
||||||
|
return new Response("unexpected", { status: 500 });
|
||||||
|
};
|
||||||
|
return { fetchImpl, calls };
|
||||||
|
}
|
||||||
|
|
||||||
|
function okToolResult(id: number, text: string): Response {
|
||||||
|
return new Response(
|
||||||
|
sse({ jsonrpc: "2.0", id, result: { content: [{ type: "text", text }] } }),
|
||||||
|
{ status: 200, headers: { "content-type": "text/event-stream" } },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
test("parseRpcPayload reads JSON bodies and SSE data lines, matching the request id", () => {
|
||||||
|
const direct = JSON.stringify({ jsonrpc: "2.0", id: 7, result: { ok: true } });
|
||||||
|
assert.deepEqual(parseRpcPayload(direct, 7).result, { ok: true });
|
||||||
|
|
||||||
|
const body =
|
||||||
|
`event: message\ndata: {"jsonrpc":"2.0","id":1,"result":{"a":1}}\n\n` +
|
||||||
|
`event: message\ndata: {"jsonrpc":"2.0","id":2,"result":{"b":2}}\n\n`;
|
||||||
|
assert.deepEqual(parseRpcPayload(body, 2).result, { b: 2 });
|
||||||
|
|
||||||
|
assert.throws(() => parseRpcPayload("not json at all", 1), ExaError);
|
||||||
|
assert.throws(() => parseRpcPayload(sse({ jsonrpc: "2.0", id: 99, result: {} }), 1), ExaError);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("exaSearch initializes a session, notifies, then posts tools/call with the session id", async () => {
|
||||||
|
resetExaSession();
|
||||||
|
const { fetchImpl, calls } = exaStub({
|
||||||
|
sessionIds: ["sid-A"],
|
||||||
|
onToolCall: (_args, id) => okToolResult(id, "search results"),
|
||||||
|
});
|
||||||
|
|
||||||
|
const text = await exaSearch({ query: "docs", numResults: 3, includeDomains: ["python.org"] }, undefined, fetchImpl);
|
||||||
|
assert.equal(text, "search results");
|
||||||
|
assert.deepEqual(calls.map((c) => c.method), ["initialize", "notifications/initialized", "tools/call"]);
|
||||||
|
|
||||||
|
const call = calls[2]!;
|
||||||
|
assert.equal(call.headers["mcp-session-id"], "sid-A");
|
||||||
|
assert.equal((call.params as { name: string }).name, "web_search_advanced_exa");
|
||||||
|
const args = (call.params as { arguments: Record<string, unknown> }).arguments;
|
||||||
|
assert.equal(args.numResults, 3);
|
||||||
|
assert.equal(args.type, "auto");
|
||||||
|
assert.deepEqual(args.includeDomains, ["python.org"]);
|
||||||
|
assert.ok((args.textMaxCharacters as number) > 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("exaSearch reuses the cached session across calls", async () => {
|
||||||
|
resetExaSession();
|
||||||
|
const { fetchImpl, calls } = exaStub({
|
||||||
|
sessionIds: ["sid-1"],
|
||||||
|
onToolCall: (args, id) => okToolResult(id, `result for ${(args as { name: string }).name}`),
|
||||||
|
});
|
||||||
|
|
||||||
|
await exaSearch({ query: "a" }, undefined, fetchImpl);
|
||||||
|
const text = await exaFetchPages(["https://example.com"], 500, undefined, fetchImpl);
|
||||||
|
assert.match(text, /web_fetch_exa/);
|
||||||
|
assert.equal(calls.length, 4);
|
||||||
|
assert.equal((calls[3]!.params as { name: string }).name, "web_fetch_exa");
|
||||||
|
assert.equal(calls[3]!.headers["mcp-session-id"], "sid-1");
|
||||||
|
assert.equal((calls[3]!.params as { arguments: { maxCharacters: number } }).arguments.maxCharacters, 500);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a 404 from tools/call drops the session, re-initializes, and retries once", async () => {
|
||||||
|
resetExaSession();
|
||||||
|
let toolCalls = 0;
|
||||||
|
const { fetchImpl, calls } = exaStub({
|
||||||
|
sessionIds: ["stale-sid", "fresh-sid"],
|
||||||
|
onToolCall: (_args, id) => {
|
||||||
|
toolCalls += 1;
|
||||||
|
return toolCalls === 1 ? new Response("Not Found", { status: 404 }) : okToolResult(id, "after re-init");
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const text = await exaSearch({ query: "retry me" }, undefined, fetchImpl);
|
||||||
|
assert.equal(text, "after re-init");
|
||||||
|
assert.deepEqual(calls.map((c) => c.method), [
|
||||||
|
"initialize",
|
||||||
|
"notifications/initialized",
|
||||||
|
"tools/call",
|
||||||
|
"initialize",
|
||||||
|
"notifications/initialized",
|
||||||
|
"tools/call",
|
||||||
|
]);
|
||||||
|
assert.equal(calls[5]!.headers["mcp-session-id"], "fresh-sid");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("isError tool results and HTTP failures raise ExaError", async () => {
|
||||||
|
resetExaSession();
|
||||||
|
const isError = exaStub({
|
||||||
|
onToolCall: (_args, id) =>
|
||||||
|
new Response(sse({ jsonrpc: "2.0", id, result: { isError: true, content: [{ type: "text", text: "bad args" }] } }), { status: 200 }),
|
||||||
|
});
|
||||||
|
await assert.rejects(exaFetchPages(["https://example.com"], undefined, undefined, isError.fetchImpl), /bad args/);
|
||||||
|
|
||||||
|
resetExaSession();
|
||||||
|
const httpFail = exaStub({ onToolCall: () => new Response("boom", { status: 503 }) });
|
||||||
|
// Session init succeeds; tools/call hits a non-404 HTTP error.
|
||||||
|
await assert.rejects(exaSearch({ query: "x" }, undefined, httpFail.fetchImpl), /HTTP 503/);
|
||||||
|
});
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import {
|
||||||
|
availableProviders,
|
||||||
|
buildRouterPrompt,
|
||||||
|
parseProviderChoice,
|
||||||
|
routeSearch,
|
||||||
|
ROUTE_CACHE_LIMIT,
|
||||||
|
type RouterContext,
|
||||||
|
type SearchProvider,
|
||||||
|
} from "../router.ts";
|
||||||
|
|
||||||
|
const ALL: SearchProvider[] = ["tavily", "exa", "keenable"];
|
||||||
|
const ALL_KEYS = () => "test-key";
|
||||||
|
const NO_KEYS = () => undefined;
|
||||||
|
|
||||||
|
function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }): RouterContext {
|
||||||
|
return {
|
||||||
|
model: { provider: "test", id: "main" },
|
||||||
|
modelRegistry: {
|
||||||
|
complete: async () => {
|
||||||
|
calls.count += 1;
|
||||||
|
return { content: [{ type: "text", text: answer }] };
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
test("availability: tavily needs a key, exa and keenable are keyless", () => {
|
||||||
|
assert.deepEqual(availableProviders(NO_KEYS), ["exa", "keenable"]);
|
||||||
|
assert.deepEqual(availableProviders(ALL_KEYS), ["tavily", "exa", "keenable"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("router prompt lists only candidate rules plus the query and caller hints", () => {
|
||||||
|
const prompt = buildRouterPrompt(
|
||||||
|
{ query: "python 3.14 release notes", site: "python.org", published_after: "2025-10-01" },
|
||||||
|
ALL,
|
||||||
|
);
|
||||||
|
assert.match(prompt, /- tavily: /);
|
||||||
|
assert.match(prompt, /- exa: /);
|
||||||
|
assert.match(prompt, /- keenable: /);
|
||||||
|
assert.match(prompt, /Query: python 3\.14 release notes/);
|
||||||
|
assert.match(prompt, /site filter.*python\.org|site: python\.org/);
|
||||||
|
assert.match(prompt, /Provider:$/);
|
||||||
|
|
||||||
|
const subset = buildRouterPrompt({ query: "hi" }, ["exa", "keenable"]);
|
||||||
|
assert.doesNotMatch(subset, /- tavily/);
|
||||||
|
assert.match(subset, /exactly one word: exa, keenable/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("parseProviderChoice takes the first available candidate token", () => {
|
||||||
|
assert.equal(parseProviderChoice("tavily", ALL), "tavily");
|
||||||
|
assert.equal(parseProviderChoice("I would pick Exa here.", ALL), "exa");
|
||||||
|
assert.equal(parseProviderChoice("keenable!", ALL), "keenable");
|
||||||
|
assert.equal(parseProviderChoice("google maybe", ALL), undefined);
|
||||||
|
// Model picked an unavailable provider but mentioned a valid one later.
|
||||||
|
assert.equal(parseProviderChoice("tavily or keenable", ["exa", "keenable"]), "keenable");
|
||||||
|
assert.equal(parseProviderChoice("tavily", ["exa", "keenable"]), undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("routeSearch honors the model pick and caches identical requests", async () => {
|
||||||
|
const calls = { count: 0 };
|
||||||
|
const ctx = ctxAnswering("exa", calls);
|
||||||
|
const cache = new Map<string, SearchProvider>();
|
||||||
|
const request = { query: "official MCP spec" };
|
||||||
|
|
||||||
|
const first = await routeSearch(request, ctx, cache, undefined, ALL_KEYS);
|
||||||
|
assert.deepEqual(first, { provider: "exa", routedBy: "model" });
|
||||||
|
|
||||||
|
const second = await routeSearch(request, ctx, cache, undefined, ALL_KEYS);
|
||||||
|
assert.deepEqual(second, { provider: "exa", routedBy: "cache" });
|
||||||
|
assert.equal(calls.count, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("routeSearch falls back to the first available provider on unroutable or failed model calls", async () => {
|
||||||
|
const cache = new Map<string, SearchProvider>();
|
||||||
|
const unparseable = await routeSearch({ query: "x" }, ctxAnswering("no idea"), cache, undefined, ALL_KEYS);
|
||||||
|
assert.deepEqual(unparseable, { provider: "tavily", routedBy: "fallback" });
|
||||||
|
|
||||||
|
const throwing: RouterContext = {
|
||||||
|
model: {},
|
||||||
|
modelRegistry: { complete: async () => { throw new Error("boom"); } },
|
||||||
|
};
|
||||||
|
const failed = await routeSearch({ query: "x" }, throwing, cache, undefined, ALL_KEYS);
|
||||||
|
assert.deepEqual(failed, { provider: "tavily", routedBy: "fallback" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("routeSearch without a tavily key excludes it from candidates", async () => {
|
||||||
|
const cache = new Map<string, SearchProvider>();
|
||||||
|
const decision = await routeSearch({ query: "x" }, ctxAnswering("tavily"), cache, undefined, NO_KEYS);
|
||||||
|
// "tavily" is not a candidate → unroutable → first available fallback.
|
||||||
|
assert.deepEqual(decision, { provider: "exa", routedBy: "fallback" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("routeSearch works without a model", async () => {
|
||||||
|
const cache = new Map<string, SearchProvider>();
|
||||||
|
const decision = await routeSearch({ query: "x" }, {}, cache, undefined, ALL_KEYS);
|
||||||
|
assert.deepEqual(decision, { provider: "tavily", routedBy: "fallback" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("route cache evicts the oldest entry at the limit", async () => {
|
||||||
|
const cache = new Map<string, SearchProvider>();
|
||||||
|
for (let i = 0; i < ROUTE_CACHE_LIMIT; i++) cache.set(`dummy-${i}`, "exa");
|
||||||
|
const decision = await routeSearch({ query: "fresh query" }, ctxAnswering("keenable"), cache, undefined, ALL_KEYS);
|
||||||
|
assert.equal(decision.provider, "keenable");
|
||||||
|
assert.equal(cache.size, ROUTE_CACHE_LIMIT);
|
||||||
|
assert.equal(cache.has("dummy-0"), false);
|
||||||
|
assert.equal(cache.has("dummy-1"), true);
|
||||||
|
});
|
||||||
Executable
+262
@@ -0,0 +1,262 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Read-modify-write the tavily/exa/keenable api_key credentials in Pi's
|
||||||
|
* auth.json. Preserves every other provider entry (OAuth tokens included),
|
||||||
|
* writes atomically with a lock directory, and keeps the file mode 600.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* node search-auth.mjs --set tavily=KEY [--set exa=KEY] [--set keenable=KEY]
|
||||||
|
* node search-auth.mjs --unset tavily [--unset exa]
|
||||||
|
* node search-auth.mjs --migrate [search.env path]
|
||||||
|
* node search-auth.mjs --list
|
||||||
|
* node search-auth.mjs --has tavily (exit 0 when set, 1 otherwise)
|
||||||
|
*
|
||||||
|
* --auth FILE overrides the auth.json location (default:
|
||||||
|
* $PI_CODING_AGENT_DIR/auth.json or ~/.pi/agent/auth.json).
|
||||||
|
*/
|
||||||
|
|
||||||
|
import {
|
||||||
|
chmodSync,
|
||||||
|
closeSync,
|
||||||
|
existsSync,
|
||||||
|
mkdirSync,
|
||||||
|
openSync,
|
||||||
|
readFileSync,
|
||||||
|
renameSync,
|
||||||
|
rmSync,
|
||||||
|
writeFileSync,
|
||||||
|
} from "node:fs";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
|
import { randomBytes } from "node:crypto";
|
||||||
|
|
||||||
|
const PROVIDERS = ["tavily", "exa", "keenable"];
|
||||||
|
const ENV_TO_PROVIDER = {
|
||||||
|
TAVILY_API_KEY: "tavily",
|
||||||
|
EXA_API_KEY: "exa",
|
||||||
|
KEENABLE_API_KEY: "keenable",
|
||||||
|
};
|
||||||
|
const SAFE_VALUE = /^[A-Za-z0-9._-]+$/;
|
||||||
|
const LOCK_RETRIES = 50;
|
||||||
|
const LOCK_DELAY_MS = 100;
|
||||||
|
|
||||||
|
function die(message) {
|
||||||
|
console.error(`错误: ${message}`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function usage() {
|
||||||
|
console.log(`用法:
|
||||||
|
node search-auth.mjs --set <provider>=<key> [--set ...]
|
||||||
|
node search-auth.mjs --unset <provider> [--unset ...]
|
||||||
|
node search-auth.mjs --migrate [search.env 路径]
|
||||||
|
node search-auth.mjs --list
|
||||||
|
node search-auth.mjs --has <provider>
|
||||||
|
|
||||||
|
选项:
|
||||||
|
--auth FILE 覆盖 auth.json 路径(默认 $PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json)`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultAuthPath(env = process.env) {
|
||||||
|
const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent");
|
||||||
|
return join(agentDir, "auth.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultSearchEnvPath(env = process.env) {
|
||||||
|
if (env.MY_PI_SEARCH_CONFIG?.trim()) return env.MY_PI_SEARCH_CONFIG.trim();
|
||||||
|
const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config");
|
||||||
|
return join(configHome, "my-pi", "search.env");
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const result = { sets: {}, unsets: new Set(), migrate: undefined, list: false, has: undefined, authPath: undefined };
|
||||||
|
for (let i = 0; i < argv.length; i++) {
|
||||||
|
const arg = argv[i];
|
||||||
|
if (arg === "--set") {
|
||||||
|
const spec = argv[++i];
|
||||||
|
const separator = spec ? spec.indexOf("=") : -1;
|
||||||
|
if (separator < 1) die("--set 需要 <provider>=<key>");
|
||||||
|
const provider = spec.slice(0, separator);
|
||||||
|
const value = spec.slice(separator + 1);
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.sets[provider] = value;
|
||||||
|
} else if (arg === "--unset") {
|
||||||
|
const provider = argv[++i];
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.unsets.add(provider);
|
||||||
|
} else if (arg === "--migrate") {
|
||||||
|
const next = argv[i + 1];
|
||||||
|
result.migrate = next && !next.startsWith("--") ? (i++, next) : true;
|
||||||
|
} else if (arg === "--list") {
|
||||||
|
result.list = true;
|
||||||
|
} else if (arg === "--has") {
|
||||||
|
const provider = argv[++i];
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.has = provider;
|
||||||
|
} else if (arg === "--auth") {
|
||||||
|
result.authPath = argv[++i];
|
||||||
|
if (!result.authPath) die("--auth 缺少 FILE");
|
||||||
|
} else if (arg === "-h" || arg === "--help") {
|
||||||
|
usage();
|
||||||
|
process.exit(0);
|
||||||
|
} else {
|
||||||
|
die(`未知参数:${arg}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
function acquireLock(authPath) {
|
||||||
|
mkdirSync(dirname(authPath), { recursive: true });
|
||||||
|
const lockDir = `${authPath}.lock`;
|
||||||
|
for (let attempt = 0; attempt < LOCK_RETRIES; attempt++) {
|
||||||
|
try {
|
||||||
|
mkdirSync(lockDir);
|
||||||
|
return () => {
|
||||||
|
try {
|
||||||
|
rmSync(lockDir, { recursive: true });
|
||||||
|
} catch {}
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code !== "EEXIST") throw error;
|
||||||
|
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, LOCK_DELAY_MS);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
die(`无法获取 ${lockDir} 锁;可能有其他进程正在写 auth.json(残留锁目录可手动删除)。`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readAuthData(authPath) {
|
||||||
|
let raw;
|
||||||
|
try {
|
||||||
|
raw = readFileSync(authPath, "utf8");
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === "ENOENT") return {};
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
let data;
|
||||||
|
try {
|
||||||
|
data = JSON.parse(raw);
|
||||||
|
} catch {
|
||||||
|
die(`auth.json 不是合法 JSON,已中止以免破坏现有凭据:${authPath}`);
|
||||||
|
}
|
||||||
|
if (!data || typeof data !== "object" || Array.isArray(data)) {
|
||||||
|
die(`auth.json 顶层必须是对象,已中止以免破坏现有凭据:${authPath}`);
|
||||||
|
}
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeAuthData(authPath, data) {
|
||||||
|
mkdirSync(dirname(authPath), { recursive: true });
|
||||||
|
const temp = join(dirname(authPath), `.auth.json.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
|
||||||
|
try {
|
||||||
|
const fd = openSync(temp, "w", 0o600);
|
||||||
|
writeFileSync(fd, `${JSON.stringify(data, null, 2)}\n`);
|
||||||
|
closeSync(fd);
|
||||||
|
renameSync(temp, authPath);
|
||||||
|
chmodSync(authPath, 0o600);
|
||||||
|
} finally {
|
||||||
|
rmSync(temp, { force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasKey(data, provider) {
|
||||||
|
const entry = data[provider];
|
||||||
|
return Boolean(entry && typeof entry === "object" && !Array.isArray(entry) && entry.type === "api_key" && entry.key);
|
||||||
|
}
|
||||||
|
|
||||||
|
function setKey(data, provider, key) {
|
||||||
|
const existing = data[provider];
|
||||||
|
const preserved =
|
||||||
|
existing && typeof existing === "object" && !Array.isArray(existing) && existing.type === "api_key"
|
||||||
|
? existing
|
||||||
|
: {};
|
||||||
|
data[provider] = { ...preserved, type: "api_key", key };
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseSearchEnv(text) {
|
||||||
|
const parsed = {};
|
||||||
|
for (const rawLine of text.split(/\r?\n/u)) {
|
||||||
|
const line = rawLine.trim().replace(/^export[ \t]+/u, "");
|
||||||
|
if (!line || line.startsWith("#")) continue;
|
||||||
|
const separator = line.indexOf("=");
|
||||||
|
if (separator < 1) continue;
|
||||||
|
const name = line.slice(0, separator).trim();
|
||||||
|
const value = line.slice(separator + 1).trim();
|
||||||
|
const provider = ENV_TO_PROVIDER[name];
|
||||||
|
if (!provider || !SAFE_VALUE.test(value)) continue;
|
||||||
|
parsed[provider] = value;
|
||||||
|
}
|
||||||
|
return parsed;
|
||||||
|
}
|
||||||
|
|
||||||
|
function validateKey(provider, key) {
|
||||||
|
if (!key || /[\r\n]/u.test(key)) die(`${provider} 的 key 不能为空且不能包含换行。`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const args = parseArgs(process.argv.slice(2));
|
||||||
|
const authPath = args.authPath ?? defaultAuthPath();
|
||||||
|
|
||||||
|
if (args.list) {
|
||||||
|
const data = readAuthData(authPath);
|
||||||
|
for (const provider of PROVIDERS) {
|
||||||
|
console.log(`${provider}: ${hasKey(data, provider) ? "已配置" : "未配置"}`);
|
||||||
|
}
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (args.has) {
|
||||||
|
process.exit(hasKey(readAuthData(authPath), args.has) ? 0 : 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const mutations = Object.keys(args.sets).length + args.unsets.size + (args.migrate ? 1 : 0);
|
||||||
|
if (mutations === 0) {
|
||||||
|
usage();
|
||||||
|
process.exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
const release = acquireLock(authPath);
|
||||||
|
try {
|
||||||
|
const data = readAuthData(authPath);
|
||||||
|
const changed = [];
|
||||||
|
|
||||||
|
for (const provider of args.unsets) {
|
||||||
|
if (provider in data) {
|
||||||
|
delete data[provider];
|
||||||
|
changed.push(`已移除 ${provider}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const [provider, key] of Object.entries(args.sets)) {
|
||||||
|
validateKey(provider, key);
|
||||||
|
setKey(data, provider, key);
|
||||||
|
changed.push(`已写入 ${provider}`);
|
||||||
|
}
|
||||||
|
if (args.migrate) {
|
||||||
|
const source = args.migrate === true ? defaultSearchEnvPath() : args.migrate;
|
||||||
|
if (!existsSync(source)) die(`迁移来源不存在:${source}`);
|
||||||
|
const parsed = parseSearchEnv(readFileSync(source, "utf8"));
|
||||||
|
const imported = [];
|
||||||
|
const skipped = [];
|
||||||
|
for (const provider of PROVIDERS) {
|
||||||
|
if (!parsed[provider]) continue;
|
||||||
|
if (provider in data) {
|
||||||
|
skipped.push(provider);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
setKey(data, provider, parsed[provider]);
|
||||||
|
imported.push(provider);
|
||||||
|
}
|
||||||
|
if (imported.length) changed.push(`已迁移 ${imported.join(", ")}`);
|
||||||
|
if (skipped.length) changed.push(`已跳过(auth.json 中已存在):${skipped.join(", ")}`);
|
||||||
|
if (!imported.length && !skipped.length) changed.push("search.env 中没有可迁移的 key");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (changed.length === 0) {
|
||||||
|
console.log("auth.json 无需修改。");
|
||||||
|
} else {
|
||||||
|
writeAuthData(authPath, data);
|
||||||
|
for (const line of changed) console.log(line);
|
||||||
|
console.log(`搜索凭据已写入 ${authPath}(权限 600),重启 Pi 后生效。`);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
release();
|
||||||
|
}
|
||||||
+67
-84
@@ -15,71 +15,54 @@ usage() {
|
|||||||
用法:
|
用法:
|
||||||
./search_config.sh
|
./search_config.sh
|
||||||
./search_config.sh --tavily KEY --exa KEY --keenable KEY
|
./search_config.sh --tavily KEY --exa KEY --keenable KEY
|
||||||
|
./search_config.sh --migrate [search.env 路径]
|
||||||
|
./search_config.sh --list
|
||||||
|
|
||||||
选项:
|
选项:
|
||||||
--tavily KEY 配置 TAVILY_API_KEY
|
--tavily KEY 配置 tavily api_key 凭据
|
||||||
--exa KEY 配置 EXA_API_KEY
|
--exa KEY 配置 exa api_key 凭据
|
||||||
--keenable KEY 配置 KEENABLE_API_KEY
|
--keenable KEY 配置 keenable api_key 凭据
|
||||||
--config FILE 覆盖配置文件路径
|
--migrate 从旧 search.env 导入缺失的 key
|
||||||
|
--list 显示各 provider 配置状态(不输出 key)
|
||||||
|
--auth FILE 覆盖 auth.json 路径
|
||||||
-h, --help 显示帮助
|
-h, --help 显示帮助
|
||||||
|
|
||||||
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互,
|
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互,
|
||||||
未指定的服务保留已有值。配置文件默认是 ~/.config/my-pi/search.env。
|
未指定的 provider 保留已有值。凭据写入 Pi 的 auth.json
|
||||||
|
($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,权限 600)。
|
||||||
|
key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR} 插值、
|
||||||
|
以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
|
|
||||||
CONFIG_FILE=${MY_PI_SEARCH_CONFIG:-${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env}
|
SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
|
||||||
|
AUTH_MJS="$SCRIPT_DIR/scripts/search-auth.mjs"
|
||||||
|
[ -f "$AUTH_MJS" ] || { warn "找不到 ${AUTH_MJS}"; exit 1; }
|
||||||
|
|
||||||
|
AUTH_FILE=
|
||||||
mode=interactive
|
mode=interactive
|
||||||
tavily_set=0
|
did_work=0
|
||||||
exa_set=0
|
|
||||||
keenable_set=0
|
|
||||||
tavily_value=
|
|
||||||
exa_value=
|
|
||||||
keenable_value=
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
run_auth() {
|
||||||
case $1 in
|
if [ -n "$AUTH_FILE" ]; then
|
||||||
--tavily)
|
node "$AUTH_MJS" --auth "$AUTH_FILE" "$@"
|
||||||
[ "$#" -ge 2 ] || { warn "--tavily 缺少 KEY"; exit 2; }
|
else
|
||||||
tavily_value=$2; tavily_set=1; mode=noninteractive; shift 2 ;;
|
node "$AUTH_MJS" "$@"
|
||||||
--exa)
|
fi
|
||||||
[ "$#" -ge 2 ] || { warn "--exa 缺少 KEY"; exit 2; }
|
|
||||||
exa_value=$2; exa_set=1; mode=noninteractive; shift 2 ;;
|
|
||||||
--keenable)
|
|
||||||
[ "$#" -ge 2 ] || { warn "--keenable 缺少 KEY"; exit 2; }
|
|
||||||
keenable_value=$2; keenable_set=1; mode=noninteractive; shift 2 ;;
|
|
||||||
--config)
|
|
||||||
[ "$#" -ge 2 ] || { warn "--config 缺少 FILE"; exit 2; }
|
|
||||||
CONFIG_FILE=$2; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) warn "未知参数:$1"; usage >&2; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
valid_key() {
|
|
||||||
case $1 in
|
|
||||||
''|*[!A-Za-z0-9._-]*) return 1 ;;
|
|
||||||
*) return 0 ;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
read_existing() {
|
has_key() {
|
||||||
existing_name=$1
|
run_auth --has "$1" >/dev/null 2>&1
|
||||||
[ -f "$CONFIG_FILE" ] || return 0
|
|
||||||
awk -v name="$existing_name" '
|
|
||||||
$1 == "export" { sub(/^export[ \t]+/, "") }
|
|
||||||
index($0, name "=") == 1 { print substr($0, length(name) + 2); exit }
|
|
||||||
' "$CONFIG_FILE"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
prompt_key() {
|
prompt_key() {
|
||||||
prompt_label=$1
|
prompt_label=$1
|
||||||
prompt_current=$2
|
prompt_configured=$2
|
||||||
if [ ! -t 0 ]; then
|
if [ ! -t 0 ]; then
|
||||||
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。"
|
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。"
|
||||||
return 2
|
return 2
|
||||||
fi
|
fi
|
||||||
if [ -n "$prompt_current" ]; then
|
if [ "$prompt_configured" -eq 1 ]; then
|
||||||
printf '%s API key(回车保留现有值): ' "$prompt_label" >&2
|
printf '%s API key(回车保留现有值): ' "$prompt_label" >&2
|
||||||
else
|
else
|
||||||
printf '%s API key(回车跳过): ' "$prompt_label" >&2
|
printf '%s API key(回车跳过): ' "$prompt_label" >&2
|
||||||
@@ -92,45 +75,45 @@ prompt_key() {
|
|||||||
printf '%s\n' "$prompt_value"
|
printf '%s\n' "$prompt_value"
|
||||||
}
|
}
|
||||||
|
|
||||||
current_tavily=$(read_existing TAVILY_API_KEY)
|
while [ "$#" -gt 0 ]; do
|
||||||
current_exa=$(read_existing EXA_API_KEY)
|
case $1 in
|
||||||
current_keenable=$(read_existing KEENABLE_API_KEY)
|
--tavily|--exa|--keenable)
|
||||||
|
[ "$#" -ge 2 ] || { warn "$1 缺少 KEY"; exit 2; }
|
||||||
if [ "$mode" = interactive ]; then
|
mode=noninteractive; did_work=1
|
||||||
tavily_value=$(prompt_key Tavily "$current_tavily") || exit $?
|
run_auth --set "${1#--}=$2" || exit $?
|
||||||
exa_value=$(prompt_key Exa "$current_exa") || exit $?
|
shift 2 ;;
|
||||||
keenable_value=$(prompt_key Keenable "$current_keenable") || exit $?
|
--migrate)
|
||||||
[ -n "$tavily_value" ] && tavily_set=1
|
mode=noninteractive; did_work=1
|
||||||
[ -n "$exa_value" ] && exa_set=1
|
if [ "$#" -ge 2 ] && [ "${2#--}" = "$2" ]; then
|
||||||
[ -n "$keenable_value" ] && keenable_set=1
|
run_auth --migrate "$2" || exit $?
|
||||||
|
shift 2
|
||||||
|
else
|
||||||
|
run_auth --migrate || exit $?
|
||||||
|
shift
|
||||||
fi
|
fi
|
||||||
|
say "如确认 auth.json 可用,可手动删除旧 search.env 文件。" ;;
|
||||||
[ "$tavily_set" -eq 1 ] || tavily_value=$current_tavily
|
--list) mode=noninteractive; did_work=1; run_auth --list || exit $?; shift ;;
|
||||||
[ "$exa_set" -eq 1 ] || exa_value=$current_exa
|
--auth)
|
||||||
[ "$keenable_set" -eq 1 ] || keenable_value=$current_keenable
|
[ "$#" -ge 2 ] || { warn "--auth 缺少 FILE"; exit 2; }
|
||||||
|
AUTH_FILE=$2; shift 2 ;;
|
||||||
for key_value in "$tavily_value" "$exa_value" "$keenable_value"; do
|
-h|--help) usage; exit 0 ;;
|
||||||
[ -z "$key_value" ] || valid_key "$key_value" || {
|
*) warn "未知参数:$1"; usage >&2; exit 2 ;;
|
||||||
warn "API key 只能包含字母、数字、点、下划线和连字符。"
|
esac
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
done
|
done
|
||||||
|
|
||||||
config_dir=${CONFIG_FILE%/*}
|
if [ "$mode" = interactive ]; then
|
||||||
[ "$config_dir" = "$CONFIG_FILE" ] && config_dir=.
|
for provider in tavily exa keenable; do
|
||||||
umask 077
|
case $provider in
|
||||||
mkdir -p "$config_dir" || exit 1
|
tavily) label=Tavily ;;
|
||||||
config_tmp=$(mktemp "$config_dir/.search.env.XXXXXX") || exit 1
|
exa) label=Exa ;;
|
||||||
trap 'rm -f "$config_tmp"' EXIT HUP INT TERM
|
keenable) label=Keenable ;;
|
||||||
{
|
esac
|
||||||
printf '# Generated by my-pi/search_config.sh. Do not commit this file.\n'
|
if has_key "$provider"; then configured=1; else configured=0; fi
|
||||||
[ -z "$tavily_value" ] || printf 'export TAVILY_API_KEY=%s\n' "$tavily_value"
|
value=$(prompt_key "$label" "$configured") || exit $?
|
||||||
[ -z "$exa_value" ] || printf 'export EXA_API_KEY=%s\n' "$exa_value"
|
if [ -n "$value" ]; then
|
||||||
[ -z "$keenable_value" ] || printf 'export KEENABLE_API_KEY=%s\n' "$keenable_value"
|
run_auth --set "$provider=$value" || exit $?
|
||||||
} > "$config_tmp" || exit 1
|
did_work=1
|
||||||
chmod 600 "$config_tmp" || exit 1
|
fi
|
||||||
mv "$config_tmp" "$CONFIG_FILE" || exit 1
|
done
|
||||||
trap - EXIT HUP INT TERM
|
[ "$did_work" -eq 1 ] || say "未提供任何 key,auth.json 保持不变。"
|
||||||
|
fi
|
||||||
say "搜索 API key 已安全写入 ${CONFIG_FILE}(权限 600)。"
|
|
||||||
say "重启 Pi 后生效;显式环境变量仍优先于该文件。"
|
|
||||||
|
|||||||
@@ -0,0 +1,208 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdir, mkdtemp, rm, utimes, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import {
|
||||||
|
buildExploreArgs,
|
||||||
|
createCodegraphExploreHandler,
|
||||||
|
findIndexedRoot,
|
||||||
|
isNotIndexedOutput,
|
||||||
|
type CodegraphExec,
|
||||||
|
type ExploreParams,
|
||||||
|
} from "../extensions/codegraph.ts";
|
||||||
|
|
||||||
|
async function makeIndexedProject(): Promise<{ root: string; cleanup: () => Promise<void> }> {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
await mkdir(join(root, ".codegraph"));
|
||||||
|
await writeFile(join(root, ".codegraph", "codegraph.db"), "");
|
||||||
|
return { root, cleanup: () => rm(root, { recursive: true, force: true }) };
|
||||||
|
}
|
||||||
|
|
||||||
|
function mockCtx(cwd: string, confirm?: () => Promise<boolean>) {
|
||||||
|
return {
|
||||||
|
cwd,
|
||||||
|
hasUI: confirm !== undefined,
|
||||||
|
ui: { confirm: confirm ?? (async () => false) },
|
||||||
|
} as never;
|
||||||
|
}
|
||||||
|
|
||||||
|
function mockExec(handler: (command: string, args: string[]) => { code?: number; stdout?: string; stderr?: string }) {
|
||||||
|
const calls: Array<{ command: string; args: string[] }> = [];
|
||||||
|
const exec: CodegraphExec = async (command, args) => {
|
||||||
|
calls.push({ command, args });
|
||||||
|
const result = handler(command, args);
|
||||||
|
return { code: result.code ?? 0, stdout: result.stdout ?? "", stderr: result.stderr ?? "", killed: false };
|
||||||
|
};
|
||||||
|
return { exec, calls };
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeHandler(exec: CodegraphExec) {
|
||||||
|
return {
|
||||||
|
handle: createCodegraphExploreHandler(exec, new Map(), new Map()),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
test("findIndexedRoot locates the nearest ancestor index", async () => {
|
||||||
|
const { root, cleanup } = await makeIndexedProject();
|
||||||
|
try {
|
||||||
|
const nested = join(root, "src", "deep");
|
||||||
|
await mkdir(nested, { recursive: true });
|
||||||
|
assert.equal(await findIndexedRoot(nested), root);
|
||||||
|
assert.equal(await findIndexedRoot(root), root);
|
||||||
|
} finally {
|
||||||
|
await cleanup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("findIndexedRoot returns undefined when no index exists", async () => {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
try {
|
||||||
|
assert.equal(await findIndexedRoot(root), undefined);
|
||||||
|
} finally {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("buildExploreArgs maps params to CLI argv", () => {
|
||||||
|
assert.deepEqual(buildExploreArgs({ query: "auth flow" }, "/repo"), ["explore", "-p", "/repo", "auth flow"]);
|
||||||
|
assert.deepEqual(
|
||||||
|
buildExploreArgs({ query: "q", maxFiles: 5, projectPath: "/ignored" }, "/repo"),
|
||||||
|
["explore", "-p", "/repo", "--max-files", "5", "q"],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("isNotIndexedOutput detects the upstream marker", () => {
|
||||||
|
assert.equal(isNotIndexedOutput("✗ CodeGraph isn't available here — no .codegraph/ index exists in /x."), true);
|
||||||
|
assert.equal(isNotIndexedOutput("Found 3 symbols"), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("explore runs the CLI and caches by database mtime", async () => {
|
||||||
|
const { root, cleanup } = await makeIndexedProject();
|
||||||
|
const { exec, calls } = mockExec(() => ({ stdout: "explore output" }));
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const first = await handle({ query: "auth" }, mockCtx(root));
|
||||||
|
assert.equal(first.content[0].text, "explore output");
|
||||||
|
const second = await handle({ query: "auth" }, mockCtx(root));
|
||||||
|
assert.equal(second.content[0].text, "explore output");
|
||||||
|
assert.equal(calls.length, 1, "identical query on unchanged index must hit the cache");
|
||||||
|
|
||||||
|
// Simulate a reindex: bump the db mtime.
|
||||||
|
await utimes(join(root, ".codegraph", "codegraph.db"), new Date(), new Date(Date.now() + 5000));
|
||||||
|
await handle({ query: "auth" }, mockCtx(root));
|
||||||
|
assert.equal(calls.length, 2, "mtime change must invalidate the cached result");
|
||||||
|
} finally {
|
||||||
|
await cleanup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("uninitialized project asks the user and indexes on approval", async () => {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
const { exec, calls } = mockExec((_command, args) => {
|
||||||
|
if (args[0] === "init") {
|
||||||
|
// Emulate codegraph init creating the index.
|
||||||
|
return { stdout: "indexed" };
|
||||||
|
}
|
||||||
|
return { stdout: "explore output" };
|
||||||
|
});
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const result = await handle({ query: "auth" }, mockCtx(root, async () => true));
|
||||||
|
assert.equal(result.content[0].text, "explore output");
|
||||||
|
assert.deepEqual(
|
||||||
|
calls.map(({ args }) => args[0]),
|
||||||
|
["init", "explore"],
|
||||||
|
);
|
||||||
|
assert.deepEqual(calls[0].args, ["init", root]);
|
||||||
|
} finally {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("uninitialized project bounces to built-in tools when declined", async () => {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
const { exec, calls } = mockExec(() => ({ stdout: "unreachable" }));
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const result = await handle({ query: "auth" }, mockCtx(root, async () => false));
|
||||||
|
assert.match(result.content[0].text ?? "", /find\/grep\/read\/LSP/);
|
||||||
|
assert.equal(calls.length, 0, "declined init must not spawn the CLI at all");
|
||||||
|
} finally {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("concurrent calls on an uninitialized project share one init decision", async () => {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
let confirmCalls = 0;
|
||||||
|
const { exec, calls } = mockExec(() => ({ stdout: "ok" }));
|
||||||
|
const pending = new Map<string, Promise<string | undefined>>();
|
||||||
|
const handle = createCodegraphExploreHandler(exec, new Map(), pending);
|
||||||
|
try {
|
||||||
|
const [a, b] = await Promise.all([
|
||||||
|
handle({ query: "one" }, mockCtx(root, async () => {
|
||||||
|
confirmCalls += 1;
|
||||||
|
return true;
|
||||||
|
})),
|
||||||
|
handle({ query: "two" }, mockCtx(root, async () => {
|
||||||
|
confirmCalls += 1;
|
||||||
|
return true;
|
||||||
|
})),
|
||||||
|
]);
|
||||||
|
assert.equal(a.content[0].text, "ok");
|
||||||
|
assert.equal(b.content[0].text, "ok");
|
||||||
|
assert.equal(confirmCalls, 1, "parallel callers must not double-prompt");
|
||||||
|
assert.equal(
|
||||||
|
calls.filter(({ args }) => args[0] === "init").length,
|
||||||
|
1,
|
||||||
|
"parallel callers must share a single init",
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("init failure is reported instead of exploring", async () => {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
const { exec } = mockExec((_command, args) =>
|
||||||
|
args[0] === "init" ? { code: 1, stderr: "boom" } : { stdout: "unreachable" },
|
||||||
|
);
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const result = await handle({ query: "auth" }, mockCtx(root, async () => true));
|
||||||
|
assert.match(result.content[0].text ?? "", /codegraph init 失败/);
|
||||||
|
} finally {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("missing CLI surfaces an install hint instead of throwing", async () => {
|
||||||
|
const { root, cleanup } = await makeIndexedProject();
|
||||||
|
const exec: CodegraphExec = async () => {
|
||||||
|
throw new Error("spawn codegraph ENOENT");
|
||||||
|
};
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const result = await handle({ query: "auth" }, mockCtx(root));
|
||||||
|
assert.match(result.content[0].text ?? "", /无法执行 codegraph CLI/);
|
||||||
|
} finally {
|
||||||
|
await cleanup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("projectPath overrides the session cwd", async () => {
|
||||||
|
const { root, cleanup } = await makeIndexedProject();
|
||||||
|
const other = await mkdtemp(join(tmpdir(), "my-pi-codegraph-"));
|
||||||
|
const { exec, calls } = mockExec(() => ({ stdout: "ok" }));
|
||||||
|
const { handle } = makeHandler(exec);
|
||||||
|
try {
|
||||||
|
const params: ExploreParams = { query: "auth", projectPath: root };
|
||||||
|
await handle(params, mockCtx(other));
|
||||||
|
assert.deepEqual(calls[0].args, ["explore", "-p", root, "auth"]);
|
||||||
|
} finally {
|
||||||
|
await cleanup();
|
||||||
|
await rm(other, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import { createExaToolApiOverride, mapExaToolName, prefixExaTool } from "../extensions/exa-tool-prefix.ts";
|
|
||||||
|
|
||||||
test("Exa tools use a provider prefix followed by the original semantic name", () => {
|
|
||||||
assert.equal(mapExaToolName("web_search_exa"), "exa_web_search");
|
|
||||||
assert.equal(mapExaToolName("web_fetch_exa"), "exa_web_fetch");
|
|
||||||
assert.equal(mapExaToolName("web_search_advanced_exa"), "exa_web_search_advanced");
|
|
||||||
assert.equal(mapExaToolName("unrelated_tool"), "unrelated_tool");
|
|
||||||
|
|
||||||
const tool = prefixExaTool({ name: "web_search_exa", label: "MCP: web_search_exa" });
|
|
||||||
assert.equal(tool.name, "exa_web_search");
|
|
||||||
assert.equal(tool.label, "Exa Web Search");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("Exa API override keeps registered and active tool names aligned", () => {
|
|
||||||
const registered: Array<{ name: string; label: string }> = [];
|
|
||||||
let active: string[] = [];
|
|
||||||
const unregistered: string[] = [];
|
|
||||||
const api = createExaToolApiOverride({
|
|
||||||
registerTool(tool: unknown) {
|
|
||||||
registered.push(tool as { name: string; label: string });
|
|
||||||
},
|
|
||||||
setActiveTools(names: string[]) {
|
|
||||||
active = names;
|
|
||||||
},
|
|
||||||
unregisterTool(name: string) {
|
|
||||||
unregistered.push(name);
|
|
||||||
return true;
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
api.registerTool({ name: "web_fetch_exa", label: "MCP: web_fetch_exa" });
|
|
||||||
api.registerTool({ name: "codegraph_explore", label: "MCP: codegraph_explore" });
|
|
||||||
api.setActiveTools(["read", "web_fetch_exa", "web_fetch_exa", "codegraph_explore"]);
|
|
||||||
api.unregisterTool("web_fetch_exa");
|
|
||||||
api.unregisterTool("codegraph_explore");
|
|
||||||
|
|
||||||
assert.equal(registered[0]?.name, "exa_web_fetch");
|
|
||||||
assert.equal(registered[0]?.label, "Exa Web Fetch");
|
|
||||||
assert.equal(registered[1]?.name, "codegraph_explore");
|
|
||||||
assert.deepEqual(active, ["read", "exa_web_fetch", "codegraph_explore"]);
|
|
||||||
assert.deepEqual(unregistered, ["exa_web_fetch", "codegraph_explore"]);
|
|
||||||
});
|
|
||||||
@@ -32,8 +32,18 @@ test("all package extensions load together without global registration conflicts
|
|||||||
allowScripts: Record<string, boolean>;
|
allowScripts: Record<string, boolean>;
|
||||||
};
|
};
|
||||||
assert.ok(
|
assert.ok(
|
||||||
|
packageJson.pi.extensions.includes("./extensions/codegraph.ts"),
|
||||||
|
"the bundle must expose the native codegraph_explore tool",
|
||||||
|
);
|
||||||
|
assert.ok(
|
||||||
|
packageJson.pi.extensions.includes("./pi-web-search/index.ts"),
|
||||||
|
"the bundle must load the unified pi-web-search tools",
|
||||||
|
);
|
||||||
|
assert.equal(packageJson.dependencies["pi-web-search"], "file:./pi-web-search");
|
||||||
|
assert.equal(
|
||||||
packageJson.pi.extensions.includes("./extensions/plugin-init.ts"),
|
packageJson.pi.extensions.includes("./extensions/plugin-init.ts"),
|
||||||
"the bundle must expose the project plugin initializer command",
|
false,
|
||||||
|
"the retired plugin initializer command must not be loaded",
|
||||||
);
|
);
|
||||||
assert.ok(
|
assert.ok(
|
||||||
packageJson.pi.extensions.includes("./pi-hermes-memory/src/index.ts"),
|
packageJson.pi.extensions.includes("./pi-hermes-memory/src/index.ts"),
|
||||||
@@ -191,12 +201,8 @@ test("all package extensions load together without global registration conflicts
|
|||||||
HOME: home,
|
HOME: home,
|
||||||
XDG_CONFIG_HOME: join(home, ".config"),
|
XDG_CONFIG_HOME: join(home, ".config"),
|
||||||
PI_CODING_AGENT_DIR: join(home, ".pi-agent"),
|
PI_CODING_AGENT_DIR: join(home, ".pi-agent"),
|
||||||
MY_PI_SEARCH_CONFIG: join(home, "missing-search.env"),
|
|
||||||
PATH: `${mockBin}:${process.env.PATH ?? ""}`,
|
PATH: `${mockBin}:${process.env.PATH ?? ""}`,
|
||||||
PI_OFFLINE: "1",
|
PI_OFFLINE: "1",
|
||||||
TAVILY_API_KEY: "",
|
|
||||||
EXA_API_KEY: "",
|
|
||||||
KEENABLE_API_KEY: "",
|
|
||||||
};
|
};
|
||||||
const runBundle = () => spawnSync("pi", args, {
|
const runBundle = () => spawnSync("pi", args, {
|
||||||
encoding: "utf8",
|
encoding: "utf8",
|
||||||
|
|||||||
@@ -1,73 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import { readFile } from "node:fs/promises";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import { createMcpConfig, EXA_TOOLS } from "../extensions/mcp-config.ts";
|
|
||||||
|
|
||||||
test("one MCP adapter config contains both Exa and CodeGraph", () => {
|
|
||||||
assert.deepEqual(createMcpConfig({}), {
|
|
||||||
settings: {
|
|
||||||
disableProxyTool: true,
|
|
||||||
scriptMode: false,
|
|
||||||
},
|
|
||||||
mcpServers: {
|
|
||||||
exa: {
|
|
||||||
url: `https://mcp.exa.ai/mcp?tools=${EXA_TOOLS.join(",")}`,
|
|
||||||
protocolVersion: "auto",
|
|
||||||
lifecycle: "eager",
|
|
||||||
directTools: EXA_TOOLS,
|
|
||||||
includeTools: EXA_TOOLS,
|
|
||||||
toolPrefix: "none",
|
|
||||||
approveTools: false,
|
|
||||||
exposeResources: false,
|
|
||||||
},
|
|
||||||
codegraph: {
|
|
||||||
command: "codegraph",
|
|
||||||
args: ["serve", "--mcp"],
|
|
||||||
lifecycle: "keep-alive",
|
|
||||||
directTools: ["codegraph_explore"],
|
|
||||||
includeTools: ["codegraph_explore"],
|
|
||||||
toolPrefix: "none",
|
|
||||||
approveTools: false,
|
|
||||||
exposeResources: false,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
test("Exa API key is resolved from the environment into a header and never placed in the URL", () => {
|
|
||||||
const config = createMcpConfig({ EXA_API_KEY: "exa-test-key" });
|
|
||||||
assert.deepEqual(config.mcpServers.exa.headers, { "x-api-key": "$env:EXA_API_KEY" });
|
|
||||||
assert.equal(config.mcpServers.exa.url?.includes("exa-test-key"), false);
|
|
||||||
assert.equal(JSON.stringify(config).includes("exa-test-key"), false);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("the package has exactly one local MCP adapter owner", async () => {
|
|
||||||
const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")) as {
|
|
||||||
pi: { extensions: string[] };
|
|
||||||
};
|
|
||||||
const localEntries = packageJson.pi.extensions.filter((entry) => !entry.startsWith("./node_modules/"));
|
|
||||||
const adapterOwners: string[] = [];
|
|
||||||
for (const entry of localEntries) {
|
|
||||||
const source = await readFile(new URL(`../${entry.slice(2)}`, import.meta.url), "utf8");
|
|
||||||
if (source.includes("createMcpAdapter") || source.includes('from "pi-mcp-adapter"')) {
|
|
||||||
adapterOwners.push(entry);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert.deepEqual(adapterOwners, ["./extensions/mcp.ts"]);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search credentials load before all search providers", async () => {
|
|
||||||
const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")) as {
|
|
||||||
pi: { extensions: string[] };
|
|
||||||
};
|
|
||||||
const entries = packageJson.pi.extensions;
|
|
||||||
const loaderIndex = entries.indexOf("./extensions/search-config.ts");
|
|
||||||
for (const provider of [
|
|
||||||
"./extensions/tavily-override.ts",
|
|
||||||
"./node_modules/@keenable/pi-search/src/index.ts",
|
|
||||||
"./extensions/mcp.ts",
|
|
||||||
]) {
|
|
||||||
assert.ok(loaderIndex >= 0 && loaderIndex < entries.indexOf(provider), `${provider} must load after search config`);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
||||||
import assert from "node:assert/strict";
|
|
||||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
|
||||||
import { homedir, tmpdir } from "node:os";
|
|
||||||
import { join, parse } from "node:path";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import pluginInitExtension, {
|
|
||||||
findMissingPluginInitializers,
|
|
||||||
initializeMissingPlugins,
|
|
||||||
validatePluginInitCwd,
|
|
||||||
} from "../extensions/plugin-init.ts";
|
|
||||||
|
|
||||||
test("initializes missing CodeGraph state after preflighting the CLI", async () => {
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
const calls: Array<{ command: string; args: string[]; cwd: string }> = [];
|
|
||||||
try {
|
|
||||||
const targets = await findMissingPluginInitializers(cwd);
|
|
||||||
assert.deepEqual(targets.map(({ name }) => name), ["CodeGraph"]);
|
|
||||||
|
|
||||||
await initializeMissingPlugins(cwd, targets, async (command, args, options) => {
|
|
||||||
calls.push({ command, args, cwd: options.cwd });
|
|
||||||
return { code: 0, stdout: "ok", stderr: "" };
|
|
||||||
});
|
|
||||||
|
|
||||||
assert.deepEqual(calls, [
|
|
||||||
{ command: "codegraph", args: ["--version"], cwd },
|
|
||||||
{ command: "codegraph", args: ["init", cwd], cwd },
|
|
||||||
]);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
test("uses the CLI's database file as the authoritative initialization marker", async () => {
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
const calls: string[] = [];
|
|
||||||
try {
|
|
||||||
await mkdir(join(cwd, ".codegraph"));
|
|
||||||
assert.deepEqual(
|
|
||||||
(await findMissingPluginInitializers(cwd)).map(({ name }) => name),
|
|
||||||
["CodeGraph"],
|
|
||||||
"a bare state directory must not count as initialized",
|
|
||||||
);
|
|
||||||
|
|
||||||
await writeFile(join(cwd, ".codegraph", "codegraph.db"), "");
|
|
||||||
const targets = await findMissingPluginInitializers(cwd);
|
|
||||||
assert.deepEqual(targets, []);
|
|
||||||
|
|
||||||
await initializeMissingPlugins(cwd, targets, async (command, args) => {
|
|
||||||
calls.push(`${command} ${args.join(" ")}`);
|
|
||||||
return { code: 0, stdout: "ok", stderr: "" };
|
|
||||||
});
|
|
||||||
assert.deepEqual(calls, ["codegraph --version"]);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects unsafe project roots", () => {
|
|
||||||
assert.throws(() => validatePluginInitCwd(parse(homedir()).root), /文件系统根目录/);
|
|
||||||
assert.throws(() => validatePluginInitCwd(homedir()), /用户主目录/);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("preflights the CLI before creating project state", async () => {
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
const calls: string[] = [];
|
|
||||||
try {
|
|
||||||
const targets = await findMissingPluginInitializers(cwd);
|
|
||||||
await assert.rejects(
|
|
||||||
initializeMissingPlugins(cwd, targets, async (command, args) => {
|
|
||||||
calls.push(`${command} ${args.join(" ")}`);
|
|
||||||
return { code: 1, stdout: "", stderr: "not installed" };
|
|
||||||
}),
|
|
||||||
/CodeGraph 检测失败:not installed/,
|
|
||||||
);
|
|
||||||
assert.deepEqual(calls, ["codegraph --version"]);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
test("plugin_init confirms the target and hot-reloads after initialization", async () => {
|
|
||||||
let command: { handler: (args: string, ctx: any) => Promise<void> } | undefined;
|
|
||||||
const calls: string[] = [];
|
|
||||||
const pi = {
|
|
||||||
registerCommand(name: string, definition: typeof command) {
|
|
||||||
assert.equal(name, "plugin_init");
|
|
||||||
command = definition;
|
|
||||||
},
|
|
||||||
exec: async (program: string, args: string[]) => {
|
|
||||||
calls.push(`${program} ${args.join(" ")}`);
|
|
||||||
return { code: 0, stdout: "ok", stderr: "" };
|
|
||||||
},
|
|
||||||
} as unknown as ExtensionAPI;
|
|
||||||
pluginInitExtension(pi);
|
|
||||||
assert.ok(command);
|
|
||||||
const registeredCommand = command!;
|
|
||||||
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
let reloaded = false;
|
|
||||||
let confirmation = "";
|
|
||||||
try {
|
|
||||||
await registeredCommand.handler("", {
|
|
||||||
cwd,
|
|
||||||
hasUI: true,
|
|
||||||
ui: {
|
|
||||||
confirm: async (_title: string, message: string) => {
|
|
||||||
confirmation = message;
|
|
||||||
return true;
|
|
||||||
},
|
|
||||||
notify() {},
|
|
||||||
},
|
|
||||||
reload: async () => {
|
|
||||||
reloaded = true;
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.match(confirmation, new RegExp(cwd.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")));
|
|
||||||
assert.deepEqual(calls, [
|
|
||||||
"codegraph --version",
|
|
||||||
`codegraph init ${cwd}`,
|
|
||||||
]);
|
|
||||||
assert.equal(reloaded, true);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
test("plugin_init reports the failure and does not reload when initialization fails", async () => {
|
|
||||||
let command: { handler: (args: string, ctx: any) => Promise<void> } | undefined;
|
|
||||||
const pi = {
|
|
||||||
registerCommand(_name: string, definition: typeof command) {
|
|
||||||
command = definition;
|
|
||||||
},
|
|
||||||
exec: async (program: string, args: string[]) => ({
|
|
||||||
code: program === "codegraph" && args[0] === "init" ? 1 : 0,
|
|
||||||
stdout: "",
|
|
||||||
stderr: "mock failure",
|
|
||||||
}),
|
|
||||||
} as unknown as ExtensionAPI;
|
|
||||||
pluginInitExtension(pi);
|
|
||||||
assert.ok(command);
|
|
||||||
const registeredCommand = command!;
|
|
||||||
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
let reloaded = false;
|
|
||||||
let error = "";
|
|
||||||
try {
|
|
||||||
await registeredCommand.handler("", {
|
|
||||||
cwd,
|
|
||||||
hasUI: true,
|
|
||||||
ui: {
|
|
||||||
confirm: async () => true,
|
|
||||||
notify(message: string, level: string) {
|
|
||||||
if (level === "error") error = message;
|
|
||||||
},
|
|
||||||
},
|
|
||||||
reload: async () => {
|
|
||||||
reloaded = true;
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.match(error, /CodeGraph 初始化失败:mock failure/);
|
|
||||||
assert.equal(reloaded, false);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
test("plugin_init stops without executing or reloading when confirmation is cancelled", async () => {
|
|
||||||
let command: { handler: (args: string, ctx: any) => Promise<void> } | undefined;
|
|
||||||
let executed = false;
|
|
||||||
const pi = {
|
|
||||||
registerCommand(_name: string, definition: typeof command) {
|
|
||||||
command = definition;
|
|
||||||
},
|
|
||||||
exec: async () => {
|
|
||||||
executed = true;
|
|
||||||
return { code: 0, stdout: "", stderr: "" };
|
|
||||||
},
|
|
||||||
} as unknown as ExtensionAPI;
|
|
||||||
pluginInitExtension(pi);
|
|
||||||
const registeredCommand = command!;
|
|
||||||
|
|
||||||
const cwd = await mkdtemp(join(tmpdir(), "my-pi-plugin-init-"));
|
|
||||||
let reloaded = false;
|
|
||||||
try {
|
|
||||||
await registeredCommand.handler("", {
|
|
||||||
cwd,
|
|
||||||
hasUI: true,
|
|
||||||
ui: { confirm: async () => false, notify() {} },
|
|
||||||
reload: async () => {
|
|
||||||
reloaded = true;
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.equal(executed, false);
|
|
||||||
assert.equal(reloaded, false);
|
|
||||||
} finally {
|
|
||||||
await rm(cwd, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
@@ -0,0 +1,252 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtemp, readFile, stat, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { spawnSync } from "node:child_process";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import {
|
||||||
|
getSearchApiKey,
|
||||||
|
getSearchAuthPath,
|
||||||
|
getSearchCredentialEnv,
|
||||||
|
parseAuthKeyTemplate,
|
||||||
|
resolveAuthKeyValue,
|
||||||
|
} from "../pi-web-search/auth.ts";
|
||||||
|
|
||||||
|
async function withAuthFile(
|
||||||
|
data: Record<string, unknown>,
|
||||||
|
fn: (authPath: string) => Promise<void> | void,
|
||||||
|
): Promise<void> {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify(data));
|
||||||
|
await fn(authPath);
|
||||||
|
}
|
||||||
|
|
||||||
|
test("auth path follows PI_CODING_AGENT_DIR and falls back to ~/.pi/agent", () => {
|
||||||
|
assert.equal(
|
||||||
|
getSearchAuthPath({ PI_CODING_AGENT_DIR: "/tmp/agent" }),
|
||||||
|
"/tmp/agent/auth.json",
|
||||||
|
);
|
||||||
|
assert.equal(getSearchAuthPath({ HOME: "/tmp/home" }), "/tmp/home/.pi/agent/auth.json");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("literal api_key credentials resolve verbatim", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{
|
||||||
|
tavily: { type: "api_key", key: "tvly-test" },
|
||||||
|
exa: { type: "api_key", key: " exa-test " },
|
||||||
|
keenable: { type: "api_key", key: "" },
|
||||||
|
anthropic: { type: "oauth", access: "a", refresh: "r", expires: 1 },
|
||||||
|
},
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-test");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath }), "exa-test");
|
||||||
|
assert.equal(getSearchApiKey("keenable", { authPath }), undefined);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("missing file, malformed json, non-api_key entries all resolve to undefined", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-"));
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath: join(dir, "absent.json") }), undefined);
|
||||||
|
const bad = join(dir, "bad.json");
|
||||||
|
await writeFile(bad, "not json{");
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath: bad }), undefined);
|
||||||
|
await withAuthFile({ tavily: { type: "oauth", access: "a", refresh: "r", expires: 1 } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
await withAuthFile({ tavily: "literal-string-entry" }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("$VAR and ${VAR} interpolate; credential env beats ambient env", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{
|
||||||
|
tavily: { type: "api_key", key: "$TAVILY_KEY" },
|
||||||
|
exa: { type: "api_key", key: "prefix-${EXA_KEY}-suffix" },
|
||||||
|
keenable: { type: "api_key", key: "$SHARED", env: { SHARED: "scoped-wins" } },
|
||||||
|
},
|
||||||
|
(authPath) => {
|
||||||
|
const env = { TAVILY_KEY: "from-env", EXA_KEY: "braced", SHARED: "ambient-loses" };
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env }), "from-env");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath, env }), "prefix-braced-suffix");
|
||||||
|
assert.equal(getSearchApiKey("keenable", { authPath, env }), "scoped-wins");
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a missing env variable resolves the whole key to undefined", async () => {
|
||||||
|
await withAuthFile({ tavily: { type: "api_key", key: "$NOPE_MISSING" } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("$$ and $! escape to literal characters", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{ tavily: { type: "api_key", key: "cost-is-$$5-and-$!literal" } },
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), "cost-is-$5-and-!literal");
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("!command executes and caches trimmed stdout", async () => {
|
||||||
|
const authPath = join(await mkdtemp(join(tmpdir(), "my-pi-auth-")), "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ tavily: { type: "api_key", key: "!printf secret-from-cmd" } }),
|
||||||
|
);
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "secret-from-cmd");
|
||||||
|
|
||||||
|
let calls = 0;
|
||||||
|
const runCommand = (command: string) => {
|
||||||
|
calls += 1;
|
||||||
|
assert.equal(command, "get-key --flag");
|
||||||
|
return "injected";
|
||||||
|
};
|
||||||
|
await withAuthFile({ exa: { type: "api_key", key: "!get-key --flag" } }, (otherPath) => {
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a failing !command resolves to undefined", async () => {
|
||||||
|
await withAuthFile({ tavily: { type: "api_key", key: "!exit 3" } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("credential env map exposes non-key config like KEENABLE_API_URL", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{ keenable: { type: "api_key", key: "k", env: { KEENABLE_API_URL: "https://keenable.example" } } },
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(
|
||||||
|
getSearchCredentialEnv("keenable", "KEENABLE_API_URL", { authPath }),
|
||||||
|
"https://keenable.example",
|
||||||
|
);
|
||||||
|
assert.equal(getSearchCredentialEnv("keenable", "MISSING", { authPath }), undefined);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("template parser handles env names, braces and escapes", () => {
|
||||||
|
assert.deepEqual(parseAuthKeyTemplate("$A_${B}-$$-$!-x"), [
|
||||||
|
{ type: "env", name: "A_" },
|
||||||
|
{ type: "env", name: "B" },
|
||||||
|
{ type: "literal", value: "-$-!-x" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
const AUTH_MJS = new URL("../scripts/search-auth.mjs", import.meta.url).pathname;
|
||||||
|
const SEARCH_SH = new URL("../search_config.sh", import.meta.url).pathname;
|
||||||
|
|
||||||
|
function runMjs(args: string[], env: Record<string, string> = {}) {
|
||||||
|
return spawnSync("node", [AUTH_MJS, ...args], { encoding: "utf8", env: { ...process.env, ...env } });
|
||||||
|
}
|
||||||
|
|
||||||
|
test("search-auth.mjs --set writes api_key entries and preserves other providers", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ anthropic: { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" } }),
|
||||||
|
);
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "tavily=tvly-new", "--set", "exa=!op read x"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.anthropic, { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" });
|
||||||
|
assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-new" });
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "!op read x" });
|
||||||
|
assert.equal((await stat(authPath)).mode & 0o777, 0o600);
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-new");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs preserves an existing credential env map on --set", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ keenable: { type: "api_key", key: "old", env: { KEENABLE_API_URL: "https://k.example" } } }),
|
||||||
|
);
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "keenable=new-key"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.keenable, {
|
||||||
|
type: "api_key",
|
||||||
|
key: "new-key",
|
||||||
|
env: { KEENABLE_API_URL: "https://k.example" },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs creates a missing auth.json and honors PI_CODING_AGENT_DIR", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const agentDir = join(dir, "agent");
|
||||||
|
const result = runMjs(["--set", "keenable=keen-1"], { PI_CODING_AGENT_DIR: agentDir });
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(join(agentDir, "auth.json"), "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.keenable, { type: "api_key", key: "keen-1" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --unset removes only the named provider", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "t" }, exa: { type: "api_key", key: "e" } }));
|
||||||
|
const result = runMjs(["--auth", authPath, "--unset", "tavily"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.equal(written.tavily, undefined);
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "e" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs refuses to clobber a malformed auth.json", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, "{oops");
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "tavily=x"]);
|
||||||
|
assert.notEqual(result.status, 0);
|
||||||
|
assert.match(result.stderr, /不是合法 JSON|abort|clobber/i);
|
||||||
|
assert.equal(await readFile(authPath, "utf8"), "{oops");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --migrate imports search.env keys without overwriting existing entries", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "existing" } }));
|
||||||
|
const envPath = join(dir, "search.env");
|
||||||
|
await writeFile(envPath, "export TAVILY_API_KEY=old\nEXA_API_KEY=exa-file\nKEENABLE_API_KEY=bad$(cmd)\n");
|
||||||
|
const result = runMjs(["--auth", authPath, "--migrate", envPath]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
assert.match(result.stdout, /已迁移 exa/);
|
||||||
|
assert.match(result.stdout, /tavily/);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.equal(written.tavily.key, "existing");
|
||||||
|
assert.equal(written.exa.key, "exa-file");
|
||||||
|
assert.equal(written.keenable, undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --has and --list never print key material", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "supersecret" } }));
|
||||||
|
assert.equal(runMjs(["--auth", authPath, "--has", "tavily"]).status, 0);
|
||||||
|
assert.equal(runMjs(["--auth", authPath, "--has", "exa"]).status, 1);
|
||||||
|
const list = runMjs(["--auth", authPath, "--list"]);
|
||||||
|
assert.equal(list.status, 0);
|
||||||
|
assert.match(list.stdout, /tavily: 已配置/);
|
||||||
|
assert.doesNotMatch(list.stdout, /supersecret/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search_config.sh writes keys through search-auth.mjs noninteractively", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
const result = spawnSync("sh", [SEARCH_SH, "--auth", authPath, "--tavily", "tvly-sh", "--exa", "exa-sh"], {
|
||||||
|
encoding: "utf8",
|
||||||
|
});
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-sh" });
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "exa-sh" });
|
||||||
|
assert.equal((await stat(authPath)).mode & 0o777, 0o600);
|
||||||
|
});
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import { mkdtemp, readFile, stat } from "node:fs/promises";
|
|
||||||
import { tmpdir } from "node:os";
|
|
||||||
import { join } from "node:path";
|
|
||||||
import { spawnSync } from "node:child_process";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import {
|
|
||||||
getSearchConfigPath,
|
|
||||||
loadSearchConfigEnv,
|
|
||||||
parseSearchConfig,
|
|
||||||
} from "../extensions/search-config.ts";
|
|
||||||
|
|
||||||
test("search config parses only supported safe assignments", () => {
|
|
||||||
assert.deepEqual(
|
|
||||||
parseSearchConfig(`
|
|
||||||
export TAVILY_API_KEY=tvly-test_1
|
|
||||||
EXA_API_KEY=exa-test.2
|
|
||||||
UNKNOWN_KEY=ignored
|
|
||||||
KEENABLE_API_KEY=$(not-allowed)
|
|
||||||
`),
|
|
||||||
{
|
|
||||||
TAVILY_API_KEY: "tvly-test_1",
|
|
||||||
EXA_API_KEY: "exa-test.2",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search config does not override explicit environment values", async () => {
|
|
||||||
const root = await mkdtemp(join(tmpdir(), "my-pi-search-config-"));
|
|
||||||
const configPath = join(root, "search.env");
|
|
||||||
await import("node:fs/promises").then(({ writeFile }) =>
|
|
||||||
writeFile(configPath, "export TAVILY_API_KEY=file-value\nexport EXA_API_KEY=exa-file\n"),
|
|
||||||
);
|
|
||||||
const env: Record<string, string | undefined> = { TAVILY_API_KEY: "explicit-value" };
|
|
||||||
const effective = loadSearchConfigEnv(env, configPath);
|
|
||||||
assert.equal(env.TAVILY_API_KEY, "explicit-value");
|
|
||||||
assert.equal(env.EXA_API_KEY, "exa-file");
|
|
||||||
assert.deepEqual(effective, { TAVILY_API_KEY: "explicit-value", EXA_API_KEY: "exa-file" });
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search config path follows XDG_CONFIG_HOME", () => {
|
|
||||||
assert.equal(getSearchConfigPath({ XDG_CONFIG_HOME: "/tmp/config" }), "/tmp/config/my-pi/search.env");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search_config.sh supports noninteractive updates and preserves omitted keys", async () => {
|
|
||||||
const home = await mkdtemp(join(tmpdir(), "my-pi-search-script-"));
|
|
||||||
const script = new URL("../search_config.sh", import.meta.url);
|
|
||||||
let result = spawnSync("sh", [script.pathname, "--tavily", "tvly-test", "--exa", "exa-test"], {
|
|
||||||
env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") },
|
|
||||||
encoding: "utf8",
|
|
||||||
});
|
|
||||||
assert.equal(result.status, 0, result.stderr);
|
|
||||||
|
|
||||||
result = spawnSync("sh", [script.pathname, "--keenable", "keen-test"], {
|
|
||||||
env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") },
|
|
||||||
encoding: "utf8",
|
|
||||||
});
|
|
||||||
assert.equal(result.status, 0, result.stderr);
|
|
||||||
|
|
||||||
const configPath = join(home, ".config", "my-pi", "search.env");
|
|
||||||
const content = await readFile(configPath, "utf8");
|
|
||||||
assert.match(content, /^export TAVILY_API_KEY=tvly-test$/m);
|
|
||||||
assert.match(content, /^export EXA_API_KEY=exa-test$/m);
|
|
||||||
assert.match(content, /^export KEENABLE_API_KEY=keen-test$/m);
|
|
||||||
assert.equal((await stat(configPath)).mode & 0o777, 0o600);
|
|
||||||
});
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import { prefixTavilyTool } from "../extensions/tavily-tool-prefix.ts";
|
|
||||||
|
|
||||||
test("Tavily override prefixes tool identity and internal tool references", () => {
|
|
||||||
const tool = prefixTavilyTool({
|
|
||||||
name: "web_search",
|
|
||||||
label: "Web Search",
|
|
||||||
description: "Use web_search, then web_fetch with Tavily.",
|
|
||||||
promptSnippet: "Call web_search.",
|
|
||||||
promptGuidelines: ["Use web_fetch after web_search."],
|
|
||||||
parameters: {} as never,
|
|
||||||
execute: async () => ({ content: [{ type: "text" as const, text: "ok" }] }),
|
|
||||||
});
|
|
||||||
|
|
||||||
assert.equal(tool.name, "tavily_web_search");
|
|
||||||
assert.equal(tool.label, "Tavily Web Search");
|
|
||||||
assert.equal(tool.description, "Use tavily_web_search, then tavily_web_fetch with Tavily.");
|
|
||||||
assert.equal(tool.promptSnippet, "Call tavily_web_search.");
|
|
||||||
assert.deepEqual(tool.promptGuidelines, ["Use tavily_web_fetch after tavily_web_search."]);
|
|
||||||
});
|
|
||||||
@@ -73,21 +73,15 @@ test("LSP routing covers navigation and post-edit diagnostics", () => {
|
|||||||
assert.match(section, /post-edit diagnostics/);
|
assert.match(section, /post-edit diagnostics/);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("online search routing selects providers by task", () => {
|
test("online search routing uses the unified web tools without naming providers", () => {
|
||||||
const tavily = buildToolRoutingSection(["tavily_web_search", "tavily_web_fetch"]);
|
const section = buildToolRoutingSection(["web_search", "web_fetch"]);
|
||||||
assert.match(tavily, /broad discovery when keywords are uncertain/);
|
assert.match(section, /Use web_search for current, real-world, or citable web information/);
|
||||||
assert.match(tavily, /news\/recent\/trending topics/);
|
assert.match(section, /provider selection is handled internally/);
|
||||||
assert.doesNotMatch(tavily, /official or API documentation/);
|
assert.match(section, /read selected pages with web_fetch/);
|
||||||
|
assert.doesNotMatch(section, /tavily|exa|keenable/iu);
|
||||||
|
|
||||||
const exa = buildToolRoutingSection(["exa_web_search", "exa_web_fetch", "exa_web_search_advanced"]);
|
const searchOnly = buildToolRoutingSection(["web_search"]);
|
||||||
assert.match(exa, /official or API documentation/);
|
assert.match(searchOnly, /Use web_search/);
|
||||||
assert.match(exa, /explicitly bound the result count and returned text length/);
|
|
||||||
assert.doesNotMatch(exa, /Chinese-language pages/);
|
|
||||||
|
|
||||||
const keenable = buildToolRoutingSection(["keenable_search", "keenable_fetch"]);
|
|
||||||
assert.match(keenable, /Chinese-language pages, policies and government notices/);
|
|
||||||
assert.match(keenable, /site-restricted search/);
|
|
||||||
assert.match(keenable, /publication-date filtering/);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("routing includes only guidance for active optional tools", () => {
|
test("routing includes only guidance for active optional tools", () => {
|
||||||
@@ -98,8 +92,7 @@ test("routing includes only guidance for active optional tools", () => {
|
|||||||
assert.doesNotMatch(section, /Use Context Mode/);
|
assert.doesNotMatch(section, /Use Context Mode/);
|
||||||
assert.doesNotMatch(section, /ctx_execute_file is confined/);
|
assert.doesNotMatch(section, /ctx_execute_file is confined/);
|
||||||
assert.doesNotMatch(section, /fresh LINE#HASH anchors/);
|
assert.doesNotMatch(section, /fresh LINE#HASH anchors/);
|
||||||
assert.doesNotMatch(section, /Choose one online search provider/);
|
assert.doesNotMatch(section, /Use web_search for current/);
|
||||||
assert.doesNotMatch(section, /broad discovery when keywords are uncertain/);
|
|
||||||
assert.match(section, /Use read only when exact source is needed/);
|
assert.match(section, /Use read only when exact source is needed/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { readFile } from "node:fs/promises";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")) as {
|
||||||
|
dependencies: Record<string, string>;
|
||||||
|
files: string[];
|
||||||
|
pi: { extensions: string[]; skills: string[] };
|
||||||
|
};
|
||||||
|
|
||||||
|
test("pi-web-search is the single search surface and reads credentials from auth.json", () => {
|
||||||
|
const entries = packageJson.pi.extensions;
|
||||||
|
assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search");
|
||||||
|
assert.equal(
|
||||||
|
entries.includes("./extensions/search-config.ts"),
|
||||||
|
false,
|
||||||
|
"search keys come from auth.json — the env loader extension is retired",
|
||||||
|
);
|
||||||
|
for (const retired of [
|
||||||
|
"./extensions/mcp.ts",
|
||||||
|
"./extensions/tavily-override.ts",
|
||||||
|
"./node_modules/@keenable/pi-search/src/index.ts",
|
||||||
|
]) {
|
||||||
|
assert.equal(entries.includes(retired), false, `${retired} must no longer be loaded`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("pi-web-search ships as a maintained file: dependency without upstream search packages", () => {
|
||||||
|
assert.equal(packageJson.dependencies["pi-web-search"], "file:./pi-web-search");
|
||||||
|
assert.ok(packageJson.files.includes("pi-web-search"));
|
||||||
|
assert.equal(packageJson.dependencies["@tavily/pi-extension"], undefined);
|
||||||
|
assert.equal(packageJson.dependencies["@keenable/pi-search"], undefined);
|
||||||
|
assert.equal(packageJson.dependencies["pi-mcp-adapter"], undefined, "Exa is a direct client now — no MCP adapter");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("no extension in the bundle loads the MCP adapter or upstream search extensions", async () => {
|
||||||
|
for (const entry of packageJson.pi.extensions.filter((e) => !e.startsWith("./node_modules/"))) {
|
||||||
|
const source = await readFile(new URL(`../${entry.slice(2)}`, import.meta.url), "utf8");
|
||||||
|
assert.equal(source.includes("pi-mcp-adapter"), false, `${entry} must not import pi-mcp-adapter`);
|
||||||
|
assert.equal(source.includes("@tavily/pi-extension"), false, `${entry} must not import @tavily/pi-extension`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the keenable skill path is gone with the upstream package", () => {
|
||||||
|
for (const skillPath of packageJson.pi.skills) {
|
||||||
|
assert.equal(skillPath.includes("@keenable"), false);
|
||||||
|
}
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user