mirror of
https://bitbucket.org/siakitem/my-pi.git
synced 2026-10-26 10:45:22 +00:00
feat(pi-web-search): 搜索凭据迁移到 Pi auth.json
三个搜索 key 改为 auth.json 中的 tavily/exa/keenable api_key 凭据, pi-web-search/auth.ts 复刻上游 resolve-config-value 语义(!command、 $VAR 插值、credential env 优先、$$/$! 转义)并在每次调用时读取, 进程环境变量与 search.env 不再被直接读取。search_config.sh 重写为 scripts/search-auth.mjs 的薄封装:加锁原子写、权限 600、保留其他 凭据条目,支持 --migrate 从旧 search.env 一次性导入。 Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
parent
ee1e6999d7
commit
0c9823ef5c
@@ -22,7 +22,7 @@
|
|||||||
- 该目录从明确 tag 导入并由本仓库直接维护,不使用 npm 预编译产物,不是 submodule,也不保留上游 `.pi` 状态;源码、测试、wrapper 与权限集成实现继续保留,但当前不属于根组合包运行时依赖,`extensions/subagents.ts` 也不在默认扩展加载列表中。
|
- 该目录从明确 tag 导入并由本仓库直接维护,不使用 npm 预编译产物,不是 submodule,也不保留上游 `.pi` 状态;源码、测试、wrapper 与权限集成实现继续保留,但当前不属于根组合包运行时依赖,`extensions/subagents.ts` 也不在默认扩展加载列表中。
|
||||||
- `extensions/fff-override.ts`:强制使用 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全;显式 CLI flag 仍遵循 FFF 官方优先级。
|
- `extensions/fff-override.ts`:强制使用 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全;显式 CLI flag 仍遵循 FFF 官方优先级。
|
||||||
- `pi-web-search/`:完全自维护的统一在线搜索扩展。对模型只暴露 `web_search` / `web_fetch`,schema 与提示词不含厂商名;`web_search` 由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`)按内置路由规则在 Tavily / Exa / Keenable 间选择并经统一参数翻译调用,`details` 记录实际 provider 与路由方式,决策按会话内 LRU 缓存,路由失败或被选厂商报错按可用性顺序降级;`web_fetch` 固定走 Exa `web_fetch_exa`,失败时静默降级 Keenable → Tavily。`clients/` 为纯 `fetch` 直连:Tavily 走 `api.tavily.com`(必须有 key),Keenable 复刻上游 client 保留 keyless `/public` 端点与 SSRF 防护,Exa 直接对 `mcp.exa.ai/mcp` 发 JSON-RPC(缓存 `mcp-session-id`,404 自动重建会话,key 只走 `x-api-key` 头),不使用 `pi-mcp-adapter` 或任何上游扩展包。
|
- `pi-web-search/`:完全自维护的统一在线搜索扩展。对模型只暴露 `web_search` / `web_fetch`,schema 与提示词不含厂商名;`web_search` 由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`)按内置路由规则在 Tavily / Exa / Keenable 间选择并经统一参数翻译调用,`details` 记录实际 provider 与路由方式,决策按会话内 LRU 缓存,路由失败或被选厂商报错按可用性顺序降级;`web_fetch` 固定走 Exa `web_fetch_exa`,失败时静默降级 Keenable → Tavily。`clients/` 为纯 `fetch` 直连:Tavily 走 `api.tavily.com`(必须有 key),Keenable 复刻上游 client 保留 keyless `/public` 端点与 SSRF 防护,Exa 直接对 `mcp.exa.ai/mcp` 发 JSON-RPC(缓存 `mcp-session-id`,404 自动重建会话,key 只走 `x-api-key` 头),不使用 `pi-mcp-adapter` 或任何上游扩展包。
|
||||||
- `search_config.sh` 与 `extensions/search-config.ts`:以交互式或参数方式把三家搜索 key 写入用户级 `search.env`(权限 `600`),并在搜索扩展初始化前加载;显式进程环境变量优先。
|
- `search_config.sh` 与 `scripts/search-auth.mjs`:以交互或参数方式把 `tavily` / `exa` / `keenable` 三条 `api_key` 凭据写入 Pi 的 `auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`);加锁后原子写、权限 `600`、保留全部其他条目,支持 `--migrate` 从旧 `search.env` 导入。`pi-web-search/auth.ts` 每次调用读取并复刻上游解析语义(`!command` 执行、`$VAR`/`${VAR}` 插值、credential `env` 优先于进程 env、`$$`/`$!` 转义);显式进程环境变量不再直接读取,env 注入通过 `"key": "$VAR"` 表达。
|
||||||
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,以 `execFile` argv 形式调用机器级 `codegraph` CLI(每次调用为短命进程,无常驻 server/daemon);未初始化项目先经 `ctx.ui.confirm` 确认再 `codegraph init` 并自动重试,拒绝或失败则把结果打回让模型改用 `find`/`grep`/`read`/LSP;结果按 `codegraph.db` mtime 做有界缓存,索引一变即失效。
|
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,以 `execFile` argv 形式调用机器级 `codegraph` CLI(每次调用为短命进程,无常驻 server/daemon);未初始化项目先经 `ctx.ui.confirm` 确认再 `codegraph init` 并自动重试,拒绝或失败则把结果打回让模型改用 `find`/`grep`/`read`/LSP;结果按 `codegraph.db` mtime 做有界缓存,索引一变即失效。
|
||||||
- `extensions/plugin-init.ts`:已移除;原 `/plugin_init` 的 CodeGraph 初始化职责由 `extensions/codegraph.ts` 在首次 `codegraph_explore` 调用内按需确认完成。
|
- `extensions/plugin-init.ts`:已移除;原 `/plugin_init` 的 CodeGraph 初始化职责由 `extensions/codegraph.ts` 在首次 `codegraph_explore` 调用内按需确认完成。
|
||||||
- `context-mode`:加载上游 Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。
|
- `context-mode`:加载上游 Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。
|
||||||
@@ -89,9 +89,9 @@
|
|||||||
- `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录,不提交到仓库,也不作为组合包运行时配置源。
|
- `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录,不提交到仓库,也不作为组合包运行时配置源。
|
||||||
- CodeGraph 是机器级依赖:`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后,扩展才可以 `codegraph init` 当前目录。init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。
|
- CodeGraph 是机器级依赖:`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后,扩展才可以 `codegraph init` 当前目录。init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。
|
||||||
- `codegraph_explore` 为原生 Pi 工具而非 MCP 工具:每次调用通过 `execFile`/`pi.exec` 以 argv 形式运行 `codegraph explore`(带超时、不经 shell),结果按 `.codegraph/codegraph.db` mtime 缓存;CLI 缺失、init 被拒绝或执行失败都不得抛出或阻止其他工具,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。
|
- `codegraph_explore` 为原生 Pi 工具而非 MCP 工具:每次调用通过 `execFile`/`pi.exec` 以 argv 形式运行 `codegraph explore`(带超时、不经 shell),结果按 `.codegraph/codegraph.db` mtime 缓存;CLI 缺失、init 被拒绝或执行失败都不得抛出或阻止其他工具,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。
|
||||||
- 组合包不加载任何 MCP adapter;Exa 由 `pi-web-search/clients/exa.ts` 以纯 `fetch` JSON-RPC 直连托管端点 `https://mcp.exa.ai/mcp`:`initialize` 一次并缓存 `mcp-session-id`,随后 `tools/call`,HTTP 404 视为会话失效自动重建并重试一次。`EXA_API_KEY` 通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。
|
- 组合包不加载任何 MCP adapter;Exa 由 `pi-web-search/clients/exa.ts` 以纯 `fetch` JSON-RPC 直连托管端点 `https://mcp.exa.ai/mcp`:`initialize` 一次并缓存 `mcp-session-id`,随后 `tools/call`,HTTP 404 视为会话失效自动重建并重试一次。`exa` 凭据通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。
|
||||||
- 在线搜索对模型只暴露 `web_search` / `web_fetch` 两个统一工具名,不得重新引入按厂商命名的工具;`web_search` 必须经 `router.ts` 的主模型分类选择 provider,`web_fetch` 固定优先 Exa。Exa 搜索统一走 `web_search_advanced_exa` 并显式约束 `numResults` 与 `textMaxCharacters`;Tavily 默认 `search_depth: basic` 且必须设置 `TAVILY_API_KEY` 才参与路由;Keenable keyless 可用,`KEENABLE_API_KEY` 仅提高速率限制。
|
- 在线搜索对模型只暴露 `web_search` / `web_fetch` 两个统一工具名,不得重新引入按厂商命名的工具;`web_search` 必须经 `router.ts` 的主模型分类选择 provider,`web_fetch` 固定优先 Exa。Exa 搜索统一走 `web_search_advanced_exa` 并显式约束 `numResults` 与 `textMaxCharacters`;Tavily 默认 `search_depth: basic` 且必须在 auth.json 存在 `tavily` api_key 凭据才参与路由;Keenable keyless 可用,`keenable` 凭据仅提高速率限制。
|
||||||
- 搜索 key 默认保存在 `${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env`,文件必须为 `600` 且不得提交;`search_config.sh` 只输出配置状态,禁止回显 key。
|
- 搜索 key 统一保存在 Pi 的 `auth.json`(`tavily`/`exa`/`keenable` 各一条 `api_key` 凭据,文件权限 `600` 且不得提交);key 支持 `!command` 与 `$VAR` 间接寻址,`search_config.sh` / `search-auth.mjs` 只输出配置状态,禁止回显 key。旧的 `search.env` 不再被读取,仅由 `--migrate` 一次性导入。
|
||||||
- `extensions/context-view.ts` 是只读的本地 effective-context observer,只读取 `pi-compaction/src/service.ts` 发布的 projection metrics,不注册生产 `context` writer;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
|
- `extensions/context-view.ts` 是只读的本地 effective-context observer,只读取 `pi-compaction/src/service.ts` 发布的 projection metrics,不注册生产 `context` writer;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
|
||||||
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
||||||
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。
|
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。
|
||||||
|
|||||||
@@ -255,14 +255,21 @@ FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash
|
|||||||
./search_config.sh --tavily '...' --exa '...' --keenable '...'
|
./search_config.sh --tavily '...' --exa '...' --keenable '...'
|
||||||
```
|
```
|
||||||
|
|
||||||
只传部分参数时保留其他服务的已有值。脚本把 key 原子写入
|
只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的
|
||||||
`${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env` 并设置权限为 `600`;
|
`auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`,权限 `600`):
|
||||||
`MY_PI_SEARCH_CONFIG` 或 `--config` 可覆盖路径。组合包在搜索扩展初始化前加载该文件,
|
每个 provider 一条 `{ "type": "api_key", "key": "..." }` 条目,原子加锁写入,
|
||||||
但调用 Pi 时显式传入的同名环境变量优先。配置文件不属于仓库,不会随安装包提交。
|
其他条目(如 OAuth 凭据)原样保留。`--list` 只显示配置状态、不回显 key;
|
||||||
|
`--migrate` 可从旧的 `search.env` 导入缺失条目;`--auth FILE` 可覆盖文件路径。
|
||||||
|
|
||||||
|
key 值支持 auth.json 原生解析语义:`$VAR` / `${VAR}` 环境变量插值、`!command`
|
||||||
|
执行命令取 stdout(如 `"!op read op://Vault/Tavily/key"`)、`$$` / `$!` 转义。
|
||||||
|
显式进程环境变量因此不再被直接读取——需要 env 注入时把 key 写成
|
||||||
|
`"$TAVILY_API_KEY"` 即可。`KEENABLE_API_URL`(非 secret 的端点覆盖)仍走
|
||||||
|
进程环境变量,也可放在 keenable 凭据的 `env` 字段中。
|
||||||
|
|
||||||
三个服务由本仓库自维护的 `pi-web-search/` 统一封装:对模型只暴露 `web_search` / `web_fetch` 两个工具,schema 与提示词中不出现任何厂商名;每次 `web_search` 先由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`,零额外配置)按内置路由规则在候选厂商中选一个,再按统一参数翻译后调用,`details` 记录实际 provider 与路由方式;选择结果按会话内 LRU 缓存,路由失败或被选厂商报错时按可用性顺序自动降级。`web_fetch` 固定调用 Exa 的 `web_fetch_exa` 提取正文(默认每页 3000 字符上限),失败时静默降级到 Keenable、再到 Tavily。
|
三个服务由本仓库自维护的 `pi-web-search/` 统一封装:对模型只暴露 `web_search` / `web_fetch` 两个工具,schema 与提示词中不出现任何厂商名;每次 `web_search` 先由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`,零额外配置)按内置路由规则在候选厂商中选一个,再按统一参数翻译后调用,`details` 记录实际 provider 与路由方式;选择结果按会话内 LRU 缓存,路由失败或被选厂商报错时按可用性顺序自动降级。`web_fetch` 固定调用 Exa 的 `web_fetch_exa` 提取正文(默认每页 3000 字符上限),失败时静默降级到 Keenable、再到 Tavily。
|
||||||
|
|
||||||
`pi-web-search/clients/` 全部是纯 `fetch` 实现:Tavily 直连 `api.tavily.com/search|extract`(必须有 `TAVILY_API_KEY`,无 key 时不参与路由);Keenable 复刻上游 client,默认走 keyless `/public` 端点,`KEENABLE_API_KEY` 只提高速率限制并保留 SSRF host 防护;Exa 直接对 `https://mcp.exa.ai/mcp` 发 JSON-RPC——initialize 一次并缓存 `mcp-session-id`、随后 `tools/call`,会话失效(HTTP 404)自动重建,key 只通过 `x-api-key` 请求头发送,不再依赖 `pi-mcp-adapter`。Exa 搜索统一走 `web_search_advanced_exa` 并显式限制结果数量与 `textMaxCharacters`,避免响应过大。
|
`pi-web-search/clients/` 全部是纯 `fetch` 实现,三个 key 统一由 `pi-web-search/auth.ts` 从 auth.json 解析(Tavily 直连 `api.tavily.com/search|extract`,必须有 `tavily` 凭据,无 key 时不参与路由;Keenable 复刻上游 client,默认走 keyless `/public` 端点,`keenable` 凭据只提高速率限制并保留 SSRF host 防护;Exa 直接对 `https://mcp.exa.ai/mcp` 发 JSON-RPC——initialize 一次并缓存 `mcp-session-id`、随后 `tools/call`,会话失效(HTTP 404)自动重建,`exa` 凭据只通过 `x-api-key` 请求头发送,不再依赖 `pi-mcp-adapter`)。Exa 搜索统一走 `web_search_advanced_exa` 并显式限制结果数量与 `textMaxCharacters`,避免响应过大。
|
||||||
|
|
||||||
### CodeGraph 是原生工具
|
### CodeGraph 是原生工具
|
||||||
|
|
||||||
|
|||||||
@@ -1,57 +0,0 @@
|
|||||||
import { readFileSync } from "node:fs";
|
|
||||||
import { homedir } from "node:os";
|
|
||||||
import { join } from "node:path";
|
|
||||||
|
|
||||||
export const SEARCH_API_KEY_NAMES = ["TAVILY_API_KEY", "EXA_API_KEY", "KEENABLE_API_KEY"] as const;
|
|
||||||
|
|
||||||
type SearchApiKeyName = (typeof SEARCH_API_KEY_NAMES)[number];
|
|
||||||
type SearchEnvironment = Record<string, string | undefined>;
|
|
||||||
|
|
||||||
const SAFE_VALUE = /^[A-Za-z0-9._-]+$/;
|
|
||||||
|
|
||||||
export function getSearchConfigPath(env: SearchEnvironment = process.env): string {
|
|
||||||
if (env.MY_PI_SEARCH_CONFIG?.trim()) {
|
|
||||||
return env.MY_PI_SEARCH_CONFIG.trim();
|
|
||||||
}
|
|
||||||
const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config");
|
|
||||||
return join(configHome, "my-pi", "search.env");
|
|
||||||
}
|
|
||||||
|
|
||||||
export function parseSearchConfig(text: string): Partial<Record<SearchApiKeyName, string>> {
|
|
||||||
const parsed: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
for (const rawLine of text.split(/\r?\n/u)) {
|
|
||||||
const line = rawLine.trim().replace(/^export[ \t]+/u, "");
|
|
||||||
if (!line || line.startsWith("#")) continue;
|
|
||||||
const separator = line.indexOf("=");
|
|
||||||
if (separator < 1) continue;
|
|
||||||
const name = line.slice(0, separator).trim() as SearchApiKeyName;
|
|
||||||
const value = line.slice(separator + 1).trim();
|
|
||||||
if (!SEARCH_API_KEY_NAMES.includes(name) || !SAFE_VALUE.test(value)) continue;
|
|
||||||
parsed[name] = value;
|
|
||||||
}
|
|
||||||
return parsed;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Load package-managed search credentials without overriding explicit process environment values. */
|
|
||||||
export function loadSearchConfigEnv(
|
|
||||||
env: SearchEnvironment = process.env,
|
|
||||||
configPath = getSearchConfigPath(env),
|
|
||||||
): Partial<Record<SearchApiKeyName, string>> {
|
|
||||||
let parsed: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
try {
|
|
||||||
parsed = parseSearchConfig(readFileSync(configPath, "utf8"));
|
|
||||||
} catch {
|
|
||||||
// Missing or unreadable config is equivalent to an empty package-managed config.
|
|
||||||
}
|
|
||||||
const effective: Partial<Record<SearchApiKeyName, string>> = {};
|
|
||||||
for (const name of SEARCH_API_KEY_NAMES) {
|
|
||||||
if (!env[name]?.trim() && parsed[name]) env[name] = parsed[name];
|
|
||||||
if (env[name]?.trim()) effective[name] = env[name]?.trim();
|
|
||||||
}
|
|
||||||
return effective;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Load search API keys before provider extensions initialize. */
|
|
||||||
export default function searchConfigExtension(): void {
|
|
||||||
loadSearchConfigEnv();
|
|
||||||
}
|
|
||||||
@@ -49,7 +49,6 @@
|
|||||||
],
|
],
|
||||||
"pi": {
|
"pi": {
|
||||||
"extensions": [
|
"extensions": [
|
||||||
"./extensions/search-config.ts",
|
|
||||||
"./extensions/fff-override.ts",
|
"./extensions/fff-override.ts",
|
||||||
"./pi-web-search/index.ts",
|
"./pi-web-search/index.ts",
|
||||||
"./extensions/codegraph.ts",
|
"./extensions/codegraph.ts",
|
||||||
|
|||||||
+11
-5
@@ -15,16 +15,22 @@ Unified web search and fetch for Pi. The main model only ever sees two tools —
|
|||||||
Plain `fetch` calls — no MCP adapter, no upstream extension wrappers:
|
Plain `fetch` calls — no MCP adapter, no upstream extension wrappers:
|
||||||
|
|
||||||
- `clients/tavily.ts` — `POST api.tavily.com/search|extract`, Bearer key.
|
- `clients/tavily.ts` — `POST api.tavily.com/search|extract`, Bearer key.
|
||||||
Requires `TAVILY_API_KEY`.
|
Requires a `tavily` credential.
|
||||||
- `clients/keenable.ts` — `api.keenable.ai`, keyless `/public` endpoints by
|
- `clients/keenable.ts` — `api.keenable.ai`, keyless `/public` endpoints by
|
||||||
default, `KEENABLE_API_KEY` raises rate limits. Adapted from
|
default, a `keenable` credential raises rate limits. Adapted from
|
||||||
`@keenable/pi-search`.
|
`@keenable/pi-search`.
|
||||||
- `clients/exa.ts` — JSON-RPC over `mcp.exa.ai/mcp` (Streamable HTTP):
|
- `clients/exa.ts` — JSON-RPC over `mcp.exa.ai/mcp` (Streamable HTTP):
|
||||||
`initialize` once, cache `mcp-session-id`, `tools/call` per request, re-init
|
`initialize` once, cache `mcp-session-id`, `tools/call` per request, re-init
|
||||||
on session expiry. `EXA_API_KEY` optional (`x-api-key` header).
|
on session expiry. An `exa` credential is optional (`x-api-key` header).
|
||||||
|
|
||||||
Keys are loaded from `~/.config/my-pi/search.env` by `extensions/search-config.ts`
|
Keys live in Pi's `auth.json` as one `api_key` credential per provider
|
||||||
(written by `search_config.sh`); explicit environment variables win.
|
(`tavily`, `exa`, `keenable`), read by `auth.ts` with upstream resolution
|
||||||
|
semantics — literal values, `$VAR`/`${VAR}` interpolation, and `!command`
|
||||||
|
indirection (e.g. `"!op read op://Vault/Tavily/key"`). `search_config.sh`
|
||||||
|
writes them (atomically, mode 600, preserving all other entries); process
|
||||||
|
environment variables are not read directly — use `"key": "$TAVILY_API_KEY"`
|
||||||
|
to indirect through env. `KEENABLE_API_URL` stays env-only, or can be stored
|
||||||
|
in the keenable credential's `env` map.
|
||||||
|
|
||||||
## Routing
|
## Routing
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,201 @@
|
|||||||
|
/**
|
||||||
|
* Search provider API keys live in Pi's own auth.json — one `api_key`
|
||||||
|
* credential per provider id (`tavily`, `exa`, `keenable`) next to Pi's
|
||||||
|
* model credentials. Key values follow upstream resolve-config-value
|
||||||
|
* semantics (replicated here because the upstream resolver is not exported):
|
||||||
|
*
|
||||||
|
* - `!command` executes the rest of the string and uses trimmed stdout,
|
||||||
|
* cached for the process lifetime (same as upstream's command cache)
|
||||||
|
* - `$NAME` / `${NAME}` interpolate environment variables; a credential-level
|
||||||
|
* `env` map wins over process env; a missing variable resolves to undefined
|
||||||
|
* - `$$` and `$!` escape to literal `$` and `!`
|
||||||
|
* - anything else is a literal key
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { execSync } from "node:child_process";
|
||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
|
||||||
|
export type SearchAuthProvider = "tavily" | "exa" | "keenable";
|
||||||
|
export const SEARCH_AUTH_PROVIDERS: readonly SearchAuthProvider[] = ["tavily", "exa", "keenable"];
|
||||||
|
|
||||||
|
/** Resolves a provider's API key; injectable so tests never touch real auth.json. */
|
||||||
|
export type SearchKeyResolver = (provider: SearchAuthProvider) => string | undefined;
|
||||||
|
|
||||||
|
export type SearchAuthEnvironment = Record<string, string | undefined>;
|
||||||
|
|
||||||
|
export interface SearchAuthOptions {
|
||||||
|
/** auth.json location; defaults to `<agent dir>/auth.json`. */
|
||||||
|
authPath?: string;
|
||||||
|
/**
|
||||||
|
* Environment used for the agent dir and `$VAR` interpolation. When
|
||||||
|
* provided it fully replaces `process.env` so tests stay isolated.
|
||||||
|
*/
|
||||||
|
env?: SearchAuthEnvironment;
|
||||||
|
/** Executes `!command` key references; defaults to a cached execSync runner. */
|
||||||
|
runCommand?: (command: string) => string | undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SearchApiKeyCredential {
|
||||||
|
type: "api_key";
|
||||||
|
key?: string;
|
||||||
|
env?: Record<string, string>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COMMAND_TIMEOUT_MS = 10_000;
|
||||||
|
const ENV_VAR_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/;
|
||||||
|
const ENV_VAR_NAME_PREFIX = /^[A-Za-z_][A-Za-z0-9_]*/;
|
||||||
|
const commandResultCache = new Map<string, string | undefined>();
|
||||||
|
|
||||||
|
/** auth.json lives in Pi's agent dir: $PI_CODING_AGENT_DIR or ~/.pi/agent. */
|
||||||
|
export function getSearchAuthPath(env: SearchAuthEnvironment = process.env): string {
|
||||||
|
const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent");
|
||||||
|
return join(agentDir, "auth.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Read the raw `api_key` credential for a provider; undefined when absent or malformed. */
|
||||||
|
export function readSearchCredential(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): SearchApiKeyCredential | undefined {
|
||||||
|
let data: unknown;
|
||||||
|
try {
|
||||||
|
data = JSON.parse(readFileSync(options.authPath ?? getSearchAuthPath(options.env), "utf8"));
|
||||||
|
} catch {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (!data || typeof data !== "object" || Array.isArray(data)) return undefined;
|
||||||
|
const entry = (data as Record<string, unknown>)[provider];
|
||||||
|
if (!entry || typeof entry !== "object" || Array.isArray(entry)) return undefined;
|
||||||
|
const credential = entry as Record<string, unknown>;
|
||||||
|
if (credential.type !== "api_key") return undefined;
|
||||||
|
if (credential.key !== undefined && typeof credential.key !== "string") return undefined;
|
||||||
|
const env = credential.env;
|
||||||
|
const scopedEnv =
|
||||||
|
env && typeof env === "object" && !Array.isArray(env)
|
||||||
|
? Object.fromEntries(
|
||||||
|
Object.entries(env as Record<string, unknown>).filter(
|
||||||
|
(pair): pair is [string, string] => typeof pair[1] === "string",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
: undefined;
|
||||||
|
return { type: "api_key", key: credential.key as string | undefined, env: scopedEnv };
|
||||||
|
}
|
||||||
|
|
||||||
|
type TemplatePart = { type: "literal"; value: string } | { type: "env"; name: string };
|
||||||
|
|
||||||
|
/** Mirrors upstream parseConfigValueTemplate: `$$`/`$!` escape, `${A}`/`$A` interpolate. */
|
||||||
|
export function parseAuthKeyTemplate(config: string): TemplatePart[] {
|
||||||
|
const parts: TemplatePart[] = [];
|
||||||
|
const appendLiteral = (value: string) => {
|
||||||
|
if (!value) return;
|
||||||
|
const previous = parts[parts.length - 1];
|
||||||
|
if (previous?.type === "literal") {
|
||||||
|
previous.value += value;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
parts.push({ type: "literal", value });
|
||||||
|
};
|
||||||
|
let index = 0;
|
||||||
|
while (index < config.length) {
|
||||||
|
const dollarIndex = config.indexOf("$", index);
|
||||||
|
if (dollarIndex < 0) {
|
||||||
|
appendLiteral(config.slice(index));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
appendLiteral(config.slice(index, dollarIndex));
|
||||||
|
const nextChar = config[dollarIndex + 1];
|
||||||
|
if (nextChar === "$" || nextChar === "!") {
|
||||||
|
appendLiteral(nextChar);
|
||||||
|
index = dollarIndex + 2;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (nextChar === "{") {
|
||||||
|
const endIndex = config.indexOf("}", dollarIndex + 2);
|
||||||
|
if (endIndex < 0) {
|
||||||
|
appendLiteral("$");
|
||||||
|
index = dollarIndex + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const name = config.slice(dollarIndex + 2, endIndex);
|
||||||
|
if (ENV_VAR_NAME.test(name)) {
|
||||||
|
parts.push({ type: "env", name });
|
||||||
|
} else {
|
||||||
|
appendLiteral(config.slice(dollarIndex, endIndex + 1));
|
||||||
|
}
|
||||||
|
index = endIndex + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const match = config.slice(dollarIndex + 1).match(ENV_VAR_NAME_PREFIX);
|
||||||
|
if (match) {
|
||||||
|
parts.push({ type: "env", name: match[0] });
|
||||||
|
index = dollarIndex + 1 + match[0].length;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
appendLiteral("$");
|
||||||
|
index = dollarIndex + 1;
|
||||||
|
}
|
||||||
|
return parts;
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultRunCommand(command: string): string | undefined {
|
||||||
|
if (commandResultCache.has(command)) return commandResultCache.get(command);
|
||||||
|
let value: string | undefined;
|
||||||
|
try {
|
||||||
|
const output = execSync(command, {
|
||||||
|
encoding: "utf-8",
|
||||||
|
timeout: COMMAND_TIMEOUT_MS,
|
||||||
|
stdio: ["ignore", "pipe", "ignore"],
|
||||||
|
});
|
||||||
|
value = output.trim() || undefined;
|
||||||
|
} catch {
|
||||||
|
value = undefined;
|
||||||
|
}
|
||||||
|
commandResultCache.set(command, value);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Mirrors upstream resolveConfigValue, including credential-scoped env precedence. */
|
||||||
|
export function resolveAuthKeyValue(
|
||||||
|
config: string,
|
||||||
|
credentialEnv: Record<string, string> | undefined,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
if (config.startsWith("!")) {
|
||||||
|
return (options.runCommand ?? defaultRunCommand)(config.slice(1));
|
||||||
|
}
|
||||||
|
const ambient = options.env ?? process.env;
|
||||||
|
let resolved = "";
|
||||||
|
for (const part of parseAuthKeyTemplate(config)) {
|
||||||
|
if (part.type === "literal") {
|
||||||
|
resolved += part.value;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const value = credentialEnv?.[part.name] || ambient[part.name];
|
||||||
|
if (value === undefined) return undefined;
|
||||||
|
resolved += value;
|
||||||
|
}
|
||||||
|
return resolved;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Provider API key from auth.json, or undefined when not configured. */
|
||||||
|
export function getSearchApiKey(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
const credential = readSearchCredential(provider, options);
|
||||||
|
if (!credential?.key) return undefined;
|
||||||
|
const resolved = resolveAuthKeyValue(credential.key, credential.env, options);
|
||||||
|
return resolved?.trim() || undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A credential-scoped config value (e.g. KEENABLE_API_URL), or undefined. */
|
||||||
|
export function getSearchCredentialEnv(
|
||||||
|
provider: SearchAuthProvider,
|
||||||
|
name: string,
|
||||||
|
options: SearchAuthOptions = {},
|
||||||
|
): string | undefined {
|
||||||
|
const value = readSearchCredential(provider, options)?.env?.[name]?.trim();
|
||||||
|
return value || undefined;
|
||||||
|
}
|
||||||
@@ -2,9 +2,12 @@
|
|||||||
* Exa client: direct JSON-RPC over the hosted endpoint https://mcp.exa.ai/mcp.
|
* Exa client: direct JSON-RPC over the hosted endpoint https://mcp.exa.ai/mcp.
|
||||||
* This is a plain HTTPS POST client (no MCP adapter, no SDK): initialize once,
|
* This is a plain HTTPS POST client (no MCP adapter, no SDK): initialize once,
|
||||||
* cache the `mcp-session-id` response header, then POST `tools/call` per call.
|
* cache the `mcp-session-id` response header, then POST `tools/call` per call.
|
||||||
* Keyless works; EXA_API_KEY is sent as the `x-api-key` header when present.
|
* Keyless works; the `exa` api_key credential from auth.json is sent as the
|
||||||
|
* `x-api-key` header when present.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey } from "../auth.ts";
|
||||||
|
|
||||||
export class ExaError extends Error {}
|
export class ExaError extends Error {}
|
||||||
|
|
||||||
const EXA_MCP_URL = "https://mcp.exa.ai/mcp?tools=web_search_exa,web_search_advanced_exa,web_fetch_exa";
|
const EXA_MCP_URL = "https://mcp.exa.ai/mcp?tools=web_search_exa,web_search_advanced_exa,web_fetch_exa";
|
||||||
@@ -41,7 +44,7 @@ function baseHeaders(): Record<string, string> {
|
|||||||
"content-type": "application/json",
|
"content-type": "application/json",
|
||||||
accept: "application/json, text/event-stream",
|
accept: "application/json, text/event-stream",
|
||||||
};
|
};
|
||||||
const key = process.env.EXA_API_KEY?.trim();
|
const key = getSearchApiKey("exa");
|
||||||
if (key) headers["x-api-key"] = key;
|
if (key) headers["x-api-key"] = key;
|
||||||
return headers;
|
return headers;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,17 +1,19 @@
|
|||||||
/**
|
/**
|
||||||
* Keenable REST client, adapted from @keenable/pi-search (keenable-client.ts).
|
* Keenable REST client, adapted from @keenable/pi-search (keenable-client.ts).
|
||||||
* Keyless by default (`/v1/*\/public`), authenticated when KEENABLE_API_KEY is
|
* Keyless by default (`/v1/*\/public`), authenticated when the `keenable`
|
||||||
* present (`/v1/*`), HTTPS-only base URL, `X-Keenable-Title` attribution.
|
* api_key credential exists in auth.json (`/v1/*`), HTTPS-only base URL,
|
||||||
|
* `X-Keenable-Title` attribution.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { isIP } from "node:net";
|
import { isIP } from "node:net";
|
||||||
|
|
||||||
|
import { getSearchApiKey, getSearchCredentialEnv } from "../auth.ts";
|
||||||
|
|
||||||
export class KeenableError extends Error {}
|
export class KeenableError extends Error {}
|
||||||
|
|
||||||
const CLIENT_VERSION = "0.1.0";
|
const CLIENT_VERSION = "0.1.0";
|
||||||
const DEFAULT_BASE_URL = "https://api.keenable.ai";
|
const DEFAULT_BASE_URL = "https://api.keenable.ai";
|
||||||
const BASE_URL_ENV = "KEENABLE_API_URL";
|
const BASE_URL_ENV = "KEENABLE_API_URL";
|
||||||
const API_KEY_ENV = "KEENABLE_API_KEY";
|
|
||||||
const ATTRIBUTION_TITLE = "Pi";
|
const ATTRIBUTION_TITLE = "Pi";
|
||||||
const USER_AGENT = `pi-web-search/${CLIENT_VERSION}`;
|
const USER_AGENT = `pi-web-search/${CLIENT_VERSION}`;
|
||||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||||
@@ -45,9 +47,15 @@ export interface KeenableFetchResult {
|
|||||||
[key: string]: unknown;
|
[key: string]: unknown;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Resolve the API base URL from `KEENABLE_API_URL`, enforcing HTTPS (http only for loopback). */
|
/**
|
||||||
|
* Resolve the API base URL from `KEENABLE_API_URL` (process env first, then
|
||||||
|
* the keenable credential's `env` map), enforcing HTTPS (http only for loopback).
|
||||||
|
*/
|
||||||
function resolveBaseUrl(): string {
|
function resolveBaseUrl(): string {
|
||||||
const base = (process.env[BASE_URL_ENV] || DEFAULT_BASE_URL).replace(/\/+$/u, "");
|
const base = (process.env[BASE_URL_ENV] || getSearchCredentialEnv("keenable", BASE_URL_ENV) || DEFAULT_BASE_URL).replace(
|
||||||
|
/\/+$/u,
|
||||||
|
"",
|
||||||
|
);
|
||||||
let parsed: URL;
|
let parsed: URL;
|
||||||
try {
|
try {
|
||||||
parsed = new URL(base);
|
parsed = new URL(base);
|
||||||
@@ -63,10 +71,9 @@ function resolveBaseUrl(): string {
|
|||||||
throw new KeenableError(`${BASE_URL_ENV} must be an https:// URL with a host, got ${base}`);
|
throw new KeenableError(`${BASE_URL_ENV} must be an https:// URL with a host, got ${base}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Non-blank API key from the env, or undefined to use the keyless public tier. */
|
/** API key from auth.json, or undefined to use the keyless public tier. */
|
||||||
function resolveApiKey(): string | undefined {
|
function resolveApiKey(): string | undefined {
|
||||||
const key = (process.env[API_KEY_ENV] ?? "").trim();
|
return getSearchApiKey("keenable");
|
||||||
return key || undefined;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
function buildHeaders(apiKey: string | undefined): Record<string, string> {
|
function buildHeaders(apiKey: string | undefined): Record<string, string> {
|
||||||
@@ -129,7 +136,7 @@ async function raiseForStatus(response: Response, keyed: boolean): Promise<void>
|
|||||||
if (response.ok) return;
|
if (response.ok) return;
|
||||||
if (!keyed && response.status === 429) {
|
if (!keyed && response.status === 429) {
|
||||||
throw new KeenableError(
|
throw new KeenableError(
|
||||||
"Keenable keyless requests hit their rate limit. Set KEENABLE_API_KEY to raise the limits.",
|
'Keenable keyless requests hit their rate limit. Add a "keenable" api_key credential to auth.json to raise the limits.',
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
let detail = "";
|
let detail = "";
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
/**
|
/**
|
||||||
* Tavily REST client: POST https://api.tavily.com/search|extract with a Bearer
|
* Tavily REST client: POST https://api.tavily.com/search|extract with a Bearer
|
||||||
* key. Requires TAVILY_API_KEY — there is no keyless tier.
|
* key from auth.json (`tavily` api_key credential) — there is no keyless tier.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey } from "../auth.ts";
|
||||||
|
|
||||||
export class TavilyError extends Error {}
|
export class TavilyError extends Error {}
|
||||||
|
|
||||||
const TAVILY_BASE_URL = "https://api.tavily.com";
|
const TAVILY_BASE_URL = "https://api.tavily.com";
|
||||||
@@ -30,8 +32,8 @@ export interface TavilySearchResult {
|
|||||||
[key: string]: unknown;
|
[key: string]: unknown;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function tavilyAvailable(env: Record<string, string | undefined> = process.env): boolean {
|
export function tavilyAvailable(resolve: (provider: "tavily") => string | undefined = getSearchApiKey): boolean {
|
||||||
return Boolean(env.TAVILY_API_KEY?.trim());
|
return Boolean(resolve("tavily"));
|
||||||
}
|
}
|
||||||
|
|
||||||
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||||
@@ -46,9 +48,9 @@ async function post(
|
|||||||
signal: AbortSignal | undefined,
|
signal: AbortSignal | undefined,
|
||||||
fetchImpl: FetchLike,
|
fetchImpl: FetchLike,
|
||||||
): Promise<Record<string, unknown>> {
|
): Promise<Record<string, unknown>> {
|
||||||
const apiKey = process.env.TAVILY_API_KEY?.trim();
|
const apiKey = getSearchApiKey("tavily");
|
||||||
if (!apiKey) {
|
if (!apiKey) {
|
||||||
throw new TavilyError("Tavily is not configured: TAVILY_API_KEY is missing.");
|
throw new TavilyError('Tavily is not configured: no "tavily" api_key credential in auth.json.');
|
||||||
}
|
}
|
||||||
let response: Response;
|
let response: Response;
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
|
|
||||||
import type { AgentToolResult, ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
import type { AgentToolResult, ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
|
||||||
import { loadSearchConfigEnv } from "../extensions/search-config.ts";
|
import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts";
|
||||||
import { exaFetchPages, exaSearch } from "./clients/exa.ts";
|
import { exaFetchPages, exaSearch } from "./clients/exa.ts";
|
||||||
import { keenableFetch, keenableSearch } from "./clients/keenable.ts";
|
import { keenableFetch, keenableSearch } from "./clients/keenable.ts";
|
||||||
import { tavilyAvailable, tavilyExtract, tavilySearch } from "./clients/tavily.ts";
|
import { tavilyAvailable, tavilyExtract, tavilySearch } from "./clients/tavily.ts";
|
||||||
@@ -108,9 +108,10 @@ export async function dispatchSearch(
|
|||||||
routeCache: Map<string, SearchProvider>,
|
routeCache: Map<string, SearchProvider>,
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
runners: Record<SearchProvider, ProviderRunner> = createProviderRunners(),
|
runners: Record<SearchProvider, ProviderRunner> = createProviderRunners(),
|
||||||
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
): Promise<AgentToolResult> {
|
): Promise<AgentToolResult> {
|
||||||
const decision = await routeSearch(params, ctx, routeCache, signal);
|
const decision = await routeSearch(params, ctx, routeCache, signal, resolve);
|
||||||
const order = [decision.provider, ...availableProviders().filter((p) => p !== decision.provider)];
|
const order = [decision.provider, ...availableProviders(resolve).filter((p) => p !== decision.provider)];
|
||||||
let lastError: unknown;
|
let lastError: unknown;
|
||||||
for (const provider of order) {
|
for (const provider of order) {
|
||||||
try {
|
try {
|
||||||
@@ -161,6 +162,7 @@ export async function dispatchFetch(
|
|||||||
params: WebFetchParams,
|
params: WebFetchParams,
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
fetchers: Record<SearchProvider, FetchRunner> = createFetchRunners(),
|
fetchers: Record<SearchProvider, FetchRunner> = createFetchRunners(),
|
||||||
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
): Promise<AgentToolResult> {
|
): Promise<AgentToolResult> {
|
||||||
const urls = normalizeUrls(params.urls);
|
const urls = normalizeUrls(params.urls);
|
||||||
if (!urls.length) {
|
if (!urls.length) {
|
||||||
@@ -171,7 +173,7 @@ export async function dispatchFetch(
|
|||||||
}
|
}
|
||||||
const maxCharacters = params.max_characters ?? DEFAULT_FETCH_MAX_CHARACTERS;
|
const maxCharacters = params.max_characters ?? DEFAULT_FETCH_MAX_CHARACTERS;
|
||||||
const order: SearchProvider[] = ["exa", "keenable"];
|
const order: SearchProvider[] = ["exa", "keenable"];
|
||||||
if (tavilyAvailable()) order.push("tavily");
|
if (tavilyAvailable(resolve)) order.push("tavily");
|
||||||
|
|
||||||
let lastError: unknown;
|
let lastError: unknown;
|
||||||
for (const provider of order) {
|
for (const provider of order) {
|
||||||
@@ -186,9 +188,6 @@ export async function dispatchFetch(
|
|||||||
}
|
}
|
||||||
|
|
||||||
export default function webSearchExtension(pi: ExtensionAPI): void {
|
export default function webSearchExtension(pi: ExtensionAPI): void {
|
||||||
// Keys may already be loaded by extensions/search-config.ts; this is a
|
|
||||||
// harmless idempotent safety net for standalone loading.
|
|
||||||
loadSearchConfigEnv();
|
|
||||||
const routeCache = new Map<string, SearchProvider>();
|
const routeCache = new Map<string, SearchProvider>();
|
||||||
|
|
||||||
pi.registerTool({
|
pi.registerTool({
|
||||||
|
|||||||
@@ -11,6 +11,7 @@
|
|||||||
},
|
},
|
||||||
"files": [
|
"files": [
|
||||||
"index.ts",
|
"index.ts",
|
||||||
|
"auth.ts",
|
||||||
"router.ts",
|
"router.ts",
|
||||||
"clients",
|
"clients",
|
||||||
"README.md",
|
"README.md",
|
||||||
|
|||||||
@@ -5,6 +5,8 @@
|
|||||||
* and failure fallbacks, never the routing itself.
|
* and failure fallbacks, never the routing itself.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts";
|
||||||
|
|
||||||
export type SearchProvider = "tavily" | "exa" | "keenable";
|
export type SearchProvider = "tavily" | "exa" | "keenable";
|
||||||
|
|
||||||
export interface SearchRequest {
|
export interface SearchRequest {
|
||||||
@@ -43,10 +45,10 @@ const PROVIDER_RULES: Record<SearchProvider, string> = {
|
|||||||
"content length.",
|
"content length.",
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Tavily requires a key; Exa and Keenable both work keyless. */
|
/** Tavily requires a key in auth.json; Exa and Keenable both work keyless. */
|
||||||
export function availableProviders(env: Record<string, string | undefined> = process.env): SearchProvider[] {
|
export function availableProviders(resolve: SearchKeyResolver = getSearchApiKey): SearchProvider[] {
|
||||||
const providers: SearchProvider[] = [];
|
const providers: SearchProvider[] = [];
|
||||||
if (env.TAVILY_API_KEY?.trim()) providers.push("tavily");
|
if (resolve("tavily")) providers.push("tavily");
|
||||||
providers.push("exa", "keenable");
|
providers.push("exa", "keenable");
|
||||||
return providers;
|
return providers;
|
||||||
}
|
}
|
||||||
@@ -111,9 +113,9 @@ export async function routeSearch(
|
|||||||
ctx: RouterContext,
|
ctx: RouterContext,
|
||||||
cache: Map<string, SearchProvider>,
|
cache: Map<string, SearchProvider>,
|
||||||
signal?: AbortSignal,
|
signal?: AbortSignal,
|
||||||
env: Record<string, string | undefined> = process.env,
|
resolve: SearchKeyResolver = getSearchApiKey,
|
||||||
): Promise<RouteDecision> {
|
): Promise<RouteDecision> {
|
||||||
const candidates = availableProviders(env);
|
const candidates = availableProviders(resolve);
|
||||||
const fallback = PROVIDER_FALLBACK_ORDER.find((provider) => candidates.includes(provider)) ?? candidates[0];
|
const fallback = PROVIDER_FALLBACK_ORDER.find((provider) => candidates.includes(provider)) ?? candidates[0];
|
||||||
if (!fallback || candidates.length <= 1) {
|
if (!fallback || candidates.length <= 1) {
|
||||||
return { provider: fallback ?? "keenable", routedBy: "fallback" };
|
return { provider: fallback ?? "keenable", routedBy: "fallback" };
|
||||||
|
|||||||
@@ -29,24 +29,8 @@ function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function withEnv<T>(key: string, value: string | undefined, fn: () => Promise<T> | T): Promise<T> | T {
|
const WITH_TAVILY = () => "k";
|
||||||
const previous = process.env[key];
|
const KEYLESS = () => undefined;
|
||||||
if (value === undefined) delete process.env[key];
|
|
||||||
else process.env[key] = value;
|
|
||||||
const restore = () => {
|
|
||||||
if (previous === undefined) delete process.env[key];
|
|
||||||
else process.env[key] = previous;
|
|
||||||
};
|
|
||||||
try {
|
|
||||||
const out = fn();
|
|
||||||
if (out instanceof Promise) return out.finally(restore);
|
|
||||||
restore();
|
|
||||||
return out;
|
|
||||||
} catch (error) {
|
|
||||||
restore();
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
test("extension registers only web_search and web_fetch, with no provider in the schema", () => {
|
test("extension registers only web_search and web_fetch, with no provider in the schema", () => {
|
||||||
const tools = new Map<string, { parameters?: { properties?: Record<string, unknown> }; description?: string }>();
|
const tools = new Map<string, { parameters?: { properties?: Record<string, unknown> }; description?: string }>();
|
||||||
@@ -64,7 +48,6 @@ test("extension registers only web_search and web_fetch, with no provider in the
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("dispatchSearch runs the model-picked provider first and records details", async () => {
|
test("dispatchSearch runs the model-picked provider first and records details", async () => {
|
||||||
await withEnv("TAVILY_API_KEY", "k", async () => {
|
|
||||||
const ran: string[] = [];
|
const ran: string[] = [];
|
||||||
const runners = createProviderRunners({
|
const runners = createProviderRunners({
|
||||||
exa: async () => {
|
exa: async () => {
|
||||||
@@ -72,15 +55,13 @@ test("dispatchSearch runs the model-picked provider first and records details",
|
|||||||
return "exa results";
|
return "exa results";
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners);
|
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
assert.deepEqual(ran, ["exa"]);
|
assert.deepEqual(ran, ["exa"]);
|
||||||
assert.equal(result.content[0]!.type === "text" && result.content[0]!.text, "exa results");
|
assert.equal(result.content[0]!.type === "text" && result.content[0]!.text, "exa results");
|
||||||
assert.deepEqual(result.details, { provider: "exa", routedBy: "model" });
|
assert.deepEqual(result.details, { provider: "exa", routedBy: "model" });
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("dispatchSearch falls back across remaining providers when the pick fails", async () => {
|
test("dispatchSearch falls back across remaining providers when the pick fails", async () => {
|
||||||
await withEnv("TAVILY_API_KEY", "k", async () => {
|
|
||||||
const ran: string[] = [];
|
const ran: string[] = [];
|
||||||
const runners = createProviderRunners({
|
const runners = createProviderRunners({
|
||||||
tavily: async () => {
|
tavily: async () => {
|
||||||
@@ -92,24 +73,21 @@ test("dispatchSearch falls back across remaining providers when the pick fails",
|
|||||||
return "exa results";
|
return "exa results";
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
const result = await dispatchSearch(PARAMS, ctxAnswering("tavily"), new Map(), undefined, runners);
|
const result = await dispatchSearch(PARAMS, ctxAnswering("tavily"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
assert.deepEqual(ran, ["tavily", "exa"]);
|
assert.deepEqual(ran, ["tavily", "exa"]);
|
||||||
assert.equal((result.details as Record<string, unknown>).provider, "exa");
|
assert.equal((result.details as Record<string, unknown>).provider, "exa");
|
||||||
assert.equal((result.details as Record<string, unknown>).routedBy, "provider-fallback");
|
assert.equal((result.details as Record<string, unknown>).routedBy, "provider-fallback");
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("dispatchSearch reports an error result when every provider fails", async () => {
|
test("dispatchSearch reports an error result when every provider fails", async () => {
|
||||||
await withEnv("TAVILY_API_KEY", "k", async () => {
|
|
||||||
const runners = createProviderRunners({
|
const runners = createProviderRunners({
|
||||||
tavily: async () => { throw new Error("t"); },
|
tavily: async () => { throw new Error("t"); },
|
||||||
exa: async () => { throw new Error("e"); },
|
exa: async () => { throw new Error("e"); },
|
||||||
keenable: async () => { throw new Error("k"); },
|
keenable: async () => { throw new Error("k"); },
|
||||||
});
|
});
|
||||||
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners);
|
const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY);
|
||||||
assert.equal((result.details as Record<string, unknown>).error, true);
|
assert.equal((result.details as Record<string, unknown>).error, true);
|
||||||
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /failed on every available provider/);
|
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /failed on every available provider/);
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("normalizeUrls accepts a string or array, trims, and deduplicates", () => {
|
test("normalizeUrls accepts a string or array, trims, and deduplicates", () => {
|
||||||
@@ -118,17 +96,15 @@ test("normalizeUrls accepts a string or array, trims, and deduplicates", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("dispatchFetch prefers exa and degrades to keenable without a model call", async () => {
|
test("dispatchFetch prefers exa and degrades to keenable without a model call", async () => {
|
||||||
await withEnv("TAVILY_API_KEY", undefined, async () => {
|
|
||||||
const fetchers = createFetchRunners({
|
const fetchers = createFetchRunners({
|
||||||
exa: async (urls) => {
|
exa: async (urls) => {
|
||||||
throw new Error(`exa down for ${urls.length}`);
|
throw new Error(`exa down for ${urls.length}`);
|
||||||
},
|
},
|
||||||
keenable: async (urls) => `keenable pages: ${urls.join(",")}`,
|
keenable: async (urls) => `keenable pages: ${urls.join(",")}`,
|
||||||
});
|
});
|
||||||
const result = await dispatchFetch({ urls: ["https://a.example", "https://b.example"] }, undefined, fetchers);
|
const result = await dispatchFetch({ urls: ["https://a.example", "https://b.example"] }, undefined, fetchers, KEYLESS);
|
||||||
assert.equal((result.details as Record<string, unknown>).provider, "keenable");
|
assert.equal((result.details as Record<string, unknown>).provider, "keenable");
|
||||||
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /keenable pages: https:\/\/a\.example,https:\/\/b\.example/);
|
assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /keenable pages: https:\/\/a\.example,https:\/\/b\.example/);
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("dispatchFetch validates input and caps the URL count", async () => {
|
test("dispatchFetch validates input and caps the URL count", async () => {
|
||||||
|
|||||||
@@ -12,7 +12,8 @@ import {
|
|||||||
} from "../router.ts";
|
} from "../router.ts";
|
||||||
|
|
||||||
const ALL: SearchProvider[] = ["tavily", "exa", "keenable"];
|
const ALL: SearchProvider[] = ["tavily", "exa", "keenable"];
|
||||||
const ENV_ALL_KEYS = { TAVILY_API_KEY: "test-key" };
|
const ALL_KEYS = () => "test-key";
|
||||||
|
const NO_KEYS = () => undefined;
|
||||||
|
|
||||||
function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }): RouterContext {
|
function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }): RouterContext {
|
||||||
return {
|
return {
|
||||||
@@ -27,8 +28,8 @@ function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }):
|
|||||||
}
|
}
|
||||||
|
|
||||||
test("availability: tavily needs a key, exa and keenable are keyless", () => {
|
test("availability: tavily needs a key, exa and keenable are keyless", () => {
|
||||||
assert.deepEqual(availableProviders({}), ["exa", "keenable"]);
|
assert.deepEqual(availableProviders(NO_KEYS), ["exa", "keenable"]);
|
||||||
assert.deepEqual(availableProviders(ENV_ALL_KEYS), ["tavily", "exa", "keenable"]);
|
assert.deepEqual(availableProviders(ALL_KEYS), ["tavily", "exa", "keenable"]);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("router prompt lists only candidate rules plus the query and caller hints", () => {
|
test("router prompt lists only candidate rules plus the query and caller hints", () => {
|
||||||
@@ -64,44 +65,44 @@ test("routeSearch honors the model pick and caches identical requests", async ()
|
|||||||
const cache = new Map<string, SearchProvider>();
|
const cache = new Map<string, SearchProvider>();
|
||||||
const request = { query: "official MCP spec" };
|
const request = { query: "official MCP spec" };
|
||||||
|
|
||||||
const first = await routeSearch(request, ctx, cache, undefined, ENV_ALL_KEYS);
|
const first = await routeSearch(request, ctx, cache, undefined, ALL_KEYS);
|
||||||
assert.deepEqual(first, { provider: "exa", routedBy: "model" });
|
assert.deepEqual(first, { provider: "exa", routedBy: "model" });
|
||||||
|
|
||||||
const second = await routeSearch(request, ctx, cache, undefined, ENV_ALL_KEYS);
|
const second = await routeSearch(request, ctx, cache, undefined, ALL_KEYS);
|
||||||
assert.deepEqual(second, { provider: "exa", routedBy: "cache" });
|
assert.deepEqual(second, { provider: "exa", routedBy: "cache" });
|
||||||
assert.equal(calls.count, 1);
|
assert.equal(calls.count, 1);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("routeSearch falls back to the first available provider on unroutable or failed model calls", async () => {
|
test("routeSearch falls back to the first available provider on unroutable or failed model calls", async () => {
|
||||||
const cache = new Map<string, SearchProvider>();
|
const cache = new Map<string, SearchProvider>();
|
||||||
const unparseable = await routeSearch({ query: "x" }, ctxAnswering("no idea"), cache, undefined, ENV_ALL_KEYS);
|
const unparseable = await routeSearch({ query: "x" }, ctxAnswering("no idea"), cache, undefined, ALL_KEYS);
|
||||||
assert.deepEqual(unparseable, { provider: "tavily", routedBy: "fallback" });
|
assert.deepEqual(unparseable, { provider: "tavily", routedBy: "fallback" });
|
||||||
|
|
||||||
const throwing: RouterContext = {
|
const throwing: RouterContext = {
|
||||||
model: {},
|
model: {},
|
||||||
modelRegistry: { complete: async () => { throw new Error("boom"); } },
|
modelRegistry: { complete: async () => { throw new Error("boom"); } },
|
||||||
};
|
};
|
||||||
const failed = await routeSearch({ query: "x" }, throwing, cache, undefined, ENV_ALL_KEYS);
|
const failed = await routeSearch({ query: "x" }, throwing, cache, undefined, ALL_KEYS);
|
||||||
assert.deepEqual(failed, { provider: "tavily", routedBy: "fallback" });
|
assert.deepEqual(failed, { provider: "tavily", routedBy: "fallback" });
|
||||||
});
|
});
|
||||||
|
|
||||||
test("routeSearch without a tavily key excludes it from candidates", async () => {
|
test("routeSearch without a tavily key excludes it from candidates", async () => {
|
||||||
const cache = new Map<string, SearchProvider>();
|
const cache = new Map<string, SearchProvider>();
|
||||||
const decision = await routeSearch({ query: "x" }, ctxAnswering("tavily"), cache, undefined, {});
|
const decision = await routeSearch({ query: "x" }, ctxAnswering("tavily"), cache, undefined, NO_KEYS);
|
||||||
// "tavily" is not a candidate → unroutable → first available fallback.
|
// "tavily" is not a candidate → unroutable → first available fallback.
|
||||||
assert.deepEqual(decision, { provider: "exa", routedBy: "fallback" });
|
assert.deepEqual(decision, { provider: "exa", routedBy: "fallback" });
|
||||||
});
|
});
|
||||||
|
|
||||||
test("routeSearch works without a model", async () => {
|
test("routeSearch works without a model", async () => {
|
||||||
const cache = new Map<string, SearchProvider>();
|
const cache = new Map<string, SearchProvider>();
|
||||||
const decision = await routeSearch({ query: "x" }, {}, cache, undefined, ENV_ALL_KEYS);
|
const decision = await routeSearch({ query: "x" }, {}, cache, undefined, ALL_KEYS);
|
||||||
assert.deepEqual(decision, { provider: "tavily", routedBy: "fallback" });
|
assert.deepEqual(decision, { provider: "tavily", routedBy: "fallback" });
|
||||||
});
|
});
|
||||||
|
|
||||||
test("route cache evicts the oldest entry at the limit", async () => {
|
test("route cache evicts the oldest entry at the limit", async () => {
|
||||||
const cache = new Map<string, SearchProvider>();
|
const cache = new Map<string, SearchProvider>();
|
||||||
for (let i = 0; i < ROUTE_CACHE_LIMIT; i++) cache.set(`dummy-${i}`, "exa");
|
for (let i = 0; i < ROUTE_CACHE_LIMIT; i++) cache.set(`dummy-${i}`, "exa");
|
||||||
const decision = await routeSearch({ query: "fresh query" }, ctxAnswering("keenable"), cache, undefined, ENV_ALL_KEYS);
|
const decision = await routeSearch({ query: "fresh query" }, ctxAnswering("keenable"), cache, undefined, ALL_KEYS);
|
||||||
assert.equal(decision.provider, "keenable");
|
assert.equal(decision.provider, "keenable");
|
||||||
assert.equal(cache.size, ROUTE_CACHE_LIMIT);
|
assert.equal(cache.size, ROUTE_CACHE_LIMIT);
|
||||||
assert.equal(cache.has("dummy-0"), false);
|
assert.equal(cache.has("dummy-0"), false);
|
||||||
|
|||||||
Executable
+262
@@ -0,0 +1,262 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Read-modify-write the tavily/exa/keenable api_key credentials in Pi's
|
||||||
|
* auth.json. Preserves every other provider entry (OAuth tokens included),
|
||||||
|
* writes atomically with a lock directory, and keeps the file mode 600.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* node search-auth.mjs --set tavily=KEY [--set exa=KEY] [--set keenable=KEY]
|
||||||
|
* node search-auth.mjs --unset tavily [--unset exa]
|
||||||
|
* node search-auth.mjs --migrate [search.env path]
|
||||||
|
* node search-auth.mjs --list
|
||||||
|
* node search-auth.mjs --has tavily (exit 0 when set, 1 otherwise)
|
||||||
|
*
|
||||||
|
* --auth FILE overrides the auth.json location (default:
|
||||||
|
* $PI_CODING_AGENT_DIR/auth.json or ~/.pi/agent/auth.json).
|
||||||
|
*/
|
||||||
|
|
||||||
|
import {
|
||||||
|
chmodSync,
|
||||||
|
closeSync,
|
||||||
|
existsSync,
|
||||||
|
mkdirSync,
|
||||||
|
openSync,
|
||||||
|
readFileSync,
|
||||||
|
renameSync,
|
||||||
|
rmSync,
|
||||||
|
writeFileSync,
|
||||||
|
} from "node:fs";
|
||||||
|
import { homedir } from "node:os";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
|
import { randomBytes } from "node:crypto";
|
||||||
|
|
||||||
|
const PROVIDERS = ["tavily", "exa", "keenable"];
|
||||||
|
const ENV_TO_PROVIDER = {
|
||||||
|
TAVILY_API_KEY: "tavily",
|
||||||
|
EXA_API_KEY: "exa",
|
||||||
|
KEENABLE_API_KEY: "keenable",
|
||||||
|
};
|
||||||
|
const SAFE_VALUE = /^[A-Za-z0-9._-]+$/;
|
||||||
|
const LOCK_RETRIES = 50;
|
||||||
|
const LOCK_DELAY_MS = 100;
|
||||||
|
|
||||||
|
function die(message) {
|
||||||
|
console.error(`错误: ${message}`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function usage() {
|
||||||
|
console.log(`用法:
|
||||||
|
node search-auth.mjs --set <provider>=<key> [--set ...]
|
||||||
|
node search-auth.mjs --unset <provider> [--unset ...]
|
||||||
|
node search-auth.mjs --migrate [search.env 路径]
|
||||||
|
node search-auth.mjs --list
|
||||||
|
node search-auth.mjs --has <provider>
|
||||||
|
|
||||||
|
选项:
|
||||||
|
--auth FILE 覆盖 auth.json 路径(默认 $PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json)`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultAuthPath(env = process.env) {
|
||||||
|
const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent");
|
||||||
|
return join(agentDir, "auth.json");
|
||||||
|
}
|
||||||
|
|
||||||
|
function defaultSearchEnvPath(env = process.env) {
|
||||||
|
if (env.MY_PI_SEARCH_CONFIG?.trim()) return env.MY_PI_SEARCH_CONFIG.trim();
|
||||||
|
const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config");
|
||||||
|
return join(configHome, "my-pi", "search.env");
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseArgs(argv) {
|
||||||
|
const result = { sets: {}, unsets: new Set(), migrate: undefined, list: false, has: undefined, authPath: undefined };
|
||||||
|
for (let i = 0; i < argv.length; i++) {
|
||||||
|
const arg = argv[i];
|
||||||
|
if (arg === "--set") {
|
||||||
|
const spec = argv[++i];
|
||||||
|
const separator = spec ? spec.indexOf("=") : -1;
|
||||||
|
if (separator < 1) die("--set 需要 <provider>=<key>");
|
||||||
|
const provider = spec.slice(0, separator);
|
||||||
|
const value = spec.slice(separator + 1);
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.sets[provider] = value;
|
||||||
|
} else if (arg === "--unset") {
|
||||||
|
const provider = argv[++i];
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.unsets.add(provider);
|
||||||
|
} else if (arg === "--migrate") {
|
||||||
|
const next = argv[i + 1];
|
||||||
|
result.migrate = next && !next.startsWith("--") ? (i++, next) : true;
|
||||||
|
} else if (arg === "--list") {
|
||||||
|
result.list = true;
|
||||||
|
} else if (arg === "--has") {
|
||||||
|
const provider = argv[++i];
|
||||||
|
if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`);
|
||||||
|
result.has = provider;
|
||||||
|
} else if (arg === "--auth") {
|
||||||
|
result.authPath = argv[++i];
|
||||||
|
if (!result.authPath) die("--auth 缺少 FILE");
|
||||||
|
} else if (arg === "-h" || arg === "--help") {
|
||||||
|
usage();
|
||||||
|
process.exit(0);
|
||||||
|
} else {
|
||||||
|
die(`未知参数:${arg}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
function acquireLock(authPath) {
|
||||||
|
mkdirSync(dirname(authPath), { recursive: true });
|
||||||
|
const lockDir = `${authPath}.lock`;
|
||||||
|
for (let attempt = 0; attempt < LOCK_RETRIES; attempt++) {
|
||||||
|
try {
|
||||||
|
mkdirSync(lockDir);
|
||||||
|
return () => {
|
||||||
|
try {
|
||||||
|
rmSync(lockDir, { recursive: true });
|
||||||
|
} catch {}
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code !== "EEXIST") throw error;
|
||||||
|
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, LOCK_DELAY_MS);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
die(`无法获取 ${lockDir} 锁;可能有其他进程正在写 auth.json(残留锁目录可手动删除)。`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readAuthData(authPath) {
|
||||||
|
let raw;
|
||||||
|
try {
|
||||||
|
raw = readFileSync(authPath, "utf8");
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === "ENOENT") return {};
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
let data;
|
||||||
|
try {
|
||||||
|
data = JSON.parse(raw);
|
||||||
|
} catch {
|
||||||
|
die(`auth.json 不是合法 JSON,已中止以免破坏现有凭据:${authPath}`);
|
||||||
|
}
|
||||||
|
if (!data || typeof data !== "object" || Array.isArray(data)) {
|
||||||
|
die(`auth.json 顶层必须是对象,已中止以免破坏现有凭据:${authPath}`);
|
||||||
|
}
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeAuthData(authPath, data) {
|
||||||
|
mkdirSync(dirname(authPath), { recursive: true });
|
||||||
|
const temp = join(dirname(authPath), `.auth.json.${process.pid}.${randomBytes(6).toString("hex")}.tmp`);
|
||||||
|
try {
|
||||||
|
const fd = openSync(temp, "w", 0o600);
|
||||||
|
writeFileSync(fd, `${JSON.stringify(data, null, 2)}\n`);
|
||||||
|
closeSync(fd);
|
||||||
|
renameSync(temp, authPath);
|
||||||
|
chmodSync(authPath, 0o600);
|
||||||
|
} finally {
|
||||||
|
rmSync(temp, { force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasKey(data, provider) {
|
||||||
|
const entry = data[provider];
|
||||||
|
return Boolean(entry && typeof entry === "object" && !Array.isArray(entry) && entry.type === "api_key" && entry.key);
|
||||||
|
}
|
||||||
|
|
||||||
|
function setKey(data, provider, key) {
|
||||||
|
const existing = data[provider];
|
||||||
|
const preserved =
|
||||||
|
existing && typeof existing === "object" && !Array.isArray(existing) && existing.type === "api_key"
|
||||||
|
? existing
|
||||||
|
: {};
|
||||||
|
data[provider] = { ...preserved, type: "api_key", key };
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseSearchEnv(text) {
|
||||||
|
const parsed = {};
|
||||||
|
for (const rawLine of text.split(/\r?\n/u)) {
|
||||||
|
const line = rawLine.trim().replace(/^export[ \t]+/u, "");
|
||||||
|
if (!line || line.startsWith("#")) continue;
|
||||||
|
const separator = line.indexOf("=");
|
||||||
|
if (separator < 1) continue;
|
||||||
|
const name = line.slice(0, separator).trim();
|
||||||
|
const value = line.slice(separator + 1).trim();
|
||||||
|
const provider = ENV_TO_PROVIDER[name];
|
||||||
|
if (!provider || !SAFE_VALUE.test(value)) continue;
|
||||||
|
parsed[provider] = value;
|
||||||
|
}
|
||||||
|
return parsed;
|
||||||
|
}
|
||||||
|
|
||||||
|
function validateKey(provider, key) {
|
||||||
|
if (!key || /[\r\n]/u.test(key)) die(`${provider} 的 key 不能为空且不能包含换行。`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const args = parseArgs(process.argv.slice(2));
|
||||||
|
const authPath = args.authPath ?? defaultAuthPath();
|
||||||
|
|
||||||
|
if (args.list) {
|
||||||
|
const data = readAuthData(authPath);
|
||||||
|
for (const provider of PROVIDERS) {
|
||||||
|
console.log(`${provider}: ${hasKey(data, provider) ? "已配置" : "未配置"}`);
|
||||||
|
}
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (args.has) {
|
||||||
|
process.exit(hasKey(readAuthData(authPath), args.has) ? 0 : 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const mutations = Object.keys(args.sets).length + args.unsets.size + (args.migrate ? 1 : 0);
|
||||||
|
if (mutations === 0) {
|
||||||
|
usage();
|
||||||
|
process.exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
const release = acquireLock(authPath);
|
||||||
|
try {
|
||||||
|
const data = readAuthData(authPath);
|
||||||
|
const changed = [];
|
||||||
|
|
||||||
|
for (const provider of args.unsets) {
|
||||||
|
if (provider in data) {
|
||||||
|
delete data[provider];
|
||||||
|
changed.push(`已移除 ${provider}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const [provider, key] of Object.entries(args.sets)) {
|
||||||
|
validateKey(provider, key);
|
||||||
|
setKey(data, provider, key);
|
||||||
|
changed.push(`已写入 ${provider}`);
|
||||||
|
}
|
||||||
|
if (args.migrate) {
|
||||||
|
const source = args.migrate === true ? defaultSearchEnvPath() : args.migrate;
|
||||||
|
if (!existsSync(source)) die(`迁移来源不存在:${source}`);
|
||||||
|
const parsed = parseSearchEnv(readFileSync(source, "utf8"));
|
||||||
|
const imported = [];
|
||||||
|
const skipped = [];
|
||||||
|
for (const provider of PROVIDERS) {
|
||||||
|
if (!parsed[provider]) continue;
|
||||||
|
if (provider in data) {
|
||||||
|
skipped.push(provider);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
setKey(data, provider, parsed[provider]);
|
||||||
|
imported.push(provider);
|
||||||
|
}
|
||||||
|
if (imported.length) changed.push(`已迁移 ${imported.join(", ")}`);
|
||||||
|
if (skipped.length) changed.push(`已跳过(auth.json 中已存在):${skipped.join(", ")}`);
|
||||||
|
if (!imported.length && !skipped.length) changed.push("search.env 中没有可迁移的 key");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (changed.length === 0) {
|
||||||
|
console.log("auth.json 无需修改。");
|
||||||
|
} else {
|
||||||
|
writeAuthData(authPath, data);
|
||||||
|
for (const line of changed) console.log(line);
|
||||||
|
console.log(`搜索凭据已写入 ${authPath}(权限 600),重启 Pi 后生效。`);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
release();
|
||||||
|
}
|
||||||
+68
-85
@@ -15,71 +15,54 @@ usage() {
|
|||||||
用法:
|
用法:
|
||||||
./search_config.sh
|
./search_config.sh
|
||||||
./search_config.sh --tavily KEY --exa KEY --keenable KEY
|
./search_config.sh --tavily KEY --exa KEY --keenable KEY
|
||||||
|
./search_config.sh --migrate [search.env 路径]
|
||||||
|
./search_config.sh --list
|
||||||
|
|
||||||
选项:
|
选项:
|
||||||
--tavily KEY 配置 TAVILY_API_KEY
|
--tavily KEY 配置 tavily api_key 凭据
|
||||||
--exa KEY 配置 EXA_API_KEY
|
--exa KEY 配置 exa api_key 凭据
|
||||||
--keenable KEY 配置 KEENABLE_API_KEY
|
--keenable KEY 配置 keenable api_key 凭据
|
||||||
--config FILE 覆盖配置文件路径
|
--migrate 从旧 search.env 导入缺失的 key
|
||||||
|
--list 显示各 provider 配置状态(不输出 key)
|
||||||
|
--auth FILE 覆盖 auth.json 路径
|
||||||
-h, --help 显示帮助
|
-h, --help 显示帮助
|
||||||
|
|
||||||
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互,
|
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互,
|
||||||
未指定的服务保留已有值。配置文件默认是 ~/.config/my-pi/search.env。
|
未指定的 provider 保留已有值。凭据写入 Pi 的 auth.json
|
||||||
|
($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,权限 600)。
|
||||||
|
key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR} 插值、
|
||||||
|
以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
|
|
||||||
CONFIG_FILE=${MY_PI_SEARCH_CONFIG:-${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env}
|
SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
|
||||||
|
AUTH_MJS="$SCRIPT_DIR/scripts/search-auth.mjs"
|
||||||
|
[ -f "$AUTH_MJS" ] || { warn "找不到 ${AUTH_MJS}"; exit 1; }
|
||||||
|
|
||||||
|
AUTH_FILE=
|
||||||
mode=interactive
|
mode=interactive
|
||||||
tavily_set=0
|
did_work=0
|
||||||
exa_set=0
|
|
||||||
keenable_set=0
|
|
||||||
tavily_value=
|
|
||||||
exa_value=
|
|
||||||
keenable_value=
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
run_auth() {
|
||||||
case $1 in
|
if [ -n "$AUTH_FILE" ]; then
|
||||||
--tavily)
|
node "$AUTH_MJS" --auth "$AUTH_FILE" "$@"
|
||||||
[ "$#" -ge 2 ] || { warn "--tavily 缺少 KEY"; exit 2; }
|
else
|
||||||
tavily_value=$2; tavily_set=1; mode=noninteractive; shift 2 ;;
|
node "$AUTH_MJS" "$@"
|
||||||
--exa)
|
fi
|
||||||
[ "$#" -ge 2 ] || { warn "--exa 缺少 KEY"; exit 2; }
|
|
||||||
exa_value=$2; exa_set=1; mode=noninteractive; shift 2 ;;
|
|
||||||
--keenable)
|
|
||||||
[ "$#" -ge 2 ] || { warn "--keenable 缺少 KEY"; exit 2; }
|
|
||||||
keenable_value=$2; keenable_set=1; mode=noninteractive; shift 2 ;;
|
|
||||||
--config)
|
|
||||||
[ "$#" -ge 2 ] || { warn "--config 缺少 FILE"; exit 2; }
|
|
||||||
CONFIG_FILE=$2; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) warn "未知参数:$1"; usage >&2; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
valid_key() {
|
|
||||||
case $1 in
|
|
||||||
''|*[!A-Za-z0-9._-]*) return 1 ;;
|
|
||||||
*) return 0 ;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
read_existing() {
|
has_key() {
|
||||||
existing_name=$1
|
run_auth --has "$1" >/dev/null 2>&1
|
||||||
[ -f "$CONFIG_FILE" ] || return 0
|
|
||||||
awk -v name="$existing_name" '
|
|
||||||
$1 == "export" { sub(/^export[ \t]+/, "") }
|
|
||||||
index($0, name "=") == 1 { print substr($0, length(name) + 2); exit }
|
|
||||||
' "$CONFIG_FILE"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
prompt_key() {
|
prompt_key() {
|
||||||
prompt_label=$1
|
prompt_label=$1
|
||||||
prompt_current=$2
|
prompt_configured=$2
|
||||||
if [ ! -t 0 ]; then
|
if [ ! -t 0 ]; then
|
||||||
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。"
|
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。"
|
||||||
return 2
|
return 2
|
||||||
fi
|
fi
|
||||||
if [ -n "$prompt_current" ]; then
|
if [ "$prompt_configured" -eq 1 ]; then
|
||||||
printf '%s API key(回车保留现有值): ' "$prompt_label" >&2
|
printf '%s API key(回车保留现有值): ' "$prompt_label" >&2
|
||||||
else
|
else
|
||||||
printf '%s API key(回车跳过): ' "$prompt_label" >&2
|
printf '%s API key(回车跳过): ' "$prompt_label" >&2
|
||||||
@@ -92,45 +75,45 @@ prompt_key() {
|
|||||||
printf '%s\n' "$prompt_value"
|
printf '%s\n' "$prompt_value"
|
||||||
}
|
}
|
||||||
|
|
||||||
current_tavily=$(read_existing TAVILY_API_KEY)
|
while [ "$#" -gt 0 ]; do
|
||||||
current_exa=$(read_existing EXA_API_KEY)
|
case $1 in
|
||||||
current_keenable=$(read_existing KEENABLE_API_KEY)
|
--tavily|--exa|--keenable)
|
||||||
|
[ "$#" -ge 2 ] || { warn "$1 缺少 KEY"; exit 2; }
|
||||||
if [ "$mode" = interactive ]; then
|
mode=noninteractive; did_work=1
|
||||||
tavily_value=$(prompt_key Tavily "$current_tavily") || exit $?
|
run_auth --set "${1#--}=$2" || exit $?
|
||||||
exa_value=$(prompt_key Exa "$current_exa") || exit $?
|
shift 2 ;;
|
||||||
keenable_value=$(prompt_key Keenable "$current_keenable") || exit $?
|
--migrate)
|
||||||
[ -n "$tavily_value" ] && tavily_set=1
|
mode=noninteractive; did_work=1
|
||||||
[ -n "$exa_value" ] && exa_set=1
|
if [ "$#" -ge 2 ] && [ "${2#--}" = "$2" ]; then
|
||||||
[ -n "$keenable_value" ] && keenable_set=1
|
run_auth --migrate "$2" || exit $?
|
||||||
fi
|
shift 2
|
||||||
|
else
|
||||||
[ "$tavily_set" -eq 1 ] || tavily_value=$current_tavily
|
run_auth --migrate || exit $?
|
||||||
[ "$exa_set" -eq 1 ] || exa_value=$current_exa
|
shift
|
||||||
[ "$keenable_set" -eq 1 ] || keenable_value=$current_keenable
|
fi
|
||||||
|
say "如确认 auth.json 可用,可手动删除旧 search.env 文件。" ;;
|
||||||
for key_value in "$tavily_value" "$exa_value" "$keenable_value"; do
|
--list) mode=noninteractive; did_work=1; run_auth --list || exit $?; shift ;;
|
||||||
[ -z "$key_value" ] || valid_key "$key_value" || {
|
--auth)
|
||||||
warn "API key 只能包含字母、数字、点、下划线和连字符。"
|
[ "$#" -ge 2 ] || { warn "--auth 缺少 FILE"; exit 2; }
|
||||||
exit 2
|
AUTH_FILE=$2; shift 2 ;;
|
||||||
}
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) warn "未知参数:$1"; usage >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
done
|
done
|
||||||
|
|
||||||
config_dir=${CONFIG_FILE%/*}
|
if [ "$mode" = interactive ]; then
|
||||||
[ "$config_dir" = "$CONFIG_FILE" ] && config_dir=.
|
for provider in tavily exa keenable; do
|
||||||
umask 077
|
case $provider in
|
||||||
mkdir -p "$config_dir" || exit 1
|
tavily) label=Tavily ;;
|
||||||
config_tmp=$(mktemp "$config_dir/.search.env.XXXXXX") || exit 1
|
exa) label=Exa ;;
|
||||||
trap 'rm -f "$config_tmp"' EXIT HUP INT TERM
|
keenable) label=Keenable ;;
|
||||||
{
|
esac
|
||||||
printf '# Generated by my-pi/search_config.sh. Do not commit this file.\n'
|
if has_key "$provider"; then configured=1; else configured=0; fi
|
||||||
[ -z "$tavily_value" ] || printf 'export TAVILY_API_KEY=%s\n' "$tavily_value"
|
value=$(prompt_key "$label" "$configured") || exit $?
|
||||||
[ -z "$exa_value" ] || printf 'export EXA_API_KEY=%s\n' "$exa_value"
|
if [ -n "$value" ]; then
|
||||||
[ -z "$keenable_value" ] || printf 'export KEENABLE_API_KEY=%s\n' "$keenable_value"
|
run_auth --set "$provider=$value" || exit $?
|
||||||
} > "$config_tmp" || exit 1
|
did_work=1
|
||||||
chmod 600 "$config_tmp" || exit 1
|
fi
|
||||||
mv "$config_tmp" "$CONFIG_FILE" || exit 1
|
done
|
||||||
trap - EXIT HUP INT TERM
|
[ "$did_work" -eq 1 ] || say "未提供任何 key,auth.json 保持不变。"
|
||||||
|
fi
|
||||||
say "搜索 API key 已安全写入 ${CONFIG_FILE}(权限 600)。"
|
|
||||||
say "重启 Pi 后生效;显式环境变量仍优先于该文件。"
|
|
||||||
|
|||||||
@@ -201,12 +201,8 @@ test("all package extensions load together without global registration conflicts
|
|||||||
HOME: home,
|
HOME: home,
|
||||||
XDG_CONFIG_HOME: join(home, ".config"),
|
XDG_CONFIG_HOME: join(home, ".config"),
|
||||||
PI_CODING_AGENT_DIR: join(home, ".pi-agent"),
|
PI_CODING_AGENT_DIR: join(home, ".pi-agent"),
|
||||||
MY_PI_SEARCH_CONFIG: join(home, "missing-search.env"),
|
|
||||||
PATH: `${mockBin}:${process.env.PATH ?? ""}`,
|
PATH: `${mockBin}:${process.env.PATH ?? ""}`,
|
||||||
PI_OFFLINE: "1",
|
PI_OFFLINE: "1",
|
||||||
TAVILY_API_KEY: "",
|
|
||||||
EXA_API_KEY: "",
|
|
||||||
KEENABLE_API_KEY: "",
|
|
||||||
};
|
};
|
||||||
const runBundle = () => spawnSync("pi", args, {
|
const runBundle = () => spawnSync("pi", args, {
|
||||||
encoding: "utf8",
|
encoding: "utf8",
|
||||||
|
|||||||
@@ -0,0 +1,252 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtemp, readFile, stat, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { spawnSync } from "node:child_process";
|
||||||
|
import test from "node:test";
|
||||||
|
|
||||||
|
import {
|
||||||
|
getSearchApiKey,
|
||||||
|
getSearchAuthPath,
|
||||||
|
getSearchCredentialEnv,
|
||||||
|
parseAuthKeyTemplate,
|
||||||
|
resolveAuthKeyValue,
|
||||||
|
} from "../pi-web-search/auth.ts";
|
||||||
|
|
||||||
|
async function withAuthFile(
|
||||||
|
data: Record<string, unknown>,
|
||||||
|
fn: (authPath: string) => Promise<void> | void,
|
||||||
|
): Promise<void> {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify(data));
|
||||||
|
await fn(authPath);
|
||||||
|
}
|
||||||
|
|
||||||
|
test("auth path follows PI_CODING_AGENT_DIR and falls back to ~/.pi/agent", () => {
|
||||||
|
assert.equal(
|
||||||
|
getSearchAuthPath({ PI_CODING_AGENT_DIR: "/tmp/agent" }),
|
||||||
|
"/tmp/agent/auth.json",
|
||||||
|
);
|
||||||
|
assert.equal(getSearchAuthPath({ HOME: "/tmp/home" }), "/tmp/home/.pi/agent/auth.json");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("literal api_key credentials resolve verbatim", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{
|
||||||
|
tavily: { type: "api_key", key: "tvly-test" },
|
||||||
|
exa: { type: "api_key", key: " exa-test " },
|
||||||
|
keenable: { type: "api_key", key: "" },
|
||||||
|
anthropic: { type: "oauth", access: "a", refresh: "r", expires: 1 },
|
||||||
|
},
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-test");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath }), "exa-test");
|
||||||
|
assert.equal(getSearchApiKey("keenable", { authPath }), undefined);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("missing file, malformed json, non-api_key entries all resolve to undefined", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-"));
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath: join(dir, "absent.json") }), undefined);
|
||||||
|
const bad = join(dir, "bad.json");
|
||||||
|
await writeFile(bad, "not json{");
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath: bad }), undefined);
|
||||||
|
await withAuthFile({ tavily: { type: "oauth", access: "a", refresh: "r", expires: 1 } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
await withAuthFile({ tavily: "literal-string-entry" }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("$VAR and ${VAR} interpolate; credential env beats ambient env", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{
|
||||||
|
tavily: { type: "api_key", key: "$TAVILY_KEY" },
|
||||||
|
exa: { type: "api_key", key: "prefix-${EXA_KEY}-suffix" },
|
||||||
|
keenable: { type: "api_key", key: "$SHARED", env: { SHARED: "scoped-wins" } },
|
||||||
|
},
|
||||||
|
(authPath) => {
|
||||||
|
const env = { TAVILY_KEY: "from-env", EXA_KEY: "braced", SHARED: "ambient-loses" };
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env }), "from-env");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath, env }), "prefix-braced-suffix");
|
||||||
|
assert.equal(getSearchApiKey("keenable", { authPath, env }), "scoped-wins");
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a missing env variable resolves the whole key to undefined", async () => {
|
||||||
|
await withAuthFile({ tavily: { type: "api_key", key: "$NOPE_MISSING" } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("$$ and $! escape to literal characters", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{ tavily: { type: "api_key", key: "cost-is-$$5-and-$!literal" } },
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), "cost-is-$5-and-!literal");
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("!command executes and caches trimmed stdout", async () => {
|
||||||
|
const authPath = join(await mkdtemp(join(tmpdir(), "my-pi-auth-")), "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ tavily: { type: "api_key", key: "!printf secret-from-cmd" } }),
|
||||||
|
);
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "secret-from-cmd");
|
||||||
|
|
||||||
|
let calls = 0;
|
||||||
|
const runCommand = (command: string) => {
|
||||||
|
calls += 1;
|
||||||
|
assert.equal(command, "get-key --flag");
|
||||||
|
return "injected";
|
||||||
|
};
|
||||||
|
await withAuthFile({ exa: { type: "api_key", key: "!get-key --flag" } }, (otherPath) => {
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected");
|
||||||
|
assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a failing !command resolves to undefined", async () => {
|
||||||
|
await withAuthFile({ tavily: { type: "api_key", key: "!exit 3" } }, (authPath) => {
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("credential env map exposes non-key config like KEENABLE_API_URL", async () => {
|
||||||
|
await withAuthFile(
|
||||||
|
{ keenable: { type: "api_key", key: "k", env: { KEENABLE_API_URL: "https://keenable.example" } } },
|
||||||
|
(authPath) => {
|
||||||
|
assert.equal(
|
||||||
|
getSearchCredentialEnv("keenable", "KEENABLE_API_URL", { authPath }),
|
||||||
|
"https://keenable.example",
|
||||||
|
);
|
||||||
|
assert.equal(getSearchCredentialEnv("keenable", "MISSING", { authPath }), undefined);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("template parser handles env names, braces and escapes", () => {
|
||||||
|
assert.deepEqual(parseAuthKeyTemplate("$A_${B}-$$-$!-x"), [
|
||||||
|
{ type: "env", name: "A_" },
|
||||||
|
{ type: "env", name: "B" },
|
||||||
|
{ type: "literal", value: "-$-!-x" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
const AUTH_MJS = new URL("../scripts/search-auth.mjs", import.meta.url).pathname;
|
||||||
|
const SEARCH_SH = new URL("../search_config.sh", import.meta.url).pathname;
|
||||||
|
|
||||||
|
function runMjs(args: string[], env: Record<string, string> = {}) {
|
||||||
|
return spawnSync("node", [AUTH_MJS, ...args], { encoding: "utf8", env: { ...process.env, ...env } });
|
||||||
|
}
|
||||||
|
|
||||||
|
test("search-auth.mjs --set writes api_key entries and preserves other providers", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ anthropic: { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" } }),
|
||||||
|
);
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "tavily=tvly-new", "--set", "exa=!op read x"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.anthropic, { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" });
|
||||||
|
assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-new" });
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "!op read x" });
|
||||||
|
assert.equal((await stat(authPath)).mode & 0o777, 0o600);
|
||||||
|
assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-new");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs preserves an existing credential env map on --set", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(
|
||||||
|
authPath,
|
||||||
|
JSON.stringify({ keenable: { type: "api_key", key: "old", env: { KEENABLE_API_URL: "https://k.example" } } }),
|
||||||
|
);
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "keenable=new-key"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.keenable, {
|
||||||
|
type: "api_key",
|
||||||
|
key: "new-key",
|
||||||
|
env: { KEENABLE_API_URL: "https://k.example" },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs creates a missing auth.json and honors PI_CODING_AGENT_DIR", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const agentDir = join(dir, "agent");
|
||||||
|
const result = runMjs(["--set", "keenable=keen-1"], { PI_CODING_AGENT_DIR: agentDir });
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(join(agentDir, "auth.json"), "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.keenable, { type: "api_key", key: "keen-1" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --unset removes only the named provider", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "t" }, exa: { type: "api_key", key: "e" } }));
|
||||||
|
const result = runMjs(["--auth", authPath, "--unset", "tavily"]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.equal(written.tavily, undefined);
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "e" });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs refuses to clobber a malformed auth.json", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, "{oops");
|
||||||
|
const result = runMjs(["--auth", authPath, "--set", "tavily=x"]);
|
||||||
|
assert.notEqual(result.status, 0);
|
||||||
|
assert.match(result.stderr, /不是合法 JSON|abort|clobber/i);
|
||||||
|
assert.equal(await readFile(authPath, "utf8"), "{oops");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --migrate imports search.env keys without overwriting existing entries", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "existing" } }));
|
||||||
|
const envPath = join(dir, "search.env");
|
||||||
|
await writeFile(envPath, "export TAVILY_API_KEY=old\nEXA_API_KEY=exa-file\nKEENABLE_API_KEY=bad$(cmd)\n");
|
||||||
|
const result = runMjs(["--auth", authPath, "--migrate", envPath]);
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
assert.match(result.stdout, /已迁移 exa/);
|
||||||
|
assert.match(result.stdout, /tavily/);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.equal(written.tavily.key, "existing");
|
||||||
|
assert.equal(written.exa.key, "exa-file");
|
||||||
|
assert.equal(written.keenable, undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search-auth.mjs --has and --list never print key material", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "supersecret" } }));
|
||||||
|
assert.equal(runMjs(["--auth", authPath, "--has", "tavily"]).status, 0);
|
||||||
|
assert.equal(runMjs(["--auth", authPath, "--has", "exa"]).status, 1);
|
||||||
|
const list = runMjs(["--auth", authPath, "--list"]);
|
||||||
|
assert.equal(list.status, 0);
|
||||||
|
assert.match(list.stdout, /tavily: 已配置/);
|
||||||
|
assert.doesNotMatch(list.stdout, /supersecret/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search_config.sh writes keys through search-auth.mjs noninteractively", async () => {
|
||||||
|
const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-"));
|
||||||
|
const authPath = join(dir, "auth.json");
|
||||||
|
const result = spawnSync("sh", [SEARCH_SH, "--auth", authPath, "--tavily", "tvly-sh", "--exa", "exa-sh"], {
|
||||||
|
encoding: "utf8",
|
||||||
|
});
|
||||||
|
assert.equal(result.status, 0, result.stderr);
|
||||||
|
const written = JSON.parse(await readFile(authPath, "utf8")) as Record<string, any>;
|
||||||
|
assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-sh" });
|
||||||
|
assert.deepEqual(written.exa, { type: "api_key", key: "exa-sh" });
|
||||||
|
assert.equal((await stat(authPath)).mode & 0o777, 0o600);
|
||||||
|
});
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import { mkdtemp, readFile, stat } from "node:fs/promises";
|
|
||||||
import { tmpdir } from "node:os";
|
|
||||||
import { join } from "node:path";
|
|
||||||
import { spawnSync } from "node:child_process";
|
|
||||||
import test from "node:test";
|
|
||||||
|
|
||||||
import {
|
|
||||||
getSearchConfigPath,
|
|
||||||
loadSearchConfigEnv,
|
|
||||||
parseSearchConfig,
|
|
||||||
} from "../extensions/search-config.ts";
|
|
||||||
|
|
||||||
test("search config parses only supported safe assignments", () => {
|
|
||||||
assert.deepEqual(
|
|
||||||
parseSearchConfig(`
|
|
||||||
export TAVILY_API_KEY=tvly-test_1
|
|
||||||
EXA_API_KEY=exa-test.2
|
|
||||||
UNKNOWN_KEY=ignored
|
|
||||||
KEENABLE_API_KEY=$(not-allowed)
|
|
||||||
`),
|
|
||||||
{
|
|
||||||
TAVILY_API_KEY: "tvly-test_1",
|
|
||||||
EXA_API_KEY: "exa-test.2",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search config does not override explicit environment values", async () => {
|
|
||||||
const root = await mkdtemp(join(tmpdir(), "my-pi-search-config-"));
|
|
||||||
const configPath = join(root, "search.env");
|
|
||||||
await import("node:fs/promises").then(({ writeFile }) =>
|
|
||||||
writeFile(configPath, "export TAVILY_API_KEY=file-value\nexport EXA_API_KEY=exa-file\n"),
|
|
||||||
);
|
|
||||||
const env: Record<string, string | undefined> = { TAVILY_API_KEY: "explicit-value" };
|
|
||||||
const effective = loadSearchConfigEnv(env, configPath);
|
|
||||||
assert.equal(env.TAVILY_API_KEY, "explicit-value");
|
|
||||||
assert.equal(env.EXA_API_KEY, "exa-file");
|
|
||||||
assert.deepEqual(effective, { TAVILY_API_KEY: "explicit-value", EXA_API_KEY: "exa-file" });
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search config path follows XDG_CONFIG_HOME", () => {
|
|
||||||
assert.equal(getSearchConfigPath({ XDG_CONFIG_HOME: "/tmp/config" }), "/tmp/config/my-pi/search.env");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("search_config.sh supports noninteractive updates and preserves omitted keys", async () => {
|
|
||||||
const home = await mkdtemp(join(tmpdir(), "my-pi-search-script-"));
|
|
||||||
const script = new URL("../search_config.sh", import.meta.url);
|
|
||||||
let result = spawnSync("sh", [script.pathname, "--tavily", "tvly-test", "--exa", "exa-test"], {
|
|
||||||
env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") },
|
|
||||||
encoding: "utf8",
|
|
||||||
});
|
|
||||||
assert.equal(result.status, 0, result.stderr);
|
|
||||||
|
|
||||||
result = spawnSync("sh", [script.pathname, "--keenable", "keen-test"], {
|
|
||||||
env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") },
|
|
||||||
encoding: "utf8",
|
|
||||||
});
|
|
||||||
assert.equal(result.status, 0, result.stderr);
|
|
||||||
|
|
||||||
const configPath = join(home, ".config", "my-pi", "search.env");
|
|
||||||
const content = await readFile(configPath, "utf8");
|
|
||||||
assert.match(content, /^export TAVILY_API_KEY=tvly-test$/m);
|
|
||||||
assert.match(content, /^export EXA_API_KEY=exa-test$/m);
|
|
||||||
assert.match(content, /^export KEENABLE_API_KEY=keen-test$/m);
|
|
||||||
assert.equal((await stat(configPath)).mode & 0o777, 0o600);
|
|
||||||
});
|
|
||||||
@@ -8,12 +8,13 @@ const packageJson = JSON.parse(await readFile(new URL("../package.json", import.
|
|||||||
pi: { extensions: string[]; skills: string[] };
|
pi: { extensions: string[]; skills: string[] };
|
||||||
};
|
};
|
||||||
|
|
||||||
test("pi-web-search is the single search surface, loading after search-config", () => {
|
test("pi-web-search is the single search surface and reads credentials from auth.json", () => {
|
||||||
const entries = packageJson.pi.extensions;
|
const entries = packageJson.pi.extensions;
|
||||||
assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search");
|
assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search");
|
||||||
assert.ok(
|
assert.equal(
|
||||||
entries.indexOf("./extensions/search-config.ts") < entries.indexOf("./pi-web-search/index.ts"),
|
entries.includes("./extensions/search-config.ts"),
|
||||||
"search credentials must load before pi-web-search",
|
false,
|
||||||
|
"search keys come from auth.json — the env loader extension is retired",
|
||||||
);
|
);
|
||||||
for (const retired of [
|
for (const retired of [
|
||||||
"./extensions/mcp.ts",
|
"./extensions/mcp.ts",
|
||||||
|
|||||||
Reference in New Issue
Block a user