diff --git a/AGENTS.md b/AGENTS.md index ece66a2..a2a7b33 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,7 +22,7 @@ - 该目录从明确 tag 导入并由本仓库直接维护,不使用 npm 预编译产物,不是 submodule,也不保留上游 `.pi` 状态;源码、测试、wrapper 与权限集成实现继续保留,但当前不属于根组合包运行时依赖,`extensions/subagents.ts` 也不在默认扩展加载列表中。 - `extensions/fff-override.ts`:强制使用 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 FFF 的 `@` 补全;显式 CLI flag 仍遵循 FFF 官方优先级。 - `pi-web-search/`:完全自维护的统一在线搜索扩展。对模型只暴露 `web_search` / `web_fetch`,schema 与提示词不含厂商名;`web_search` 由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`)按内置路由规则在 Tavily / Exa / Keenable 间选择并经统一参数翻译调用,`details` 记录实际 provider 与路由方式,决策按会话内 LRU 缓存,路由失败或被选厂商报错按可用性顺序降级;`web_fetch` 固定走 Exa `web_fetch_exa`,失败时静默降级 Keenable → Tavily。`clients/` 为纯 `fetch` 直连:Tavily 走 `api.tavily.com`(必须有 key),Keenable 复刻上游 client 保留 keyless `/public` 端点与 SSRF 防护,Exa 直接对 `mcp.exa.ai/mcp` 发 JSON-RPC(缓存 `mcp-session-id`,404 自动重建会话,key 只走 `x-api-key` 头),不使用 `pi-mcp-adapter` 或任何上游扩展包。 -- `search_config.sh` 与 `extensions/search-config.ts`:以交互式或参数方式把三家搜索 key 写入用户级 `search.env`(权限 `600`),并在搜索扩展初始化前加载;显式进程环境变量优先。 +- `search_config.sh` 与 `scripts/search-auth.mjs`:以交互或参数方式把 `tavily` / `exa` / `keenable` 三条 `api_key` 凭据写入 Pi 的 `auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`);加锁后原子写、权限 `600`、保留全部其他条目,支持 `--migrate` 从旧 `search.env` 导入。`pi-web-search/auth.ts` 每次调用读取并复刻上游解析语义(`!command` 执行、`$VAR`/`${VAR}` 插值、credential `env` 优先于进程 env、`$$`/`$!` 转义);显式进程环境变量不再直接读取,env 注入通过 `"key": "$VAR"` 表达。 - `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,以 `execFile` argv 形式调用机器级 `codegraph` CLI(每次调用为短命进程,无常驻 server/daemon);未初始化项目先经 `ctx.ui.confirm` 确认再 `codegraph init` 并自动重试,拒绝或失败则把结果打回让模型改用 `find`/`grep`/`read`/LSP;结果按 `codegraph.db` mtime 做有界缓存,索引一变即失效。 - `extensions/plugin-init.ts`:已移除;原 `/plugin_init` 的 CodeGraph 初始化职责由 `extensions/codegraph.ts` 在首次 `codegraph_explore` 调用内按需确认完成。 - `context-mode`:加载上游 Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性。 @@ -89,9 +89,9 @@ - `.pi-debug/` 是 `/dump-system-prompt` 生成的本地诊断目录,不提交到仓库,也不作为组合包运行时配置源。 - CodeGraph 是机器级依赖:`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后,扩展才可以 `codegraph init` 当前目录。init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。 - `codegraph_explore` 为原生 Pi 工具而非 MCP 工具:每次调用通过 `execFile`/`pi.exec` 以 argv 形式运行 `codegraph explore`(带超时、不经 shell),结果按 `.codegraph/codegraph.db` mtime 缓存;CLI 缺失、init 被拒绝或执行失败都不得抛出或阻止其他工具,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。 -- 组合包不加载任何 MCP adapter;Exa 由 `pi-web-search/clients/exa.ts` 以纯 `fetch` JSON-RPC 直连托管端点 `https://mcp.exa.ai/mcp`:`initialize` 一次并缓存 `mcp-session-id`,随后 `tools/call`,HTTP 404 视为会话失效自动重建并重试一次。`EXA_API_KEY` 通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。 -- 在线搜索对模型只暴露 `web_search` / `web_fetch` 两个统一工具名,不得重新引入按厂商命名的工具;`web_search` 必须经 `router.ts` 的主模型分类选择 provider,`web_fetch` 固定优先 Exa。Exa 搜索统一走 `web_search_advanced_exa` 并显式约束 `numResults` 与 `textMaxCharacters`;Tavily 默认 `search_depth: basic` 且必须设置 `TAVILY_API_KEY` 才参与路由;Keenable keyless 可用,`KEENABLE_API_KEY` 仅提高速率限制。 -- 搜索 key 默认保存在 `${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env`,文件必须为 `600` 且不得提交;`search_config.sh` 只输出配置状态,禁止回显 key。 +- 组合包不加载任何 MCP adapter;Exa 由 `pi-web-search/clients/exa.ts` 以纯 `fetch` JSON-RPC 直连托管端点 `https://mcp.exa.ai/mcp`:`initialize` 一次并缓存 `mcp-session-id`,随后 `tools/call`,HTTP 404 视为会话失效自动重建并重试一次。`exa` 凭据通过 `x-api-key` 请求头发送,不得放入 URL、仓库文件或日志。 +- 在线搜索对模型只暴露 `web_search` / `web_fetch` 两个统一工具名,不得重新引入按厂商命名的工具;`web_search` 必须经 `router.ts` 的主模型分类选择 provider,`web_fetch` 固定优先 Exa。Exa 搜索统一走 `web_search_advanced_exa` 并显式约束 `numResults` 与 `textMaxCharacters`;Tavily 默认 `search_depth: basic` 且必须在 auth.json 存在 `tavily` api_key 凭据才参与路由;Keenable keyless 可用,`keenable` 凭据仅提高速率限制。 +- 搜索 key 统一保存在 Pi 的 `auth.json`(`tavily`/`exa`/`keenable` 各一条 `api_key` 凭据,文件权限 `600` 且不得提交);key 支持 `!command` 与 `$VAR` 间接寻址,`search_config.sh` / `search-auth.mjs` 只输出配置状态,禁止回显 key。旧的 `search.env` 不再被读取,仅由 `--migrate` 一次性导入。 - `extensions/context-view.ts` 是只读的本地 effective-context observer,只读取 `pi-compaction/src/service.ts` 发布的 projection metrics,不注册生产 `context` writer;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。 - `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。 - 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。 diff --git a/README.md b/README.md index 8d2b26b..de1320f 100644 --- a/README.md +++ b/README.md @@ -255,14 +255,21 @@ FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash ./search_config.sh --tavily '...' --exa '...' --keenable '...' ``` -只传部分参数时保留其他服务的已有值。脚本把 key 原子写入 -`${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env` 并设置权限为 `600`; -`MY_PI_SEARCH_CONFIG` 或 `--config` 可覆盖路径。组合包在搜索扩展初始化前加载该文件, -但调用 Pi 时显式传入的同名环境变量优先。配置文件不属于仓库,不会随安装包提交。 +只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的 +`auth.json`(`$PI_CODING_AGENT_DIR/auth.json`,默认 `~/.pi/agent/auth.json`,权限 `600`): +每个 provider 一条 `{ "type": "api_key", "key": "..." }` 条目,原子加锁写入, +其他条目(如 OAuth 凭据)原样保留。`--list` 只显示配置状态、不回显 key; +`--migrate` 可从旧的 `search.env` 导入缺失条目;`--auth FILE` 可覆盖文件路径。 + +key 值支持 auth.json 原生解析语义:`$VAR` / `${VAR}` 环境变量插值、`!command` +执行命令取 stdout(如 `"!op read op://Vault/Tavily/key"`)、`$$` / `$!` 转义。 +显式进程环境变量因此不再被直接读取——需要 env 注入时把 key 写成 +`"$TAVILY_API_KEY"` 即可。`KEENABLE_API_URL`(非 secret 的端点覆盖)仍走 +进程环境变量,也可放在 keenable 凭据的 `env` 字段中。 三个服务由本仓库自维护的 `pi-web-search/` 统一封装:对模型只暴露 `web_search` / `web_fetch` 两个工具,schema 与提示词中不出现任何厂商名;每次 `web_search` 先由当前主模型(`ctx.model` + `ctx.modelRegistry.complete`,零额外配置)按内置路由规则在候选厂商中选一个,再按统一参数翻译后调用,`details` 记录实际 provider 与路由方式;选择结果按会话内 LRU 缓存,路由失败或被选厂商报错时按可用性顺序自动降级。`web_fetch` 固定调用 Exa 的 `web_fetch_exa` 提取正文(默认每页 3000 字符上限),失败时静默降级到 Keenable、再到 Tavily。 -`pi-web-search/clients/` 全部是纯 `fetch` 实现:Tavily 直连 `api.tavily.com/search|extract`(必须有 `TAVILY_API_KEY`,无 key 时不参与路由);Keenable 复刻上游 client,默认走 keyless `/public` 端点,`KEENABLE_API_KEY` 只提高速率限制并保留 SSRF host 防护;Exa 直接对 `https://mcp.exa.ai/mcp` 发 JSON-RPC——initialize 一次并缓存 `mcp-session-id`、随后 `tools/call`,会话失效(HTTP 404)自动重建,key 只通过 `x-api-key` 请求头发送,不再依赖 `pi-mcp-adapter`。Exa 搜索统一走 `web_search_advanced_exa` 并显式限制结果数量与 `textMaxCharacters`,避免响应过大。 +`pi-web-search/clients/` 全部是纯 `fetch` 实现,三个 key 统一由 `pi-web-search/auth.ts` 从 auth.json 解析(Tavily 直连 `api.tavily.com/search|extract`,必须有 `tavily` 凭据,无 key 时不参与路由;Keenable 复刻上游 client,默认走 keyless `/public` 端点,`keenable` 凭据只提高速率限制并保留 SSRF host 防护;Exa 直接对 `https://mcp.exa.ai/mcp` 发 JSON-RPC——initialize 一次并缓存 `mcp-session-id`、随后 `tools/call`,会话失效(HTTP 404)自动重建,`exa` 凭据只通过 `x-api-key` 请求头发送,不再依赖 `pi-mcp-adapter`)。Exa 搜索统一走 `web_search_advanced_exa` 并显式限制结果数量与 `textMaxCharacters`,避免响应过大。 ### CodeGraph 是原生工具 diff --git a/extensions/search-config.ts b/extensions/search-config.ts deleted file mode 100644 index 0134f8b..0000000 --- a/extensions/search-config.ts +++ /dev/null @@ -1,57 +0,0 @@ -import { readFileSync } from "node:fs"; -import { homedir } from "node:os"; -import { join } from "node:path"; - -export const SEARCH_API_KEY_NAMES = ["TAVILY_API_KEY", "EXA_API_KEY", "KEENABLE_API_KEY"] as const; - -type SearchApiKeyName = (typeof SEARCH_API_KEY_NAMES)[number]; -type SearchEnvironment = Record; - -const SAFE_VALUE = /^[A-Za-z0-9._-]+$/; - -export function getSearchConfigPath(env: SearchEnvironment = process.env): string { - if (env.MY_PI_SEARCH_CONFIG?.trim()) { - return env.MY_PI_SEARCH_CONFIG.trim(); - } - const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config"); - return join(configHome, "my-pi", "search.env"); -} - -export function parseSearchConfig(text: string): Partial> { - const parsed: Partial> = {}; - for (const rawLine of text.split(/\r?\n/u)) { - const line = rawLine.trim().replace(/^export[ \t]+/u, ""); - if (!line || line.startsWith("#")) continue; - const separator = line.indexOf("="); - if (separator < 1) continue; - const name = line.slice(0, separator).trim() as SearchApiKeyName; - const value = line.slice(separator + 1).trim(); - if (!SEARCH_API_KEY_NAMES.includes(name) || !SAFE_VALUE.test(value)) continue; - parsed[name] = value; - } - return parsed; -} - -/** Load package-managed search credentials without overriding explicit process environment values. */ -export function loadSearchConfigEnv( - env: SearchEnvironment = process.env, - configPath = getSearchConfigPath(env), -): Partial> { - let parsed: Partial> = {}; - try { - parsed = parseSearchConfig(readFileSync(configPath, "utf8")); - } catch { - // Missing or unreadable config is equivalent to an empty package-managed config. - } - const effective: Partial> = {}; - for (const name of SEARCH_API_KEY_NAMES) { - if (!env[name]?.trim() && parsed[name]) env[name] = parsed[name]; - if (env[name]?.trim()) effective[name] = env[name]?.trim(); - } - return effective; -} - -/** Load search API keys before provider extensions initialize. */ -export default function searchConfigExtension(): void { - loadSearchConfigEnv(); -} diff --git a/package.json b/package.json index 15fa1ad..58a5fbc 100644 --- a/package.json +++ b/package.json @@ -49,7 +49,6 @@ ], "pi": { "extensions": [ - "./extensions/search-config.ts", "./extensions/fff-override.ts", "./pi-web-search/index.ts", "./extensions/codegraph.ts", diff --git a/pi-web-search/README.md b/pi-web-search/README.md index 5ad2c9b..6cf48a9 100644 --- a/pi-web-search/README.md +++ b/pi-web-search/README.md @@ -15,16 +15,22 @@ Unified web search and fetch for Pi. The main model only ever sees two tools — Plain `fetch` calls — no MCP adapter, no upstream extension wrappers: - `clients/tavily.ts` — `POST api.tavily.com/search|extract`, Bearer key. - Requires `TAVILY_API_KEY`. + Requires a `tavily` credential. - `clients/keenable.ts` — `api.keenable.ai`, keyless `/public` endpoints by - default, `KEENABLE_API_KEY` raises rate limits. Adapted from + default, a `keenable` credential raises rate limits. Adapted from `@keenable/pi-search`. - `clients/exa.ts` — JSON-RPC over `mcp.exa.ai/mcp` (Streamable HTTP): `initialize` once, cache `mcp-session-id`, `tools/call` per request, re-init - on session expiry. `EXA_API_KEY` optional (`x-api-key` header). + on session expiry. An `exa` credential is optional (`x-api-key` header). -Keys are loaded from `~/.config/my-pi/search.env` by `extensions/search-config.ts` -(written by `search_config.sh`); explicit environment variables win. +Keys live in Pi's `auth.json` as one `api_key` credential per provider +(`tavily`, `exa`, `keenable`), read by `auth.ts` with upstream resolution +semantics — literal values, `$VAR`/`${VAR}` interpolation, and `!command` +indirection (e.g. `"!op read op://Vault/Tavily/key"`). `search_config.sh` +writes them (atomically, mode 600, preserving all other entries); process +environment variables are not read directly — use `"key": "$TAVILY_API_KEY"` +to indirect through env. `KEENABLE_API_URL` stays env-only, or can be stored +in the keenable credential's `env` map. ## Routing diff --git a/pi-web-search/auth.ts b/pi-web-search/auth.ts new file mode 100644 index 0000000..53bb754 --- /dev/null +++ b/pi-web-search/auth.ts @@ -0,0 +1,201 @@ +/** + * Search provider API keys live in Pi's own auth.json — one `api_key` + * credential per provider id (`tavily`, `exa`, `keenable`) next to Pi's + * model credentials. Key values follow upstream resolve-config-value + * semantics (replicated here because the upstream resolver is not exported): + * + * - `!command` executes the rest of the string and uses trimmed stdout, + * cached for the process lifetime (same as upstream's command cache) + * - `$NAME` / `${NAME}` interpolate environment variables; a credential-level + * `env` map wins over process env; a missing variable resolves to undefined + * - `$$` and `$!` escape to literal `$` and `!` + * - anything else is a literal key + */ + +import { execSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; + +export type SearchAuthProvider = "tavily" | "exa" | "keenable"; +export const SEARCH_AUTH_PROVIDERS: readonly SearchAuthProvider[] = ["tavily", "exa", "keenable"]; + +/** Resolves a provider's API key; injectable so tests never touch real auth.json. */ +export type SearchKeyResolver = (provider: SearchAuthProvider) => string | undefined; + +export type SearchAuthEnvironment = Record; + +export interface SearchAuthOptions { + /** auth.json location; defaults to `/auth.json`. */ + authPath?: string; + /** + * Environment used for the agent dir and `$VAR` interpolation. When + * provided it fully replaces `process.env` so tests stay isolated. + */ + env?: SearchAuthEnvironment; + /** Executes `!command` key references; defaults to a cached execSync runner. */ + runCommand?: (command: string) => string | undefined; +} + +export interface SearchApiKeyCredential { + type: "api_key"; + key?: string; + env?: Record; +} + +const COMMAND_TIMEOUT_MS = 10_000; +const ENV_VAR_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/; +const ENV_VAR_NAME_PREFIX = /^[A-Za-z_][A-Za-z0-9_]*/; +const commandResultCache = new Map(); + +/** auth.json lives in Pi's agent dir: $PI_CODING_AGENT_DIR or ~/.pi/agent. */ +export function getSearchAuthPath(env: SearchAuthEnvironment = process.env): string { + const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent"); + return join(agentDir, "auth.json"); +} + +/** Read the raw `api_key` credential for a provider; undefined when absent or malformed. */ +export function readSearchCredential( + provider: SearchAuthProvider, + options: SearchAuthOptions = {}, +): SearchApiKeyCredential | undefined { + let data: unknown; + try { + data = JSON.parse(readFileSync(options.authPath ?? getSearchAuthPath(options.env), "utf8")); + } catch { + return undefined; + } + if (!data || typeof data !== "object" || Array.isArray(data)) return undefined; + const entry = (data as Record)[provider]; + if (!entry || typeof entry !== "object" || Array.isArray(entry)) return undefined; + const credential = entry as Record; + if (credential.type !== "api_key") return undefined; + if (credential.key !== undefined && typeof credential.key !== "string") return undefined; + const env = credential.env; + const scopedEnv = + env && typeof env === "object" && !Array.isArray(env) + ? Object.fromEntries( + Object.entries(env as Record).filter( + (pair): pair is [string, string] => typeof pair[1] === "string", + ), + ) + : undefined; + return { type: "api_key", key: credential.key as string | undefined, env: scopedEnv }; +} + +type TemplatePart = { type: "literal"; value: string } | { type: "env"; name: string }; + +/** Mirrors upstream parseConfigValueTemplate: `$$`/`$!` escape, `${A}`/`$A` interpolate. */ +export function parseAuthKeyTemplate(config: string): TemplatePart[] { + const parts: TemplatePart[] = []; + const appendLiteral = (value: string) => { + if (!value) return; + const previous = parts[parts.length - 1]; + if (previous?.type === "literal") { + previous.value += value; + return; + } + parts.push({ type: "literal", value }); + }; + let index = 0; + while (index < config.length) { + const dollarIndex = config.indexOf("$", index); + if (dollarIndex < 0) { + appendLiteral(config.slice(index)); + break; + } + appendLiteral(config.slice(index, dollarIndex)); + const nextChar = config[dollarIndex + 1]; + if (nextChar === "$" || nextChar === "!") { + appendLiteral(nextChar); + index = dollarIndex + 2; + continue; + } + if (nextChar === "{") { + const endIndex = config.indexOf("}", dollarIndex + 2); + if (endIndex < 0) { + appendLiteral("$"); + index = dollarIndex + 1; + continue; + } + const name = config.slice(dollarIndex + 2, endIndex); + if (ENV_VAR_NAME.test(name)) { + parts.push({ type: "env", name }); + } else { + appendLiteral(config.slice(dollarIndex, endIndex + 1)); + } + index = endIndex + 1; + continue; + } + const match = config.slice(dollarIndex + 1).match(ENV_VAR_NAME_PREFIX); + if (match) { + parts.push({ type: "env", name: match[0] }); + index = dollarIndex + 1 + match[0].length; + continue; + } + appendLiteral("$"); + index = dollarIndex + 1; + } + return parts; +} + +function defaultRunCommand(command: string): string | undefined { + if (commandResultCache.has(command)) return commandResultCache.get(command); + let value: string | undefined; + try { + const output = execSync(command, { + encoding: "utf-8", + timeout: COMMAND_TIMEOUT_MS, + stdio: ["ignore", "pipe", "ignore"], + }); + value = output.trim() || undefined; + } catch { + value = undefined; + } + commandResultCache.set(command, value); + return value; +} + +/** Mirrors upstream resolveConfigValue, including credential-scoped env precedence. */ +export function resolveAuthKeyValue( + config: string, + credentialEnv: Record | undefined, + options: SearchAuthOptions = {}, +): string | undefined { + if (config.startsWith("!")) { + return (options.runCommand ?? defaultRunCommand)(config.slice(1)); + } + const ambient = options.env ?? process.env; + let resolved = ""; + for (const part of parseAuthKeyTemplate(config)) { + if (part.type === "literal") { + resolved += part.value; + continue; + } + const value = credentialEnv?.[part.name] || ambient[part.name]; + if (value === undefined) return undefined; + resolved += value; + } + return resolved; +} + +/** Provider API key from auth.json, or undefined when not configured. */ +export function getSearchApiKey( + provider: SearchAuthProvider, + options: SearchAuthOptions = {}, +): string | undefined { + const credential = readSearchCredential(provider, options); + if (!credential?.key) return undefined; + const resolved = resolveAuthKeyValue(credential.key, credential.env, options); + return resolved?.trim() || undefined; +} + +/** A credential-scoped config value (e.g. KEENABLE_API_URL), or undefined. */ +export function getSearchCredentialEnv( + provider: SearchAuthProvider, + name: string, + options: SearchAuthOptions = {}, +): string | undefined { + const value = readSearchCredential(provider, options)?.env?.[name]?.trim(); + return value || undefined; +} diff --git a/pi-web-search/clients/exa.ts b/pi-web-search/clients/exa.ts index e2feb8b..e827424 100644 --- a/pi-web-search/clients/exa.ts +++ b/pi-web-search/clients/exa.ts @@ -2,9 +2,12 @@ * Exa client: direct JSON-RPC over the hosted endpoint https://mcp.exa.ai/mcp. * This is a plain HTTPS POST client (no MCP adapter, no SDK): initialize once, * cache the `mcp-session-id` response header, then POST `tools/call` per call. - * Keyless works; EXA_API_KEY is sent as the `x-api-key` header when present. + * Keyless works; the `exa` api_key credential from auth.json is sent as the + * `x-api-key` header when present. */ +import { getSearchApiKey } from "../auth.ts"; + export class ExaError extends Error {} const EXA_MCP_URL = "https://mcp.exa.ai/mcp?tools=web_search_exa,web_search_advanced_exa,web_fetch_exa"; @@ -41,7 +44,7 @@ function baseHeaders(): Record { "content-type": "application/json", accept: "application/json, text/event-stream", }; - const key = process.env.EXA_API_KEY?.trim(); + const key = getSearchApiKey("exa"); if (key) headers["x-api-key"] = key; return headers; } diff --git a/pi-web-search/clients/keenable.ts b/pi-web-search/clients/keenable.ts index 70b6324..4f68008 100644 --- a/pi-web-search/clients/keenable.ts +++ b/pi-web-search/clients/keenable.ts @@ -1,17 +1,19 @@ /** * Keenable REST client, adapted from @keenable/pi-search (keenable-client.ts). - * Keyless by default (`/v1/*\/public`), authenticated when KEENABLE_API_KEY is - * present (`/v1/*`), HTTPS-only base URL, `X-Keenable-Title` attribution. + * Keyless by default (`/v1/*\/public`), authenticated when the `keenable` + * api_key credential exists in auth.json (`/v1/*`), HTTPS-only base URL, + * `X-Keenable-Title` attribution. */ import { isIP } from "node:net"; +import { getSearchApiKey, getSearchCredentialEnv } from "../auth.ts"; + export class KeenableError extends Error {} const CLIENT_VERSION = "0.1.0"; const DEFAULT_BASE_URL = "https://api.keenable.ai"; const BASE_URL_ENV = "KEENABLE_API_URL"; -const API_KEY_ENV = "KEENABLE_API_KEY"; const ATTRIBUTION_TITLE = "Pi"; const USER_AGENT = `pi-web-search/${CLIENT_VERSION}`; const DEFAULT_TIMEOUT_MS = 30_000; @@ -45,9 +47,15 @@ export interface KeenableFetchResult { [key: string]: unknown; } -/** Resolve the API base URL from `KEENABLE_API_URL`, enforcing HTTPS (http only for loopback). */ +/** + * Resolve the API base URL from `KEENABLE_API_URL` (process env first, then + * the keenable credential's `env` map), enforcing HTTPS (http only for loopback). + */ function resolveBaseUrl(): string { - const base = (process.env[BASE_URL_ENV] || DEFAULT_BASE_URL).replace(/\/+$/u, ""); + const base = (process.env[BASE_URL_ENV] || getSearchCredentialEnv("keenable", BASE_URL_ENV) || DEFAULT_BASE_URL).replace( + /\/+$/u, + "", + ); let parsed: URL; try { parsed = new URL(base); @@ -63,10 +71,9 @@ function resolveBaseUrl(): string { throw new KeenableError(`${BASE_URL_ENV} must be an https:// URL with a host, got ${base}`); } -/** Non-blank API key from the env, or undefined to use the keyless public tier. */ +/** API key from auth.json, or undefined to use the keyless public tier. */ function resolveApiKey(): string | undefined { - const key = (process.env[API_KEY_ENV] ?? "").trim(); - return key || undefined; + return getSearchApiKey("keenable"); } function buildHeaders(apiKey: string | undefined): Record { @@ -129,7 +136,7 @@ async function raiseForStatus(response: Response, keyed: boolean): Promise if (response.ok) return; if (!keyed && response.status === 429) { throw new KeenableError( - "Keenable keyless requests hit their rate limit. Set KEENABLE_API_KEY to raise the limits.", + 'Keenable keyless requests hit their rate limit. Add a "keenable" api_key credential to auth.json to raise the limits.', ); } let detail = ""; diff --git a/pi-web-search/clients/tavily.ts b/pi-web-search/clients/tavily.ts index fc1a2fb..0df016e 100644 --- a/pi-web-search/clients/tavily.ts +++ b/pi-web-search/clients/tavily.ts @@ -1,8 +1,10 @@ /** * Tavily REST client: POST https://api.tavily.com/search|extract with a Bearer - * key. Requires TAVILY_API_KEY — there is no keyless tier. + * key from auth.json (`tavily` api_key credential) — there is no keyless tier. */ +import { getSearchApiKey } from "../auth.ts"; + export class TavilyError extends Error {} const TAVILY_BASE_URL = "https://api.tavily.com"; @@ -30,8 +32,8 @@ export interface TavilySearchResult { [key: string]: unknown; } -export function tavilyAvailable(env: Record = process.env): boolean { - return Boolean(env.TAVILY_API_KEY?.trim()); +export function tavilyAvailable(resolve: (provider: "tavily") => string | undefined = getSearchApiKey): boolean { + return Boolean(resolve("tavily")); } function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal { @@ -46,9 +48,9 @@ async function post( signal: AbortSignal | undefined, fetchImpl: FetchLike, ): Promise> { - const apiKey = process.env.TAVILY_API_KEY?.trim(); + const apiKey = getSearchApiKey("tavily"); if (!apiKey) { - throw new TavilyError("Tavily is not configured: TAVILY_API_KEY is missing."); + throw new TavilyError('Tavily is not configured: no "tavily" api_key credential in auth.json.'); } let response: Response; try { diff --git a/pi-web-search/index.ts b/pi-web-search/index.ts index 4fbac61..f33de04 100644 --- a/pi-web-search/index.ts +++ b/pi-web-search/index.ts @@ -9,7 +9,7 @@ import type { AgentToolResult, ExtensionAPI } from "@earendil-works/pi-coding-agent"; -import { loadSearchConfigEnv } from "../extensions/search-config.ts"; +import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts"; import { exaFetchPages, exaSearch } from "./clients/exa.ts"; import { keenableFetch, keenableSearch } from "./clients/keenable.ts"; import { tavilyAvailable, tavilyExtract, tavilySearch } from "./clients/tavily.ts"; @@ -108,9 +108,10 @@ export async function dispatchSearch( routeCache: Map, signal?: AbortSignal, runners: Record = createProviderRunners(), + resolve: SearchKeyResolver = getSearchApiKey, ): Promise { - const decision = await routeSearch(params, ctx, routeCache, signal); - const order = [decision.provider, ...availableProviders().filter((p) => p !== decision.provider)]; + const decision = await routeSearch(params, ctx, routeCache, signal, resolve); + const order = [decision.provider, ...availableProviders(resolve).filter((p) => p !== decision.provider)]; let lastError: unknown; for (const provider of order) { try { @@ -161,6 +162,7 @@ export async function dispatchFetch( params: WebFetchParams, signal?: AbortSignal, fetchers: Record = createFetchRunners(), + resolve: SearchKeyResolver = getSearchApiKey, ): Promise { const urls = normalizeUrls(params.urls); if (!urls.length) { @@ -171,7 +173,7 @@ export async function dispatchFetch( } const maxCharacters = params.max_characters ?? DEFAULT_FETCH_MAX_CHARACTERS; const order: SearchProvider[] = ["exa", "keenable"]; - if (tavilyAvailable()) order.push("tavily"); + if (tavilyAvailable(resolve)) order.push("tavily"); let lastError: unknown; for (const provider of order) { @@ -186,9 +188,6 @@ export async function dispatchFetch( } export default function webSearchExtension(pi: ExtensionAPI): void { - // Keys may already be loaded by extensions/search-config.ts; this is a - // harmless idempotent safety net for standalone loading. - loadSearchConfigEnv(); const routeCache = new Map(); pi.registerTool({ diff --git a/pi-web-search/package.json b/pi-web-search/package.json index cc92048..6056001 100644 --- a/pi-web-search/package.json +++ b/pi-web-search/package.json @@ -11,6 +11,7 @@ }, "files": [ "index.ts", + "auth.ts", "router.ts", "clients", "README.md", diff --git a/pi-web-search/router.ts b/pi-web-search/router.ts index 1517ef3..95946c3 100644 --- a/pi-web-search/router.ts +++ b/pi-web-search/router.ts @@ -5,6 +5,8 @@ * and failure fallbacks, never the routing itself. */ +import { getSearchApiKey, type SearchKeyResolver } from "./auth.ts"; + export type SearchProvider = "tavily" | "exa" | "keenable"; export interface SearchRequest { @@ -43,10 +45,10 @@ const PROVIDER_RULES: Record = { "content length.", }; -/** Tavily requires a key; Exa and Keenable both work keyless. */ -export function availableProviders(env: Record = process.env): SearchProvider[] { +/** Tavily requires a key in auth.json; Exa and Keenable both work keyless. */ +export function availableProviders(resolve: SearchKeyResolver = getSearchApiKey): SearchProvider[] { const providers: SearchProvider[] = []; - if (env.TAVILY_API_KEY?.trim()) providers.push("tavily"); + if (resolve("tavily")) providers.push("tavily"); providers.push("exa", "keenable"); return providers; } @@ -111,9 +113,9 @@ export async function routeSearch( ctx: RouterContext, cache: Map, signal?: AbortSignal, - env: Record = process.env, + resolve: SearchKeyResolver = getSearchApiKey, ): Promise { - const candidates = availableProviders(env); + const candidates = availableProviders(resolve); const fallback = PROVIDER_FALLBACK_ORDER.find((provider) => candidates.includes(provider)) ?? candidates[0]; if (!fallback || candidates.length <= 1) { return { provider: fallback ?? "keenable", routedBy: "fallback" }; diff --git a/pi-web-search/test/dispatch.test.ts b/pi-web-search/test/dispatch.test.ts index cfe7469..3c5663e 100644 --- a/pi-web-search/test/dispatch.test.ts +++ b/pi-web-search/test/dispatch.test.ts @@ -29,24 +29,8 @@ function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }) { }; } -function withEnv(key: string, value: string | undefined, fn: () => Promise | T): Promise | T { - const previous = process.env[key]; - if (value === undefined) delete process.env[key]; - else process.env[key] = value; - const restore = () => { - if (previous === undefined) delete process.env[key]; - else process.env[key] = previous; - }; - try { - const out = fn(); - if (out instanceof Promise) return out.finally(restore); - restore(); - return out; - } catch (error) { - restore(); - throw error; - } -} +const WITH_TAVILY = () => "k"; +const KEYLESS = () => undefined; test("extension registers only web_search and web_fetch, with no provider in the schema", () => { const tools = new Map }; description?: string }>(); @@ -64,52 +48,46 @@ test("extension registers only web_search and web_fetch, with no provider in the }); test("dispatchSearch runs the model-picked provider first and records details", async () => { - await withEnv("TAVILY_API_KEY", "k", async () => { - const ran: string[] = []; - const runners = createProviderRunners({ - exa: async () => { - ran.push("exa"); - return "exa results"; - }, - }); - const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners); - assert.deepEqual(ran, ["exa"]); - assert.equal(result.content[0]!.type === "text" && result.content[0]!.text, "exa results"); - assert.deepEqual(result.details, { provider: "exa", routedBy: "model" }); + const ran: string[] = []; + const runners = createProviderRunners({ + exa: async () => { + ran.push("exa"); + return "exa results"; + }, }); + const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY); + assert.deepEqual(ran, ["exa"]); + assert.equal(result.content[0]!.type === "text" && result.content[0]!.text, "exa results"); + assert.deepEqual(result.details, { provider: "exa", routedBy: "model" }); }); test("dispatchSearch falls back across remaining providers when the pick fails", async () => { - await withEnv("TAVILY_API_KEY", "k", async () => { - const ran: string[] = []; - const runners = createProviderRunners({ - tavily: async () => { - ran.push("tavily"); - throw new Error("tavily down"); - }, - exa: async () => { - ran.push("exa"); - return "exa results"; - }, - }); - const result = await dispatchSearch(PARAMS, ctxAnswering("tavily"), new Map(), undefined, runners); - assert.deepEqual(ran, ["tavily", "exa"]); - assert.equal((result.details as Record).provider, "exa"); - assert.equal((result.details as Record).routedBy, "provider-fallback"); + const ran: string[] = []; + const runners = createProviderRunners({ + tavily: async () => { + ran.push("tavily"); + throw new Error("tavily down"); + }, + exa: async () => { + ran.push("exa"); + return "exa results"; + }, }); + const result = await dispatchSearch(PARAMS, ctxAnswering("tavily"), new Map(), undefined, runners, WITH_TAVILY); + assert.deepEqual(ran, ["tavily", "exa"]); + assert.equal((result.details as Record).provider, "exa"); + assert.equal((result.details as Record).routedBy, "provider-fallback"); }); test("dispatchSearch reports an error result when every provider fails", async () => { - await withEnv("TAVILY_API_KEY", "k", async () => { - const runners = createProviderRunners({ - tavily: async () => { throw new Error("t"); }, - exa: async () => { throw new Error("e"); }, - keenable: async () => { throw new Error("k"); }, - }); - const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners); - assert.equal((result.details as Record).error, true); - assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /failed on every available provider/); + const runners = createProviderRunners({ + tavily: async () => { throw new Error("t"); }, + exa: async () => { throw new Error("e"); }, + keenable: async () => { throw new Error("k"); }, }); + const result = await dispatchSearch(PARAMS, ctxAnswering("exa"), new Map(), undefined, runners, WITH_TAVILY); + assert.equal((result.details as Record).error, true); + assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /failed on every available provider/); }); test("normalizeUrls accepts a string or array, trims, and deduplicates", () => { @@ -118,17 +96,15 @@ test("normalizeUrls accepts a string or array, trims, and deduplicates", () => { }); test("dispatchFetch prefers exa and degrades to keenable without a model call", async () => { - await withEnv("TAVILY_API_KEY", undefined, async () => { - const fetchers = createFetchRunners({ - exa: async (urls) => { - throw new Error(`exa down for ${urls.length}`); - }, - keenable: async (urls) => `keenable pages: ${urls.join(",")}`, - }); - const result = await dispatchFetch({ urls: ["https://a.example", "https://b.example"] }, undefined, fetchers); - assert.equal((result.details as Record).provider, "keenable"); - assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /keenable pages: https:\/\/a\.example,https:\/\/b\.example/); + const fetchers = createFetchRunners({ + exa: async (urls) => { + throw new Error(`exa down for ${urls.length}`); + }, + keenable: async (urls) => `keenable pages: ${urls.join(",")}`, }); + const result = await dispatchFetch({ urls: ["https://a.example", "https://b.example"] }, undefined, fetchers, KEYLESS); + assert.equal((result.details as Record).provider, "keenable"); + assert.match(result.content[0]!.type === "text" ? result.content[0]!.text : "", /keenable pages: https:\/\/a\.example,https:\/\/b\.example/); }); test("dispatchFetch validates input and caps the URL count", async () => { diff --git a/pi-web-search/test/router.test.ts b/pi-web-search/test/router.test.ts index 344031b..83b0581 100644 --- a/pi-web-search/test/router.test.ts +++ b/pi-web-search/test/router.test.ts @@ -12,7 +12,8 @@ import { } from "../router.ts"; const ALL: SearchProvider[] = ["tavily", "exa", "keenable"]; -const ENV_ALL_KEYS = { TAVILY_API_KEY: "test-key" }; +const ALL_KEYS = () => "test-key"; +const NO_KEYS = () => undefined; function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }): RouterContext { return { @@ -27,8 +28,8 @@ function ctxAnswering(answer: string, calls: { count: number } = { count: 0 }): } test("availability: tavily needs a key, exa and keenable are keyless", () => { - assert.deepEqual(availableProviders({}), ["exa", "keenable"]); - assert.deepEqual(availableProviders(ENV_ALL_KEYS), ["tavily", "exa", "keenable"]); + assert.deepEqual(availableProviders(NO_KEYS), ["exa", "keenable"]); + assert.deepEqual(availableProviders(ALL_KEYS), ["tavily", "exa", "keenable"]); }); test("router prompt lists only candidate rules plus the query and caller hints", () => { @@ -64,44 +65,44 @@ test("routeSearch honors the model pick and caches identical requests", async () const cache = new Map(); const request = { query: "official MCP spec" }; - const first = await routeSearch(request, ctx, cache, undefined, ENV_ALL_KEYS); + const first = await routeSearch(request, ctx, cache, undefined, ALL_KEYS); assert.deepEqual(first, { provider: "exa", routedBy: "model" }); - const second = await routeSearch(request, ctx, cache, undefined, ENV_ALL_KEYS); + const second = await routeSearch(request, ctx, cache, undefined, ALL_KEYS); assert.deepEqual(second, { provider: "exa", routedBy: "cache" }); assert.equal(calls.count, 1); }); test("routeSearch falls back to the first available provider on unroutable or failed model calls", async () => { const cache = new Map(); - const unparseable = await routeSearch({ query: "x" }, ctxAnswering("no idea"), cache, undefined, ENV_ALL_KEYS); + const unparseable = await routeSearch({ query: "x" }, ctxAnswering("no idea"), cache, undefined, ALL_KEYS); assert.deepEqual(unparseable, { provider: "tavily", routedBy: "fallback" }); const throwing: RouterContext = { model: {}, modelRegistry: { complete: async () => { throw new Error("boom"); } }, }; - const failed = await routeSearch({ query: "x" }, throwing, cache, undefined, ENV_ALL_KEYS); + const failed = await routeSearch({ query: "x" }, throwing, cache, undefined, ALL_KEYS); assert.deepEqual(failed, { provider: "tavily", routedBy: "fallback" }); }); test("routeSearch without a tavily key excludes it from candidates", async () => { const cache = new Map(); - const decision = await routeSearch({ query: "x" }, ctxAnswering("tavily"), cache, undefined, {}); + const decision = await routeSearch({ query: "x" }, ctxAnswering("tavily"), cache, undefined, NO_KEYS); // "tavily" is not a candidate → unroutable → first available fallback. assert.deepEqual(decision, { provider: "exa", routedBy: "fallback" }); }); test("routeSearch works without a model", async () => { const cache = new Map(); - const decision = await routeSearch({ query: "x" }, {}, cache, undefined, ENV_ALL_KEYS); + const decision = await routeSearch({ query: "x" }, {}, cache, undefined, ALL_KEYS); assert.deepEqual(decision, { provider: "tavily", routedBy: "fallback" }); }); test("route cache evicts the oldest entry at the limit", async () => { const cache = new Map(); for (let i = 0; i < ROUTE_CACHE_LIMIT; i++) cache.set(`dummy-${i}`, "exa"); - const decision = await routeSearch({ query: "fresh query" }, ctxAnswering("keenable"), cache, undefined, ENV_ALL_KEYS); + const decision = await routeSearch({ query: "fresh query" }, ctxAnswering("keenable"), cache, undefined, ALL_KEYS); assert.equal(decision.provider, "keenable"); assert.equal(cache.size, ROUTE_CACHE_LIMIT); assert.equal(cache.has("dummy-0"), false); diff --git a/scripts/search-auth.mjs b/scripts/search-auth.mjs new file mode 100755 index 0000000..244df97 --- /dev/null +++ b/scripts/search-auth.mjs @@ -0,0 +1,262 @@ +#!/usr/bin/env node +/** + * Read-modify-write the tavily/exa/keenable api_key credentials in Pi's + * auth.json. Preserves every other provider entry (OAuth tokens included), + * writes atomically with a lock directory, and keeps the file mode 600. + * + * Usage: + * node search-auth.mjs --set tavily=KEY [--set exa=KEY] [--set keenable=KEY] + * node search-auth.mjs --unset tavily [--unset exa] + * node search-auth.mjs --migrate [search.env path] + * node search-auth.mjs --list + * node search-auth.mjs --has tavily (exit 0 when set, 1 otherwise) + * + * --auth FILE overrides the auth.json location (default: + * $PI_CODING_AGENT_DIR/auth.json or ~/.pi/agent/auth.json). + */ + +import { + chmodSync, + closeSync, + existsSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { homedir } from "node:os"; +import { dirname, join } from "node:path"; +import { randomBytes } from "node:crypto"; + +const PROVIDERS = ["tavily", "exa", "keenable"]; +const ENV_TO_PROVIDER = { + TAVILY_API_KEY: "tavily", + EXA_API_KEY: "exa", + KEENABLE_API_KEY: "keenable", +}; +const SAFE_VALUE = /^[A-Za-z0-9._-]+$/; +const LOCK_RETRIES = 50; +const LOCK_DELAY_MS = 100; + +function die(message) { + console.error(`错误: ${message}`); + process.exit(1); +} + +function usage() { + console.log(`用法: + node search-auth.mjs --set = [--set ...] + node search-auth.mjs --unset [--unset ...] + node search-auth.mjs --migrate [search.env 路径] + node search-auth.mjs --list + node search-auth.mjs --has + +选项: + --auth FILE 覆盖 auth.json 路径(默认 $PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json)`); +} + +function defaultAuthPath(env = process.env) { + const agentDir = env.PI_CODING_AGENT_DIR?.trim() || join(env.HOME?.trim() || homedir(), ".pi", "agent"); + return join(agentDir, "auth.json"); +} + +function defaultSearchEnvPath(env = process.env) { + if (env.MY_PI_SEARCH_CONFIG?.trim()) return env.MY_PI_SEARCH_CONFIG.trim(); + const configHome = env.XDG_CONFIG_HOME?.trim() || join(env.HOME?.trim() || homedir(), ".config"); + return join(configHome, "my-pi", "search.env"); +} + +function parseArgs(argv) { + const result = { sets: {}, unsets: new Set(), migrate: undefined, list: false, has: undefined, authPath: undefined }; + for (let i = 0; i < argv.length; i++) { + const arg = argv[i]; + if (arg === "--set") { + const spec = argv[++i]; + const separator = spec ? spec.indexOf("=") : -1; + if (separator < 1) die("--set 需要 ="); + const provider = spec.slice(0, separator); + const value = spec.slice(separator + 1); + if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`); + result.sets[provider] = value; + } else if (arg === "--unset") { + const provider = argv[++i]; + if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`); + result.unsets.add(provider); + } else if (arg === "--migrate") { + const next = argv[i + 1]; + result.migrate = next && !next.startsWith("--") ? (i++, next) : true; + } else if (arg === "--list") { + result.list = true; + } else if (arg === "--has") { + const provider = argv[++i]; + if (!PROVIDERS.includes(provider)) die(`未知 provider:${provider}(支持 ${PROVIDERS.join("/")})`); + result.has = provider; + } else if (arg === "--auth") { + result.authPath = argv[++i]; + if (!result.authPath) die("--auth 缺少 FILE"); + } else if (arg === "-h" || arg === "--help") { + usage(); + process.exit(0); + } else { + die(`未知参数:${arg}`); + } + } + return result; +} + +function acquireLock(authPath) { + mkdirSync(dirname(authPath), { recursive: true }); + const lockDir = `${authPath}.lock`; + for (let attempt = 0; attempt < LOCK_RETRIES; attempt++) { + try { + mkdirSync(lockDir); + return () => { + try { + rmSync(lockDir, { recursive: true }); + } catch {} + }; + } catch (error) { + if (error.code !== "EEXIST") throw error; + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, LOCK_DELAY_MS); + } + } + die(`无法获取 ${lockDir} 锁;可能有其他进程正在写 auth.json(残留锁目录可手动删除)。`); +} + +function readAuthData(authPath) { + let raw; + try { + raw = readFileSync(authPath, "utf8"); + } catch (error) { + if (error.code === "ENOENT") return {}; + throw error; + } + let data; + try { + data = JSON.parse(raw); + } catch { + die(`auth.json 不是合法 JSON,已中止以免破坏现有凭据:${authPath}`); + } + if (!data || typeof data !== "object" || Array.isArray(data)) { + die(`auth.json 顶层必须是对象,已中止以免破坏现有凭据:${authPath}`); + } + return data; +} + +function writeAuthData(authPath, data) { + mkdirSync(dirname(authPath), { recursive: true }); + const temp = join(dirname(authPath), `.auth.json.${process.pid}.${randomBytes(6).toString("hex")}.tmp`); + try { + const fd = openSync(temp, "w", 0o600); + writeFileSync(fd, `${JSON.stringify(data, null, 2)}\n`); + closeSync(fd); + renameSync(temp, authPath); + chmodSync(authPath, 0o600); + } finally { + rmSync(temp, { force: true }); + } +} + +function hasKey(data, provider) { + const entry = data[provider]; + return Boolean(entry && typeof entry === "object" && !Array.isArray(entry) && entry.type === "api_key" && entry.key); +} + +function setKey(data, provider, key) { + const existing = data[provider]; + const preserved = + existing && typeof existing === "object" && !Array.isArray(existing) && existing.type === "api_key" + ? existing + : {}; + data[provider] = { ...preserved, type: "api_key", key }; +} + +function parseSearchEnv(text) { + const parsed = {}; + for (const rawLine of text.split(/\r?\n/u)) { + const line = rawLine.trim().replace(/^export[ \t]+/u, ""); + if (!line || line.startsWith("#")) continue; + const separator = line.indexOf("="); + if (separator < 1) continue; + const name = line.slice(0, separator).trim(); + const value = line.slice(separator + 1).trim(); + const provider = ENV_TO_PROVIDER[name]; + if (!provider || !SAFE_VALUE.test(value)) continue; + parsed[provider] = value; + } + return parsed; +} + +function validateKey(provider, key) { + if (!key || /[\r\n]/u.test(key)) die(`${provider} 的 key 不能为空且不能包含换行。`); +} + +const args = parseArgs(process.argv.slice(2)); +const authPath = args.authPath ?? defaultAuthPath(); + +if (args.list) { + const data = readAuthData(authPath); + for (const provider of PROVIDERS) { + console.log(`${provider}: ${hasKey(data, provider) ? "已配置" : "未配置"}`); + } + process.exit(0); +} + +if (args.has) { + process.exit(hasKey(readAuthData(authPath), args.has) ? 0 : 1); +} + +const mutations = Object.keys(args.sets).length + args.unsets.size + (args.migrate ? 1 : 0); +if (mutations === 0) { + usage(); + process.exit(2); +} + +const release = acquireLock(authPath); +try { + const data = readAuthData(authPath); + const changed = []; + + for (const provider of args.unsets) { + if (provider in data) { + delete data[provider]; + changed.push(`已移除 ${provider}`); + } + } + for (const [provider, key] of Object.entries(args.sets)) { + validateKey(provider, key); + setKey(data, provider, key); + changed.push(`已写入 ${provider}`); + } + if (args.migrate) { + const source = args.migrate === true ? defaultSearchEnvPath() : args.migrate; + if (!existsSync(source)) die(`迁移来源不存在:${source}`); + const parsed = parseSearchEnv(readFileSync(source, "utf8")); + const imported = []; + const skipped = []; + for (const provider of PROVIDERS) { + if (!parsed[provider]) continue; + if (provider in data) { + skipped.push(provider); + continue; + } + setKey(data, provider, parsed[provider]); + imported.push(provider); + } + if (imported.length) changed.push(`已迁移 ${imported.join(", ")}`); + if (skipped.length) changed.push(`已跳过(auth.json 中已存在):${skipped.join(", ")}`); + if (!imported.length && !skipped.length) changed.push("search.env 中没有可迁移的 key"); + } + + if (changed.length === 0) { + console.log("auth.json 无需修改。"); + } else { + writeAuthData(authPath, data); + for (const line of changed) console.log(line); + console.log(`搜索凭据已写入 ${authPath}(权限 600),重启 Pi 后生效。`); + } +} finally { + release(); +} diff --git a/search_config.sh b/search_config.sh index 3124c4f..ef9906d 100755 --- a/search_config.sh +++ b/search_config.sh @@ -15,71 +15,54 @@ usage() { 用法: ./search_config.sh ./search_config.sh --tavily KEY --exa KEY --keenable KEY + ./search_config.sh --migrate [search.env 路径] + ./search_config.sh --list 选项: - --tavily KEY 配置 TAVILY_API_KEY - --exa KEY 配置 EXA_API_KEY - --keenable KEY 配置 KEENABLE_API_KEY - --config FILE 覆盖配置文件路径 + --tavily KEY 配置 tavily api_key 凭据 + --exa KEY 配置 exa api_key 凭据 + --keenable KEY 配置 keenable api_key 凭据 + --migrate 从旧 search.env 导入缺失的 key + --list 显示各 provider 配置状态(不输出 key) + --auth FILE 覆盖 auth.json 路径 -h, --help 显示帮助 不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互, -未指定的服务保留已有值。配置文件默认是 ~/.config/my-pi/search.env。 +未指定的 provider 保留已有值。凭据写入 Pi 的 auth.json +($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,权限 600)。 +key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR} 插值、 +以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。 EOF } -CONFIG_FILE=${MY_PI_SEARCH_CONFIG:-${XDG_CONFIG_HOME:-$HOME/.config}/my-pi/search.env} +SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd) +AUTH_MJS="$SCRIPT_DIR/scripts/search-auth.mjs" +[ -f "$AUTH_MJS" ] || { warn "找不到 ${AUTH_MJS}"; exit 1; } + +AUTH_FILE= mode=interactive -tavily_set=0 -exa_set=0 -keenable_set=0 -tavily_value= -exa_value= -keenable_value= +did_work=0 -while [ "$#" -gt 0 ]; do - case $1 in - --tavily) - [ "$#" -ge 2 ] || { warn "--tavily 缺少 KEY"; exit 2; } - tavily_value=$2; tavily_set=1; mode=noninteractive; shift 2 ;; - --exa) - [ "$#" -ge 2 ] || { warn "--exa 缺少 KEY"; exit 2; } - exa_value=$2; exa_set=1; mode=noninteractive; shift 2 ;; - --keenable) - [ "$#" -ge 2 ] || { warn "--keenable 缺少 KEY"; exit 2; } - keenable_value=$2; keenable_set=1; mode=noninteractive; shift 2 ;; - --config) - [ "$#" -ge 2 ] || { warn "--config 缺少 FILE"; exit 2; } - CONFIG_FILE=$2; shift 2 ;; - -h|--help) usage; exit 0 ;; - *) warn "未知参数:$1"; usage >&2; exit 2 ;; - esac -done - -valid_key() { - case $1 in - ''|*[!A-Za-z0-9._-]*) return 1 ;; - *) return 0 ;; - esac +run_auth() { + if [ -n "$AUTH_FILE" ]; then + node "$AUTH_MJS" --auth "$AUTH_FILE" "$@" + else + node "$AUTH_MJS" "$@" + fi } -read_existing() { - existing_name=$1 - [ -f "$CONFIG_FILE" ] || return 0 - awk -v name="$existing_name" ' - $1 == "export" { sub(/^export[ \t]+/, "") } - index($0, name "=") == 1 { print substr($0, length(name) + 2); exit } - ' "$CONFIG_FILE" +has_key() { + run_auth --has "$1" >/dev/null 2>&1 } prompt_key() { prompt_label=$1 - prompt_current=$2 + prompt_configured=$2 if [ ! -t 0 ]; then warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。" return 2 fi - if [ -n "$prompt_current" ]; then + if [ "$prompt_configured" -eq 1 ]; then printf '%s API key(回车保留现有值): ' "$prompt_label" >&2 else printf '%s API key(回车跳过): ' "$prompt_label" >&2 @@ -92,45 +75,45 @@ prompt_key() { printf '%s\n' "$prompt_value" } -current_tavily=$(read_existing TAVILY_API_KEY) -current_exa=$(read_existing EXA_API_KEY) -current_keenable=$(read_existing KEENABLE_API_KEY) - -if [ "$mode" = interactive ]; then - tavily_value=$(prompt_key Tavily "$current_tavily") || exit $? - exa_value=$(prompt_key Exa "$current_exa") || exit $? - keenable_value=$(prompt_key Keenable "$current_keenable") || exit $? - [ -n "$tavily_value" ] && tavily_set=1 - [ -n "$exa_value" ] && exa_set=1 - [ -n "$keenable_value" ] && keenable_set=1 -fi - -[ "$tavily_set" -eq 1 ] || tavily_value=$current_tavily -[ "$exa_set" -eq 1 ] || exa_value=$current_exa -[ "$keenable_set" -eq 1 ] || keenable_value=$current_keenable - -for key_value in "$tavily_value" "$exa_value" "$keenable_value"; do - [ -z "$key_value" ] || valid_key "$key_value" || { - warn "API key 只能包含字母、数字、点、下划线和连字符。" - exit 2 - } +while [ "$#" -gt 0 ]; do + case $1 in + --tavily|--exa|--keenable) + [ "$#" -ge 2 ] || { warn "$1 缺少 KEY"; exit 2; } + mode=noninteractive; did_work=1 + run_auth --set "${1#--}=$2" || exit $? + shift 2 ;; + --migrate) + mode=noninteractive; did_work=1 + if [ "$#" -ge 2 ] && [ "${2#--}" = "$2" ]; then + run_auth --migrate "$2" || exit $? + shift 2 + else + run_auth --migrate || exit $? + shift + fi + say "如确认 auth.json 可用,可手动删除旧 search.env 文件。" ;; + --list) mode=noninteractive; did_work=1; run_auth --list || exit $?; shift ;; + --auth) + [ "$#" -ge 2 ] || { warn "--auth 缺少 FILE"; exit 2; } + AUTH_FILE=$2; shift 2 ;; + -h|--help) usage; exit 0 ;; + *) warn "未知参数:$1"; usage >&2; exit 2 ;; + esac done -config_dir=${CONFIG_FILE%/*} -[ "$config_dir" = "$CONFIG_FILE" ] && config_dir=. -umask 077 -mkdir -p "$config_dir" || exit 1 -config_tmp=$(mktemp "$config_dir/.search.env.XXXXXX") || exit 1 -trap 'rm -f "$config_tmp"' EXIT HUP INT TERM -{ - printf '# Generated by my-pi/search_config.sh. Do not commit this file.\n' - [ -z "$tavily_value" ] || printf 'export TAVILY_API_KEY=%s\n' "$tavily_value" - [ -z "$exa_value" ] || printf 'export EXA_API_KEY=%s\n' "$exa_value" - [ -z "$keenable_value" ] || printf 'export KEENABLE_API_KEY=%s\n' "$keenable_value" -} > "$config_tmp" || exit 1 -chmod 600 "$config_tmp" || exit 1 -mv "$config_tmp" "$CONFIG_FILE" || exit 1 -trap - EXIT HUP INT TERM - -say "搜索 API key 已安全写入 ${CONFIG_FILE}(权限 600)。" -say "重启 Pi 后生效;显式环境变量仍优先于该文件。" +if [ "$mode" = interactive ]; then + for provider in tavily exa keenable; do + case $provider in + tavily) label=Tavily ;; + exa) label=Exa ;; + keenable) label=Keenable ;; + esac + if has_key "$provider"; then configured=1; else configured=0; fi + value=$(prompt_key "$label" "$configured") || exit $? + if [ -n "$value" ]; then + run_auth --set "$provider=$value" || exit $? + did_work=1 + fi + done + [ "$did_work" -eq 1 ] || say "未提供任何 key,auth.json 保持不变。" +fi diff --git a/tests/extension-load.test.ts b/tests/extension-load.test.ts index 76cb3ee..b86d7f1 100644 --- a/tests/extension-load.test.ts +++ b/tests/extension-load.test.ts @@ -201,12 +201,8 @@ test("all package extensions load together without global registration conflicts HOME: home, XDG_CONFIG_HOME: join(home, ".config"), PI_CODING_AGENT_DIR: join(home, ".pi-agent"), - MY_PI_SEARCH_CONFIG: join(home, "missing-search.env"), PATH: `${mockBin}:${process.env.PATH ?? ""}`, PI_OFFLINE: "1", - TAVILY_API_KEY: "", - EXA_API_KEY: "", - KEENABLE_API_KEY: "", }; const runBundle = () => spawnSync("pi", args, { encoding: "utf8", diff --git a/tests/search-auth.test.ts b/tests/search-auth.test.ts new file mode 100644 index 0000000..0893fc6 --- /dev/null +++ b/tests/search-auth.test.ts @@ -0,0 +1,252 @@ +import assert from "node:assert/strict"; +import { mkdtemp, readFile, stat, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { spawnSync } from "node:child_process"; +import test from "node:test"; + +import { + getSearchApiKey, + getSearchAuthPath, + getSearchCredentialEnv, + parseAuthKeyTemplate, + resolveAuthKeyValue, +} from "../pi-web-search/auth.ts"; + +async function withAuthFile( + data: Record, + fn: (authPath: string) => Promise | void, +): Promise { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-")); + const authPath = join(dir, "auth.json"); + await writeFile(authPath, JSON.stringify(data)); + await fn(authPath); +} + +test("auth path follows PI_CODING_AGENT_DIR and falls back to ~/.pi/agent", () => { + assert.equal( + getSearchAuthPath({ PI_CODING_AGENT_DIR: "/tmp/agent" }), + "/tmp/agent/auth.json", + ); + assert.equal(getSearchAuthPath({ HOME: "/tmp/home" }), "/tmp/home/.pi/agent/auth.json"); +}); + +test("literal api_key credentials resolve verbatim", async () => { + await withAuthFile( + { + tavily: { type: "api_key", key: "tvly-test" }, + exa: { type: "api_key", key: " exa-test " }, + keenable: { type: "api_key", key: "" }, + anthropic: { type: "oauth", access: "a", refresh: "r", expires: 1 }, + }, + (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-test"); + assert.equal(getSearchApiKey("exa", { authPath }), "exa-test"); + assert.equal(getSearchApiKey("keenable", { authPath }), undefined); + }, + ); +}); + +test("missing file, malformed json, non-api_key entries all resolve to undefined", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-")); + assert.equal(getSearchApiKey("tavily", { authPath: join(dir, "absent.json") }), undefined); + const bad = join(dir, "bad.json"); + await writeFile(bad, "not json{"); + assert.equal(getSearchApiKey("tavily", { authPath: bad }), undefined); + await withAuthFile({ tavily: { type: "oauth", access: "a", refresh: "r", expires: 1 } }, (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath }), undefined); + }); + await withAuthFile({ tavily: "literal-string-entry" }, (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath }), undefined); + }); +}); + +test("$VAR and ${VAR} interpolate; credential env beats ambient env", async () => { + await withAuthFile( + { + tavily: { type: "api_key", key: "$TAVILY_KEY" }, + exa: { type: "api_key", key: "prefix-${EXA_KEY}-suffix" }, + keenable: { type: "api_key", key: "$SHARED", env: { SHARED: "scoped-wins" } }, + }, + (authPath) => { + const env = { TAVILY_KEY: "from-env", EXA_KEY: "braced", SHARED: "ambient-loses" }; + assert.equal(getSearchApiKey("tavily", { authPath, env }), "from-env"); + assert.equal(getSearchApiKey("exa", { authPath, env }), "prefix-braced-suffix"); + assert.equal(getSearchApiKey("keenable", { authPath, env }), "scoped-wins"); + }, + ); +}); + +test("a missing env variable resolves the whole key to undefined", async () => { + await withAuthFile({ tavily: { type: "api_key", key: "$NOPE_MISSING" } }, (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), undefined); + }); +}); + +test("$$ and $! escape to literal characters", async () => { + await withAuthFile( + { tavily: { type: "api_key", key: "cost-is-$$5-and-$!literal" } }, + (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath, env: {} }), "cost-is-$5-and-!literal"); + }, + ); +}); + +test("!command executes and caches trimmed stdout", async () => { + const authPath = join(await mkdtemp(join(tmpdir(), "my-pi-auth-")), "auth.json"); + await writeFile( + authPath, + JSON.stringify({ tavily: { type: "api_key", key: "!printf secret-from-cmd" } }), + ); + assert.equal(getSearchApiKey("tavily", { authPath }), "secret-from-cmd"); + + let calls = 0; + const runCommand = (command: string) => { + calls += 1; + assert.equal(command, "get-key --flag"); + return "injected"; + }; + await withAuthFile({ exa: { type: "api_key", key: "!get-key --flag" } }, (otherPath) => { + assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected"); + assert.equal(getSearchApiKey("exa", { authPath: otherPath, runCommand }), "injected"); + }); +}); + +test("a failing !command resolves to undefined", async () => { + await withAuthFile({ tavily: { type: "api_key", key: "!exit 3" } }, (authPath) => { + assert.equal(getSearchApiKey("tavily", { authPath }), undefined); + }); +}); + +test("credential env map exposes non-key config like KEENABLE_API_URL", async () => { + await withAuthFile( + { keenable: { type: "api_key", key: "k", env: { KEENABLE_API_URL: "https://keenable.example" } } }, + (authPath) => { + assert.equal( + getSearchCredentialEnv("keenable", "KEENABLE_API_URL", { authPath }), + "https://keenable.example", + ); + assert.equal(getSearchCredentialEnv("keenable", "MISSING", { authPath }), undefined); + }, + ); +}); + +test("template parser handles env names, braces and escapes", () => { + assert.deepEqual(parseAuthKeyTemplate("$A_${B}-$$-$!-x"), [ + { type: "env", name: "A_" }, + { type: "env", name: "B" }, + { type: "literal", value: "-$-!-x" }, + ]); +}); + +const AUTH_MJS = new URL("../scripts/search-auth.mjs", import.meta.url).pathname; +const SEARCH_SH = new URL("../search_config.sh", import.meta.url).pathname; + +function runMjs(args: string[], env: Record = {}) { + return spawnSync("node", [AUTH_MJS, ...args], { encoding: "utf8", env: { ...process.env, ...env } }); +} + +test("search-auth.mjs --set writes api_key entries and preserves other providers", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile( + authPath, + JSON.stringify({ anthropic: { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" } }), + ); + const result = runMjs(["--auth", authPath, "--set", "tavily=tvly-new", "--set", "exa=!op read x"]); + assert.equal(result.status, 0, result.stderr); + const written = JSON.parse(await readFile(authPath, "utf8")) as Record; + assert.deepEqual(written.anthropic, { type: "oauth", access: "tok", refresh: "ref", expires: 42, accountId: "acc" }); + assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-new" }); + assert.deepEqual(written.exa, { type: "api_key", key: "!op read x" }); + assert.equal((await stat(authPath)).mode & 0o777, 0o600); + assert.equal(getSearchApiKey("tavily", { authPath }), "tvly-new"); +}); + +test("search-auth.mjs preserves an existing credential env map on --set", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile( + authPath, + JSON.stringify({ keenable: { type: "api_key", key: "old", env: { KEENABLE_API_URL: "https://k.example" } } }), + ); + const result = runMjs(["--auth", authPath, "--set", "keenable=new-key"]); + assert.equal(result.status, 0, result.stderr); + const written = JSON.parse(await readFile(authPath, "utf8")) as Record; + assert.deepEqual(written.keenable, { + type: "api_key", + key: "new-key", + env: { KEENABLE_API_URL: "https://k.example" }, + }); +}); + +test("search-auth.mjs creates a missing auth.json and honors PI_CODING_AGENT_DIR", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const agentDir = join(dir, "agent"); + const result = runMjs(["--set", "keenable=keen-1"], { PI_CODING_AGENT_DIR: agentDir }); + assert.equal(result.status, 0, result.stderr); + const written = JSON.parse(await readFile(join(agentDir, "auth.json"), "utf8")) as Record; + assert.deepEqual(written.keenable, { type: "api_key", key: "keen-1" }); +}); + +test("search-auth.mjs --unset removes only the named provider", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "t" }, exa: { type: "api_key", key: "e" } })); + const result = runMjs(["--auth", authPath, "--unset", "tavily"]); + assert.equal(result.status, 0, result.stderr); + const written = JSON.parse(await readFile(authPath, "utf8")) as Record; + assert.equal(written.tavily, undefined); + assert.deepEqual(written.exa, { type: "api_key", key: "e" }); +}); + +test("search-auth.mjs refuses to clobber a malformed auth.json", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile(authPath, "{oops"); + const result = runMjs(["--auth", authPath, "--set", "tavily=x"]); + assert.notEqual(result.status, 0); + assert.match(result.stderr, /不是合法 JSON|abort|clobber/i); + assert.equal(await readFile(authPath, "utf8"), "{oops"); +}); + +test("search-auth.mjs --migrate imports search.env keys without overwriting existing entries", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "existing" } })); + const envPath = join(dir, "search.env"); + await writeFile(envPath, "export TAVILY_API_KEY=old\nEXA_API_KEY=exa-file\nKEENABLE_API_KEY=bad$(cmd)\n"); + const result = runMjs(["--auth", authPath, "--migrate", envPath]); + assert.equal(result.status, 0, result.stderr); + assert.match(result.stdout, /已迁移 exa/); + assert.match(result.stdout, /tavily/); + const written = JSON.parse(await readFile(authPath, "utf8")) as Record; + assert.equal(written.tavily.key, "existing"); + assert.equal(written.exa.key, "exa-file"); + assert.equal(written.keenable, undefined); +}); + +test("search-auth.mjs --has and --list never print key material", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + await writeFile(authPath, JSON.stringify({ tavily: { type: "api_key", key: "supersecret" } })); + assert.equal(runMjs(["--auth", authPath, "--has", "tavily"]).status, 0); + assert.equal(runMjs(["--auth", authPath, "--has", "exa"]).status, 1); + const list = runMjs(["--auth", authPath, "--list"]); + assert.equal(list.status, 0); + assert.match(list.stdout, /tavily: 已配置/); + assert.doesNotMatch(list.stdout, /supersecret/); +}); + +test("search_config.sh writes keys through search-auth.mjs noninteractively", async () => { + const dir = await mkdtemp(join(tmpdir(), "my-pi-auth-write-")); + const authPath = join(dir, "auth.json"); + const result = spawnSync("sh", [SEARCH_SH, "--auth", authPath, "--tavily", "tvly-sh", "--exa", "exa-sh"], { + encoding: "utf8", + }); + assert.equal(result.status, 0, result.stderr); + const written = JSON.parse(await readFile(authPath, "utf8")) as Record; + assert.deepEqual(written.tavily, { type: "api_key", key: "tvly-sh" }); + assert.deepEqual(written.exa, { type: "api_key", key: "exa-sh" }); + assert.equal((await stat(authPath)).mode & 0o777, 0o600); +}); diff --git a/tests/search-config.test.ts b/tests/search-config.test.ts deleted file mode 100644 index f65adb9..0000000 --- a/tests/search-config.test.ts +++ /dev/null @@ -1,67 +0,0 @@ -import assert from "node:assert/strict"; -import { mkdtemp, readFile, stat } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { spawnSync } from "node:child_process"; -import test from "node:test"; - -import { - getSearchConfigPath, - loadSearchConfigEnv, - parseSearchConfig, -} from "../extensions/search-config.ts"; - -test("search config parses only supported safe assignments", () => { - assert.deepEqual( - parseSearchConfig(` -export TAVILY_API_KEY=tvly-test_1 -EXA_API_KEY=exa-test.2 -UNKNOWN_KEY=ignored -KEENABLE_API_KEY=$(not-allowed) -`), - { - TAVILY_API_KEY: "tvly-test_1", - EXA_API_KEY: "exa-test.2", - }, - ); -}); - -test("search config does not override explicit environment values", async () => { - const root = await mkdtemp(join(tmpdir(), "my-pi-search-config-")); - const configPath = join(root, "search.env"); - await import("node:fs/promises").then(({ writeFile }) => - writeFile(configPath, "export TAVILY_API_KEY=file-value\nexport EXA_API_KEY=exa-file\n"), - ); - const env: Record = { TAVILY_API_KEY: "explicit-value" }; - const effective = loadSearchConfigEnv(env, configPath); - assert.equal(env.TAVILY_API_KEY, "explicit-value"); - assert.equal(env.EXA_API_KEY, "exa-file"); - assert.deepEqual(effective, { TAVILY_API_KEY: "explicit-value", EXA_API_KEY: "exa-file" }); -}); - -test("search config path follows XDG_CONFIG_HOME", () => { - assert.equal(getSearchConfigPath({ XDG_CONFIG_HOME: "/tmp/config" }), "/tmp/config/my-pi/search.env"); -}); - -test("search_config.sh supports noninteractive updates and preserves omitted keys", async () => { - const home = await mkdtemp(join(tmpdir(), "my-pi-search-script-")); - const script = new URL("../search_config.sh", import.meta.url); - let result = spawnSync("sh", [script.pathname, "--tavily", "tvly-test", "--exa", "exa-test"], { - env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") }, - encoding: "utf8", - }); - assert.equal(result.status, 0, result.stderr); - - result = spawnSync("sh", [script.pathname, "--keenable", "keen-test"], { - env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, ".config") }, - encoding: "utf8", - }); - assert.equal(result.status, 0, result.stderr); - - const configPath = join(home, ".config", "my-pi", "search.env"); - const content = await readFile(configPath, "utf8"); - assert.match(content, /^export TAVILY_API_KEY=tvly-test$/m); - assert.match(content, /^export EXA_API_KEY=exa-test$/m); - assert.match(content, /^export KEENABLE_API_KEY=keen-test$/m); - assert.equal((await stat(configPath)).mode & 0o777, 0o600); -}); diff --git a/tests/web-search.test.ts b/tests/web-search.test.ts index 34bb6b0..ec67eee 100644 --- a/tests/web-search.test.ts +++ b/tests/web-search.test.ts @@ -8,12 +8,13 @@ const packageJson = JSON.parse(await readFile(new URL("../package.json", import. pi: { extensions: string[]; skills: string[] }; }; -test("pi-web-search is the single search surface, loading after search-config", () => { +test("pi-web-search is the single search surface and reads credentials from auth.json", () => { const entries = packageJson.pi.extensions; assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search"); - assert.ok( - entries.indexOf("./extensions/search-config.ts") < entries.indexOf("./pi-web-search/index.ts"), - "search credentials must load before pi-web-search", + assert.equal( + entries.includes("./extensions/search-config.ts"), + false, + "search keys come from auth.json — the env loader extension is retired", ); for (const retired of [ "./extensions/mcp.ts",