Compare commits

...
5 Commits
Author SHA1 Message Date
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 56cbc83296 refactor: 移除 context-mode,Chrome 快照改为直接返回
context-mode 的 ctx_execute 系工具以真实 shell 执行且权限为 allow,绕过了
bash 策略与 protectedWrites;其每轮注入、独立 MCP 子进程与 skills 也与组合包
提示词单一所有权冲突。大输出职责已由 pi-overflow(入口截断)与
pi-compaction(压力遮蔽)承担:
- 删除 context-mode 依赖、wrapper/过滤器、skills,以及 ctx_* 的权限、
  tools.jsonc、路由规则、tool-search 常驻与 4 个分组
- better-sqlite3@12 仅为 context-mode 传递依赖,随之移除根 allowScripts
- pi-chrome:按用户决定移除快照隐私规则与 .pi/chrome-context 临时文件机制,
  快照 JSON 直接作为工具结果返回、details 只留小型元数据,超长由 pi-overflow
  截断并用 overflow_query 提取;授权生命周期不变

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:45:33 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 2a6bf8c929 feat(pi-overflow): 新增统一的新工具结果展示截断扩展
此前截断分散在多层且互相冲突:Pi bash 保留尾部 50KB,RTK 兜底截断又只留
开头 12000 字符(砍掉报错与完整输出路径),pi-chrome 截开头 30K,web_fetch
各 provider 不一致。新扩展统一负责"新结果给模型看多少":
- capture.ts 最先加载暂存原文,index.ts 最后加载(system-prompt 之前)
- 超过 24K 字符时原文落盘 <agentDir>/my-pi/pi-overflow/(700/600、原子写),
  优先取 bash fullOutputPath 完整文件;模型看到 头 + 中段信号行 + 尾 的有界
  视图,行号基于落盘原文、结尾永远可见
- overflow_query:按 id 正则搜索、行区间、jsonPath 只读查询,id 校验防穿越
- 收编:RTK truncate 默认关闭、删除 pi-chrome truncateText、web_fetch 默认
  每页 20000(上限 100000)、web_search max_results 限定 1–20
- pi-compaction 遮蔽带 details.overflow.id 的结果时在占位符提示 overflow_query
- 登记 tools.jsonc / routing / 权限 allow / tool-search 常驻与分组

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:25:44 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> a6d8416c76 refactor(pi-compaction): 改为压力触发的观察遮蔽与空闲阈值整段摘要
原实现对所有工具结果(含 read)在下一次请求即替换为约 4K 的首尾存根,
不看上下文压力,导致模型看不到大文件中段;后台逐条语义摘要又反复改写
历史、打碎 prompt 缓存。参照 Anthropic 上下文编辑、Manus 与 JetBrains
观察遮蔽研究重构为:
- L2:上下文达到阈值时在 turn_end 批量提交 Pi 原生 context_edit,把最近
  keepRecentTurns 轮之外的旧 toolResult 换成确定性占位符;trigger/target
  回差与 minClearTokens 保证少触发、一次清够;原始条目保留,可经
  context_tree_query 取回
- L3:空闲 agent_settled 且超过绝对阈值时调用一次 ctx.compact
- 阈值按绝对 token 计并受窗口比例约束,默认针对 1M 窗口(DeepSeek V4.1
  Flash):遮蔽 200K→100K、整段摘要 400K
- 删除逐条存根、语义 worker、soft/hard/target 比例与自行取消 Pi 压缩的门控;
  Pi 原生阈值压缩保留为最终安全网

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:51:36 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> a254d4fd68 fix: 根 files 改为子包运行时白名单,并删除过期提示词快照
整目录列入子包时,pi-subagents 上游 .npmignore 覆盖 .gitignore,导致
嵌套 node_modules(含第二套 Pi runtime)与 dist 被打包:64.9MB/19,570
文件降至 1.2MB/471 文件。新增 tests/package-files.test.ts 守护:无嵌套
node_modules/dist、扩展入口与 file: 依赖清单在包内、包内源码相对引用闭包。
同时删除无引用的 pi-default-system-prompt.md 旧快照(可用 /dump-system-prompt)。

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:16:47 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> e996f59469 refactor: notify/footer 持久配置收编到 config/,chrome bridge 地址固定
仓库规则要求 config/ 为唯一配置来源、不以环境变量作持久默认值:
- pi-notify 改读 config/pi-notify.json(逐字段校验回退,复用原 parsers 钳制)
- pi-minimal-footer 改读 config/pi-minimal-footer.json
- pi-chrome 移除 PI_CHROME_BRIDGE_HOST/PORT,固定 127.0.0.1:17318
  (浏览器端 service_worker.js 硬编码同一地址,单侧可配只会断连)

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:16:47 -07:00
93 changed files with 3776 additions and 3144 deletions
+4 -4
View File
@@ -18,10 +18,10 @@
| `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` | | `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` |
| `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 | | `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 |
| `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 | | `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 |
| `pi-compaction/` | 增量 Session segment、确定性压缩、后台语义摘要、effective token、native compaction | | `pi-compaction/` | 压力触发的观察遮蔽(Pi context_edit)、阈值整段摘要、effective token、native compaction |
| `pi-overflow/` | 新工具结果统一截断:超阈值原文落盘 + 有界视图 + `overflow_query` |
| `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep`/`multi_grep` 与 `@` 补全 | | `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep`/`multi_grep` 与 `@` 补全 |
| `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) | | `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) |
| `extensions/context-mode.ts` | 经 `prompt-guard.ts` 加载 context-mode adapter 与 skills(`context-mode-filter.ts` 过滤冲突锚点) |
| `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) | | `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) |
| `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` | | `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` |
| `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) | | `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) |
@@ -35,8 +35,8 @@
## 全局硬规则 ## 全局硬规则
- 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。 - 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。
- 加载顺序:`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。 - 加载顺序:`pi-overflow/capture.ts` 必须排在最前(先于 RTK 等结果改写扩展暂存原文),`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `pi-overflow/index.ts` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。
- 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前仅 `better-sqlite3@12.11.1`;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不新增放行)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。 - 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前无放行项;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不经根 `allowScripts`)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。根 `files` 对子包按运行时子路径白名单列出(源码、package.json、README/LICENSE/CHANGELOG/UPSTREAM),不得整目录列入,以免打入嵌套 node_modules、dist 和测试;由 `tests/package-files.test.ts` 守护。
- 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。 - 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。
- 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。 - 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。
- 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。 - 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。
+31 -27
View File
@@ -4,7 +4,6 @@
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。 - `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。 - 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
- `context-mode@1.0.169`:通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。
- 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。 - 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。 - `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。 - `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
@@ -12,11 +11,12 @@
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。 - `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
- 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。 - 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。
- 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。 - 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。
- 本仓库维护的 `pi-compaction`:以增量投影统一确定性工具结果压缩、后台语义摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。 - 本仓库维护的 `pi-compaction`:压力触发时以 Pi `context_edit` 批量遮蔽旧工具结果、空闲时按阈值触发整段摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。
- 本仓库自维护的 `pi-overflow`:统一的新工具结果展示截断——超阈值原文落盘、向模型展示「头 + 信号行 + 尾」的有界视图,并提供 `overflow_query` 按 id 搜索、行区间或 JSON 路径取回原文。
- 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。 - 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。
- 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。 - 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。
- 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。 - 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。
- 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;大体积页面快照写入 Session 临时文件并交给 Context Mode 隔离分析,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。 - 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;页面快照 JSON 直接作为工具结果返回,超长时由 pi-overflow 截断并用 `overflow_query` 查询,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。
- 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。 - 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。
- 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。 - 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。
- 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。 - 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。
@@ -141,7 +141,7 @@ CodeGraph 的更新或遥测设置。
在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。 在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。
日志、测试/构建输出、工作区内大文件的探索/分析/总结以及不可预测的大输出优先交给 Context Mode。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。 日志、测试/构建输出以及不可预测的大输出由 pi-overflow 截断展示,原文可用 `overflow_query` 按 id 检索。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。
使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。 使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。
@@ -149,7 +149,7 @@ CodeGraph 的更新或遥测设置。
`extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。 `extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。
其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP、Context Mode)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。 其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。
`config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。 `config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。
@@ -159,7 +159,7 @@ CodeGraph 的更新或遥测设置。
授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。 授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。
`chrome_snapshot`、`chrome_find` 和带 `includeSnapshot=true` 的交互不会再把完整页面结构直接塞进模型上下文:完整 JSON 只写入当前项目 `.pi/chrome-context/` 下按 Session/目标隔离的临时文件,工具结果仅返回相对路径和小型摘要。模型应立即用 `ctx_execute_file` 从该文件提取下一步所需的 uid、文本或状态,不能用 `read` / `cat` 重新读入全文。每个目标只保留最新快照;页面操作后旧快照失效,Session 结束、`/chrome revoke` 或授权过期时清理。 `chrome_snapshot` 和带 `includeSnapshot=true` 的交互把页面快照 JSON 直接作为工具结果返回,`details` 只保留元素数、匹配数等小型元数据;`chrome_find` 只返回紧凑的排序匹配。快照超长时结果会带 `[pi-overflow id=…]` 标记,模型用 `overflow_query` 的 `jsonPath`(如 `elements[0].uid`)或 `search` 提取所需的 uid、文本或状态,而不是重复抓取快照。页面交互后旧快照即失效,需要重新抓取;早期版本写入 Session 临时文件的快照机制已移除。
### 按需加载工具 schema ### 按需加载工具 schema
@@ -167,7 +167,7 @@ CodeGraph 的更新或遥测设置。
本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。 本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。
新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、`ctx_execute`、`ctx_execute_file`、`ctx_batch_execute` 和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Context Mode 执行/知识库/观测/管理、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。 新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、六个记忆工具、`context_tree_query`、`overflow_query` 和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。
标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。 标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。
@@ -180,16 +180,14 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
"alwaysEnabled": [ "alwaysEnabled": [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
"memory_replace", "memory_replace",
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query" "context_tree_query",
"overflow_query"
], ],
"showToolSearchFooterStatus": false, "showToolSearchFooterStatus": false,
"maxActiveGroups": 5, "maxActiveGroups": 5,
@@ -205,7 +203,7 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。 组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。
Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位、再用 Context Mode 隔离分析,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。 Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。
### 搜索归 FFF ### 搜索归 FFF
@@ -220,14 +218,12 @@ RTK 保留 Bash、read、build、test、lint 和 Git 输出压缩。默认 `read
仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有 仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有
`commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。 `commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。
### 大输出归 Context Mode ### 大输出归 pi-overflow
Context Mode 直接加载 npm 包内置的 Pi adapter 和 skills,不需要额外写入 新工具结果的展示截断统一由 `pi-overflow` 负责:超阈值的原文落盘、模型只看到
`~/.pi/agent/mcp.json`,也不需要全局安装 `context-mode` 命令。它提供 `ctx_*` 工具, 「头 + 信号行 + 尾」的有界视图,需要原文时用 `overflow_query` 检索(详见下文
将批量读取、命令研究和网页内容先在隔离进程中处理,再把收敛后的结果送入上下文。 「Pi Overflow」小节)。FFF 仍负责精确字面搜索;RTK 继续对 Bash、build、test、lint
和 Git 输出做语义压缩,其兜底硬截断已关闭。
FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash、build、test、lint
和 Git 输出。Context Mode 不替换这两个扩展的现有配置。
### SSH 服务器配置 ### SSH 服务器配置
@@ -366,23 +362,31 @@ compaction 和排队 follow-up 完成前不会提前通知;同时监听 `pi-pe
`permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的 `permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的
决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`, 决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`,
并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty 并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty
返回原始来源;仍可通过环境变量改用 `unfocused` 或 `invisible`。 返回原始来源;仍可通过 `visibility` 改用 `unfocused` 或 `invisible`。
默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制 默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制
当前 Session 的两类通知;持久默认通过 `PI_NOTIFY_ENABLED`、`PI_NOTIFY_MIN_SECONDS`、 当前 Session 的两类通知;持久默认通过组合包 `config/pi-notify.json` 的 `enabled`、`minSeconds`、
`PI_NOTIFY_VISIBILITY`、`PI_NOTIFY_ACTION`、`PI_NOTIFY_MESSAGE_SOURCE` 等环境变量配置。 `visibility`、`action`、`messageSource` 等字段配置。`messageSource: "none"`
`PI_NOTIFY_MESSAGE_SOURCE=none` 会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要 会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要
`set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见 `set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见
[`pi-notify/README.md`](pi-notify/README.md)。 [`pi-notify/README.md`](pi-notify/README.md)。
### Pi Compaction ### Pi Compaction
组合包只加载仓库内维护的 `pi-compaction/`。它以增量 Segment Index 管理已经进入 Session 的历史:已闭合工具结果先同步替换为有界的确定性证据 stub,大于阈值的结果再由单 Session 有界后台 worker 生成语义摘要;Provider 请求不会等待后台摘要。原始工具证据不重复写入 custom entry,可通过 `context_tree_query` 使用 source key 或 tool call ID 有界恢复。 组合包只加载仓库内维护的 `pi-compaction/`。它管理已经进入 Session 的历史投影:上下文压力达到阈值时,在 `turn_end` 批量提交 Pi 原生 `context_edit`,把最近 `keepRecentTurns` 轮之外的大型工具结果遮蔽为指向 `context_tree_query` 的单行占位符;遮蔽成批发生、受 trigger/target 回差与 `minClearTokens` 约束。原始工具结果保留在 Session 中(UI、导出不变),可通过 `context_tree_query` 按 tool call ID 有界恢复。
普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。Pi Core 的 raw threshold 只负责唤醒:只有 effective token 下界达到默认 90% 才执行 threshold compaction;manual 与 overflow 始终允许继续并在 native 失败时回退 Pi 默认 compaction。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。 普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。整段摘要只在空闲 `agent_settled` 且达到绝对 token 阈值(默认 400K,受窗口比例约束,272K 窗口时约为 90%)时经 `ctx.compact` 触发一次;Pi 自身的阈值压缩作为最终安全网始终有效,`session_before_compact` 不再按扩展自身阈值取消。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。
唯一配置位于已安装组合包的 `config/pi-compaction.json`,默认 soft/hard/target 比例为 72%/90%/55%,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和后台任务。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。 唯一配置位于已安装组合包的 `config/pi-compaction.json`:`masking` 段为遮蔽触发/目标阈值、`keepRecentTurns`、`minEntryTokens` 与 `excludeTools`,`summary` 段为整段摘要阈值与 thinking 档位,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和已遮蔽结果数。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。
原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。 原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。
本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 Context Mode 职责互补:Context Mode 防止大型原始内容进入会话,Pi Compaction 管理已经进入会话的长期历史。 本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 pi-overflow 职责互补:pi-overflow 在新工具结果进入会话时即截断展示,Pi Compaction 管理已经进入会话的长期历史。
### Pi Overflow
组合包自维护的 `pi-overflow/` 统一负责「每个新工具结果给模型看多少」。它有两个入口:`capture.ts` 排在 `pi.extensions` 最前,只把原始结果文本暂存到进程内 Map(容量 64),绝不修改结果;`index.ts` 排在 `system-prompt.ts` 之前、所有其他扩展之后,此时 RTK 等 handler 已经完成变换——它用最终可见文本判断是否超阈值,落盘的原文取 bash 的 `details.fullOutputPath` 文件、暂存原文或当前文本三者中的最完整版本,视图与其中的行号则一律基于落盘原文生成,保证与 `overflow_query` 读取的内容一致。
当最终文本仍超过 `thresholdChars`(默认 24000)时,模型看到的是确定性的「头部 40 行 + 中段匹配 error/warn 等信号行(至多 40 条)+ 尾部 120 行」视图,并写入 `details.overflow.id`;当上游已把结果压短但原文超阈值时,只追加一行 `[pi-overflow id=…]` 恢复标记。原文以 `sha1(sessionId+toolCallId)` 前 12 位 hex 为 id 存入 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),session 启动时按 `retentionDays`/`maxStoreBytes` 清理。模型可用 `overflow_query` 按 id 正则搜索(带上下文合并)、按行区间或 `jsonPath` 读取;`read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用永远豁免。配置为组合包内 `config/pi-overflow.json`,详见 [`pi-overflow/README.md`](pi-overflow/README.md)。
RTK 的兜底截断与 pi-chrome 的 30K 字符截断已收编进本扩展:`config/rtk-optimizer.json` 中 `outputCompaction.truncate.enabled` 默认关闭,chrome_* 工具返回完整文本;pi-web-search 的 `web_fetch` 默认 `max_characters` 提高到 20000(上限 100000),`web_search` 的 `max_results` 约束在 1–20。
+13 -13
View File
@@ -1,20 +1,20 @@
{ {
"enabled": true, "enabled": true,
"softRatio": 0.72, "masking": {
"hardRatio": 0.9,
"targetRatio": 0.55,
"deterministic": {
"enabled": true, "enabled": true,
"maxStubTokens": 1024 "triggerTokens": 200000,
"triggerRatio": 0.6,
"targetTokens": 100000,
"targetRatio": 0.35,
"minClearTokens": 50000,
"keepRecentTurns": 10,
"minEntryTokens": 256,
"excludeTools": ["ask_user_question"]
}, },
"semantic": { "summary": {
"enabled": true, "triggerTokens": 400000,
"model": "default", "triggerRatio": 0.9,
"thinking": "low", "thinking": "low"
"concurrency": 1,
"minInputChars": 4000,
"maxInputChars": 24000,
"maxRetries": 2
}, },
"native": { "native": {
"enabled": true "enabled": true
+4
View File
@@ -0,0 +1,4 @@
{
"showCwd": true,
"showBranch": true
}
+11
View File
@@ -0,0 +1,11 @@
{
"enabled": true,
"title": "π - {project}",
"body": "{message}",
"messageSource": "assistant",
"messageMax": 80,
"minSeconds": 3,
"visibility": "always",
"action": "focus",
"soundCommand": ""
}
+13
View File
@@ -0,0 +1,13 @@
{
"enabled": true,
"thresholdChars": 24000,
"headLines": 40,
"tailLines": 120,
"maxSignalLines": 40,
"signalPattern": "\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b",
"maxLineChars": 500,
"exemptTools": ["read", "edit", "write", "overflow_query", "ask_user_question"],
"retentionDays": 7,
"maxStoreBytes": 1073741824,
"maxEntryBytes": 67108864
}
+3 -8
View File
@@ -45,13 +45,6 @@
"lsp_references": "allow", "lsp_references": "allow",
"lsp_hover": "allow", "lsp_hover": "allow",
"lsp_symbols": "allow", "lsp_symbols": "allow",
"ctx_execute": "allow",
"ctx_execute_file": "allow",
"ctx_batch_execute": "allow",
"ctx_search": "allow",
"ctx_index": "allow",
"ctx_stats": "allow",
"ctx_doctor": "allow",
"tool_search": "allow", "tool_search": "allow",
"ask_user_question": "allow", "ask_user_question": "allow",
// Launching, steering or reading a subagent has no side effect itself; // Launching, steering or reading a subagent has no side effect itself;
@@ -86,10 +79,12 @@
// index and never touches the workspace. It is pinned in Tool Search's // index and never touches the workspace. It is pinned in Tool Search's
// alwaysEnabled set, so it must not pay an AutoReview round trip per turn. // alwaysEnabled set, so it must not pay an AutoReview round trip per turn.
"context_tree_query": "allow", "context_tree_query": "allow",
// pi-overflow's inspector reads only the extension's own stored output
// directory (id-validated); pinned in Tool Search's alwaysEnabled set.
"overflow_query": "allow",
// Model-chosen URLs can exfiltrate data — same reason curl/wget ask. // Model-chosen URLs can exfiltrate data — same reason curl/wget ask.
"web_fetch": "ask", "web_fetch": "ask",
"ctx_fetch_and_index": "ask",
"ssh_connect": "ask", "ssh_connect": "ask",
"ssh_cd": "ask", "ssh_cd": "ask",
+4 -12
View File
@@ -50,20 +50,12 @@
"text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches." "text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches."
}, },
{ {
"whenAll": ["web_fetch", "ctx_fetch_and_index"], "whenAll": ["chrome_snapshot", "overflow_query"],
"text": "Read a web page with web_fetch. Use ctx_fetch_and_index only when the content should be indexed into the context-mode knowledge base for repeated ctx_search queries." "text": "chrome_snapshot (and Chrome actions with includeSnapshot=true) return the page snapshot JSON directly. When it is large the result carries a [pi-overflow id=…] marker: extract only the uids, text, or state you need with overflow_query (jsonPath or search) instead of taking another snapshot, and take a fresh snapshot after any page interaction because the previous one is stale."
}, },
{ {
"whenAny": ["ctx_execute", "ctx_execute_file", "ctx_batch_execute"], "whenAny": ["overflow_query"],
"text": "Use Context Mode for logs, test/build output, generated data, large-file exploration/analysis/summarization, and any command whose output may be large or unpredictable. Prefer one ctx_batch_execute over several ctx_execute calls for multi-command research, and print only the derived answer needed for the task." "text": "When a tool result starts with or ends with a [pi-overflow id=…] marker, the full original output is stored; inspect it with overflow_query (search, line ranges, or jsonPath) instead of re-running the command or dumping it again."
},
{
"whenAny": ["ctx_execute_file"],
"text": "ctx_execute_file analyzes a file and never edits it; make changes with read + edit. It is confined to the current project root: do not call it for absolute paths outside the workspace, ../ traversal, or symlinks that resolve outside the workspace; host permission approval does not bypass this boundary. Use another explicitly authorized tool for those files."
},
{
"whenAll": ["chrome_snapshot", "ctx_execute_file"],
"text": "After chrome_snapshot or a Chrome action with includeSnapshot=true, immediately analyze the returned workspace-relative capture path with ctx_execute_file; never use read or cat for the raw JSON. Print only the uids, text, or state needed for the next action, and take a fresh snapshot after any interaction because the previous capture may be stale."
}, },
{ {
"whenAny": ["read"], "whenAny": ["read"],
+1 -12
View File
@@ -43,18 +43,6 @@
"web_search": { "inherit": true }, "web_search": { "inherit": true },
"web_fetch": { "inherit": true }, "web_fetch": { "inherit": true },
"ctx_execute": { "inherit": true },
"ctx_execute_file": { "inherit": true },
"ctx_batch_execute": { "inherit": true },
"ctx_index": { "inherit": true },
"ctx_search": { "inherit": true },
"ctx_fetch_and_index": { "inherit": true },
"ctx_stats": { "inherit": true },
"ctx_doctor": { "inherit": true },
"ctx_insight": { "inherit": true },
"ctx_upgrade": { "inherit": true },
"ctx_purge": { "inherit": true },
"memory_search": { "inherit": true }, "memory_search": { "inherit": true },
"session_search": { "inherit": true }, "session_search": { "inherit": true },
"memory_add": { "inherit": true }, "memory_add": { "inherit": true },
@@ -62,6 +50,7 @@
"memory_remove": { "inherit": true }, "memory_remove": { "inherit": true },
"skill_manage": { "inherit": true }, "skill_manage": { "inherit": true },
"context_tree_query": { "inherit": true }, "context_tree_query": { "inherit": true },
"overflow_query": { "inherit": true },
"ask_user_question": { "inherit": true }, "ask_user_question": { "inherit": true },
+1 -1
View File
@@ -11,7 +11,7 @@
"enabled": false "enabled": false
}, },
"truncate": { "truncate": {
"enabled": true, "enabled": false,
"maxChars": 12000 "maxChars": 12000
}, },
"sourceCodeFilteringEnabled": false, "sourceCodeFilteringEnabled": false,
+2 -4
View File
@@ -2,16 +2,14 @@
"alwaysEnabled": [ "alwaysEnabled": [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
"memory_replace", "memory_replace",
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query" "context_tree_query",
"overflow_query"
], ],
"showToolSearchFooterStatus": false, "showToolSearchFooterStatus": false,
"maxActiveGroups": 5, "maxActiveGroups": 5,
-1
View File
@@ -27,7 +27,6 @@
- `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 `@` 补全;FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。 - `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 `@` 补全;FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。
- `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。 - `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。
- CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。 - CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。
- `context-mode.ts` + `context-mode-filter.ts`:经 `prompt-guard.ts` 代理加载上游 `context-mode` Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性;Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文。filter 只丢弃其 context 注入中与本组合包路由重复且冲突的锚点段。
- `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。 - `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
- `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。 - `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。
- `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。 - `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。
-36
View File
@@ -1,36 +0,0 @@
/**
* Pure filter applied after context-mode's `context` handler: its upstream
* build appends one `{ role: "user", content }` message whose parts are
* `"\n\n"`-joined and whose first part is a routing anchor that duplicates —
* and contradicts — the bundle-owned `config/prompt/routing.jsonc`. The anchor
* part is stripped; remaining parts (active memory, resume snapshot) stay.
* No other message is touched; when nothing changes the input array is
* returned unchanged (same reference).
*/
export const CONTEXT_MODE_ANCHOR_PREFIX = "context-mode active. Hierarchy:";
type MessageLike = { role?: unknown; content?: unknown } & Record<string, unknown>;
export function filterContextModeMessages<T extends MessageLike>(
before: readonly T[],
after: readonly T[],
): T[] {
if (after.length <= before.length) return after as T[];
let result: T[] | null = null;
for (let i = after.length - 1; i >= before.length; i--) {
const message = after[i];
if (message === null || typeof message !== "object") continue;
if (message.role !== "user" || typeof message.content !== "string") continue;
const parts = message.content.split("\n\n");
const kept = parts.filter((part) => !part.startsWith(CONTEXT_MODE_ANCHOR_PREFIX));
if (kept.length === parts.length) continue;
result ??= after.slice();
if (kept.length === 0) {
result.splice(i, 1);
} else {
result[i] = { ...message, content: kept.join("\n\n") };
}
}
return result ?? (after as T[]);
}
-18
View File
@@ -1,18 +0,0 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
// The adapter is a prebuilt .js bundle, not a TypeScript source.
import piExtension from "../node_modules/context-mode/build/adapters/pi/extension.js";
import { filterContextModeMessages } from "./context-mode-filter.ts";
import { guardExtensionApi } from "./prompt-guard.ts";
/**
* Load context-mode through the prompt guard: the bundle keeps sole ownership
* of the system prompt, so the upstream routing anchor appended to the
* `context` message list is filtered out (bundle routing lives in
* `config/prompt/routing.jsonc`), while upstream active-memory/resume parts
* pass through untouched. The upstream code this filter depends on is pinned
* by `tests/system-prompt.test.ts` against the exact `context-mode` version.
*/
export default function bundledContextModeExtension(pi: ExtensionAPI): void {
piExtension(guardExtensionApi(pi, { contextFilter: filterContextModeMessages }));
}
+1 -1
View File
@@ -17,7 +17,7 @@ export default function contextView(pi: ExtensionAPI): void {
`Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`, `Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`,
`Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`, `Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`,
`Segments: ${formatNumber(metrics.segmentCount)}`, `Segments: ${formatNumber(metrics.segmentCount)}`,
`Semantic jobs: ${formatNumber(metrics.semanticJobs)}`, `Masked tool results: ${formatNumber(metrics.maskedResults)}`,
`Generation: ${formatNumber(metrics.generation)}`, `Generation: ${formatNumber(metrics.generation)}`,
].join("\n") ].join("\n")
: "No pi-compaction projection is available yet. Send one message to build the first snapshot."; : "No pi-compaction projection is available yet. Send one message to build the first snapshot.";
+39 -1526
View File
File diff suppressed because it is too large Load Diff
+85 -19
View File
@@ -10,15 +10,76 @@
], ],
"files": [ "files": [
"extensions", "extensions",
"pi-ask-user", "pi-ask-user/index.ts",
"pi-compaction", "pi-ask-user/src",
"pi-chrome", "pi-ask-user/package.json",
"pi-hermes-memory", "pi-ask-user/README.md",
"pi-rtk-optimizer", "pi-ask-user/LICENSE",
"pi-minimal-footer", "pi-ask-user/UPSTREAM.md",
"pi-subagents", "pi-compaction/index.ts",
"pi-notify", "pi-compaction/src",
"pi-ssh", "pi-compaction/package.json",
"pi-compaction/README.md",
"pi-compaction/CHANGELOG.md",
"pi-compaction/LICENSES",
"pi-compaction/UPSTREAM.md",
"pi-chrome/extensions",
"pi-chrome/package.json",
"pi-chrome/README.md",
"pi-chrome/CHANGELOG.md",
"pi-chrome/SECURITY.md",
"pi-chrome/LICENSE",
"pi-chrome/UPSTREAM.md",
"pi-hermes-memory/src",
"pi-hermes-memory/package.json",
"pi-hermes-memory/README.md",
"pi-hermes-memory/CHANGELOG.md",
"pi-hermes-memory/LICENSE",
"pi-hermes-memory/UPSTREAM.md",
"pi-rtk-optimizer/index.ts",
"pi-rtk-optimizer/src",
"pi-rtk-optimizer/package.json",
"pi-rtk-optimizer/README.md",
"pi-rtk-optimizer/CHANGELOG.md",
"pi-rtk-optimizer/LICENSE",
"pi-rtk-optimizer/UPSTREAM.md",
"pi-minimal-footer/index.ts",
"pi-minimal-footer/status.ts",
"pi-minimal-footer/settings.ts",
"pi-minimal-footer/assets",
"pi-minimal-footer/package.json",
"pi-minimal-footer/README.md",
"pi-minimal-footer/LICENSE",
"pi-minimal-footer/UPSTREAM.md",
"pi-subagents/src",
"pi-subagents/package.json",
"pi-subagents/README.md",
"pi-subagents/CHANGELOG.md",
"pi-subagents/SECURITY.md",
"pi-subagents/LICENSE",
"pi-subagents/UPSTREAM.md",
"pi-overflow/index.ts",
"pi-overflow/capture.ts",
"pi-overflow/settings.ts",
"pi-overflow/src",
"pi-overflow/package.json",
"pi-overflow/README.md",
"pi-overflow/LICENSE",
"pi-overflow/UPSTREAM.md",
"pi-notify/src",
"pi-notify/package.json",
"pi-notify/README.md",
"pi-notify/LICENSE",
"pi-notify/UPSTREAM.md",
"pi-ssh/index.ts",
"pi-ssh/permission-integration.ts",
"pi-ssh/src",
"pi-ssh/scripts",
"pi-ssh/package.json",
"pi-ssh/README.md",
"pi-ssh/CHANGELOG.md",
"pi-ssh/LICENSE",
"pi-ssh/UPSTREAM.md",
"pi-tool-search/extensions", "pi-tool-search/extensions",
"pi-tool-search/docs", "pi-tool-search/docs",
"pi-tool-search/package.json", "pi-tool-search/package.json",
@@ -26,8 +87,18 @@
"pi-tool-search/CHANGELOG.md", "pi-tool-search/CHANGELOG.md",
"pi-tool-search/LICENSE", "pi-tool-search/LICENSE",
"pi-tool-search/UPSTREAM.md", "pi-tool-search/UPSTREAM.md",
"pi-web-search", "pi-web-search/index.ts",
"pi-worktree", "pi-web-search/auth.ts",
"pi-web-search/router.ts",
"pi-web-search/clients",
"pi-web-search/package.json",
"pi-web-search/README.md",
"pi-web-search/LICENSE",
"pi-worktree/index.ts",
"pi-worktree/src",
"pi-worktree/package.json",
"pi-worktree/README.md",
"pi-worktree/LICENSE",
"pi-permission-system/LICENSES", "pi-permission-system/LICENSES",
"pi-permission-system/docs/auto-review", "pi-permission-system/docs/auto-review",
"pi-permission-system/src", "pi-permission-system/src",
@@ -47,12 +118,12 @@
], ],
"pi": { "pi": {
"extensions": [ "extensions": [
"./pi-overflow/capture.ts",
"./extensions/fff-override.ts", "./extensions/fff-override.ts",
"./pi-web-search/index.ts", "./pi-web-search/index.ts",
"./extensions/codegraph.ts", "./extensions/codegraph.ts",
"./pi-rtk-optimizer/index.ts", "./pi-rtk-optimizer/index.ts",
"./extensions/hashline.ts", "./extensions/hashline.ts",
"./extensions/context-mode.ts",
"./extensions/lsp.ts", "./extensions/lsp.ts",
"./pi-hermes-memory/src/index.ts", "./pi-hermes-memory/src/index.ts",
"./pi-compaction/index.ts", "./pi-compaction/index.ts",
@@ -66,10 +137,8 @@
"./pi-worktree/index.ts", "./pi-worktree/index.ts",
"./pi-chrome/extensions/chrome-profile-bridge/index.ts", "./pi-chrome/extensions/chrome-profile-bridge/index.ts",
"./pi-tool-search/extensions/index.ts", "./pi-tool-search/extensions/index.ts",
"./pi-overflow/index.ts",
"./extensions/system-prompt.ts" "./extensions/system-prompt.ts"
],
"skills": [
"./node_modules/context-mode/skills"
] ]
}, },
"dependencies": { "dependencies": {
@@ -78,13 +147,13 @@
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer", "@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
"@smoose/pi-notify": "file:./pi-notify", "@smoose/pi-notify": "file:./pi-notify",
"@tintinweb/pi-subagents": "file:./pi-subagents", "@tintinweb/pi-subagents": "file:./pi-subagents",
"context-mode": "1.0.169",
"jiti": "2.7.0", "jiti": "2.7.0",
"pi-ask-user": "file:./pi-ask-user", "pi-ask-user": "file:./pi-ask-user",
"pi-chrome": "file:./pi-chrome", "pi-chrome": "file:./pi-chrome",
"pi-compaction": "file:./pi-compaction", "pi-compaction": "file:./pi-compaction",
"pi-hashline-edit": "0.8.3", "pi-hashline-edit": "0.8.3",
"pi-hermes-memory": "file:./pi-hermes-memory", "pi-hermes-memory": "file:./pi-hermes-memory",
"pi-overflow": "file:./pi-overflow",
"pi-lsp": "0.1.7", "pi-lsp": "0.1.7",
"pi-ssh": "file:./pi-ssh", "pi-ssh": "file:./pi-ssh",
"pi-tool-search": "file:./pi-tool-search", "pi-tool-search": "file:./pi-tool-search",
@@ -117,8 +186,5 @@
}, },
"engines": { "engines": {
"node": ">=22.19.0" "node": ">=22.19.0"
},
"allowScripts": {
"better-sqlite3@12.11.1": true
} }
} }
+2 -3
View File
@@ -1,9 +1,8 @@
# my-pi 维护约定(pi-chrome) # my-pi 维护约定(pi-chrome)
- 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,Session 快照存储位于 `src/session-snapshot-store.ts`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。 - 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。
- 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。 - 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。
- 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。 - 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。
- 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。 - 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
- 完整 Chrome snapshot 不得进入普通工具文本或 `details`;只能原子写入当前项目 `.pi/chrome-context/<session>/<target>/snapshot.json`,使用哈希目录、POSIX `700` 目录 / `600` 文件,并只返回工作区相对路径和有界元数据。快照不得跨 Session 持久化,Session 启动/关闭、撤销和授权过期必须清理。 - `chrome_snapshot` 与带 `includeSnapshot=true` 的动作把快照 JSON 直接放进工具结果文本返回;超长结果的展示截断由 pi-overflow 统一负责(原文落盘,模型用 `overflow_query` 的 jsonPath/search 提取)。`details` 只放元素数、匹配数等小型元数据,不得把完整快照写进 `details`(会进入会话文件),也不再写 Session 临时快照文件。`chrome_find` 只返回有界匹配摘要,不保存文件。
- `chrome_snapshot` 和带 `includeSnapshot=true` 的动作必须引导模型立即使用 `ctx_execute_file`,不得使用 `read` / `cat` 把原始 JSON 重新塞入上下文;任何可能改变页面或目标的成功操作必须使旧快照失效,`chrome_find` 只可额外返回有界匹配摘要。
- 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。 - 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。
+4 -10
View File
@@ -6,7 +6,7 @@
```text ```text
You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI." You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
Agent: chrome_tab(list) → chrome_snapshot(...) → ctx_execute_file(capture) → chrome_screenshot(...) Agent: chrome_tab(list) → chrome_snapshot(...) → overflow_query(jsonPath) → chrome_screenshot(...)
✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png ✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
You: [keeps coding — agent never asked you to log in] You: [keeps coding — agent never asked you to log in]
``` ```
@@ -74,17 +74,11 @@ Second doctor run should show all checks passing.
Tool parameters and gotchas are documented inline in Pi. Tool parameters and gotchas are documented inline in Pi.
### Session-local snapshots and Context Mode ### Snapshots and pi-overflow
In this maintained build, `chrome_snapshot`, `chrome_find`, and actions with `includeSnapshot=true` keep the full structured page snapshot out of the normal tool result. They atomically save only the latest capture for each target under: In this maintained build, `chrome_snapshot` and actions with `includeSnapshot=true` return the full structured page snapshot as JSON directly in the tool result. `chrome_find` returns only up to 20 compact ranked matches.
```text Large snapshots are bounded by the bundle's `pi-overflow` extension: the result keeps a bounded view and the full original is stored, marked with `[pi-overflow id=…]`. Extract the uids, text, or state you need with `overflow_query` — a `jsonPath` such as `elements[0].uid` or `summary.focused`, or a regex `search` — instead of taking another snapshot. Any page interaction makes earlier snapshots stale, so take a fresh one (or pass `includeSnapshot=true` on the action) when the page has changed.
.pi/chrome-context/<session-hash>/<target-hash>/snapshot.json
```
The tool returns a workspace-relative path plus bounded metadata. Analyze that path immediately with `ctx_execute_file` and print only the uids, text, or state needed for the next browser action. Do **not** use `read` or `cat` on the capture, because that would put the raw page back into model context.
Captures are deliberately ephemeral: a successful page interaction invalidates the old target capture, and session shutdown, `/chrome revoke`, or authorization expiry removes the session files. `chrome_find` still returns up to 20 compact ranked matches for direct use while storing the full capture privately.
--- ---
+4 -5
View File
@@ -15,10 +15,9 @@
导入后的本地改动: 导入后的本地改动:
- Chrome 快照以原子方式写入当前项目 `.pi/chrome-context/` 目录下的 owner-only、Session 临时文件,而不是把原始页面结构返回给模型上下文。 - Chrome 快照与 `includeSnapshot=true` 的动作结果把快照 JSON 直接放入工具文本返回,`details` 只保留元素数/匹配数等小型元数据;展示截断由组合包 pi-overflow 统一负责(原文落盘 + `overflow_query` 查询)。`chrome_find` 只返回紧凑排序匹配,不保存文件。早期版本曾把快照原子写入 `.pi/chrome-context/` 临时文件并引导 `ctx_execute_file` 分析;该 Session 临时文件机制已随 context-mode 一起移除,各项目中残留的 `.pi/chrome-context/` 目录不再创建,可手动删除。
- 每个 Session/目标只保留最新快照;浏览器交互会使陈旧捕获失效,Session 关闭时移除该 Session 目录。
- `chrome_find` 返回紧凑的排序匹配,同时把完整结构化快照保留在 Session 本地捕获中,供可选的 `ctx_execute_file` 分析使用。
- click/type/fill/key 上的 `includeSnapshot=true` 会保存动作后的快照并返回其捕获路径,而不是把快照嵌入工具结果。
- 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。 - 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。
- 移除 `PI_CHROME_BRIDGE_HOST`/`PI_CHROME_BRIDGE_PORT` 环境变量,host/port 固定为 `127.0.0.1:17318`(Companion 扩展 `service_worker.js` 同样硬编码该地址,单侧可配只会断连且非回环 host 有安全风险)。
- 移除 `MAX_TEXT_CHARS`/`truncateText` 的 30K 字符文本截断:chrome_* 工具返回完整文本,超长结果的展示截断由 pi-overflow 统一负责;`MAX_ELEMENTS` 等条数上限保留不变。
同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期、隐私和 Context Mode 集成。 同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期和组合包集成改动。
@@ -4,11 +4,6 @@ import { existsSync, readFileSync, statSync } from "node:fs";
import { mkdir, writeFile } from "node:fs/promises"; import { mkdir, writeFile } from "node:fs/promises";
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
import { dirname, join, resolve } from "node:path"; import { dirname, join, resolve } from "node:path";
import {
SessionSnapshotStore,
type SnapshotCaptureMetadata,
type SnapshotTarget,
} from "../../src/session-snapshot-store.ts";
/** /**
* Existing-profile Chrome bridge for pi. * Existing-profile Chrome bridge for pi.
@@ -64,17 +59,11 @@ const PI_CHROME_GLOBAL_KEY = "__piChromeProfileBridgeLoaded__";
// reload) so a /reload — which tears down and re-evaluates the module — does not silently drop // reload) so a /reload — which tears down and re-evaluates the module — does not silently drop
// an active /chrome authorize grant. // an active /chrome authorize grant.
const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__"; const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__";
const DEFAULT_HOST = process.env.PI_CHROME_BRIDGE_HOST ?? "127.0.0.1"; const DEFAULT_HOST = "127.0.0.1";
const DEFAULT_PORT = Number(process.env.PI_CHROME_BRIDGE_PORT ?? "17318"); const DEFAULT_PORT = 17318;
const DEFAULT_TIMEOUT_MS = 30_000; const DEFAULT_TIMEOUT_MS = 30_000;
const MAX_TEXT_CHARS = 30_000;
const MAX_ELEMENTS = 80; const MAX_ELEMENTS = 80;
function truncateText(text: string, maxChars = MAX_TEXT_CHARS): string {
if (text.length <= maxChars) return text;
return `${text.slice(0, maxChars)}\n\n[truncated ${text.length - maxChars} characters]`;
}
function safeJson(value: unknown): string { function safeJson(value: unknown): string {
return JSON.stringify(value, null, 2); return JSON.stringify(value, null, 2);
} }
@@ -95,13 +84,28 @@ function snapshotFromIncludedResult(raw: unknown): unknown {
return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined; return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined;
} }
export function formatSnapshotCapture(metadata: SnapshotCaptureMetadata): string { /**
const kib = Math.max(1, Math.ceil(metadata.bytes / 1024)); * Small, session-file-safe metadata describing a returned snapshot. The full
return [ * snapshot goes to the model as text — details must stay small because Pi
`Chrome snapshot saved to ${metadata.path} (${kib} KiB, ${metadata.elementCount} elements${metadata.matchCount ? `, ${metadata.matchCount} matches` : ""}).`, * persists them in the session file.
"Analyze it immediately with ctx_execute_file; FILE_CONTENT is JSON and the page data is under .snapshot.", */
"This capture is replaced by the next snapshot for the same target and becomes stale after page interaction.", export function snapshotStats(snapshot: unknown): { elementCount: number; matchCount: number } {
].join("\n"); const object = snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
? snapshot as { elements?: unknown; matches?: unknown }
: {};
return {
elementCount: Array.isArray(object.elements) ? object.elements.length : 0,
matchCount: Array.isArray(object.matches) ? object.matches.length : 0,
};
}
/**
* Action summary followed by the inline snapshot JSON. Oversized results are
* bounded by pi-overflow and recoverable through overflow_query, so no local
* capture file is written.
*/
export function formatActionResultText(text: string, snapshot: unknown | undefined): string {
return snapshot === undefined ? text : `${text}\n\n${safeJson(snapshot)}`;
} }
export function formatChromeFindMatches(snapshot: any): string { export function formatChromeFindMatches(snapshot: any): string {
@@ -142,7 +146,7 @@ function formatChromeInspect(inspect: any): string {
lines.push("\n## Ancestors"); lines.push("\n## Ancestors");
for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`); for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`);
} }
return truncateText(lines.join("\n")); return lines.join("\n");
} }
function extensionRoot(): string { function extensionRoot(): string {
@@ -648,7 +652,6 @@ export default function (pi: ExtensionAPI): void {
let countdownInterval: NodeJS.Timeout | undefined; let countdownInterval: NodeJS.Timeout | undefined;
// Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy. // Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy.
let sessionCtx: ExtensionContext | undefined; let sessionCtx: ExtensionContext | undefined;
let snapshotStore: SessionSnapshotStore | undefined;
const clearAuthExpiryTimer = (): void => { const clearAuthExpiryTimer = (): void => {
if (!authExpiryTimer) return; if (!authExpiryTimer) return;
@@ -722,7 +725,6 @@ export default function (pi: ExtensionAPI): void {
persistAuth(); persistAuth();
// Revoking control ends pi-chrome's automation for this session; tidy up the target we own. // Revoking control ends pi-chrome's automation for this session; tidy up the target we own.
cleanupAutomationTargetBestEffort(); cleanupAutomationTargetBestEffort();
void snapshotStore?.clear().catch(() => undefined);
}; };
const authSummary = (): string => { const authSummary = (): string => {
@@ -853,45 +855,22 @@ export default function (pi: ExtensionAPI): void {
return { ...params, foreground: !background } as T; return { ...params, foreground: !background } as T;
}; };
const snapshotTargetFrom = (params: Record<string, unknown>): SnapshotTarget => ({ const formatActionResult = (raw: unknown, params: Record<string, unknown>, text: string): ToolTextResult => {
targetId: typeof params.targetId === "string" ? params.targetId : undefined,
urlIncludes: typeof params.urlIncludes === "string" ? params.urlIncludes : undefined,
titleIncludes: typeof params.titleIncludes === "string" ? params.titleIncludes : undefined,
});
const saveSnapshotCapture = async (snapshot: unknown, params: Record<string, unknown>): Promise<SnapshotCaptureMetadata> => {
if (!snapshotStore) throw new Error("Chrome snapshot storage is unavailable before session_start.");
const stored = await snapshotStore.save(snapshot, snapshotTargetFrom(params));
return stored.metadata;
};
const invalidateSnapshotCapture = async (params?: Record<string, unknown>): Promise<void> => {
if (!snapshotStore) return;
if (!params) await snapshotStore.clear();
else await snapshotStore.invalidate(snapshotTargetFrom(params));
};
const formatActionWithCapture = async (raw: unknown, params: Record<string, unknown>, text: string): Promise<ToolTextResult> => {
const snapshot = snapshotFromIncludedResult(raw); const snapshot = snapshotFromIncludedResult(raw);
const result = params.includeSnapshot && raw && typeof raw === "object" const result = params.includeSnapshot && raw && typeof raw === "object"
? (raw as { result?: unknown }).result ? (raw as { result?: unknown }).result
: raw; : raw;
if (!snapshot) { if (!snapshot) {
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text }], details: { result: result as Json } }; return { content: [{ type: "text", text }], details: { result: result as Json } };
} }
const capture = await saveSnapshotCapture(snapshot, params);
return { return {
content: [{ type: "text", text: `${text}\n\n${formatSnapshotCapture(capture)}` }], content: [{ type: "text", text: formatActionResultText(text, snapshot) }],
details: { result: result as Json, capture }, details: { result: result as Json, snapshot: snapshotStats(snapshot) },
}; };
}; };
pi.on("session_start", async (_event, ctx) => { pi.on("session_start", async (_event, ctx) => {
sessionCtx = ctx; sessionCtx = ctx;
const sessionId = ctx.sessionManager.getSessionId?.() || "unknown-session";
snapshotStore = new SessionSnapshotStore(workspaceCwd(ctx), sessionId);
await snapshotStore.clear();
await bridge.start(); await bridge.start();
// Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis. // Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis.
if (chromeControlAuthorized()) { if (chromeControlAuthorized()) {
@@ -909,10 +888,6 @@ export default function (pi: ExtensionAPI): void {
pi.on("session_shutdown", async (event) => { pi.on("session_shutdown", async (event) => {
clearAuthExpiryTimer(); clearAuthExpiryTimer();
clearCountdownInterval(); clearCountdownInterval();
// Snapshot files are intentionally ephemeral. A reload must take a fresh observation too,
// because the browser page may have changed while the extension was reloading.
await snapshotStore?.clear().catch(() => undefined);
snapshotStore = undefined;
// Keep the owned automation target across /reload, but clean it up on real session end. // Keep the owned automation target across /reload, but clean it up on real session end.
if (event?.reason !== "reload") cleanupAutomationTargetBestEffort(); if (event?.reason !== "reload") cleanupAutomationTargetBestEffort();
bridge.stop(); bridge.stop();
@@ -939,12 +914,12 @@ Capability model (important):
- Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates. - Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates.
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached. - Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP. - \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP.
- \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` are written to a private session-local JSON capture; raw page data is never returned inline. The tool result gives the path for immediate \`ctx_execute_file\` analysis. - \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` return the page snapshot as JSON directly in the tool result. When the result is large it carries a \`[pi-overflow id=…]\` marker — extract only the uids, text, or state you need with \`overflow_query\` (jsonPath such as \`elements[0].uid\` or \`summary.focused\`, or a \`search\`) instead of taking another snapshot.
Usage rules: Usage rules:
1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying. 1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying.
2. Call \`chrome_snapshot\`, then immediately call \`ctx_execute_file\` on the returned path and print only the uids/state needed for the next action. Never use read/cat on the full capture. 2. Call \`chrome_snapshot\` and read the returned JSON (or query a pi-overflow marker with \`overflow_query\`); keep only the uids/state needed for the next action.
3. Prefer \`uid\` over \`selector\`. After any page interaction, treat the old capture as stale. Use \`includeSnapshot=true\` when a fresh post-action capture is needed. 3. Prefer \`uid\` over \`selector\`. After any page interaction, earlier snapshots are stale — take a fresh one (or pass \`includeSnapshot=true\` on the action) instead of reusing old uids.
4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm. 4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm.
5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now). 5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now).
6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work. 6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work.
@@ -1337,7 +1312,6 @@ Usage rules:
const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs."; const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs.";
return { content: [{ type: "text", text }], details: { tabs } }; return { content: [{ type: "text", text }], details: { tabs } };
} }
if (params.action !== "version") await invalidateSnapshotCapture();
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1346,8 +1320,8 @@ Usage rules:
name: "chrome_snapshot", name: "chrome_snapshot",
label: "Chrome Snapshot", label: "Chrome Snapshot",
description: description:
"Capture the current Chrome page into a private, session-local JSON file instead of placing the raw DOM snapshot in model context. The result returns a path for immediate ctx_execute_file analysis. Each target keeps only its latest capture; page interaction makes it stale. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.", "Capture the current Chrome page and return the snapshot as JSON directly in the result. When it is large the result carries a [pi-overflow id=…] marker — extract only the uids, text, or state you need with overflow_query (jsonPath or search). Page interaction makes a snapshot stale; take a fresh one after navigating or acting. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.",
promptSnippet: "Save the current Chrome page to a session-local JSON capture, then analyze that path with ctx_execute_file for only the required uids and state.", promptSnippet: "Capture the current Chrome page and return its JSON snapshot; query large results via overflow_query.",
parameters: Type.Object({ parameters: Type.Object({
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1372,8 +1346,7 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params); return { content: [{ type: "text", text: safeJson(snapshot) }], details: { snapshot: snapshotStats(snapshot) } };
return { content: [{ type: "text", text: formatSnapshotCapture(capture) }], details: { capture } };
}, },
}); });
@@ -1381,8 +1354,8 @@ Usage rules:
name: "chrome_find", name: "chrome_find",
label: "Chrome Find", label: "Chrome Find",
description: description:
"Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates; the full structured snapshot is saved to the same private session-local file used by chrome_snapshot for optional ctx_execute_file analysis.", "Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates.",
promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches while keeping the full snapshot out of model context.", promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches.",
parameters: Type.Object({ parameters: Type.Object({
query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }), query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }),
mode: Type.Optional(StringEnum(snapshotModeValues)), mode: Type.Optional(StringEnum(snapshotModeValues)),
@@ -1403,10 +1376,9 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params);
return { return {
content: [{ type: "text", text: `${formatChromeFindMatches(snapshot)}\n\n${formatSnapshotCapture(capture)}` }], content: [{ type: "text", text: formatChromeFindMatches(snapshot) }],
details: { capture }, details: { snapshot: snapshotStats(snapshot) },
}; };
}, },
}); });
@@ -1452,9 +1424,8 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params); const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${safeJson(snapshot)}`;
const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${formatSnapshotCapture(capture)}`; return { content: [{ type: "text", text }], details: { snapshot: snapshotStats(snapshot), fallback: "page.snapshot" } };
return { content: [{ type: "text", text }], details: { capture, fallback: "page.snapshot" } };
} }
}, },
}); });
@@ -1481,7 +1452,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal); const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } };
}, },
}); });
@@ -1506,13 +1476,12 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
const text = value === undefined const text = value === undefined
? "undefined" ? "undefined"
: typeof value === "string" : typeof value === "string"
? value ? value
: safeJson(value) ?? "undefined"; : safeJson(value) ?? "undefined";
return { content: [{ type: "text", text: truncateText(text) }], details: { value: value as Json } }; return { content: [{ type: "text", text }], details: { value: value as Json } };
}, },
}); });
@@ -1520,7 +1489,7 @@ Usage rules:
name: "chrome_click", name: "chrome_click",
label: "Chrome Click", label: "Chrome Click",
description: description:
"Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-click snapshot to the session-local file and return its ctx_execute_file path, never the raw snapshot.", "Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. Earlier snapshots become stale after the click. Pass includeSnapshot=true to get a fresh post-click snapshot JSON inline.",
promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.", promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.",
parameters: Type.Object({ parameters: Type.Object({
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })), uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })),
@@ -1528,7 +1497,7 @@ Usage rules:
x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })), x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })),
y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })), y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })),
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })), domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-click snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-click snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1545,7 +1514,7 @@ Usage rules:
const summary = summarizeActionResult(result); const summary = summarizeActionResult(result);
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`; const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`; const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1553,13 +1522,13 @@ Usage rules:
name: "chrome_type", name: "chrome_type",
label: "Chrome Type", label: "Chrome Type",
description: description:
"Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-type snapshot and return its ctx_execute_file path.", "Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. Earlier snapshots become stale after typing. Pass includeSnapshot=true to get a fresh post-type snapshot JSON inline.",
promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.", promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.",
parameters: Type.Object({ parameters: Type.Object({
text: Type.String(), text: Type.String(),
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })), uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })), selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-type snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-type snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
pressEnter: Type.Optional(Type.Boolean()), pressEnter: Type.Optional(Type.Boolean()),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
@@ -1578,7 +1547,7 @@ Usage rules:
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : ""; const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
const base = `Typed ${params.text.length} character(s)${into}.`; const base = `Typed ${params.text.length} character(s)${into}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1586,7 +1555,7 @@ Usage rules:
name: "chrome_fill", name: "chrome_fill",
label: "Chrome Fill", label: "Chrome Fill",
description: description:
"Set the full value of a text input, textarea, or contenteditable element using Chrome input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-fill snapshot and return its ctx_execute_file path.", "Set the full value of a text input, textarea, or contenteditable element using Chrome input. Earlier snapshots become stale after filling. Pass includeSnapshot=true to get a fresh post-fill snapshot JSON inline.",
promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.", promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.",
parameters: Type.Object({ parameters: Type.Object({
text: Type.String(), text: Type.String(),
@@ -1594,7 +1563,7 @@ Usage rules:
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })), selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })), submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })), domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-fill snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-fill snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1612,7 +1581,7 @@ Usage rules:
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : ""; const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
const base = `Filled ${params.text.length} character(s)${into}.`; const base = `Filled ${params.text.length} character(s)${into}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1620,7 +1589,7 @@ Usage rules:
name: "chrome_key", name: "chrome_key",
label: "Chrome Key", label: "Chrome Key",
description: description:
"Send a keyboard key to an existing Chrome tab. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-key snapshot and return its ctx_execute_file path. Runs in the background by default; pass background=false so the user can watch.", "Send a keyboard key to an existing Chrome tab. Earlier snapshots become stale after the key press. Pass includeSnapshot=true to get a fresh post-key snapshot JSON inline. Runs in the background by default; pass background=false so the user can watch.",
promptSnippet: "Press keys in Chrome through the companion extension.", promptSnippet: "Press keys in Chrome through the companion extension.",
parameters: Type.Object({ parameters: Type.Object({
key: Type.String(), key: Type.String(),
@@ -1630,7 +1599,7 @@ Usage rules:
altKey: Type.Optional(Type.Boolean()), altKey: Type.Optional(Type.Boolean()),
metaKey: Type.Optional(Type.Boolean()), metaKey: Type.Optional(Type.Boolean()),
}, { description: "Modifier keys to hold while pressing the key (chord)." })), }, { description: "Modifier keys to hold while pressing the key (chord)." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-key snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-key snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1647,7 +1616,7 @@ Usage rules:
const summary = summarizeActionResult(result); const summary = summarizeActionResult(result);
const base = `Pressed ${params.key}.`; const base = `Pressed ${params.key}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1669,7 +1638,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal); const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } };
}, },
}); });
@@ -1691,7 +1659,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1713,7 +1681,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1733,7 +1701,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1813,7 +1781,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } };
}, },
}); });
@@ -1840,7 +1807,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } };
}, },
}); });
@@ -1863,7 +1829,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`; const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } };
}, },
@@ -1887,7 +1852,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } };
}, },
}); });
@@ -1910,7 +1874,6 @@ Usage rules:
const cwd = workspaceCwd(ctx); const cwd = workspaceCwd(ctx);
const paths = params.paths.map((p) => resolve(cwd, p)); const paths = params.paths.map((p) => resolve(cwd, p));
const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } };
}, },
}); });
+1 -2
View File
@@ -2,7 +2,7 @@
"name": "pi-chrome", "name": "pi-chrome",
"version": "0.15.46", "version": "0.15.46",
"scripts": { "scripts": {
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/session-snapshot-store.test.ts", "test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/snapshot-result.test.ts",
"version": "node scripts/sync-manifest-version.js", "version": "node scripts/sync-manifest-version.js",
"prepublishOnly": "node scripts/sync-manifest-version.js" "prepublishOnly": "node scripts/sync-manifest-version.js"
}, },
@@ -52,7 +52,6 @@
"type": "commonjs", "type": "commonjs",
"files": [ "files": [
"extensions", "extensions",
"src",
"test", "test",
"docs", "docs",
"test-suite", "test-suite",
-140
View File
@@ -1,140 +0,0 @@
import { createHash, randomUUID } from "node:crypto";
import { chmod, mkdir, rename, rm, writeFile } from "node:fs/promises";
import { join, relative, resolve, sep } from "node:path";
type SnapshotObject = Record<string, unknown>;
export type SnapshotTarget = {
targetId?: string;
urlIncludes?: string;
titleIncludes?: string;
};
export type SnapshotCaptureMetadata = {
version: 1;
captureId: string;
capturedAt: string;
path: string;
bytes: number;
targetKey: string;
targetId?: string | number;
mode?: string;
elementCount: number;
matchCount: number;
staleAfterInteraction: true;
};
export type StoredSnapshotCapture = {
metadata: SnapshotCaptureMetadata;
absolutePath: string;
};
export type SnapshotCaptureFile = {
version: 1;
captureId: string;
capturedAt: string;
targetKey: string;
snapshot: unknown;
};
function hashSegment(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
function normalizeRelativePath(path: string): string {
return path.split(sep).join("/");
}
function snapshotObject(snapshot: unknown): SnapshotObject | undefined {
return snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
? snapshot as SnapshotObject
: undefined;
}
export function snapshotTargetKey(target: SnapshotTarget = {}): string {
if (target.targetId) return `target:${target.targetId}`;
if (target.urlIncludes) return `url:${target.urlIncludes}`;
if (target.titleIncludes) return `title:${target.titleIncludes}`;
return "default";
}
export class SessionSnapshotStore {
readonly workspaceRoot: string;
readonly sessionRoot: string;
readonly sessionHash: string;
constructor(workspaceRoot: string, sessionId: string) {
this.workspaceRoot = resolve(workspaceRoot);
this.sessionHash = hashSegment(sessionId || "unknown-session");
this.sessionRoot = join(this.workspaceRoot, ".pi", "chrome-context", this.sessionHash);
}
async save(snapshot: unknown, target: SnapshotTarget = {}): Promise<StoredSnapshotCapture> {
const targetKey = snapshotTargetKey(target);
const targetDirectory = join(this.sessionRoot, hashSegment(targetKey));
const absolutePath = join(targetDirectory, "snapshot.json");
const temporaryPath = join(targetDirectory, `.snapshot.${process.pid}.${randomUUID()}.tmp`);
const captureId = randomUUID();
const capturedAt = new Date().toISOString();
const capture: SnapshotCaptureFile = {
version: 1,
captureId,
capturedAt,
targetKey,
snapshot,
};
const serialized = `${JSON.stringify(capture, null, 2)}\n`;
await mkdir(targetDirectory, { recursive: true, mode: 0o700 });
await chmod(this.sessionRoot, 0o700).catch(() => undefined);
await chmod(targetDirectory, 0o700).catch(() => undefined);
try {
await writeFile(temporaryPath, serialized, { encoding: "utf8", mode: 0o600, flag: "wx" });
await chmod(temporaryPath, 0o600).catch(() => undefined);
await rename(temporaryPath, absolutePath);
await chmod(absolutePath, 0o600).catch(() => undefined);
} catch (error) {
await rm(temporaryPath, { force: true }).catch(() => undefined);
throw error;
}
const object = snapshotObject(snapshot);
const elements = Array.isArray(object?.elements) ? object.elements : [];
const matches = Array.isArray(object?.matches) ? object.matches : [];
const targetId = typeof object?.targetId === "string" || typeof object?.targetId === "number"
? object.targetId
: undefined;
const mode = typeof object?.mode === "string" ? object.mode : undefined;
const path = normalizeRelativePath(relative(this.workspaceRoot, absolutePath));
return {
absolutePath,
metadata: {
version: 1,
captureId,
capturedAt,
path,
bytes: Buffer.byteLength(serialized),
targetKey,
targetId,
mode,
elementCount: elements.length,
matchCount: matches.length,
staleAfterInteraction: true,
},
};
}
async invalidate(target?: SnapshotTarget): Promise<void> {
if (!target) {
await this.clear();
return;
}
const targetDirectory = join(this.sessionRoot, hashSegment(snapshotTargetKey(target)));
await rm(targetDirectory, { recursive: true, force: true });
}
async clear(): Promise<void> {
await rm(this.sessionRoot, { recursive: true, force: true });
}
}
@@ -1,100 +0,0 @@
import assert from "node:assert/strict";
import { mkdtemp, readFile, stat } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { SessionSnapshotStore, snapshotTargetKey } from "../src/session-snapshot-store.ts";
import { CHROME_TOOL_NAMES, formatChromeFindMatches, formatSnapshotCapture } from "../extensions/chrome-profile-bridge/index.ts";
test("snapshotTargetKey prefers explicit target selectors", () => {
assert.equal(snapshotTargetKey(), "default");
assert.equal(snapshotTargetKey({ titleIncludes: "Inbox" }), "title:Inbox");
assert.equal(snapshotTargetKey({ urlIncludes: "example.com" }), "url:example.com");
assert.equal(snapshotTargetKey({ targetId: "42", urlIncludes: "ignored" }), "target:42");
});
test("SessionSnapshotStore atomically keeps only the latest capture per target", async () => {
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-snapshot-"));
const store = new SessionSnapshotStore(workspace, "session/private/id");
const first = await store.save({ mode: "interactive", targetId: 42, elements: [{ uid: "el-1" }], matches: [] });
const second = await store.save({ mode: "changes", targetId: 42, elements: [{ uid: "el-2" }], matches: [{ uid: "el-2" }] });
assert.equal(first.metadata.path, second.metadata.path);
assert.notEqual(first.metadata.captureId, second.metadata.captureId);
assert.equal(second.metadata.elementCount, 1);
assert.equal(second.metadata.matchCount, 1);
assert.equal(second.metadata.targetId, 42);
assert.ok(!second.metadata.path.includes("private"));
const stored = JSON.parse(await readFile(second.absolutePath, "utf8"));
assert.equal(stored.version, 1);
assert.equal(stored.captureId, second.metadata.captureId);
assert.equal(stored.snapshot.elements[0].uid, "el-2");
if (process.platform !== "win32") {
assert.equal((await stat(store.sessionRoot)).mode & 0o777, 0o700);
assert.equal((await stat(second.absolutePath)).mode & 0o777, 0o600);
}
await store.clear();
await assert.rejects(stat(second.absolutePath), { code: "ENOENT" });
});
test("target invalidation and session cleanup do not affect another session", async () => {
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-isolation-"));
const firstSession = new SessionSnapshotStore(workspace, "session-a");
const secondSession = new SessionSnapshotStore(workspace, "session-b");
const firstTarget = { targetId: "101" };
const secondTarget = { targetId: "202" };
const first = await firstSession.save({ elements: [] }, firstTarget);
const otherTarget = await firstSession.save({ elements: [] }, secondTarget);
const otherSession = await secondSession.save({ elements: [] }, firstTarget);
await firstSession.invalidate(firstTarget);
await assert.rejects(stat(first.absolutePath), { code: "ENOENT" });
await stat(otherTarget.absolutePath);
await stat(otherSession.absolutePath);
await firstSession.clear();
await assert.rejects(stat(otherTarget.absolutePath), { code: "ENOENT" });
await stat(otherSession.absolutePath);
await secondSession.clear();
});
test("snapshot result formatting stays bounded and points to Context Mode", () => {
const text = formatSnapshotCapture({
version: 1,
captureId: "capture-1",
capturedAt: "2026-08-28T00:00:00.000Z",
path: ".pi/chrome-context/session/target/snapshot.json",
bytes: 80_000,
targetKey: "target:42",
targetId: 42,
mode: "interactive",
elementCount: 80,
matchCount: 3,
staleAfterInteraction: true,
});
assert.match(text, /ctx_execute_file/);
assert.match(text, /\.pi\/chrome-context\/session\/target\/snapshot\.json/);
assert.doesNotMatch(text, /raw-secret-page-content/);
});
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
const matches = Array.from({ length: 25 }, (_, index) => ({
kind: "element",
uid: `el-${index}`,
role: "button",
label: `Button ${index}`,
rect: { x: index, y: index, width: 10, height: 10 },
}));
const text = formatChromeFindMatches({ query: "button", matches });
assert.match(text, /el-0/);
assert.match(text, /el-19/);
assert.doesNotMatch(text, /el-20/);
assert.equal(CHROME_TOOL_NAMES.length, 21);
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
});
+48
View File
@@ -0,0 +1,48 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
CHROME_TOOL_NAMES,
formatActionResultText,
formatChromeFindMatches,
snapshotStats,
} from "../extensions/chrome-profile-bridge/index.ts";
test("formatActionResultText appends the inline snapshot JSON", () => {
const snapshot = { elements: [{ uid: "el-1", role: "button", label: "Save" }], matches: [] };
const text = formatActionResultText("Clicked el-1", snapshot);
const [summary, json] = text.split("\n\n");
assert.equal(summary, "Clicked el-1");
assert.deepEqual(JSON.parse(json), snapshot);
});
test("formatActionResultText returns the summary alone without a snapshot", () => {
assert.equal(formatActionResultText("Clicked el-1", undefined), "Clicked el-1");
});
test("snapshotStats stays small: only element and match counts", () => {
assert.deepEqual(
snapshotStats({ elements: [{ uid: "a" }, { uid: "b" }], matches: [{ uid: "a" }] }),
{ elementCount: 2, matchCount: 1 },
);
assert.deepEqual(snapshotStats({ elements: "nope" }), { elementCount: 0, matchCount: 0 });
assert.deepEqual(snapshotStats(undefined), { elementCount: 0, matchCount: 0 });
assert.deepEqual(snapshotStats([1, 2, 3]), { elementCount: 0, matchCount: 0 });
});
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
const matches = Array.from({ length: 25 }, (_, index) => ({
kind: "element",
uid: `el-${index}`,
role: "button",
label: `Button ${index}`,
rect: { x: index, y: index, width: 10, height: 10 },
}));
const text = formatChromeFindMatches({ query: "button", matches });
assert.match(text, /el-0/);
assert.match(text, /el-19/);
assert.doesNotMatch(text, /el-20/);
assert.equal(CHROME_TOOL_NAMES.length, 21);
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
});
+13 -9
View File
@@ -1,19 +1,23 @@
# my-pi 维护约定(pi-compaction) # my-pi 维护约定(pi-compaction)
- 扩展入口是 `pi-compaction/index.ts`,增量索引、投影、后台摘要、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。 - 扩展入口是 `pi-compaction/index.ts`,遮蔽规划、投影索引、遮蔽台账、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。
- 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。 - 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。
- `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune` 或 `openai-codex-native-compaction` 状态;切换后应使用新 Session。配置不写回 `settings.json`,默认 soft/hard/target 比例为 72%/90%/55%。 - `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune`、`openai-codex-native-compaction` 或旧 `segment-summary` 状态;切换后应使用新 Session。配置不写回 `settings.json`。
- Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage;background worker 不得在 Provider 请求中被 await。 - Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。 - `pi-compaction` 负责已经进入 Session 的历史投影,与 pi-overflow 的新结果入口截断职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。 - L2 遮蔽只用 Pi 原生 `context_edit`(`{ type: "context_edit", targetId, replacement }`)处理保护窗口外的旧 toolResult:成批提交、受 trigger/target 回差与 `minClearTokens` 约束,不逐条即时替换;不为遮蔽写 `segment-summary` 等 custom entry,原始条目、UI 与导出保持不变。
- Segment custom entry 只能保存摘要、hash 和 source reference,不得复制完整原始工具输出。`context_tree_query` 恢复结果必须有界,native checkpoint 的 portable summary 也必须有界。 - 不得改写最近 `keepRecentTurns` 轮;不得遮蔽 user/assistant 消息、`excludeTools` 中的工具、`SKILL.md` 的 `read` 结果、已遮蔽或已省略的条目、以及小于 `minEntryTokens` 的结果。
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。Pi Core raw threshold 只负责唤醒,只有 effective lower bound 达到 `hardRatio` 才执行 threshold compaction;manual 与 overflow 不得被阻断。 - 遮蔽占位符必须确定性、单行、以 `MASK_PREFIX`(`"[pi-compaction] Earlier "`)开头,并在文本中指向 `context_tree_query` 恢复;`context_tree_query` 与 native checkpoint 的 portable summary 都必须有界。被遮蔽的 toolResult 若带 `details.overflow.id`(pi-overflow 已存原文),占位符末尾追加 `overflow_query` 提示——只读该公开字段,不得 import pi-overflow 内部实现。
- L3 整段摘要只能从空闲的 `agent_settled` 通过 `ctx.compact` 触发(`ctx.compact` 会中止当前运行,非空闲时调用会破坏运行);同一 leaf 不重复触发。阈值以绝对 token 计并受窗口比例约束(`effectiveThreshold = min(tokens, floor(ratio × contextWindow))`)。
- `context` 钩子不得改写消息——非 checkpoint 路径只更新 snapshot 并返回 undefined;唯一例外是 native checkpoint 路径:checkpoint 之后的条目在投影时必须应用各 target 最新的 `context_edit`(null 省略、字符串规范成 text block),因为 `sessionEntryToContextMessages` 不处理 edits。
- `session_before_compact` 只负责生成 portable/native 摘要,不得按自身阈值取消 Pi 的压缩(Pi 的 `contextTokens > window - reserveTokens` 是最终安全网)。
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。遮蔽台账在 `session_start` / `session_tree` / `session_compact` 从当前 branch 的 `context_edit` 条目重建,指标暴露 `maskedResults` 与 `reclaimedTokens`。
- Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。 - Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。
- 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认在 turn boundary 达到 effective 下界 90% 时触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。 - 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认由 L3 阈值或 Pi 自身阈值触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。
- SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。 - SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。
- 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。 - 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。
## 验证 ## 验证
- 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。 - 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。
- 修改 projection、token accounting、threshold、checkpoint、模型切换、后台并发或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。 - 修改 projection、token accounting、遮蔽规划/台账、threshold、checkpoint、模型切换或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。
+36 -31
View File
@@ -1,19 +1,22 @@
# pi-compaction # pi-compaction
High-performance unified context projection and compaction for Pi. Pressure-aware context masking and threshold compaction for Pi.
## Responsibilities ## Responsibilities
- incrementally index completed tool calls; Pi context management is layered; this extension owns the middle two layers:
- replace large tool results with deterministic bounded evidence stubs;
- pass small tool results through unchanged whenever a stub would not be shorter;
- refine large stubs with non-blocking semantic summaries;
- expose one effective context projection to normal provider requests and Codex/SubAPI native compaction;
- cancel Pi's raw threshold compaction until the effective lower-bound reaches the configured threshold;
- retain original tool evidence through `context_tree_query`;
- publish read-only effective metrics for the footer and context observer.
Context Mode remains responsible for preventing large external data from entering the session. RTK remains responsible for ordinary tool-output cleanup. - **L1 — entry truncation** (future `pi-overflow`): truncate new tool results as they are written. Not implemented here.
- **L2 — observation masking** (this extension): under real context pressure, replace old tool results outside a protected recent window with deterministic one-line placeholders via Pi `context_edit` entries. Originals stay in the session untouched and are recoverable with `context_tree_query`.
- **L3 — whole-context summary** (this extension): when the session idles (`agent_settled`) above the summary threshold, trigger `ctx.compact` once per leaf.
- **Pi native threshold compaction** is the final safety net and is never blocked; this extension no longer cancels `session_before_compact` based on its own thresholds.
Additionally it:
- publishes one effective-context snapshot for normal provider requests and Codex/SubAPI native compaction;
- maintains a masked-results ledger so `/context`, `/compaction-status`, the footer, and AutoReview see the same placeholders the model sees.
pi-overflow bounds new tool results as they arrive. RTK remains responsible for ordinary tool-output cleanup.
## Configuration ## Configuration
@@ -22,21 +25,21 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
```json ```json
{ {
"enabled": true, "enabled": true,
"softRatio": 0.72, "masking": {
"hardRatio": 0.9,
"targetRatio": 0.55,
"deterministic": {
"enabled": true, "enabled": true,
"maxStubTokens": 1024 "triggerTokens": 200000,
"triggerRatio": 0.6,
"targetTokens": 100000,
"targetRatio": 0.35,
"minClearTokens": 50000,
"keepRecentTurns": 10,
"minEntryTokens": 256,
"excludeTools": ["ask_user_question"]
}, },
"semantic": { "summary": {
"enabled": true, "triggerTokens": 400000,
"model": "default", "triggerRatio": 0.9,
"thinking": "low", "thinking": "low"
"concurrency": 1,
"minInputChars": 4000,
"maxInputChars": 24000,
"maxRetries": 2
}, },
"native": { "native": {
"enabled": true "enabled": true
@@ -44,7 +47,9 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
} }
``` ```
Invalid or missing fields are normalized in memory. All thresholds are absolute token counts constrained by a context-window ratio: `effectiveThreshold = min(tokens, floor(ratio × contextWindow))`. Masking fires when `currentTokens ≥ triggerEff` and clears the oldest eligible results until `currentTokens - cleared ≤ targetEff`; if the clearable volume stays under `min(minClearTokens, triggerEff - targetEff)` nothing is masked, so masking is rare, batched, and hysteretic. `masking.target` is normalized below `masking.trigger` and `summary.trigger` above `masking.trigger` when a config would violate that ordering. Invalid or missing fields fall back to defaults in memory.
Masking never touches the most recent `keepRecentTurns` assistant turns, user/assistant messages, `excludeTools`, `SKILL.md` reads, entries already masked, or results smaller than `minEntryTokens`.
## Native providers ## Native providers
@@ -67,17 +72,17 @@ The gateway must have a compatible deployed version, model and available upstrea
## Session format ## Session format
The only custom entry type is `pi-compaction`, version 1. Segment summary entries contain hashes, summaries, and source references; they never duplicate complete raw tool output. Native checkpoints use Pi compaction-entry details with the same kind/version. The extension writes no `segment-summary` or other custom entries for masking: L2 uses Pi's native `context_edit` entries (`{ type: "context_edit", targetId, replacement }`), which alter only future model context. The raw session entries — and therefore UI, exports, and the evidence ledger — are unchanged; every masked result remains recoverable through `context_tree_query`. Masking placeholders are deterministic single lines starting with `[pi-compaction] Earlier `.
Projection replaces a tool result only when the resulting summary plus its `Source:` recovery line is strictly shorter than the original text content. Small tool results are therefore passed through byte-for-byte and keep their original content parts, including any non-text parts. Because a stub is only larger than the text it would replace when the original is short, this comparison is the sole gate; no separate size option exists. Every indexed result remains recoverable through `context_tree_query` whether or not the message was replaced. Compaction entries still carry `pi-compaction` version 1 details; native checkpoints use the same kind/version. This implementation intentionally does not read legacy `context-prune-*`, `openai-codex-native-compaction`, or older `segment-summary` entries. Start a new session after switching to this extension.
This implementation intentionally does not read legacy `context-prune-*` or `openai-codex-native-compaction` entries. Start a new session after switching to this extension.
## Interfaces ## Interfaces
- `context_tree_query`: recover original bounded tool evidence by source key or tool call ID. - `context_tree_query`: recover the original output of a tool result that pi-compaction cleared from context, by tool call ID (1–20 ids per call, bounded output). Use it when a `[pi-compaction] Earlier` placeholder lacks evidence needed for the current task.
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, and pending semantic jobs. - pi-overflow linkage: when the masked tool result carries `details.overflow.id` (pi-overflow stored the oversized original), the placeholder ends with `Full original stored: overflow_query {"id":"…"}`. pi-compaction only reads that public field and never imports pi-overflow internals.
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, masked results, and generation.
- Service `my-pi.pi-compaction.service.v1`: `getSnapshot`, `getMetrics`, and `query(sessionId, ids)` — `query` returns a placeholder record only for tool call IDs that are currently masked, so consumers such as the permission system's AutoReview see the same evidence the model does.
## Performance model ## Performance model
The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots. The `context` hook performs no model call and no session write. Semantic summaries run in a bounded background worker and become visible only at later safe event boundaries. The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots; the `context` hook performs no model call, no session write, and returns the message list unchanged except on the native-checkpoint path, where post-checkpoint `context_edit` entries are applied during projection. Masking drafts are produced in `turn_end` in one batch and only when both the trigger and the `minClear` hysteresis pass, so the hot path stays allocation-free for unprompted turns.
+5
View File
@@ -29,3 +29,8 @@
- 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。 - 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。
- 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。 - 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。
- 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。 - 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。
## 本地改动记录
- 上下文管理重构为 L2/L3:L2 在上下文压力下经 `turn_end` 提交 Pi 原生 `context_edit`,把保护窗口外的旧 toolResult 批量遮蔽为指向 `context_tree_query` 的确定性占位符(原始条目保留、不写 custom entry);L3 只在空闲 `agent_settled` 且达到绝对 token 阈值(受窗口比例约束)时调用一次 `ctx.compact`。删除逐条确定性存根、后台语义 worker、`softRatio`/`hardRatio`/`targetRatio` 配置与 `session_before_compact` 的自行阈值取消;Pi 原生阈值压缩保留为最终安全网。
- L2 遮蔽占位符联动 pi-overflow:被遮蔽的 toolResult 若携带 `details.overflow.id`,占位符末尾追加 `overflow_query {"id":"…"}` 恢复提示。
+174 -76
View File
@@ -14,6 +14,12 @@ import {
type SessionEntry, type SessionEntry,
} from "@earendil-works/pi-coding-agent"; } from "@earendil-works/pi-coding-agent";
import { DEFAULT_CONFIG, loadConfig } from "./src/config.js"; import { DEFAULT_CONFIG, loadConfig } from "./src/config.js";
import {
applyContextEdits,
planMasking,
rebuildMaskedLedger,
type MaskedLedgerEntry,
} from "./src/masking.js";
import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js"; import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js";
import { import {
buildCompactionRequestBody, buildCompactionRequestBody,
@@ -32,27 +38,28 @@ import {
type NativeCompactionDetails, type NativeCompactionDetails,
type ResponseItem, type ResponseItem,
} from "./src/native-compaction.js"; } from "./src/native-compaction.js";
import { SegmentIndex } from "./src/segment-index.js"; import { contentText, SegmentIndex } from "./src/segment-index.js";
import { SemanticWorker } from "./src/semantic-worker.js";
import { shouldCompact } from "./src/token-accounting.js";
import { import {
CompactionServiceRegistry, CompactionServiceRegistry,
publishCompactionService, publishCompactionService,
unpublishCompactionService, unpublishCompactionService,
} from "./src/service.js"; } from "./src/service.js";
import { effectiveThreshold } from "./src/token-accounting.js";
import { import {
COMPACTION_CUSTOM_TYPE, COMPACTION_CUSTOM_TYPE,
COMPACTION_VERSION, COMPACTION_VERSION,
QUERY_TOOL_NAME, QUERY_TOOL_NAME,
type MaskedResultRecord,
type PiCompactionConfig, type PiCompactionConfig,
type ProjectionSnapshot, type ProjectionSnapshot,
type ToolHistoryRecord,
} from "./src/types.js"; } from "./src/types.js";
interface SessionRuntime { interface SessionRuntime {
index: SegmentIndex; index: SegmentIndex;
config: PiCompactionConfig; config: PiCompactionConfig;
codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] }; codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] };
masked: Map<string, MaskedLedgerEntry>;
lastAutoCompactLeafId: string | null;
} }
interface CachedPayload { interface CachedPayload {
@@ -64,10 +71,6 @@ function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error); return error instanceof Error ? error.message : String(error);
} }
function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
interface EnvironmentValue { interface EnvironmentValue {
hash: string; hash: string;
tokens: number; tokens: number;
@@ -81,6 +84,10 @@ interface EnvironmentCacheEntry extends EnvironmentValue {
tools: unknown[]; tools: unknown[];
} }
function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
function environment( function environment(
pi: ExtensionAPI, pi: ExtensionAPI,
ctx: ExtensionContext, ctx: ExtensionContext,
@@ -182,34 +189,46 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const runtimes = new Map<string, SessionRuntime>(); const runtimes = new Map<string, SessionRuntime>();
const payloads = new Map<string, CachedPayload>(); const payloads = new Map<string, CachedPayload>();
const environments = new Map<string, EnvironmentCacheEntry>(); const environments = new Map<string, EnvironmentCacheEntry>();
const worker = new SemanticWorker();
const service = new CompactionServiceRegistry(); const service = new CompactionServiceRegistry();
publishCompactionService(service); publishCompactionService(service);
const maskedRecords = (sessionId: string, ids: string[]): MaskedResultRecord[] => {
const masked = runtimes.get(sessionId)?.masked;
if (!masked) return [];
return ids.flatMap((id) => {
const entry = masked.get(id);
return entry
? [{
toolCallId: id,
resultTimestamp: entry.resultTimestamp,
deterministicSummary: entry.placeholder,
}]
: [];
});
};
const runtimeFor = (ctx: ExtensionContext): SessionRuntime => { const runtimeFor = (ctx: ExtensionContext): SessionRuntime => {
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
let runtime = runtimes.get(sessionId); let runtime = runtimes.get(sessionId);
if (!runtime) { if (!runtime) {
const index = new SegmentIndex(sessionId); runtime = {
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], DEFAULT_CONFIG); index: new SegmentIndex(sessionId),
runtime = { index, config: structuredClone(DEFAULT_CONFIG) }; config: structuredClone(DEFAULT_CONFIG),
masked: rebuildLedger(ctx),
lastAutoCompactLeafId: null,
};
runtimes.set(sessionId, runtime); runtimes.set(sessionId, runtime);
service.setSession(sessionId, (ids) => index.query(ids)); service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
} }
return runtime; return runtime;
}; };
const persist = (record: unknown): void => { const ledgerStats = (runtime: SessionRuntime): { maskedResults: number; reclaimedTokens: number } => {
pi.appendEntry(COMPACTION_CUSTOM_TYPE, record); let reclaimedTokens = 0;
}; for (const entry of runtime.masked.values()) {
reclaimedTokens += Math.max(0, entry.originalTokens - entry.placeholderTokens);
const drainSemantic = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
const sessionId = ctx.sessionManager.getSessionId();
for (const result of worker.drain(sessionId)) {
const record = runtime.index.applySemantic(result.sourceKey, result.sourceHash, result.summary);
if (record) persist(record);
} }
runtime.index.setSemanticJobs(worker.count(sessionId)); return { maskedResults: runtime.masked.size, reclaimedTokens };
}; };
const projectMessages = ( const projectMessages = (
@@ -218,16 +237,32 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
messages: readonly any[], messages: readonly any[],
): ProjectionSnapshot => { ): ProjectionSnapshot => {
const env = environment(pi, ctx, environments); const env = environment(pi, ctx, environments);
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId())); const stats = ledgerStats(runtime);
const snapshot = runtime.index.project(messages, { const snapshot = runtime.index.project(messages, {
environmentHash: env.hash, environmentHash: env.hash,
environmentTokens: env.tokens, environmentTokens: env.tokens,
contextWindow: env.contextWindow, contextWindow: env.contextWindow,
maskedResults: stats.maskedResults,
reclaimedTokens: stats.reclaimedTokens,
}); });
service.setSnapshot(snapshot); service.setSnapshot(snapshot);
return snapshot; return snapshot;
}; };
const rebuildLedger = (ctx: ExtensionContext): Map<string, MaskedLedgerEntry> =>
rebuildMaskedLedger(
ctx.sessionManager.getBranch(),
ctx.sessionManager.getLeafId(),
(message) => estimateTokens(message as any),
);
const hydrateRuntime = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
runtime.index.hydrate();
runtime.masked = rebuildLedger(ctx);
runtime.lastAutoCompactLeafId = null;
runtime.codexCache = undefined;
};
const branchMessages = ( const branchMessages = (
ctx: ExtensionContext, ctx: ExtensionContext,
runtime: SessionRuntime, runtime: SessionRuntime,
@@ -237,9 +272,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const checkpoint = findNativeCheckpoint(branch); const checkpoint = findNativeCheckpoint(branch);
let messages: any[]; let messages: any[];
if (checkpoint.status === "valid") { if (checkpoint.status === "valid") {
messages = branch // sessionEntryToContextMessages sees only raw entries; applyContextEdits
.slice(checkpoint.checkpoint.entryIndex + 1) // additionally replays the latest context_edit per target (omissions and
.flatMap((entry) => sessionEntryToContextMessages(entry)); // placeholder replacements included).
messages = applyContextEdits(branch.slice(checkpoint.checkpoint.entryIndex + 1));
if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) { if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) {
messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages]; messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages];
} }
@@ -278,15 +314,22 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
publishCompactionService(service); publishCompactionService(service);
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
const config = await loadConfig(); const config = await loadConfig();
const index = new SegmentIndex(sessionId); const runtime = runtimes.get(sessionId) ?? {
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], config); index: new SegmentIndex(sessionId),
runtimes.set(sessionId, { index, config }); config,
service.setSession(sessionId, (ids) => index.query(ids)); masked: new Map<string, MaskedLedgerEntry>(),
lastAutoCompactLeafId: null,
};
runtime.config = config;
runtime.index.hydrate();
runtime.masked = rebuildLedger(ctx);
runtime.lastAutoCompactLeafId = null;
runtimes.set(sessionId, runtime);
service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
}); });
pi.on("session_shutdown", (_event, ctx) => { pi.on("session_shutdown", (_event, ctx) => {
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
worker.cancelSession(sessionId);
runtimes.delete(sessionId); runtimes.delete(sessionId);
payloads.delete(sessionId); payloads.delete(sessionId);
environments.delete(sessionId); environments.delete(sessionId);
@@ -296,15 +339,13 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("session_tree", (_event, ctx) => { pi.on("session_tree", (_event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config); hydrateRuntime(ctx, runtime);
runtime.codexCache = undefined;
service.clearSnapshot(ctx.sessionManager.getSessionId()); service.clearSnapshot(ctx.sessionManager.getSessionId());
}); });
pi.on("session_compact", (_event, ctx) => { pi.on("session_compact", (_event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config); hydrateRuntime(ctx, runtime);
runtime.codexCache = undefined;
service.clearSnapshot(ctx.sessionManager.getSessionId()); service.clearSnapshot(ctx.sessionManager.getSessionId());
payloads.delete(ctx.sessionManager.getSessionId()); payloads.delete(ctx.sessionManager.getSessionId());
}); });
@@ -319,21 +360,9 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("turn_end", (event, ctx) => { pi.on("turn_end", (event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
drainSemantic(ctx, runtime); if (!runtime.config.enabled) return undefined;
const previous = runtime.index.getSnapshot();
const captured = runtime.index.captureTurn(event.message, event.toolResults, runtime.config);
for (const record of captured) persist(runtime.index.toPersisted(record));
const pressure = previous && previous.contextWindow > 0
? previous.upperBound / previous.contextWindow >= runtime.config.softRatio
: false;
const semanticCandidates = pressure
? runtime.index.allRecords()
: captured.filter((record) => record.rawText.length >= runtime.config.semantic.maxInputChars);
for (const record of semanticCandidates) {
worker.enqueue(ctx.sessionManager.getSessionId(), record, ctx, runtime.config);
}
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId()));
const previous = runtime.index.getSnapshot();
const usage = ctx.getContextUsage(); const usage = ctx.getContextUsage();
if (usage?.tokens && previous) { if (usage?.tokens && previous) {
const env = environment(pi, ctx, environments); const env = environment(pi, ctx, environments);
@@ -343,6 +372,50 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
env.hash, env.hash,
); );
} }
const currentTokens = usage?.tokens
?? (event.context?.contextMessages ?? []).reduce((sum, message) => sum + estimateTokens(message as any), 0)
+ environment(pi, ctx, environments).tokens;
const plan = planMasking({
contextEntries: event.context?.contextEntries ?? [],
currentTokens,
contextWindow: ctx.model?.contextWindow ?? 0,
masking: runtime.config.masking,
estimate: (message) => estimateTokens(message as any),
});
if (!plan.drafts.length) return undefined;
for (const result of plan.masked) {
runtime.masked.set(result.toolCallId, {
placeholder: result.placeholder,
originalTokens: result.originalTokens,
placeholderTokens: result.placeholderTokens,
resultTimestamp: result.resultTimestamp,
});
}
return { entries: [...(event.entries ?? []), ...plan.drafts] };
});
pi.on("agent_settled", (_event, ctx) => {
const runtime = runtimeFor(ctx);
if (!runtime.config.enabled || ctx.isIdle?.() === false) return;
const snapshot = runtime.index.getSnapshot();
const tokens = ctx.getContextUsage()?.tokens ?? snapshot?.effectiveTokens;
if (tokens == null) return;
const contextWindow = ctx.model?.contextWindow ?? snapshot?.contextWindow ?? 0;
const threshold = effectiveThreshold(
runtime.config.summary.triggerTokens,
runtime.config.summary.triggerRatio,
contextWindow,
);
if (tokens < threshold) return;
const leafId = ctx.sessionManager.getLeafId();
if (leafId !== null && runtime.lastAutoCompactLeafId === leafId) return;
runtime.lastAutoCompactLeafId = leafId;
ctx.compact({
onError: (error) => {
if (ctx.hasUI) ctx.ui.notify(`pi-compaction auto summary failed: ${errorMessage(error)}`, "warning");
},
});
}); });
pi.on("context", (event, ctx) => { pi.on("context", (event, ctx) => {
@@ -358,7 +431,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const projected = branchMessages(ctx, runtime, branch); const projected = branchMessages(ctx, runtime, branch);
return { messages: projected.snapshot.messages as any }; return { messages: projected.snapshot.messages as any };
} }
return { messages: projectMessages(ctx, runtime, event.messages).messages as any }; // Non-checkpoint path only refreshes the shared snapshot; the message list
// is returned to Pi unchanged — masking is append-only via context_edit.
projectMessages(ctx, runtime, event.messages);
return undefined;
}); });
pi.on("before_provider_headers", (event, ctx) => { pi.on("before_provider_headers", (event, ctx) => {
@@ -387,7 +463,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("session_before_compact", async (event, ctx) => { pi.on("session_before_compact", async (event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
if (!runtime.config.enabled) return undefined; if (!runtime.config.enabled) return undefined;
drainSemantic(ctx, runtime);
const { snapshot, checkpoint } = branchMessages( const { snapshot, checkpoint } = branchMessages(
ctx, ctx,
runtime, runtime,
@@ -395,8 +470,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.reason === "overflow" && event.willRetry, event.reason === "overflow" && event.willRetry,
); );
if (!shouldCompact(event.reason, snapshot, runtime.config)) return { cancel: true };
const model = ctx.model; const model = ctx.model;
if (!model) return undefined; if (!model) return undefined;
if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) { if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) {
@@ -412,7 +485,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.signal, event.signal,
event.customInstructions, event.customInstructions,
undefined, undefined,
runtime.config.semantic.thinking, runtime.config.summary.thinking,
); );
return { return {
compaction: { compaction: {
@@ -463,7 +536,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
replacementHistory: buildReplacementHistory( replacementHistory: buildReplacementHistory(
input, input,
remote.compactionItem, remote.compactionItem,
Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * Math.max(0.05, runtime.config.targetRatio - 0.3)))), Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * 0.25))),
), ),
portableSummary: portableSummary(snapshot.messages), portableSummary: portableSummary(snapshot.messages),
sourceFingerprint: snapshot.fingerprint, sourceFingerprint: snapshot.fingerprint,
@@ -504,7 +577,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.signal, event.signal,
event.preparation.customInstructions, event.preparation.customInstructions,
undefined, undefined,
runtime.config.semantic.thinking, runtime.config.summary.thinking,
undefined, undefined,
undefined, undefined,
undefined, undefined,
@@ -531,38 +604,64 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.registerTool({ pi.registerTool({
name: QUERY_TOOL_NAME, name: QUERY_TOOL_NAME,
label: "Query Compacted Tool History", label: "Query Compacted Tool History",
description: "Recover original tool outputs by source key or tool call id after pi-compaction replaced them with bounded summaries.", description: "Recover the original output of a tool result that pi-compaction cleared from context, by tool call id.",
promptSnippet: "Recover original compacted tool evidence", promptSnippet: "Recover original tool output cleared from context",
promptGuidelines: [ promptGuidelines: [
"Use context_tree_query only when a pi-compaction summary lacks evidence needed for the current task.", "Use context_tree_query when you see a \"[pi-compaction] Earlier\" placeholder and still need the original tool output.",
], ],
parameters: Type.Object({ parameters: Type.Object({
toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }), toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }),
}), }),
async execute(_id, params, _signal, _update, ctx) { async execute(_id, params, _signal, _update, ctx) {
const records = service.query(ctx.sessionManager.getSessionId(), params.toolCallIds); const branch = ctx.sessionManager.getBranch() as SessionEntry[];
const blocks = records.map((record: ToolHistoryRecord) => { const calls = new Map<string, { toolName: string; args: unknown }>();
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, records.length))); const results = new Map<string, any>();
const args = truncateHead(JSON.stringify(record.args), { maxLines: 20, maxBytes: 2_048 }).content; for (const entry of branch) {
const truncated = truncateHead(record.rawText, { const message = (entry as any)?.message;
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, records.length))), if ((entry as any)?.type !== "message" || !message) continue;
if (message.role === "assistant" && Array.isArray(message.content)) {
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
calls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.arguments ?? block.input ?? block.args ?? {},
});
}
}
} else if (message.role === "toolResult" && typeof message.toolCallId === "string") {
results.set(message.toolCallId, message);
}
}
const found: string[] = [];
const blocks: string[] = [];
for (const id of params.toolCallIds) {
const result = results.get(id);
if (!result) continue;
found.push(id);
const call = calls.get(id) ?? { toolName: String(result.toolName ?? "unknown"), args: {} };
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, params.toolCallIds.length)));
const args = truncateHead(JSON.stringify(call.args ?? {}), { maxLines: 20, maxBytes: 2_048 }).content;
const rawText = contentText(result.content);
const truncated = truncateHead(rawText, {
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, params.toolCallIds.length))),
maxBytes: perResultBytes, maxBytes: perResultBytes,
}); });
return [ blocks.push([
`## ${record.sourceKey}`, `## ${id}`,
`Tool: ${record.toolName}`, `Tool: ${call.toolName}`,
`Args: ${args}`, `Args: ${args}`,
`Status: ${record.isError ? "ERROR" : "OK"}`, `Status: ${result.isError ? "ERROR" : "OK"}`,
"", "",
truncated.content, truncated.content,
...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []), ...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []),
].join("\n"); ].join("\n"));
}); }
const missing = params.toolCallIds.filter((id: string) => !records.some((record) => record.sourceKey === id || record.toolCallId === id)); const missing = params.toolCallIds.filter((id: string) => !results.has(id));
for (const id of missing) blocks.push(`## ${id}\n(not found)`); for (const id of missing) blocks.push(`## ${id}\n(not found)`);
return { return {
content: [{ type: "text", text: blocks.join("\n\n---\n\n") }], content: [{ type: "text", text: blocks.join("\n\n---\n\n") }],
details: { sourceKeys: records.map((record) => record.sourceKey), missing }, details: { toolCallIds: found, missing },
}; };
}, },
}); });
@@ -576,12 +675,11 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
`Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`, `Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`,
`Raw: ${metrics.rawTokens}/${metrics.contextWindow}`, `Raw: ${metrics.rawTokens}/${metrics.contextWindow}`,
`Reclaimed: ${metrics.reclaimedTokens}`, `Reclaimed: ${metrics.reclaimedTokens}`,
`Segments: ${metrics.segmentCount}; semantic jobs: ${metrics.semanticJobs}; generation: ${metrics.generation}`, `Segments: ${metrics.segmentCount}; masked tool results: ${metrics.maskedResults}; generation: ${metrics.generation}`,
].join("\n") ].join("\n")
: "pi-compaction has no projection snapshot yet."; : "pi-compaction has no projection snapshot yet.";
if (ctx.hasUI) ctx.ui.notify(text, "info"); if (ctx.hasUI) ctx.ui.notify(text, "info");
else process.stderr.write(`${text}\n`); else process.stderr.write(`${text}\n`);
}, },
}); });
} }
+58 -35
View File
@@ -12,27 +12,29 @@ export const CONFIG_PATH = fileURLToPath(
export const DEFAULT_CONFIG: PiCompactionConfig = { export const DEFAULT_CONFIG: PiCompactionConfig = {
enabled: true, enabled: true,
softRatio: 0.72, masking: {
hardRatio: 0.9,
targetRatio: 0.55,
deterministic: {
enabled: true, enabled: true,
maxStubTokens: 1_024, triggerTokens: 200_000,
triggerRatio: 0.6,
targetTokens: 100_000,
targetRatio: 0.35,
minClearTokens: 50_000,
keepRecentTurns: 10,
minEntryTokens: 256,
excludeTools: ["ask_user_question"],
}, },
semantic: { summary: {
enabled: true, triggerTokens: 400_000,
model: "default", triggerRatio: 0.9,
thinking: "low", thinking: "low",
concurrency: 1,
minInputChars: 4_000,
maxInputChars: 24_000,
maxRetries: 2,
}, },
native: { native: {
enabled: true, enabled: true,
}, },
}; };
const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh"];
function ratio(value: unknown, fallback: number): number { function ratio(value: unknown, fallback: number): number {
return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback; return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback;
} }
@@ -41,46 +43,67 @@ function integer(value: unknown, fallback: number, min = 0, max = Number.MAX_SAF
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback; return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback;
} }
function positiveInteger(value: unknown, fallback: number): number {
return integer(value, fallback, 1);
}
function object(value: unknown): Record<string, unknown> { function object(value: unknown): Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value) return value !== null && typeof value === "object" && !Array.isArray(value)
? value as Record<string, unknown> ? value as Record<string, unknown>
: {}; : {};
} }
function stringArray(value: unknown, fallback: string[]): string[] {
if (!Array.isArray(value) || !value.every((entry) => typeof entry === "string")) return fallback;
return value.map((entry) => entry.trim()).filter((entry) => entry.length > 0);
}
export function normalizeConfig(value: unknown): PiCompactionConfig { export function normalizeConfig(value: unknown): PiCompactionConfig {
const input = object(value); const input = object(value);
const deterministic = object(input.deterministic); const masking = object(input.masking);
const semantic = object(input.semantic); const summary = object(input.summary);
const native = object(input.native); const native = object(input.native);
const softRatio = ratio(input.softRatio, DEFAULT_CONFIG.softRatio);
const hardRatio = Math.max(softRatio + 0.01, ratio(input.hardRatio, DEFAULT_CONFIG.hardRatio));
const targetRatio = Math.min(softRatio - 0.01, ratio(input.targetRatio, DEFAULT_CONFIG.targetRatio));
const thinking = ["off", "minimal", "low", "medium", "high", "xhigh"].includes(String(semantic.thinking))
? semantic.thinking as PiCompactionConfig["semantic"]["thinking"]
: DEFAULT_CONFIG.semantic.thinking;
return { const normalized: PiCompactionConfig = {
enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled, enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled,
softRatio, masking: {
hardRatio: Math.min(0.99, hardRatio), enabled: typeof masking.enabled === "boolean" ? masking.enabled : DEFAULT_CONFIG.masking.enabled,
targetRatio: Math.max(0.1, targetRatio), triggerTokens: positiveInteger(masking.triggerTokens, DEFAULT_CONFIG.masking.triggerTokens),
deterministic: { triggerRatio: ratio(masking.triggerRatio, DEFAULT_CONFIG.masking.triggerRatio),
enabled: typeof deterministic.enabled === "boolean" ? deterministic.enabled : DEFAULT_CONFIG.deterministic.enabled, targetTokens: positiveInteger(masking.targetTokens, DEFAULT_CONFIG.masking.targetTokens),
maxStubTokens: integer(deterministic.maxStubTokens, DEFAULT_CONFIG.deterministic.maxStubTokens, 32, 4_096), targetRatio: ratio(masking.targetRatio, DEFAULT_CONFIG.masking.targetRatio),
minClearTokens: positiveInteger(masking.minClearTokens, DEFAULT_CONFIG.masking.minClearTokens),
keepRecentTurns: integer(masking.keepRecentTurns, DEFAULT_CONFIG.masking.keepRecentTurns, 1, 100),
minEntryTokens: integer(masking.minEntryTokens, DEFAULT_CONFIG.masking.minEntryTokens, 0, 100_000),
excludeTools: stringArray(masking.excludeTools, DEFAULT_CONFIG.masking.excludeTools),
}, },
semantic: { summary: {
enabled: typeof semantic.enabled === "boolean" ? semantic.enabled : DEFAULT_CONFIG.semantic.enabled, triggerTokens: positiveInteger(summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens),
model: typeof semantic.model === "string" && semantic.model.trim() ? semantic.model.trim() : DEFAULT_CONFIG.semantic.model, triggerRatio: ratio(summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio),
thinking, thinking: THINKING_LEVELS.includes(String(summary.thinking))
concurrency: integer(semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency, 1, 4), ? summary.thinking as PiCompactionConfig["summary"]["thinking"]
minInputChars: integer(semantic.minInputChars, DEFAULT_CONFIG.semantic.minInputChars, 0, 1_000_000), : DEFAULT_CONFIG.summary.thinking,
maxInputChars: integer(semantic.maxInputChars, DEFAULT_CONFIG.semantic.maxInputChars, 1_000, 1_000_000),
maxRetries: integer(semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries, 0, 5),
}, },
native: { native: {
enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled, enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled,
}, },
}; };
// Trigger/target must keep their hysteresis ordering; fix up rather than
// letting a degenerate pair mask-or-compact every turn.
if (normalized.masking.targetTokens >= normalized.masking.triggerTokens) {
normalized.masking.targetTokens = Math.floor(normalized.masking.triggerTokens / 2);
}
if (normalized.masking.targetRatio >= normalized.masking.triggerRatio) {
normalized.masking.targetRatio = normalized.masking.triggerRatio / 2;
}
if (normalized.summary.triggerTokens <= normalized.masking.triggerTokens) {
normalized.summary.triggerTokens = normalized.masking.triggerTokens * 2;
}
if (normalized.summary.triggerRatio <= normalized.masking.triggerRatio) {
normalized.summary.triggerRatio = Math.min(0.99, normalized.masking.triggerRatio + 0.01);
}
return normalized;
} }
export async function loadConfig( export async function loadConfig(
+257
View File
@@ -0,0 +1,257 @@
import {
buildContextEntries,
sessionEntryToContextMessages,
type ContextEditEntryDraft,
} from "@earendil-works/pi-coding-agent";
import { contentText, timestampOf } from "./segment-index.js";
import { effectiveThreshold } from "./token-accounting.js";
import type { MaskingConfig } from "./types.js";
// Every placeholder written through a context_edit starts with this prefix so
// re-masking, ledger rebuilds, and the model-facing recovery hint all key off
// one marker.
export const MASK_PREFIX = "[pi-compaction] Earlier ";
const MAX_PLACEHOLDER_ARGS_CHARS = 300;
const EDITABLE_ROLES = new Set(["user", "assistant", "toolResult", "custom"]);
export interface MaskingProjectedEntry {
sourceEntry: { id?: string; type?: string; message?: any };
messages: any[];
}
export interface MaskedResult {
toolCallId: string;
entryId: string;
placeholder: string;
originalTokens: number;
placeholderTokens: number;
resultTimestamp: number;
}
export interface MaskingPlan {
drafts: ContextEditEntryDraft[];
clearedTokens: number;
masked: MaskedResult[];
}
export interface MaskedLedgerEntry {
placeholder: string;
originalTokens: number;
placeholderTokens: number;
resultTimestamp: number;
}
const EMPTY_PLAN: MaskingPlan = { drafts: [], clearedTokens: 0, masked: [] };
function json(value: unknown): string {
try {
return JSON.stringify(value) ?? "undefined";
} catch {
return "[unserializable]";
}
}
function maskPlaceholder(
toolName: string,
args: unknown,
toolCallId: string,
originalTokens: number,
overflowId?: string,
): string {
let argsText = json(args);
if (argsText.length > MAX_PLACEHOLDER_ARGS_CHARS) {
argsText = `${argsText.slice(0, MAX_PLACEHOLDER_ARGS_CHARS)}…`;
}
// `details.overflow.id` is pi-overflow's public contract — read the field
// only, never import its implementation.
const overflowHint = typeof overflowId === "string" && overflowId.length > 0
? ` Full original stored: overflow_query {"id":"${overflowId}"}.`
: "";
return `${MASK_PREFIX}${toolName} result cleared to save context (~${originalTokens} tokens). ` +
`Call args: ${argsText}. Re-run the call if still needed, or recover the original with ` +
`context_tree_query {"toolCallIds":["${toolCallId}"]}.${overflowHint}`;
}
function isSkillRead(toolName: string, args: unknown): boolean {
if (toolName !== "read" || args === null || typeof args !== "object") return false;
const record = args as Record<string, unknown>;
const raw = record.path ?? record.filePath ?? record.file;
if (typeof raw !== "string") return false;
return (raw.split(/[\\/]/).pop() ?? raw) === "SKILL.md";
}
function toolResultCalls(entries: readonly MaskingProjectedEntry[]): Map<string, { toolName: string; args: unknown }> {
const calls = new Map<string, { toolName: string; args: unknown }>();
for (const entry of entries) {
for (const message of entry.messages ?? []) {
if (message?.role !== "assistant" || !Array.isArray(message.content)) continue;
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
calls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.arguments ?? block.input ?? block.args ?? {},
});
}
}
}
}
return calls;
}
export function planMasking(params: {
contextEntries: readonly MaskingProjectedEntry[];
currentTokens: number;
contextWindow: number;
masking: MaskingConfig;
estimate: (message: any) => number;
}): MaskingPlan {
const { masking } = params;
const triggerEff = effectiveThreshold(masking.triggerTokens, masking.triggerRatio, params.contextWindow);
if (!masking.enabled || params.currentTokens < triggerEff) return EMPTY_PLAN;
const targetEff = effectiveThreshold(masking.targetTokens, masking.targetRatio, params.contextWindow);
const minClear = Math.min(masking.minClearTokens, Math.max(0, triggerEff - targetEff));
const entries = params.contextEntries;
// The last keepRecentTurns assistant messages and everything after them stay
// untouched; when fewer turns exist, the whole context is protected.
let protectedIndex = 0;
let turns = 0;
for (let index = entries.length - 1; index >= 0; index--) {
const entry = entries[index];
if (entry?.sourceEntry?.type === "message" && (entry.messages ?? []).some((m) => m?.role === "assistant")) {
turns++;
if (turns >= masking.keepRecentTurns) {
protectedIndex = index;
break;
}
}
}
const calls = toolResultCalls(entries);
const needed = params.currentTokens - targetEff;
const masked: MaskedResult[] = [];
let clearedTokens = 0;
for (let index = 0; index < protectedIndex && clearedTokens < needed; index++) {
const entry = entries[index];
const entryId = entry?.sourceEntry?.id;
if (entry?.sourceEntry?.type !== "message" || typeof entryId !== "string") continue;
for (const message of entry.messages ?? []) {
if (clearedTokens >= needed) break;
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
const call = calls.get(message.toolCallId);
const toolName = call?.toolName ?? String(message.toolName ?? "unknown");
if (masking.excludeTools.includes(toolName)) continue;
const args = call?.args ?? {};
if (isSkillRead(toolName, args)) continue;
if (contentText(message.content).startsWith(MASK_PREFIX)) continue;
const originalTokens = params.estimate(message);
if (originalTokens < masking.minEntryTokens) continue;
const overflowId = (message.details as { overflow?: { id?: unknown } } | undefined)?.overflow?.id;
const placeholder = maskPlaceholder(
toolName,
args,
message.toolCallId,
originalTokens,
typeof overflowId === "string" ? overflowId : undefined,
);
const placeholderTokens = params.estimate({ ...message, content: [{ type: "text", text: placeholder }] });
clearedTokens += originalTokens - placeholderTokens;
masked.push({
toolCallId: message.toolCallId,
entryId,
placeholder,
originalTokens,
placeholderTokens,
resultTimestamp: timestampOf(message.timestamp),
});
}
}
if (clearedTokens < minClear) return EMPTY_PLAN;
return {
drafts: masked.map((result) => ({
type: "context_edit",
targetId: result.entryId,
replacement: { content: result.placeholder },
})),
clearedTokens,
masked,
};
}
/**
* Rebuild the masked-results ledger from a session branch after hydration:
* the latest context_edit per target wins, only edits carrying this
* extension's placeholder prefix count as masked, and only targets that are
* still part of the active context (post-compaction kept set) are counted.
*/
export function rebuildMaskedLedger(
branch: readonly any[],
leafId: string | null | undefined,
estimate: (message: any) => number,
): Map<string, MaskedLedgerEntry> {
const latestEdits = new Map<string, any>();
const targets = new Map<string, any>();
for (const entry of branch) {
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
latestEdits.set(entry.targetId, entry);
} else if (entry?.type === "message" && typeof entry.id === "string") {
targets.set(entry.id, entry);
}
}
const contextIds = new Set(
buildContextEntries(branch as any, leafId).map((entry) => entry.id),
);
const masked = new Map<string, MaskedLedgerEntry>();
for (const [targetId, edit] of latestEdits) {
if (!contextIds.has(targetId)) continue;
const text = contentText(edit.replacement?.content);
if (!text.startsWith(MASK_PREFIX)) continue;
const message = targets.get(targetId)?.message;
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
masked.set(message.toolCallId, {
placeholder: text,
originalTokens: estimate(message),
placeholderTokens: estimate({ ...message, content: [{ type: "text", text }] }),
resultTimestamp: timestampOf(message.timestamp),
});
}
return masked;
}
/**
* Apply the latest context_edit per target the way buildSessionProjection does
* for message entries `sessionEntryToContextMessages` alone would miss: null
* omits the entry, a string replacement becomes one text block for
* assistant/toolResult roles.
*/
export function applyContextEdits(entries: readonly any[]): any[] {
const edits = new Map<string, { content: unknown } | null>();
for (const entry of entries) {
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
edits.set(entry.targetId, entry.replacement ?? null);
}
}
const projected: any[] = [];
for (const entry of entries) {
const messages = sessionEntryToContextMessages(entry);
const edit = edits.get(entry?.id);
if (edit === undefined) {
projected.push(...messages);
continue;
}
if (edit === null) continue;
for (const message of messages) {
if (!EDITABLE_ROLES.has(message?.role)) {
projected.push(message);
continue;
}
const content = (message.role === "assistant" || message.role === "toolResult")
&& typeof edit.content === "string"
? [{ type: "text", text: edit.content }]
: edit.content;
projected.push({ ...message, content });
}
}
return projected;
}
+31 -246
View File
@@ -1,16 +1,8 @@
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { estimateTokens } from "@earendil-works/pi-coding-agent"; import { estimateTokens } from "@earendil-works/pi-coding-agent";
import { import type { ProjectionSnapshot, UsageBaseline } from "./types.js";
COMPACTION_CUSTOM_TYPE,
COMPACTION_VERSION,
type PiCompactionConfig,
type ProjectionSnapshot,
type SegmentSummaryRecord,
type ToolHistoryRecord,
type UsageBaseline,
} from "./types.js";
function timestampOf(value: unknown): number { export function timestampOf(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value; if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string") { if (typeof value === "string") {
const parsed = Date.parse(value); const parsed = Date.parse(value);
@@ -19,10 +11,6 @@ function timestampOf(value: unknown): number {
return 0; return 0;
} }
export function sourceKey(toolCallId: string, timestamp: unknown): string {
return `${toolCallId}@${timestampOf(timestamp)}`;
}
export function contentText(content: unknown): string { export function contentText(content: unknown): string {
if (typeof content === "string") return content; if (typeof content === "string") return content;
if (!Array.isArray(content)) return ""; if (!Array.isArray(content)) return "";
@@ -37,42 +25,6 @@ function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20); return createHash("sha256").update(value).digest("hex").slice(0, 20);
} }
function json(value: unknown): string {
try {
return JSON.stringify(value) ?? "undefined";
} catch {
return "[unserializable]";
}
}
function boundedEvidence(text: string, maxChars: number): string {
const normalized = text.replace(/\u001b\[[0-9;]*m/g, "").trim();
if (normalized.length <= maxChars) return normalized;
const marker = `\n… ${normalized.length - maxChars} chars omitted …\n`;
const available = Math.max(0, maxChars - marker.length);
const head = Math.ceil(available * 0.7);
const tail = Math.max(0, available - head);
return `${normalized.slice(0, head)}${marker}${tail ? normalized.slice(-tail) : ""}`;
}
export function deterministicSummary(params: {
toolName: string;
args: unknown;
rawText: string;
isError: boolean;
maxStubTokens: number;
}): string {
const maxChars = params.maxStubTokens * 4;
const argsText = boundedEvidence(json(params.args), Math.min(600, Math.floor(maxChars * 0.25)));
const evidenceBudget = Math.max(128, maxChars - argsText.length - 120);
const evidence = boundedEvidence(params.rawText || "(no output)", evidenceBudget);
return [
`[pi-compaction tool=${params.toolName} status=${params.isError ? "error" : "ok"}]`,
`Args: ${argsText}`,
`Evidence:\n${evidence}`,
].join("\n");
}
function messageKey(message: any): string { function messageKey(message: any): string {
const role = String(message?.role ?? "unknown"); const role = String(message?.role ?? "unknown");
const timestamp = timestampOf(message?.timestamp); const timestamp = timestampOf(message?.timestamp);
@@ -85,24 +37,14 @@ function messageKey(message: any): string {
return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`; return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`;
} }
function isSummaryRecord(value: unknown): value is SegmentSummaryRecord { /**
if (!value || typeof value !== "object") return false; * Incremental context projection: tracks the branch's message list, estimates
const record = value as Partial<SegmentSummaryRecord>; * tokens, and maintains the usage-backed effective-token baseline. Projection
return record.version === COMPACTION_VERSION * is identity — pi-compaction never rewrites messages on this path; context
&& record.kind === "segment-summary" * pressure reduction happens through append-only context_edit entries.
&& typeof record.sourceKey === "string" */
&& typeof record.toolCallId === "string"
&& typeof record.resultTimestamp === "number"
&& typeof record.toolName === "string"
&& typeof record.sourceHash === "string"
&& typeof record.deterministicSummary === "string";
}
export class SegmentIndex { export class SegmentIndex {
private readonly records = new Map<string, ToolHistoryRecord>();
private readonly ids = new Map<string, string[]>();
private generation = 0; private generation = 0;
private semanticJobs = 0;
private usageBaseline?: UsageBaseline; private usageBaseline?: UsageBaseline;
private sourceMessages: any[] = []; private sourceMessages: any[] = [];
private sourceKeys: string[] = []; private sourceKeys: string[] = [];
@@ -117,67 +59,13 @@ export class SegmentIndex {
this.sessionId = sessionId; this.sessionId = sessionId;
} }
hydrate(branch: readonly any[], config: PiCompactionConfig): void { hydrate(): void {
this.reset(); this.reset();
if (!config.enabled) return;
const summaries = new Map<string, SegmentSummaryRecord>();
for (const entry of branch) {
if (entry?.type === "custom" && entry.customType === COMPACTION_CUSTOM_TYPE && isSummaryRecord(entry.data)) {
summaries.set(entry.data.sourceKey, entry.data);
}
}
const pendingCalls = new Map<string, { toolName: string; args: unknown }>();
for (const entry of branch) {
if (entry?.type !== "message") continue;
const message = entry.message;
if (message?.role === "assistant" && Array.isArray(message.content)) {
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
pendingCalls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.input ?? block.arguments ?? block.args ?? {},
});
}
}
} else if (message?.role === "toolResult" && typeof message.toolCallId === "string") {
const call = pendingCalls.get(message.toolCallId) ?? {
toolName: String(message.toolName ?? "unknown"),
args: {},
};
const key = sourceKey(message.toolCallId, message.timestamp);
const rawText = contentText(message.content);
const persisted = summaries.get(key);
const record: ToolHistoryRecord = {
sourceKey: key,
toolCallId: message.toolCallId,
resultTimestamp: timestampOf(message.timestamp),
toolName: call.toolName,
args: call.args,
isError: Boolean(message.isError),
rawText,
sourceHash: hash(`${call.toolName}\0${json(call.args)}\0${rawText}`),
deterministicSummary: persisted?.deterministicSummary ?? deterministicSummary({
toolName: call.toolName,
args: call.args,
rawText,
isError: Boolean(message.isError),
maxStubTokens: config.deterministic.maxStubTokens,
}),
semanticSummary: persisted?.semanticSummary,
};
this.put(record);
pendingCalls.delete(message.toolCallId);
}
}
this.generation++; this.generation++;
} }
reset(): void { reset(): void {
this.records.clear();
this.ids.clear();
this.generation = 0; this.generation = 0;
this.semanticJobs = 0;
this.usageBaseline = undefined; this.usageBaseline = undefined;
this.sourceMessages = []; this.sourceMessages = [];
this.sourceKeys = []; this.sourceKeys = [];
@@ -187,86 +75,6 @@ export class SegmentIndex {
this.cached = undefined; this.cached = undefined;
} }
private put(record: ToolHistoryRecord): void {
this.records.set(record.sourceKey, record);
const keys = this.ids.get(record.toolCallId) ?? [];
if (!keys.includes(record.sourceKey)) keys.push(record.sourceKey);
this.ids.set(record.toolCallId, keys);
}
captureTurn(message: any, toolResults: any[], config: PiCompactionConfig): ToolHistoryRecord[] {
if (!config.enabled || !config.deterministic.enabled || !Array.isArray(message?.content)) return [];
const calls = new Map<string, any>();
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") calls.set(block.id, block);
}
const captured: ToolHistoryRecord[] = [];
for (const result of toolResults ?? []) {
if (!result || typeof result.toolCallId !== "string") continue;
const call = calls.get(result.toolCallId);
if (!call) continue;
const rawText = contentText(result.content);
const toolName = String(call.name ?? result.toolName ?? "unknown");
const args = call.input ?? call.arguments ?? call.args ?? {};
const key = sourceKey(result.toolCallId, result.timestamp);
const record: ToolHistoryRecord = {
sourceKey: key,
toolCallId: result.toolCallId,
resultTimestamp: timestampOf(result.timestamp),
toolName,
args,
isError: Boolean(result.isError),
rawText,
sourceHash: hash(`${toolName}\0${json(args)}\0${rawText}`),
deterministicSummary: deterministicSummary({
toolName,
args,
rawText,
isError: Boolean(result.isError),
maxStubTokens: config.deterministic.maxStubTokens,
}),
};
this.put(record);
captured.push(record);
}
if (captured.length) {
this.generation++;
this.cached = undefined;
}
return captured;
}
applySemantic(sourceKeyValue: string, sourceHash: string, summary: string): SegmentSummaryRecord | undefined {
const record = this.records.get(sourceKeyValue);
if (!record || record.sourceHash !== sourceHash || !summary.trim()) return undefined;
record.semanticSummary = summary.trim();
this.generation++;
this.cached = undefined;
return this.toPersisted(record);
}
toPersisted(record: ToolHistoryRecord): SegmentSummaryRecord {
const effective = record.semanticSummary ?? record.deterministicSummary;
return {
version: COMPACTION_VERSION,
kind: "segment-summary",
sourceKey: record.sourceKey,
toolCallId: record.toolCallId,
resultTimestamp: record.resultTimestamp,
toolName: record.toolName,
sourceHash: record.sourceHash,
deterministicSummary: record.deterministicSummary,
...(record.semanticSummary ? { semanticSummary: record.semanticSummary } : {}),
rawChars: record.rawText.length,
effectiveChars: effective.length,
};
}
setSemanticJobs(count: number): void {
this.semanticJobs = Math.max(0, count);
if (this.cached) this.cached = { ...this.cached, semanticJobs: this.semanticJobs };
}
recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void { recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void {
if (!Number.isFinite(tokens) || tokens <= 0) return; if (!Number.isFinite(tokens) || tokens <= 0) return;
this.usageBaseline = { tokens, messageEstimate, environmentHash }; this.usageBaseline = { tokens, messageEstimate, environmentHash };
@@ -277,11 +85,27 @@ export class SegmentIndex {
environmentHash: string; environmentHash: string;
environmentTokens: number; environmentTokens: number;
contextWindow: number; contextWindow: number;
maskedResults: number;
reclaimedTokens: number;
}): ProjectionSnapshot { }): ProjectionSnapshot {
const sameTail = messages.length === this.sourceMessages.length const sameTail = messages.length === this.sourceMessages.length
&& messages.length > 0 && messages.length > 0
&& messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1]; && messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1];
if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) return this.cached; if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) {
if (this.cached.maskedResults === params.maskedResults
&& this.cached.reclaimedTokens === params.reclaimedTokens) {
return this.cached;
}
// Masking stats changed but the message list did not: publish a new
// immutable snapshot object instead of re-projecting or mutating.
this.cached = {
...this.cached,
maskedResults: params.maskedResults,
reclaimedTokens: params.reclaimedTokens,
rawTokens: this.cached.effectiveTokens + params.reclaimedTokens,
};
return this.cached;
}
const appendOnly = this.sourceMessages.length > 0 const appendOnly = this.sourceMessages.length > 0
&& messages.length >= this.sourceMessages.length && messages.length >= this.sourceMessages.length
@@ -301,17 +125,14 @@ export class SegmentIndex {
for (let index = start; index < messages.length; index++) { for (let index = start; index < messages.length; index++) {
const message = messages[index]; const message = messages[index];
const projected = this.projectMessage(message);
this.sourceMessages.push(message); this.sourceMessages.push(message);
this.sourceKeys.push(messageKey(message)); this.sourceKeys.push(messageKey(message));
this.projectedMessages.push(projected); this.projectedMessages.push(message);
this.rawEstimates.push(estimateTokens(message as any)); this.rawEstimates.push(estimateTokens(message as any));
this.effectiveEstimates.push(estimateTokens(projected as any)); this.effectiveEstimates.push(estimateTokens(message as any));
} }
const rawMessageTokens = this.rawEstimates.reduce((sum, value) => sum + value, 0);
const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0); const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0);
const rawTokens = rawMessageTokens + params.environmentTokens;
const baseline = this.usageBaseline; const baseline = this.usageBaseline;
const usageBacked = baseline?.environmentHash === params.environmentHash; const usageBacked = baseline?.environmentHash === params.environmentHash;
const effectiveTokens = usageBacked const effectiveTokens = usageBacked
@@ -329,55 +150,19 @@ export class SegmentIndex {
environmentHash: params.environmentHash, environmentHash: params.environmentHash,
messages: this.projectedMessages, messages: this.projectedMessages,
messageEstimate: effectiveMessageTokens, messageEstimate: effectiveMessageTokens,
rawTokens, rawTokens: effectiveTokens + params.reclaimedTokens,
effectiveTokens, effectiveTokens,
reclaimedTokens: Math.max(0, rawTokens - effectiveTokens), reclaimedTokens: params.reclaimedTokens,
contextWindow: params.contextWindow, contextWindow: params.contextWindow,
confidence, confidence,
lowerBound, lowerBound,
upperBound, upperBound,
segmentCount: this.sourceMessages.length, segmentCount: this.sourceMessages.length,
semanticJobs: this.semanticJobs, maskedResults: params.maskedResults,
}; };
return this.cached; return this.cached;
} }
private projectMessage(message: any): any {
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") return message;
const record = this.records.get(sourceKey(message.toolCallId, message.timestamp));
if (!record) return message;
const original = contentText(message.content);
const summary = record.semanticSummary ?? record.deterministicSummary;
const projected = `${summary}\nSource: ${record.sourceKey}; recover with context_tree_query.`;
if (projected.length >= original.length) return message;
return {
...message,
content: [{ type: "text", text: projected }],
};
}
query(ids: string[]): ToolHistoryRecord[] {
const found: ToolHistoryRecord[] = [];
const seen = new Set<string>();
for (const id of ids) {
const direct = this.records.get(id);
const keys = direct ? [id] : this.ids.get(id) ?? [];
for (const key of keys) {
if (seen.has(key)) continue;
const record = this.records.get(key);
if (record) {
found.push(record);
seen.add(key);
}
}
}
return found;
}
allRecords(): ToolHistoryRecord[] {
return [...this.records.values()];
}
getSnapshot(): ProjectionSnapshot | undefined { getSnapshot(): ProjectionSnapshot | undefined {
return this.cached; return this.cached;
} }
-189
View File
@@ -1,189 +0,0 @@
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import type { PiCompactionConfig, ToolHistoryRecord } from "./types.js";
interface Job {
sessionId: string;
sourceKey: string;
sourceHash: string;
toolName: string;
args: unknown;
isError: boolean;
rawText: string;
ctx: ExtensionContext;
config: PiCompactionConfig;
controller: AbortController;
}
export interface SemanticResult {
sessionId: string;
sourceKey: string;
sourceHash: string;
summary: string;
}
function modelFor(config: PiCompactionConfig, ctx: ExtensionContext): any {
if (config.semantic.model === "default") return ctx.model;
const slash = config.semantic.model.indexOf("/");
if (slash <= 0) return ctx.model;
return ctx.modelRegistry.find(
config.semantic.model.slice(0, slash),
config.semantic.model.slice(slash + 1),
) ?? ctx.model;
}
function serialize(job: Job): string {
const args = (() => {
try { return JSON.stringify(job.args); } catch { return "[unserializable]"; }
})();
const max = job.config.semantic.maxInputChars;
const raw = job.rawText.length <= max
? job.rawText
: `${job.rawText.slice(0, Math.floor(max * 0.7))}\n… omitted …\n${job.rawText.slice(-Math.floor(max * 0.3))}`;
return `Summarize this completed coding-agent tool call for future context.\nKeep concrete outcomes, paths, identifiers, values, and exact errors.\nUse at most 6 concise bullets. Do not give advice and do not call tools.\n\nTool: ${job.toolName}\nStatus: ${job.isError ? "ERROR" : "OK"}\nArgs: ${args}\nOutput:\n${raw}`;
}
async function run(job: Job): Promise<string> {
const model = modelFor(job.config, job.ctx);
if (!model) throw new Error("No summarizer model selected");
const message = await job.ctx.modelRegistry.complete(model, {
messages: [{
role: "user",
content: [{ type: "text", text: serialize(job) }],
timestamp: Date.now(),
}],
} as any, {
signal: job.controller.signal,
reasoningEffort: job.config.semantic.thinking === "off" ? undefined : job.config.semantic.thinking,
} as any);
if (message.stopReason === "length" || message.stopReason === "error" || message.stopReason === "aborted") {
throw new Error(message.errorMessage ?? `Semantic compaction stopped: ${message.stopReason}`);
}
const text = message.content
.flatMap((part: any) => part.type === "text" && typeof part.text === "string" ? [part.text] : [])
.join("\n")
.trim();
if (!text) throw new Error("Semantic compaction returned an empty summary");
return text;
}
function abortableDelay(ms: number, signal: AbortSignal): Promise<void> {
if (signal.aborted) return Promise.reject(new Error("Semantic compaction aborted"));
return new Promise((resolve, reject) => {
const timer = setTimeout(done, ms);
function done(): void {
signal.removeEventListener("abort", aborted);
resolve();
}
function aborted(): void {
clearTimeout(timer);
signal.removeEventListener("abort", aborted);
reject(new Error("Semantic compaction aborted"));
}
signal.addEventListener("abort", aborted, { once: true });
});
}
export class SemanticWorker {
private readonly queue: Job[] = [];
private readonly active = new Map<string, Job>();
private readonly ready: SemanticResult[] = [];
private readonly queuedKeys = new Set<string>();
enqueue(sessionId: string, record: ToolHistoryRecord, ctx: ExtensionContext, config: PiCompactionConfig): void {
if (!config.semantic.enabled || record.semanticSummary || record.rawText.length < config.semantic.minInputChars) return;
const key = `${sessionId}:${record.sourceKey}:${record.sourceHash}`;
if (this.queuedKeys.has(key)) return;
this.queuedKeys.add(key);
this.queue.push({
sessionId,
sourceKey: record.sourceKey,
sourceHash: record.sourceHash,
toolName: record.toolName,
args: record.args,
isError: record.isError,
rawText: record.rawText,
ctx,
config,
controller: new AbortController(),
});
this.pump(config.semantic.concurrency);
}
private pump(limit: number): void {
while (this.active.size < limit && this.queue.length > 0) {
const job = this.queue.shift()!;
const key = `${job.sessionId}:${job.sourceKey}:${job.sourceHash}`;
this.active.set(key, job);
void this.execute(key, job, limit);
}
}
private async execute(key: string, job: Job, limit: number): Promise<void> {
let lastError: unknown;
try {
for (let attempt = 0; attempt <= job.config.semantic.maxRetries; attempt++) {
try {
const summary = await run(job);
if (!job.controller.signal.aborted) {
this.ready.push({
sessionId: job.sessionId,
sourceKey: job.sourceKey,
sourceHash: job.sourceHash,
summary,
});
}
return;
} catch (error) {
lastError = error;
if (job.controller.signal.aborted || attempt === job.config.semantic.maxRetries) break;
await abortableDelay(Math.min(8_000, 1_000 * 2 ** attempt), job.controller.signal);
}
}
if (!job.controller.signal.aborted) {
console.error(`pi-compaction: semantic summary failed for ${job.sourceKey}:`, lastError);
}
} finally {
this.active.delete(key);
this.queuedKeys.delete(key);
this.pump(limit);
}
}
drain(sessionId: string): SemanticResult[] {
const drained: SemanticResult[] = [];
for (let index = this.ready.length - 1; index >= 0; index--) {
if (this.ready[index]!.sessionId !== sessionId) continue;
drained.push(this.ready[index]!);
this.ready.splice(index, 1);
}
return drained.reverse();
}
count(sessionId: string): number {
return this.queue.filter((job) => job.sessionId === sessionId).length
+ [...this.active.values()].filter((job) => job.sessionId === sessionId).length;
}
cancelSession(sessionId: string): void {
for (const job of this.active.values()) {
if (job.sessionId === sessionId) job.controller.abort();
}
for (let index = this.queue.length - 1; index >= 0; index--) {
if (this.queue[index]!.sessionId === sessionId) {
const job = this.queue[index]!;
this.queuedKeys.delete(`${job.sessionId}:${job.sourceKey}:${job.sourceHash}`);
this.queue.splice(index, 1);
}
}
for (let index = this.ready.length - 1; index >= 0; index--) {
if (this.ready[index]!.sessionId === sessionId) this.ready.splice(index, 1);
}
}
cancelAll(): void {
for (const job of this.active.values()) job.controller.abort();
this.queue.length = 0;
this.ready.length = 0;
this.queuedKeys.clear();
}
}
+5 -5
View File
@@ -2,8 +2,8 @@ import {
SERVICE_SYMBOL, SERVICE_SYMBOL,
type CompactionMetrics, type CompactionMetrics,
type CompactionService, type CompactionService,
type MaskedResultRecord,
type ProjectionSnapshot, type ProjectionSnapshot,
type ToolHistoryRecord,
} from "./types.js"; } from "./types.js";
interface GlobalWithCompactionService { interface GlobalWithCompactionService {
@@ -14,11 +14,11 @@ const root = globalThis as GlobalWithCompactionService;
export class CompactionServiceRegistry implements CompactionService { export class CompactionServiceRegistry implements CompactionService {
private readonly snapshots = new Map<string, ProjectionSnapshot>(); private readonly snapshots = new Map<string, ProjectionSnapshot>();
private readonly queryHandlers = new Map<string, (ids: string[]) => ToolHistoryRecord[]>(); private readonly queryHandlers = new Map<string, (ids: string[]) => MaskedResultRecord[]>();
setSession( setSession(
sessionId: string, sessionId: string,
query: (ids: string[]) => ToolHistoryRecord[], query: (ids: string[]) => MaskedResultRecord[],
): void { ): void {
this.queryHandlers.set(sessionId, query); this.queryHandlers.set(sessionId, query);
} }
@@ -58,11 +58,11 @@ export class CompactionServiceRegistry implements CompactionService {
percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0, percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0,
confidence: snapshot.confidence, confidence: snapshot.confidence,
segmentCount: snapshot.segmentCount, segmentCount: snapshot.segmentCount,
semanticJobs: snapshot.semanticJobs, maskedResults: snapshot.maskedResults,
}; };
} }
query(sessionId: string, ids: string[]): ToolHistoryRecord[] { query(sessionId: string, ids: string[]): MaskedResultRecord[] {
return this.queryHandlers.get(sessionId)?.(ids) ?? []; return this.queryHandlers.get(sessionId)?.(ids) ?? [];
} }
} }
+4 -12
View File
@@ -1,13 +1,5 @@
import type { PiCompactionConfig, ProjectionSnapshot } from "./types.js"; // Effective trigger for one configured (tokens, ratio) pair: the absolute
// token figure wins on large windows, the ratio caps it on small ones.
export type CompactionReason = "manual" | "threshold" | "overflow"; export function effectiveThreshold(tokens: number, ratio: number, contextWindow: number): number {
return contextWindow > 0 ? Math.min(tokens, Math.floor(ratio * contextWindow)) : tokens;
export function shouldCompact(
reason: CompactionReason,
snapshot: Pick<ProjectionSnapshot, "lowerBound" | "contextWindow">,
config: Pick<PiCompactionConfig, "hardRatio">,
): boolean {
if (reason === "manual" || reason === "overflow") return true;
if (snapshot.contextWindow <= 0) return true;
return snapshot.lowerBound / snapshot.contextWindow >= config.hardRatio;
} }
+30 -48
View File
@@ -5,58 +5,33 @@ export const COMPACTION_VERSION = 1 as const;
export const QUERY_TOOL_NAME = "context_tree_query"; export const QUERY_TOOL_NAME = "context_tree_query";
export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1"); export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1");
export interface MaskingConfig {
enabled: boolean;
triggerTokens: number;
triggerRatio: number;
targetTokens: number;
targetRatio: number;
minClearTokens: number;
keepRecentTurns: number;
minEntryTokens: number;
excludeTools: string[];
}
export interface SummaryConfig {
triggerTokens: number;
triggerRatio: number;
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
}
export interface PiCompactionConfig { export interface PiCompactionConfig {
enabled: boolean; enabled: boolean;
softRatio: number; masking: MaskingConfig;
hardRatio: number; summary: SummaryConfig;
targetRatio: number;
deterministic: {
enabled: boolean;
maxStubTokens: number;
};
semantic: {
enabled: boolean;
model: string;
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
concurrency: number;
minInputChars: number;
maxInputChars: number;
maxRetries: number;
};
native: { native: {
enabled: boolean; enabled: boolean;
}; };
} }
export interface SegmentSummaryRecord {
version: typeof COMPACTION_VERSION;
kind: "segment-summary";
sourceKey: string;
toolCallId: string;
resultTimestamp: number;
toolName: string;
sourceHash: string;
deterministicSummary: string;
semanticSummary?: string;
rawChars: number;
effectiveChars: number;
}
export type PiCompactionRecord = SegmentSummaryRecord;
export interface ToolHistoryRecord {
sourceKey: string;
toolCallId: string;
resultTimestamp: number;
toolName: string;
args: unknown;
isError: boolean;
rawText: string;
sourceHash: string;
deterministicSummary: string;
semanticSummary?: string;
}
export interface UsageBaseline { export interface UsageBaseline {
tokens: number; tokens: number;
messageEstimate: number; messageEstimate: number;
@@ -78,7 +53,7 @@ export interface ProjectionSnapshot {
lowerBound: number; lowerBound: number;
upperBound: number; upperBound: number;
segmentCount: number; segmentCount: number;
semanticJobs: number; maskedResults: number;
codexItems?: ResponseItem[]; codexItems?: ResponseItem[];
} }
@@ -99,6 +74,13 @@ export interface PortableCheckpointDetails {
sourceFingerprint: string; sourceFingerprint: string;
} }
/** What the service returns for one masked tool result — the same placeholder the model sees. */
export interface MaskedResultRecord {
toolCallId: string;
resultTimestamp: number;
deterministicSummary: string;
}
export interface CompactionMetrics { export interface CompactionMetrics {
sessionId: string; sessionId: string;
generation: number; generation: number;
@@ -109,11 +91,11 @@ export interface CompactionMetrics {
percent: number; percent: number;
confidence: ProjectionSnapshot["confidence"]; confidence: ProjectionSnapshot["confidence"];
segmentCount: number; segmentCount: number;
semanticJobs: number; maskedResults: number;
} }
export interface CompactionService { export interface CompactionService {
getSnapshot(sessionId: string): ProjectionSnapshot | undefined; getSnapshot(sessionId: string): ProjectionSnapshot | undefined;
getMetrics(sessionId: string): CompactionMetrics | undefined; getMetrics(sessionId: string): CompactionMetrics | undefined;
query(sessionId: string, ids: string[]): ToolHistoryRecord[]; query(sessionId: string, ids: string[]): MaskedResultRecord[];
} }
+68 -20
View File
@@ -2,34 +2,82 @@ import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts"; import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts";
test("normalizes one greenfield piCompaction config without legacy aliases", () => { test("normalizes a full pi-compaction config", () => {
const config = normalizeConfig({ const config = normalizeConfig({
enabled: false, enabled: false,
softRatio: 0.7, masking: {
hardRatio: 0.91, enabled: false,
targetRatio: 0.5, triggerTokens: 150_000,
deterministic: { enabled: false, maxStubTokens: 512 }, triggerRatio: 0.5,
semantic: { enabled: false, concurrency: 2, maxRetries: 0 }, targetTokens: 60_000,
targetRatio: 0.2,
minClearTokens: 30_000,
keepRecentTurns: 5,
minEntryTokens: 512,
excludeTools: ["bash", "read"],
},
summary: { triggerTokens: 300_000, triggerRatio: 0.8, thinking: "medium" },
native: { enabled: false }, native: { enabled: false },
}); });
assert.equal(config.enabled, false); assert.equal(config.enabled, false);
assert.equal(config.deterministic.enabled, false); assert.deepEqual(config.masking, {
assert.equal(config.deterministic.maxStubTokens, 512); enabled: false,
assert.equal(config.semantic.enabled, false); triggerTokens: 150_000,
assert.equal(config.semantic.maxRetries, 0); triggerRatio: 0.5,
targetTokens: 60_000,
targetRatio: 0.2,
minClearTokens: 30_000,
keepRecentTurns: 5,
minEntryTokens: 512,
excludeTools: ["bash", "read"],
});
assert.equal(config.summary.triggerTokens, 300_000);
assert.equal(config.summary.triggerRatio, 0.8);
assert.equal(config.summary.thinking, "medium");
assert.equal(config.native.enabled, false); assert.equal(config.native.enabled, false);
}); });
test("invalid ratios and bounds fail safe to bounded defaults", () => { test("invalid fields fail safe to bounded defaults", () => {
const config = normalizeConfig({ const config = normalizeConfig({
softRatio: 2, enabled: "yes",
hardRatio: -1, masking: {
targetRatio: 0.99, triggerTokens: -1,
semantic: { concurrency: 50, maxRetries: -1 }, triggerRatio: 2,
targetTokens: "a lot",
targetRatio: 0,
minClearTokens: 0,
keepRecentTurns: 500,
minEntryTokens: -3,
excludeTools: "read",
},
summary: { triggerTokens: Number.NaN, triggerRatio: 1.5, thinking: "insane" },
native: { enabled: 1 },
}); });
assert.equal(config.softRatio, DEFAULT_CONFIG.softRatio); assert.equal(config.enabled, DEFAULT_CONFIG.enabled);
assert.ok(config.targetRatio < config.softRatio); assert.deepEqual(config.masking, DEFAULT_CONFIG.masking);
assert.ok(config.hardRatio > config.softRatio); assert.equal(config.summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens);
assert.equal(config.semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency); assert.equal(config.summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio);
assert.equal(config.semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries); assert.equal(config.summary.thinking, DEFAULT_CONFIG.summary.thinking);
assert.equal(config.native.enabled, DEFAULT_CONFIG.native.enabled);
});
test("masking target is corrected below trigger and summary above masking", () => {
const config = normalizeConfig({
masking: { triggerTokens: 80_000, triggerRatio: 0.5, targetTokens: 120_000, targetRatio: 0.9 },
summary: { triggerTokens: 50_000, triggerRatio: 0.4 },
});
assert.equal(config.masking.targetTokens, 40_000);
assert.equal(config.masking.targetRatio, 0.25);
assert.equal(config.summary.triggerTokens, 160_000);
assert.ok(config.summary.triggerRatio > config.masking.triggerRatio);
});
test("minClearTokens and excludeTools accept valid overrides", () => {
const config = normalizeConfig({
masking: { minClearTokens: 1, keepRecentTurns: 1, minEntryTokens: 0, excludeTools: [] },
});
assert.equal(config.masking.minClearTokens, 1);
assert.equal(config.masking.keepRecentTurns, 1);
assert.equal(config.masking.minEntryTokens, 0);
assert.deepEqual(config.masking.excludeTools, []);
}); });
+225 -61
View File
@@ -1,7 +1,8 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { performance } from "node:perf_hooks";
import piCompaction from "../index.ts"; import piCompaction from "../index.ts";
import { getCompactionService } from "../src/service.ts";
import { MASK_PREFIX } from "../src/masking.ts";
function harness() { function harness() {
const handlers = new Map<string, Function[]>(); const handlers = new Map<string, Function[]>();
@@ -24,90 +25,253 @@ function harness() {
return { pi, handlers, tools, commands, entries }; return { pi, handlers, tools, commands, entries };
} }
function context(branch: any[] = []) { function context(branch: any[] = [], tokens = 1_000) {
return { const compactCalls: any[] = [];
const sessionId = `session-${Math.random().toString(36).slice(2)}`;
const ctx = {
sessionManager: { sessionManager: {
getSessionId: () => "session-test", getSessionId: () => sessionId,
getBranch: () => branch, getBranch: () => branch,
getLeafId: () => "leaf-1",
}, },
model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 }, model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 },
modelRegistry: {}, modelRegistry: {},
getSystemPrompt: () => "system", getSystemPrompt: () => "system",
getContextUsage: () => ({ tokens: 1_000, contextWindow: 100_000, percent: 1 }), getContextUsage: () => ({ tokens, contextWindow: 100_000, percent: 1 }),
compact(options: unknown) { compactCalls.push(options); },
isIdle: () => true,
hasUI: false, hasUI: false,
abort() {}, abort() {},
} as any; } as any;
return { ctx, compactCalls };
} }
test("one runtime projects tool evidence and recovers it through context_tree_query", async () => { function projectedEntry(id: string, message: any) {
return { sourceEntry: { id, type: "message", message }, messages: [message] };
}
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
return {
role: "assistant",
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
timestamp: id,
};
}
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
return {
role: "toolResult",
toolCallId,
toolName,
content: [{ type: "text", text }],
timestamp: id,
};
}
// Default config: masking trigger = min(200K, 60% of window); at the harness
// 100K window that is 60K tokens and the target is 35K, so 60K used needs
// ~25K tokens of maskable results outside the 10-turn protected window.
function pressurizedContextEntries() {
const entries = [
projectedEntry("a1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }])),
projectedEntry("r1", toolResultMessage("r1", "c1", `failure evidence\n${"x".repeat(120_000)}\nstack tail`)),
];
for (let index = 0; index < 10; index++) {
entries.push(projectedEntry(`keep-${index}`, assistantMessage(`keep-${index}`, [])));
}
return entries;
}
test("turn_end masks old tool results under pressure and preserves prior drafts", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context([], 60_000);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const assistant = { const prior = { type: "custom", customType: "other-extension", data: {} };
role: "assistant", const result = await h.handlers.get("turn_end")![0]!({
content: [{ type: "toolCall", id: "call-1", name: "bash", arguments: { command: "npm test" } }], entries: [prior],
timestamp: 10, message: assistantMessage("a1", []),
}; toolResults: [],
const result = { context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
role: "toolResult", }, ctx);
toolCallId: "call-1",
toolName: "bash",
content: [{ type: "text", text: `failure evidence\n${"x".repeat(12_000)}\nstack tail` }],
isError: true,
timestamp: 11,
};
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx);
assert.match(projected.messages[1].content[0].text, /pi-compaction tool=bash/); assert.ok(result?.entries);
const cacheStart = performance.now(); assert.strictEqual(result.entries[0], prior);
for (let iteration = 0; iteration < 1_000; iteration++) { const edits = result.entries.slice(1);
await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx); assert.equal(edits.length, 1);
} assert.equal(edits[0]!.type, "context_edit");
const cacheAverageMs = (performance.now() - cacheStart) / 1_000; assert.equal(edits[0]!.targetId, "r1");
assert.ok(cacheAverageMs < 2, `full context handler cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`); assert.ok(edits[0]!.replacement.content.startsWith(MASK_PREFIX));
assert.equal(h.entries.length, 1); // Masking is append-only context_edit; nothing is written as custom entries.
assert.equal(h.entries[0]!.customType, "pi-compaction"); assert.equal(h.entries.length, 0);
const query = h.tools.get("context_tree_query");
const recovered = await query.execute("q", { toolCallIds: ["call-1"] }, new AbortController().signal, undefined, ctx);
assert.match(recovered.content[0].text, /failure evidence/);
await h.handlers.get("session_shutdown")![0]!({}, ctx); await h.handlers.get("session_shutdown")![0]!({}, ctx);
}); });
test("small tool results are passed through without a stub wrapper", async () => { test("turn_end below the masking trigger returns undefined", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context([], 40_000);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const assistant = { const result = await h.handlers.get("turn_end")![0]!({
role: "assistant", entries: [],
content: [{ type: "toolCall", id: "call-small", name: "bash", arguments: { command: "ls" } }], message: assistantMessage("a1", []),
timestamp: 20, toolResults: [],
}; context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
const result = { }, ctx);
role: "toolResult", assert.equal(result, undefined);
toolCallId: "call-small", await h.handlers.get("session_shutdown")![0]!({}, ctx);
toolName: "bash", });
content: [{ type: "text", text: "ok\nfile-a.ts\nfile-b.ts" }],
timestamp: 21,
};
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx); test("context hook returns messages untouched on the non-checkpoint path", async () => {
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx); const h = harness();
piCompaction(h.pi);
const { ctx } = context();
await h.handlers.get("session_start")![0]!({}, ctx);
assert.strictEqual(projected.messages[1].content[0].text, "ok\nfile-a.ts\nfile-b.ts"); const messages = [assistantMessage("a1", []), toolResultMessage("r1", "c1", "visible\nfile-a.ts")];
assert.doesNotMatch(projected.messages[1].content[0].text, /pi-compaction tool=/); const projected = await h.handlers.get("context")![0]!({ messages }, ctx);
assert.equal(projected, undefined);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("agent_settled compacts once above the summary threshold and never repeats on the same leaf", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx, compactCalls } = context([], 95_000);
await h.handlers.get("session_start")![0]!({}, ctx);
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 1);
assert.equal(typeof compactCalls[0].onError, "function");
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 1);
ctx.sessionManager.getLeafId = () => "leaf-2";
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 2);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("agent_settled stays idle below the summary threshold", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx, compactCalls } = context([], 50_000);
await h.handlers.get("session_start")![0]!({}, ctx);
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 0);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("context_tree_query recovers the original tool output from the branch", async () => {
const h = harness();
piCompaction(h.pi);
const branch = [
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "failure evidence\nstack tail") },
];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
// Original evidence stays recoverable even when the message is passed through.
const query = h.tools.get("context_tree_query")!; const query = h.tools.get("context_tree_query")!;
const recovered = await query.execute("q", { toolCallIds: ["call-small"] }, new AbortController().signal, undefined, ctx); const recovered = await query.execute("q", { toolCallIds: ["c1"] }, new AbortController().signal, undefined, ctx);
assert.match(recovered.content[0].text, /file-a\.ts/); assert.match(recovered.content[0].text, /failure evidence/);
assert.match(recovered.content[0].text, /stack tail/);
assert.deepEqual(recovered.details.missing, []);
const missing = await query.execute("q", { toolCallIds: ["nope"] }, new AbortController().signal, undefined, ctx);
assert.match(missing.content[0].text, /not found/);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("service query returns placeholders only for masked tool call ids", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx } = context([], 60_000);
await h.handlers.get("session_start")![0]!({}, ctx);
const sessionId = ctx.sessionManager.getSessionId();
await h.handlers.get("turn_end")![0]!({
entries: [],
message: assistantMessage("a1", []),
toolResults: [],
context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
}, ctx);
const records = getCompactionService()!.query(sessionId, ["c1", "never-masked"]);
assert.equal(records.length, 1);
assert.equal(records[0]!.toolCallId, "c1");
assert.ok(records[0]!.deterministicSummary.startsWith(MASK_PREFIX));
assert.equal(typeof records[0]!.resultTimestamp, "number");
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("hydrating a branch with this extension's context_edit rebuilds masking metrics", async () => {
const h = harness();
piCompaction(h.pi);
const placeholder = `${MASK_PREFIX}bash result cleared to save context (~1_000 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["c1"]}.`;
const branch = [
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "x".repeat(8_000)) },
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: placeholder } },
];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
const sessionId = ctx.sessionManager.getSessionId();
await h.handlers.get("context")![0]!({ messages: [toolResultMessage("r1", "c1", "x".repeat(8_000))] }, ctx);
const metrics = getCompactionService()!.getMetrics(sessionId);
assert.equal(metrics!.maskedResults, 1);
assert.ok(metrics!.reclaimedTokens > 0);
assert.ok(metrics!.rawTokens > metrics!.effectiveTokens);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("session_before_compact no longer cancels threshold compactions", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx } = context();
await h.handlers.get("session_start")![0]!({}, ctx);
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: false, error: "no credentials" });
const result = await h.handlers.get("session_before_compact")![0]!({
reason: "threshold", branchEntries: [], preparation: { firstKeptEntryId: "u1" },
signal: new AbortController().signal,
}, ctx);
assert.equal(result, undefined);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("native checkpoint projection applies context edits made after the checkpoint", async () => {
const h = harness();
piCompaction(h.pi);
const details = {
kind: "pi-compaction",
version: 1,
checkpoint: "native",
modelKey: "test:test:model",
replacementHistory: [{ type: "compaction", encrypted_content: "opaque" }],
portableSummary: "checkpoint summary",
sourceFingerprint: "fp",
};
const branch = [
{ type: "compaction", id: "c1", parentId: null, timestamp: "t", summary: "marker", firstKeptEntryId: "r1", tokensBefore: 1_000, details },
{ type: "message", id: "r1", parentId: "c1", timestamp: "t", message: toolResultMessage("r1", "c1", "ORIGINAL-CONTENT-HERE") },
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: `${MASK_PREFIX}bash result cleared to save context (~5 tokens). Call args: {}.` } },
{ type: "message", id: "u1", parentId: "e1", timestamp: "t", message: { role: "user", content: "next question", timestamp: 30 } },
] as any[];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
const serialized = JSON.stringify(projected.messages);
assert.doesNotMatch(serialized, /ORIGINAL-CONTENT-HERE/);
assert.match(serialized, /result cleared to save context/);
assert.match(serialized, /next question/);
await h.handlers.get("session_shutdown")![0]!({}, ctx); await h.handlers.get("session_shutdown")![0]!({}, ctx);
}); });
@@ -127,7 +291,7 @@ test("model switching after a native checkpoint projects the portable checkpoint
{ type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details }, { type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details },
{ type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } }, { type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } },
] as any[]; ] as any[];
const ctx = context(branch); const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx); const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
assert.match(projected.messages[0].content[0].text, /decisions and files/); assert.match(projected.messages[0].content[0].text, /decisions and files/);
@@ -143,7 +307,7 @@ for (const provider of ["sub-api", "openai-codex"]) {
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Remember the test decision.", timestamp: 1 }, message: { role: "user", content: "Remember the test decision.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
ctx.model = { ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses", provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
id: "selected-model", input: ["text"], contextWindow: 100_000, id: "selected-model", input: ["text"], contextWindow: 100_000,
@@ -231,7 +395,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Test only.", timestamp: 1 }, message: { role: "user", content: "Test only.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" }; ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" };
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" }); ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" });
const responses = [ const responses = [
@@ -266,7 +430,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => { test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context();
try { try {
for (const model of [ for (const model of [
{ provider: "other", api: "openai-responses", id: "test" }, { provider: "other", api: "openai-responses", id: "test" },
@@ -302,7 +466,7 @@ test("native runtime delegates authentication to each provider without blocking
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 }, message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
const provider = scenario.provider ?? "sub-api"; const provider = scenario.provider ?? "sub-api";
ctx.model = { ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses", provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
+209
View File
@@ -0,0 +1,209 @@
import test from "node:test";
import assert from "node:assert/strict";
import { MASK_PREFIX, planMasking, rebuildMaskedLedger } from "../src/masking.ts";
import { effectiveThreshold } from "../src/token-accounting.ts";
import type { MaskingConfig } from "../src/types.ts";
const MASKING: MaskingConfig = {
enabled: true,
triggerTokens: 1_000,
triggerRatio: 0.9,
targetTokens: 400,
targetRatio: 0.35,
minClearTokens: 200,
keepRecentTurns: 2,
minEntryTokens: 10,
excludeTools: ["ask_user_question"],
};
const estimate = (message: any): number => Math.ceil(JSON.stringify(message).length / 4);
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
return {
role: "assistant",
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
timestamp: id,
};
}
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
return {
role: "toolResult",
toolCallId,
toolName,
content: [{ type: "text", text }],
timestamp: id,
};
}
function entry(id: string, message: any, messages = [message]) {
return { sourceEntry: { id, type: "message", message }, messages };
}
function protect(turns = MASKING.keepRecentTurns) {
return Array.from({ length: turns }, (_unused, index) =>
entry(`recent-${index}`, { role: "assistant", content: [{ type: "text", text: "ok" }], timestamp: `recent-${index}` }));
}
test("below the effective trigger plans no drafts", () => {
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "ls" } }])),
entry("e2", toolResultMessage("r1", "c1", "x".repeat(8_000))),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 999, contextWindow: 0, masking: MASKING, estimate });
assert.equal(plan.drafts.length, 0);
assert.equal(plan.masked.length, 0);
});
test("above the trigger masks oldest first until context fits under target", () => {
const entries = [
entry("e1", assistantMessage("a1", [
{ id: "c1", name: "bash", args: { command: "one" } },
{ id: "c2", name: "bash", args: { command: "two" } },
{ id: "c3", name: "bash", args: { command: "three" } },
])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000))),
entry("e4", toolResultMessage("r3", "c3", "c".repeat(4_000))),
entry("e5", toolResultMessage("r4", "c4", "recent result".repeat(200))),
...protect(),
];
// needed = 2_000 - 400 = 1_600; each result clears ~980 tokens.
const plan = planMasking({ contextEntries: entries, currentTokens: 2_000, contextWindow: 0, masking: MASKING, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e2", "e3"]);
});
test("results inside the recent keepRecentTurns window are never masked", () => {
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(4_000))),
entry("recent-a", assistantMessage("ra", [{ id: "rc", name: "bash", args: {} }])),
entry("recent-r", toolResultMessage("rr", "rc", "b".repeat(4_000))),
entry("recent-b", assistantMessage("rb", [])),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e1"]);
});
test("excluded tools, SKILL.md reads, already-masked and small results are skipped", () => {
const entries = [
entry("e1", assistantMessage("a1", [
{ id: "c1", name: "ask_user_question", args: { question: "x" } },
{ id: "c2", name: "read", args: { path: "/repo/skills/foo/SKILL.md" } },
{ id: "c3", name: "bash", args: { command: "old" } },
{ id: "c4", name: "bash", args: { command: "tiny" } },
])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000), "ask_user_question")),
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000), "read")),
entry("e4", toolResultMessage("r3", "c3", `${MASK_PREFIX}bash result cleared to save context (~900 tokens).`), ),
entry("e5", toolResultMessage("r4", "c4", "tiny")),
entry("e6", toolResultMessage("r5", "c5", "d".repeat(4_000))),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 100, minEntryTokens: 100 };
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e6"]);
assert.equal(plan.masked[0]!.toolCallId, "c5");
});
test("when clearable volume stays under minClear nothing is masked", () => {
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(600))),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
// ~130 tokens clearable, below minClearTokens (200) → not worth invalidating cache.
assert.equal(plan.drafts.length, 0);
});
test("placeholder is deterministic, names the tool and call id, and truncates args at 300 chars", () => {
const longArgs = { command: `run ${"v".repeat(600)}` };
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: longArgs }])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 1 };
const first = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
const second = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.equal(first.drafts.length, 1);
const placeholder = first.masked[0]!.placeholder;
assert.equal(placeholder, second.masked[0]!.placeholder);
assert.ok(placeholder.startsWith(MASK_PREFIX));
assert.match(placeholder, /bash result cleared/);
assert.match(placeholder, /"toolCallIds":\["c1"\]/);
const argsText = placeholder.match(/Call args: (.*)\. Re-run/)?.[1] ?? "";
assert.ok(argsText.length <= 301, `args excerpt ${argsText.length} exceeded 300+ellipsis`);
assert.ok(placeholder.includes("…"));
});
test("a masked result carrying details.overflow.id gets an overflow_query hint in the placeholder", () => {
const withOverflow = {
...toolResultMessage("r1", "c1", "a".repeat(4_000)),
details: { overflow: { id: "0123456789ab", totalLines: 5, totalBytes: 4000, truncated: true } },
};
const withoutOverflow = toolResultMessage("r2", "c2", "b".repeat(4_000));
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash" }, { id: "c2", name: "bash" }])),
entry("e2", withOverflow),
entry("e3", withoutOverflow),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 1 };
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.equal(plan.masked.length, 2);
const first = plan.masked.find((m) => m.toolCallId === "c1")!.placeholder;
const second = plan.masked.find((m) => m.toolCallId === "c2")!.placeholder;
assert.ok(first.includes(` Full original stored: overflow_query {"id":"0123456789ab"}.`));
assert.ok(!second.includes("overflow_query"));
});
test("the small-window ratio caps both trigger and target", () => {
const masking: MaskingConfig = {
...MASKING,
triggerTokens: 200_000,
triggerRatio: 0.6,
targetTokens: 100_000,
targetRatio: 0.35,
minClearTokens: 50_000,
};
assert.equal(effectiveThreshold(masking.triggerTokens, masking.triggerRatio, 272_000), 163_200);
assert.equal(effectiveThreshold(masking.targetTokens, masking.targetRatio, 272_000), 95_200);
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(300_000))),
...protect(),
];
const below = planMasking({ contextEntries: entries, currentTokens: 163_199, contextWindow: 272_000, masking, estimate });
assert.equal(below.drafts.length, 0);
const above = planMasking({ contextEntries: entries, currentTokens: 163_200, contextWindow: 272_000, masking, estimate });
assert.equal(above.drafts.length, 1);
});
test("ledger rebuild counts only masked targets still inside the active context", () => {
const placeholder = (id: string) =>
`${MASK_PREFIX}bash result cleared to save context (~100 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["${id}"]}.`;
const branch = [
{ type: "message", id: "u1", parentId: null, timestamp: "t", message: { role: "user", content: "start", timestamp: 1 } },
{ type: "message", id: "r-old", parentId: "u1", timestamp: "t", message: toolResultMessage("m-old", "c-old", "o".repeat(2_000)) },
{ type: "context_edit", id: "e-old", parentId: "r-old", timestamp: "t", targetId: "r-old", replacement: { content: placeholder("c-old") } },
{ type: "message", id: "m-keep", parentId: "e-old", timestamp: "t", message: { role: "user", content: "kept question", timestamp: 2 } },
{ type: "compaction", id: "c1", parentId: "m-keep", timestamp: "t", summary: "summary", firstKeptEntryId: "m-keep", tokensBefore: 9_999 },
{ type: "message", id: "a-keep", parentId: "c1", timestamp: "t", message: assistantMessage("a-keep", []) },
{ type: "message", id: "r-new", parentId: "a-keep", timestamp: "t", message: toolResultMessage("m-new", "c-new", "n".repeat(2_000)) },
{ type: "context_edit", id: "e-new", parentId: "r-new", timestamp: "t", targetId: "r-new", replacement: { content: placeholder("c-new") } },
];
const ledger = rebuildMaskedLedger(branch, "e-new", estimate);
assert.equal(ledger.size, 1);
assert.ok(ledger.has("c-new"));
assert.ok(!ledger.has("c-old"));
});
test("user and assistant messages are never masked", () => {
const entries = [
entry("e1", { role: "user", content: "u".repeat(8_000), timestamp: "u1" }),
entry("e2", { role: "assistant", content: [{ type: "text", text: "a".repeat(8_000) }], timestamp: "a1" }),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 9_000, contextWindow: 0, masking: { ...MASKING, keepRecentTurns: 1, minClearTokens: 1 }, estimate });
assert.equal(plan.drafts.length, 0);
});
+3 -2
View File
@@ -10,9 +10,9 @@ function message(index: number) {
test("cached projection and append-only delta stay inside the hot-path budget", () => { test("cached projection and append-only delta stay inside the hot-path budget", () => {
const index = new SegmentIndex("perf"); const index = new SegmentIndex("perf");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const messages = Array.from({ length: 10_000 }, (_, value) => message(value)); const messages = Array.from({ length: 10_000 }, (_, value) => message(value));
const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000 }; const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
index.project(messages, params); index.project(messages, params);
const cacheStart = performance.now(); const cacheStart = performance.now();
@@ -24,6 +24,7 @@ test("cached projection and append-only delta stay inside the hot-path budget",
index.project(appended, params); index.project(appended, params);
const deltaMs = performance.now() - deltaStart; const deltaMs = performance.now() - deltaStart;
console.log(`cache-hit average ${cacheAverageMs.toFixed(3)}ms; delta projection ${deltaMs.toFixed(3)}ms`);
assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`); assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`);
assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`); assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`);
}); });
+69 -40
View File
@@ -1,7 +1,6 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { DEFAULT_CONFIG } from "../src/config.ts"; import { SegmentIndex } from "../src/segment-index.ts";
import { SegmentIndex, deterministicSummary } from "../src/segment-index.ts";
const assistant = { const assistant = {
role: "assistant", role: "assistant",
@@ -17,66 +16,96 @@ const result = {
timestamp: 11, timestamp: 11,
}; };
test("deterministic summary is bounded and preserves head/tail evidence", () => { // Regression: a ~14.5K-char read result projected under a 1M window must reach
const summary = deterministicSummary({ // the model byte-for-byte unchanged — masking only happens via context_edit
toolName: "bash", // drafts under real pressure, never on the hot projection path.
args: { command: "npm test" }, test("projection is byte-for-byte identity for a 300-line read result under a 1M window", () => {
rawText: result.content[0]!.text, const readBody = Array.from({ length: 300 }, (_unused, index) => `line ${index + 1}: ${"y".repeat(38)}`).join("\n");
isError: true, const readResult = {
maxStubTokens: 256, role: "toolResult",
toolCallId: "call-read",
toolName: "read",
content: [{ type: "text", text: readBody }],
timestamp: 12,
};
assert.ok(readBody.length > 14_000 && readBody.length < 15_000, `fixture ${readBody.length} chars`);
const index = new SegmentIndex("session-1");
index.hydrate();
const snapshot = index.project([assistant, readResult], {
environmentHash: "env",
environmentTokens: 100,
contextWindow: 1_000_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
assert.match(summary, /tool=bash status=error/);
assert.match(summary, /failure/); assert.strictEqual(snapshot.messages[1], readResult);
assert.match(summary, /stack tail/); assert.strictEqual(snapshot.messages[1]!.content[0].text, readBody);
assert.ok(summary.length <= 1_100); assert.ok(snapshot.messages[1]!.content[0].text.includes("line 150:"));
assert.equal(snapshot.reclaimedTokens, 0);
assert.equal(snapshot.maskedResults, 0);
}); });
test("projection incrementally replaces tool results and caches identical input", () => { test("projection caches identical input and handles append-only deltas", () => {
const index = new SegmentIndex("session-1"); const index = new SegmentIndex("session-2");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const records = index.captureTurn(assistant, [result], DEFAULT_CONFIG);
assert.equal(records.length, 1);
const messages = [assistant, result]; const messages = [assistant, result];
const first = index.project(messages, { const params = { environmentHash: "env", environmentTokens: 100, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
environmentHash: "env", const first = index.project(messages, params);
environmentTokens: 100, const second = index.project(messages, params);
contextWindow: 300_000,
});
const second = index.project(messages, {
environmentHash: "env",
environmentTokens: 100,
contextWindow: 300_000,
});
assert.strictEqual(first, second); assert.strictEqual(first, second);
assert.ok(first.effectiveTokens < first.rawTokens); assert.ok(first.rawTokens > 0);
assert.match(first.messages[1]!.content[0].text, /context_tree_query/);
const record = records[0]!; const appended = index.project([...messages, { role: "user", content: "next", timestamp: 13 }], params);
assert.ok(index.applySemantic(record.sourceKey, record.sourceHash, "- npm test failed\n- stack tail")); assert.notStrictEqual(appended, first);
const semantic = index.project(messages, { assert.equal(appended.messages.length, 3);
assert.strictEqual(appended.messages[1], result);
});
test("changed masking stats publish a new snapshot without mutating the old one", () => {
const index = new SegmentIndex("session-stats");
index.hydrate();
const params = {
environmentHash: "env", environmentHash: "env",
environmentTokens: 100, environmentTokens: 100,
contextWindow: 300_000, contextWindow: 300_000,
}); maskedResults: 0,
assert.notStrictEqual(semantic, first); reclaimedTokens: 0,
assert.match(semantic.messages[1]!.content[0].text, /npm test failed/); };
const first = index.project([assistant, result], params);
const same = index.project([assistant, result], params);
assert.strictEqual(same, first);
const updated = index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 });
assert.notStrictEqual(updated, first);
assert.equal(updated.maskedResults, 1);
assert.equal(updated.reclaimedTokens, 500);
assert.equal(updated.rawTokens, updated.effectiveTokens + 500);
// The previously published object stays immutable.
assert.equal(first.maskedResults, 0);
assert.equal(first.reclaimedTokens, 0);
// Unchanged stats keep returning the same cached object.
assert.strictEqual(index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 }), updated);
}); });
test("usage-backed baseline is adjusted only by projected delta", () => { test("usage-backed baseline is adjusted only by projected delta", () => {
const index = new SegmentIndex("session-2"); const index = new SegmentIndex("session-3");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const initial = index.project([assistant], { const initial = index.project([assistant], {
environmentHash: "env", environmentHash: "env",
environmentTokens: 50, environmentTokens: 50,
contextWindow: 100_000, contextWindow: 100_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
index.recordUsage(1_000, initial.messageEstimate, "env"); index.recordUsage(1_000, initial.messageEstimate, "env");
const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], { const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], {
environmentHash: "env", environmentHash: "env",
environmentTokens: 50, environmentTokens: 50,
contextWindow: 100_000, contextWindow: 100_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
assert.equal(next.confidence, "usage-backed"); assert.equal(next.confidence, "usage-backed");
assert.ok(next.effectiveTokens >= 1_000); assert.ok(next.effectiveTokens >= 1_000);
+11 -13
View File
@@ -1,19 +1,17 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { shouldCompact } from "../src/token-accounting.ts"; import { effectiveThreshold } from "../src/token-accounting.ts";
test("raw pressure cannot trigger compaction while effective lower-bound stays below 90%", () => { test("absolute token threshold wins on large windows", () => {
const config = { hardRatio: 0.9 }; assert.equal(effectiveThreshold(200_000, 0.6, 1_000_000), 200_000);
const snapshot = { lowerBound: 231_000, contextWindow: 300_000 };
assert.equal(shouldCompact("threshold", snapshot, config), false);
assert.equal(shouldCompact("manual", snapshot, config), true);
assert.equal(shouldCompact("overflow", snapshot, config), true);
}); });
test("effective lower-bound at the configured threshold compacts", () => { test("window ratio caps the threshold on small windows", () => {
assert.equal(shouldCompact( assert.equal(effectiveThreshold(200_000, 0.6, 272_000), 163_200);
"threshold", assert.equal(effectiveThreshold(100_000, 0.35, 272_000), 95_200);
{ lowerBound: 270_000, contextWindow: 300_000 }, });
{ hardRatio: 0.9 },
), true); test("a missing context window falls back to the absolute threshold", () => {
assert.equal(effectiveThreshold(200_000, 0.6, 0), 200_000);
assert.equal(effectiveThreshold(200_000, 0.6, -1), 200_000);
}); });
-31
View File
@@ -1,31 +0,0 @@
You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.
Available tools:
- read: Read file contents
- bash: Execute bash commands (ls, grep, find, etc.)
- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call
- write: Create or overwrite files
In addition to the tools above, you may have access to other custom tools depending on the project.
Guidelines:
- Use bash for file operations like ls, rg, find
- Use read to examine files instead of cat or sed.
- You can inspect PI_* environment variables for current model and session details.
- Use edit for precise changes (edits[].oldText must match exactly)
- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls
- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.
- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.
- Use write only for new files or complete rewrites.
- Be concise in your responses
- Show file paths clearly when working with files
Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):
- Main documentation: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/README.md
- Additional docs: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/docs
- Examples: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/examples (extensions, custom tools, SDK)
- When reading pi docs or examples, resolve docs/... under Additional docs and examples/... under Examples, not the current working directory
- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md), environment variables (docs/environment-variables.md)
- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing
- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)
Current working directory: /Users/sia/code/my-pi
+1 -1
View File
@@ -19,7 +19,7 @@
### 依赖 ### 依赖
- `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,与 Context Mode 使用的根 `better-sqlite3@12.11.1` 并存;因此根 `allowScripts` 无需新增条目,也不得删除既有的 `better-sqlite3@12.11.1` 放行。 - `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,根 `allowScripts` 无需任何 `better-sqlite3` 放行条目(context-mode 移除后根依赖已不再包含 better-sqlite3@12)。
- `strip-ansi@7.2.0`:解析到根 `node_modules`。 - `strip-ansi@7.2.0`:解析到根 `node_modules`。
### 本地改动 ### 本地改动
+6 -6
View File
@@ -34,14 +34,14 @@ pi install npm:@ogulcancelik/pi-minimal-footer
## Configuration ## Configuration
Environment variables (all optional): Persistent defaults are read from `config/pi-minimal-footer.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
| Variable | Description | Default | | Field | Description | Default |
| ------------------------------- | -------------------------------------------------------- | ------- | | ------------ | -------------------------------------------------------- | ------- |
| `PI_MINIMAL_FOOTER_SHOW_CWD` | Show current working directory in footer status line | `1` | | `showCwd` | Show current working directory in footer status line | `true` |
| `PI_MINIMAL_FOOTER_SHOW_BRANCH` | Show git branch/dirty/ahead/behind in footer status line | `1` | | `showBranch` | Show git branch/dirty/ahead/behind in footer status line | `true` |
Accepted false values: `0`, `false`, `no`, `off` (case-insensitive). Only `true`/`false` values are honoured; anything else falls back to the default for that field.
## How it works ## How it works
+1 -1
View File
@@ -5,4 +5,4 @@
- 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10` - 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10`
- 导入日期:2026-08-19 - 导入日期:2026-08-19
本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段)。 本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段),并把 showCwd/showBranch 持久配置由 `PI_MINIMAL_FOOTER_*` 环境变量改为组合包 `config/pi-minimal-footer.json`。
+3 -16
View File
@@ -15,6 +15,7 @@ import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path"; import { join } from "node:path";
import { homedir } from "node:os"; import { homedir } from "node:os";
import { collectFooterStatusItems } from "./status.ts"; import { collectFooterStatusItems } from "./status.ts";
import { readFooterSettings } from "./settings.ts";
// ============ Types ============ // ============ Types ============
@@ -43,19 +44,6 @@ interface GitCache {
const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes
const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider
// ============ Env Flags ============
function parseBooleanEnv(value: string | undefined, fallback: boolean): boolean {
if (value === undefined) return fallback;
const normalized = value.trim().toLowerCase();
if (!normalized) return fallback;
if (["1", "true", "yes", "on"].includes(normalized)) return true;
if (["0", "false", "no", "off"].includes(normalized)) return false;
return fallback;
}
// ============ Identity Cache ============ // ============ Identity Cache ============
let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved
@@ -754,9 +742,8 @@ export default function (pi: ExtensionAPI) {
const BAR_FILLED = "━"; const BAR_FILLED = "━";
const BAR_EMPTY = "─"; const BAR_EMPTY = "─";
// Optional visibility toggles (default: enabled) // Optional visibility toggles from bundle config (default: enabled)
const showCwd = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_CWD, true); const { showCwd, showBranch } = readFooterSettings();
const showBranch = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_BRANCH, true);
function formatTokenCount(tokens: number): string { function formatTokenCount(tokens: number): string {
if (tokens >= 1_000_000) { if (tokens >= 1_000_000) {
+30
View File
@@ -0,0 +1,30 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
export interface FooterSettings {
showCwd: boolean;
showBranch: boolean;
}
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-minimal-footer.json` from inside the installed bundle (resolved
// relative to this source file), never environment variables.
export const FOOTER_CONFIG_PATH = fileURLToPath(
new URL("../config/pi-minimal-footer.json", import.meta.url),
);
export function readFooterSettings(configPath: string = FOOTER_CONFIG_PATH): FooterSettings {
try {
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
const raw =
parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: {};
return {
showCwd: typeof raw.showCwd === "boolean" ? raw.showCwd : true,
showBranch: typeof raw.showBranch === "boolean" ? raw.showBranch : true,
};
} catch {
return { showCwd: true, showBranch: true };
}
}
+51
View File
@@ -0,0 +1,51 @@
import assert from "node:assert/strict";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, it } from "node:test";
import { readFooterSettings } from "../settings.ts";
let tempDir: string | undefined;
afterEach(() => {
if (tempDir) {
rmSync(tempDir, { recursive: true, force: true });
tempDir = undefined;
}
});
function configPath(contents: string): string {
tempDir = mkdtempSync(join(tmpdir(), "pi-minimal-footer-settings-"));
const path = join(tempDir, "pi-minimal-footer.json");
writeFileSync(path, contents);
return path;
}
const expectedDefaults = { showCwd: true, showBranch: true };
describe("readFooterSettings", () => {
it("returns all defaults when the config file is missing", () => {
assert.deepEqual(readFooterSettings("/does/not/exist/pi-minimal-footer.json"), expectedDefaults);
});
it("returns all defaults for malformed or non-object JSON", () => {
for (const contents of ["not json", "[1,2]", "42"]) {
assert.deepEqual(readFooterSettings(configPath(contents)), expectedDefaults, contents);
}
});
it("adopts explicit boolean values", () => {
assert.deepEqual(
readFooterSettings(configPath(JSON.stringify({ showCwd: false, showBranch: false }))),
{ showCwd: false, showBranch: false },
);
});
it("falls back per-field for non-boolean values", () => {
assert.deepEqual(
readFooterSettings(configPath(JSON.stringify({ showCwd: "no", showBranch: 0 }))),
expectedDefaults,
);
});
});
+1 -1
View File
@@ -5,7 +5,7 @@
- Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。 - Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。
- 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。 - 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。
- 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。 - 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。
- `/notify on|off` 只覆盖当前 Session;持久默认来自 `PI_NOTIFY_*` 环境变量。`PI_NOTIFY_MESSAGE_SOURCE=none` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。 - `/notify on|off` 只覆盖当前 Session;持久默认来自组合包 `config/pi-notify.json`(`src/settings.ts` 读取,文件缺失或字段非法逐字段回退默认)。`messageSource: "none"` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。
- 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。 - 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。
## 验证 ## 验证
+19 -15
View File
@@ -28,19 +28,21 @@ Do not separately install upstream when using the bundle, or notifications will
- `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus. - `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus.
- `/notify status`: show the active backend, visibility policy, and duration threshold. - `/notify status`: show the active backend, visibility policy, and duration threshold.
Command overrides are session-local. Use environment variables for persistent defaults. Command overrides are session-local. Persistent defaults come from the bundle's `config/pi-notify.json`.
## Configuration ## Configuration
- `PI_NOTIFY_ENABLED`: `1`/`true` or `0`/`false`; default enabled. Persistent defaults are read once per session from `config/pi-notify.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
- `PI_NOTIFY_TITLE`: title template shared by completion and permission notifications; default `π - {project}`.
- `PI_NOTIFY_BODY`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary. - `enabled`: `true`/`false`; default `true`.
- `PI_NOTIFY_MESSAGE_SOURCE`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details. - `title`: title template shared by completion and permission notifications; default `π - {project}`.
- `PI_NOTIFY_MESSAGE_MAX`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`. - `body`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary.
- `PI_NOTIFY_MIN_SECONDS`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts. - `messageSource`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details.
- `PI_NOTIFY_VISIBILITY`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`. - `messageMax`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`.
- `PI_NOTIFY_ACTION`: Kitty click action `focus` or `none`; default `focus`. - `minSeconds`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts.
- `PI_NOTIFY_SOUND_CMD`: optional shell command run after the notification. This is trusted local configuration and executes through a shell. - `visibility`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`.
- `action`: Kitty click action `focus` or `none`; default `focus`.
- `soundCommand`: optional shell command run after the notification. This is trusted local configuration and executes through a shell.
Visibility policies: Visibility policies:
@@ -70,12 +72,14 @@ The active tmux client type is checked so a stale `KITTY_WINDOW_ID` cannot force
- `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict. - `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict.
- `{tool_error_count}`: observed tool error count. - `{tool_error_count}`: observed tool error count.
Example for multiple projects without exposing assistant text on the lock screen: Example for multiple projects without exposing assistant text on the lock screen (`config/pi-notify.json`):
```bash ```json
export PI_NOTIFY_TITLE='π {status_icon} · {project}' {
export PI_NOTIFY_MESSAGE_SOURCE=none "title": "π {status_icon} · {project}",
export PI_NOTIFY_VISIBILITY=unfocused "messageSource": "none",
"visibility": "unfocused"
}
``` ```
Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications. Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications.
+1
View File
@@ -18,5 +18,6 @@
- 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。 - 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。
- 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。 - 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。
- 新增生命周期、多窗口元数据、回退、解析器和配置测试。 - 新增生命周期、多窗口元数据、回退、解析器和配置测试。
- 持久配置由 `PI_NOTIFY_*` 环境变量改为组合包 `config/pi-notify.json`(`src/settings.ts` 读取并逐字段回退默认值)。
上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。 上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。
+11 -17
View File
@@ -1,13 +1,6 @@
import type { ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import { extractMessageText, truncateText } from "./messages.ts"; import { extractMessageText, truncateText } from "./messages.ts";
import { import { readNotifySettings, type NotifySettings } from "./settings.ts";
parseEnabled,
parseKittyVisibility,
parseMessageMax,
parseMessageSource,
parseMinDurationMs,
parseNotificationAction,
} from "./parsers.ts";
import { renderTemplate } from "./template.ts"; import { renderTemplate } from "./template.ts";
import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts"; import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts";
@@ -20,12 +13,13 @@ export function getConfig(
messages: unknown[] | undefined, messages: unknown[] | undefined,
runtime: NotificationRuntimeValues, runtime: NotificationRuntimeValues,
now = Date.now(), now = Date.now(),
settings: NotifySettings = readNotifySettings(),
): NotifyConfig { ): NotifyConfig {
const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt); const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt);
const messageMax = parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX); const messageMax = settings.messageMax;
const userMessage = truncateText(extractMessageText(messages, "user"), messageMax); const userMessage = truncateText(extractMessageText(messages, "user"), messageMax);
const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax); const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax);
const messageSource = parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE); const messageSource = settings.messageSource;
const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : ""; const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : "";
const message = selectedMessage || "Ready for input"; const message = selectedMessage || "Ready for input";
const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0); const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0);
@@ -39,15 +33,15 @@ export function getConfig(
}; };
return { return {
enabled: parseEnabled(process.env.PI_NOTIFY_ENABLED), enabled: settings.enabled,
title: renderTemplate(process.env.PI_NOTIFY_TITLE || DEFAULT_TITLE, ctx, durationMs, values), title: renderTemplate(settings.title || DEFAULT_TITLE, ctx, durationMs, values),
body: renderTemplate(process.env.PI_NOTIFY_BODY || DEFAULT_BODY, ctx, durationMs, values), body: renderTemplate(settings.body || DEFAULT_BODY, ctx, durationMs, values),
durationMs, durationMs,
minDurationMs: parseMinDurationMs(process.env.PI_NOTIFY_MIN_SECONDS), minDurationMs: settings.minDurationMs,
notificationId: runtime.notificationId, notificationId: runtime.notificationId,
visibility: parseKittyVisibility(process.env.PI_NOTIFY_VISIBILITY), visibility: settings.visibility,
action: parseNotificationAction(process.env.PI_NOTIFY_ACTION), action: settings.action,
soundCommand: process.env.PI_NOTIFY_SOUND_CMD?.trim() || undefined, soundCommand: settings.soundCommand,
}; };
} }
+10 -7
View File
@@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { getConfig, shouldNotify } from "./config.ts"; import { getConfig, shouldNotify } from "./config.ts";
import { truncateText } from "./messages.ts"; import { truncateText } from "./messages.ts";
import { parseMessageMax, parseMessageSource } from "./parsers.ts"; import { readNotifySettings, type NotifySettings } from "./settings.ts";
import { getNotificationBackend, sendNotification } from "./terminal.ts"; import { getNotificationBackend, sendNotification } from "./terminal.ts";
import type { NotifyConfig } from "./types.ts"; import type { NotifyConfig } from "./types.ts";
@@ -11,13 +11,14 @@ type ExtensionDependencies = {
createNotificationId: () => string; createNotificationId: () => string;
notify: (config: NotifyConfig) => void; notify: (config: NotifyConfig) => void;
getBackend: typeof getNotificationBackend; getBackend: typeof getNotificationBackend;
settings: NotifySettings;
}; };
const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt"; const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt";
function formatPermissionPromptBody(raw: unknown): string | undefined { function formatPermissionPromptBody(raw: unknown, settings: NotifySettings): string | undefined {
if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined; if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined;
if (parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE) === "none") { if (settings.messageSource === "none") {
return "Permission confirmation required"; return "Permission confirmation required";
} }
@@ -29,7 +30,7 @@ function formatPermissionPromptBody(raw: unknown): string | undefined {
if (!value) return labels.join(" · "); if (!value) return labels.join(" · ");
const normalizedValue = value.replace(/\s+/g, " ").trim(); const normalizedValue = value.replace(/\s+/g, " ").trim();
return `${labels.join(" · ")}\n${truncateText(normalizedValue, parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX))}`; return `${labels.join(" · ")}\n${truncateText(normalizedValue, settings.messageMax)}`;
} }
function readNonEmptyString(value: unknown): string | undefined { function readNonEmptyString(value: unknown): string | undefined {
@@ -47,6 +48,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`); const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`);
const notify = dependencies.notify ?? sendNotification; const notify = dependencies.notify ?? sendNotification;
const getBackend = dependencies.getBackend ?? getNotificationBackend; const getBackend = dependencies.getBackend ?? getNotificationBackend;
const settings = dependencies.settings ?? readNotifySettings();
return function piNotify(pi: ExtensionAPI): void { return function piNotify(pi: ExtensionAPI): void {
const notificationId = createNotificationId(); const notificationId = createNotificationId();
@@ -81,7 +83,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
pi.on("agent_settled", (_event, ctx) => { pi.on("agent_settled", (_event, ctx) => {
try { try {
const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now()); const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now(), settings);
if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config); if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config);
} finally { } finally {
resetRun(); resetRun();
@@ -89,7 +91,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
}); });
const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => { const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => {
const body = formatPermissionPromptBody(raw); const body = formatPermissionPromptBody(raw, settings);
if (!body || currentCtx?.mode !== "tui") return; if (!body || currentCtx?.mode !== "tui") return;
const config = getConfig( const config = getConfig(
@@ -98,6 +100,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
undefined, undefined,
{ notificationId, toolErrorCount }, { notificationId, toolErrorCount },
now(), now(),
settings,
); );
const permissionConfig = { ...config, body, minDurationMs: 0 }; const permissionConfig = { ...config, body, minDurationMs: 0 };
if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig); if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig);
@@ -119,7 +122,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
return; return;
} }
const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now()); const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now(), settings);
if (action === "test") { if (action === "test") {
if (ctx.mode !== "tui") { if (ctx.mode !== "tui") {
ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning"); ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning");
+63
View File
@@ -0,0 +1,63 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import {
parseEnabled,
parseKittyVisibility,
parseMessageMax,
parseMessageSource,
parseMinDurationMs,
parseNotificationAction,
} from "./parsers.ts";
import type { KittyVisibility, MessageSource, NotificationAction } from "./types.ts";
export interface NotifySettings {
enabled: boolean;
title: string;
body: string;
messageSource: MessageSource;
messageMax: number;
minDurationMs: number;
visibility: KittyVisibility;
action: NotificationAction;
soundCommand?: string;
}
const DEFAULT_TITLE = "π - {project}";
const DEFAULT_BODY = "{message}";
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-notify.json` from inside the installed bundle (resolved relative
// to this source file), never environment variables.
export const NOTIFY_CONFIG_PATH = fileURLToPath(new URL("../../config/pi-notify.json", import.meta.url));
function readConfigFile(configPath: string): Record<string, unknown> {
try {
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
return parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: {};
} catch {
return {};
}
}
function optionalString(value: unknown): string | undefined {
if (typeof value === "string") return value;
if (typeof value === "number" || typeof value === "boolean") return String(value);
return undefined;
}
export function readNotifySettings(configPath: string = NOTIFY_CONFIG_PATH): NotifySettings {
const raw = readConfigFile(configPath);
return {
enabled: parseEnabled(optionalString(raw.enabled)),
title: typeof raw.title === "string" && raw.title ? raw.title : DEFAULT_TITLE,
body: typeof raw.body === "string" && raw.body ? raw.body : DEFAULT_BODY,
messageSource: parseMessageSource(optionalString(raw.messageSource)),
messageMax: parseMessageMax(optionalString(raw.messageMax)),
minDurationMs: parseMinDurationMs(optionalString(raw.minSeconds)),
visibility: parseKittyVisibility(optionalString(raw.visibility)),
action: parseNotificationAction(optionalString(raw.action)),
soundCommand: typeof raw.soundCommand === "string" ? raw.soundCommand.trim() || undefined : undefined,
};
}
+24 -34
View File
@@ -1,28 +1,18 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { afterEach, describe, it } from "node:test"; import { describe, it } from "node:test";
import type { ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import { getConfig, shouldNotify } from "../src/config.ts"; import { getConfig, shouldNotify } from "../src/config.ts";
import type { NotifySettings } from "../src/settings.ts";
const envKeys = [ const defaultSettings = (): NotifySettings => ({
"PI_NOTIFY_ENABLED", enabled: true,
"PI_NOTIFY_TITLE", title: "π - {project}",
"PI_NOTIFY_BODY", body: "{message}",
"PI_NOTIFY_MESSAGE_SOURCE", messageSource: "assistant",
"PI_NOTIFY_MESSAGE_MAX", messageMax: 80,
"PI_NOTIFY_MIN_SECONDS", minDurationMs: 3_000,
"PI_NOTIFY_VISIBILITY", visibility: "always",
"PI_NOTIFY_ACTION", action: "focus",
"PI_NOTIFY_SOUND_CMD",
] as const;
const savedEnv = new Map(envKeys.map((key) => [key, process.env[key]]));
afterEach(() => {
for (const key of envKeys) {
const value = savedEnv.get(key);
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
}); });
function context(): ExtensionContext { function context(): ExtensionContext {
@@ -45,6 +35,7 @@ describe("config", () => {
], ],
runtime, runtime,
5_000, 5_000,
defaultSettings(),
); );
assert.deepEqual(config, { assert.deepEqual(config, {
@@ -62,19 +53,20 @@ describe("config", () => {
}); });
it("supports privacy, threshold, focus, and status template settings", () => { it("supports privacy, threshold, focus, and status template settings", () => {
process.env.PI_NOTIFY_ENABLED = "off";
process.env.PI_NOTIFY_TITLE = "{status_icon} {status} {tool_error_count}";
process.env.PI_NOTIFY_BODY = "{message}";
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none";
process.env.PI_NOTIFY_MIN_SECONDS = "10.5";
process.env.PI_NOTIFY_VISIBILITY = "invisible";
process.env.PI_NOTIFY_ACTION = "none";
process.env.PI_NOTIFY_SOUND_CMD = " say done ";
const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], { const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], {
notificationId: "pi-session", notificationId: "pi-session",
toolErrorCount: 2, toolErrorCount: 2,
}, 2_000); }, 2_000, {
...defaultSettings(),
enabled: false,
title: "{status_icon} {status} {tool_error_count}",
body: "{message}",
messageSource: "none",
minDurationMs: 10_500,
visibility: "invisible",
action: "none",
soundCommand: "say done",
});
assert.equal(config.title, "⚠ ready · 2 tool errors 2"); assert.equal(config.title, "⚠ ready · 2 tool errors 2");
assert.equal(config.body, "Ready for input"); assert.equal(config.body, "Ready for input");
@@ -88,15 +80,13 @@ describe("config", () => {
}); });
it("applies selected-message truncation and a session override", () => { it("applies selected-message truncation and a session override", () => {
process.env.PI_NOTIFY_MESSAGE_SOURCE = "user";
process.env.PI_NOTIFY_MESSAGE_MAX = "5";
process.env.PI_NOTIFY_MIN_SECONDS = "0";
const config = getConfig( const config = getConfig(
context(), context(),
1_000, 1_000,
[{ role: "user", content: "hello world" }], [{ role: "user", content: "hello world" }],
runtime, runtime,
2_000, 2_000,
{ ...defaultSettings(), messageSource: "user", messageMax: 5, minDurationMs: 0 },
); );
assert.equal(config.body, "hell…"); assert.equal(config.body, "hell…");
+17 -18
View File
@@ -1,23 +1,24 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { afterEach, describe, it } from "node:test"; import { describe, it } from "node:test";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { createPiNotifyExtension } from "../src/index.ts"; import { createPiNotifyExtension } from "../src/index.ts";
import type { NotifySettings } from "../src/settings.ts";
import type { NotifyConfig } from "../src/types.ts"; import type { NotifyConfig } from "../src/types.ts";
type Handler = (...args: any[]) => any; type Handler = (...args: any[]) => any;
const savedEnv = new Map( const DEFAULT_SETTINGS: NotifySettings = {
["PI_NOTIFY_MIN_SECONDS", "PI_NOTIFY_MESSAGE_SOURCE"].map((key) => [key, process.env[key]]), enabled: true,
); title: "π - {project}",
body: "{message}",
messageSource: "assistant",
messageMax: 80,
minDurationMs: 3_000,
visibility: "always",
action: "focus",
};
afterEach(() => { function harness(mode: ExtensionContext["mode"] = "tui", settings: Partial<NotifySettings> = {}) {
for (const [key, value] of savedEnv) {
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
});
function harness(mode: ExtensionContext["mode"] = "tui") {
const handlers = new Map<string, Handler>(); const handlers = new Map<string, Handler>();
const eventHandlers = new Map<string, Handler>(); const eventHandlers = new Map<string, Handler>();
const commands = new Map<string, { handler: Handler }>(); const commands = new Map<string, { handler: Handler }>();
@@ -45,6 +46,7 @@ function harness(mode: ExtensionContext["mode"] = "tui") {
createNotificationId: () => "pi-session-test", createNotificationId: () => "pi-session-test",
notify: (config) => notifications.push(config), notify: (config) => notifications.push(config),
getBackend: () => "kitty", getBackend: () => "kitty",
settings: { ...DEFAULT_SETTINGS, ...settings },
})(pi); })(pi);
const ctx = { const ctx = {
@@ -93,9 +95,8 @@ describe("pi-notify extension", () => {
}); });
it("does not write terminal notifications in RPC or non-interactive modes", async () => { it("does not write terminal notifications in RPC or non-interactive modes", async () => {
process.env.PI_NOTIFY_MIN_SECONDS = "0";
for (const mode of ["rpc", "json", "print"] as const) { for (const mode of ["rpc", "json", "print"] as const) {
const h = harness(mode); const h = harness(mode, { minDurationMs: 0 });
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
await h.handlers.get("agent_start")?.({}, h.ctx); await h.handlers.get("agent_start")?.({}, h.ctx);
await h.handlers.get("agent_settled")?.({}, h.ctx); await h.handlers.get("agent_settled")?.({}, h.ctx);
@@ -123,8 +124,7 @@ describe("pi-notify extension", () => {
}); });
it("notifies immediately when permission confirmation needs human input", async () => { it("notifies immediately when permission confirmation needs human input", async () => {
process.env.PI_NOTIFY_MIN_SECONDS = "3600"; const h = harness("tui", { minDurationMs: 3_600_000 });
const h = harness();
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
h.eventHandlers.get("permissions:ui_prompt")?.({ h.eventHandlers.get("permissions:ui_prompt")?.({
@@ -144,8 +144,7 @@ describe("pi-notify extension", () => {
}); });
it("keeps permission details private and honors the session off override", async () => { it("keeps permission details private and honors the session off override", async () => {
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none"; const h = harness("tui", { messageSource: "none" });
const h = harness();
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" }; const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" };
+107
View File
@@ -0,0 +1,107 @@
import assert from "node:assert/strict";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, it } from "node:test";
import { readNotifySettings } from "../src/settings.ts";
let tempDir: string | undefined;
afterEach(() => {
if (tempDir) {
rmSync(tempDir, { recursive: true, force: true });
tempDir = undefined;
}
});
function configPath(contents: string): string {
tempDir = mkdtempSync(join(tmpdir(), "pi-notify-settings-"));
const path = join(tempDir, "pi-notify.json");
writeFileSync(path, contents);
return path;
}
const expectedDefaults = {
enabled: true,
title: "π - {project}",
body: "{message}",
messageSource: "assistant",
messageMax: 80,
minDurationMs: 3_000,
visibility: "always",
action: "focus",
soundCommand: undefined,
};
describe("readNotifySettings", () => {
it("returns all defaults when the config file is missing", () => {
assert.deepEqual(readNotifySettings("/does/not/exist/pi-notify.json"), expectedDefaults);
});
it("returns all defaults for malformed or non-object JSON", () => {
for (const contents of ["not json", "[1,2]", "\"hi\"", "42"]) {
assert.deepEqual(readNotifySettings(configPath(contents)), expectedDefaults, contents);
}
});
it("adopts valid values from the config file", () => {
const settings = readNotifySettings(configPath(JSON.stringify({
enabled: false,
title: "π {status_icon} · {project}",
body: "{status}",
messageSource: "none",
messageMax: 120,
minSeconds: 30,
visibility: "unfocused",
action: "none",
soundCommand: "afplay /done.aiff",
})));
assert.deepEqual(settings, {
enabled: false,
title: "π {status_icon} · {project}",
body: "{status}",
messageSource: "none",
messageMax: 120,
minDurationMs: 30_000,
visibility: "unfocused",
action: "none",
soundCommand: "afplay /done.aiff",
});
});
it("falls back per-field when values have the wrong type or value", () => {
const settings = readNotifySettings(configPath(JSON.stringify({
enabled: {},
title: "",
body: null,
messageSource: ["assistant"],
messageMax: "bogus",
minSeconds: "bogus",
visibility: "bogus",
action: {},
})));
assert.deepEqual(settings, expectedDefaults);
});
it("clamps messageMax and minSeconds to their bounded ranges", () => {
const settings = readNotifySettings(configPath(JSON.stringify({ messageMax: 999, minSeconds: 4000 })));
assert.equal(settings.messageMax, 500);
assert.equal(settings.minDurationMs, 3_600_000);
const floor = readNotifySettings(configPath(JSON.stringify({ messageMax: -5, minSeconds: -1 })));
assert.equal(floor.messageMax, 0);
assert.equal(floor.minDurationMs, 0);
});
it("treats an empty or whitespace soundCommand as absent and trims values", () => {
for (const value of ["", " "]) {
assert.equal(readNotifySettings(configPath(JSON.stringify({ soundCommand: value }))).soundCommand, undefined);
}
assert.equal(
readNotifySettings(configPath(JSON.stringify({ soundCommand: " afplay /done.aiff " }))).soundCommand,
"afplay /done.aiff",
);
});
});
+18
View File
@@ -0,0 +1,18 @@
# my-pi 维护约定(pi-overflow)
- 完全自维护(无上游)的新结果展示截断扩展,负责"每个新 tool_result 给模型看多少":`capture.ts` 最先加载只暂存原文,`index.ts` 最后加载(在 `system-prompt.ts` 之前)生成有界视图并注册 `overflow_query`。
- 根 `pi.extensions` 顺序是硬约束:`./pi-overflow/capture.ts` 必须第一,`./pi-overflow/index.ts` 必须在所有其他扩展之后、`./extensions/system-prompt.ts` 之前;跨扩展暂存走 `globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]`,容量 64 条、淘汰最旧。
- `tool_result` handler 链中 capture 不得修改结果;index 改写 content 时必须原样回传 `structuredContent`、保留 image 部分、合并而非丢弃已有 `details`(如 RTK 压缩元数据、bash `fullOutputPath`)。
- `read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用(`parentToolCallId`)永远豁免;read 靠自身 offset/limit 分页,不得破坏 hashline 锚点。
- 视图与行号一律基于刚落盘的存储原文(不是 current 文本):文件来源用流式逐行扫描收集 head/环形 tail/signals,文本来源切同构片段;视图必须确定性、不超 `thresholdChars`、结尾永远可见;落盘原文优先取 `details.fullOutputPath` 文件(超 `maxEntryBytes` 只留头尾各半),其次暂存原文,再次当前文本。
- 超预算收缩优先级为 tail > signals > head:head 先缩到 min(10, headLines),signals 逐步减半至 10 再到 0,再按约 35/65 缩 head/tail,按字符截断兜底。
- signal 扫描与 `overflow_query` 的 search 正则只匹配每行前 10,000 字符;query 输出每行按 `maxLineChars` 截断。
- 存储位于 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),id 为 `sha1(sessionId + "\0" + toolCallId)` 前 12 位 hex;`overflow_query` 的 id 必须匹配 `^[0-9a-f]{12}$` 防路径穿越;`session_start` 按 `retentionDays`/`maxStoreBytes` 清理并删除超过 1 小时的 `*.tmp-*` 残留,失败不得抛出。
- 持久配置只读组合包 `config/pi-overflow.json`(`settings.ts` 逐字段校验、非法回退默认、signalPattern 编译失败回退默认且不敏感大小写);不得读写 `settings.json` 或以环境变量作持久默认。
- `overflow_query` 只读本扩展自己的存储目录;非法 id/正则/JSON/路径一律返回清楚错误文本,不得抛出。
- 与 pi-compaction 的唯一联动是被遮蔽 toolResult 的 `details.overflow.id` 公开字段——pi-compaction 只读它、不得 import 本包任何实现。
- 测试必须注入临时存储目录,不得写真实 agent 目录。
## 验证
- 在本目录内至少运行 `npm test` 与 `npm run check`;涉及加载入口、`pi.extensions` 顺序或 `files` 白名单时还要运行根 `node --test tests/extension-load.test.ts` 与 `tests/package-files.test.ts`。
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 sia
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+66
View File
@@ -0,0 +1,66 @@
# pi-overflow
Bounded display for oversized tool results. When a tool result exceeds
`thresholdChars`, pi-overflow persists the full original to disk and shows the
model a deterministic view — head lines, signal lines (error/fail/warn/…), and
tail lines — plus an `overflow_query` tool to search or read the stored
original by id.
## Two-stage load order
- `capture.ts` — **first** in `pi.extensions`. A `tool_result` handler that
never modifies anything; it stashes the untouched original text in
`globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]` (capacity 64) so the
bound stage can still reach the pre-RTK original.
- `index.ts` — **last** content extension, right before `system-prompt.ts`.
Its `tool_result` handler sees the final text after every other handler
(RTK semantic compaction included) has run, then:
- resolves the *original*: `details.fullOutputPath` file (streamed, capped
at `maxEntryBytes` head+tail) → stashed raw text → current text;
- if the current text still exceeds `thresholdChars`, replaces it with the
bounded view **built from the stored file** — line numbers and totals in
the view are exactly what `overflow_query` will read, even when RTK or
bash already cut the in-context text (file sources are scanned
line-by-line in a stream, never held in memory);
- if the current text fits but the original did not (an earlier extension
already cut it), keeps the text and appends a one-line
`[pi-overflow id=…]` recovery marker;
- preserves image parts and `structuredContent`, and merges
`details.overflow = { id, totalLines, totalBytes, truncated }` into the
existing details.
- Exempt tools (`read`, `edit`, `write`, `overflow_query`,
`ask_user_question`) and nested calls (`parentToolCallId`) are never
touched.
## Stored originals
`<agentDir>/my-pi/pi-overflow/<id>.txt` + `<id>.json` metadata, where
`id = sha1(sessionId + "\0" + toolCallId)[0:12]`. Directories are `0700`,
files `0600`, writes are atomic. `session_start` cleans entries older than
`retentionDays` and evicts oldest-first past `maxStoreBytes`.
## `overflow_query`
- `id` (required, `^[0-9a-f]{12}$` — path traversal safe)
- `search` — case-insensitive regex; hits come back with `context` lines and
adjacent hits merge; capped by `maxResults`
- `lines: { start, end }` — 1-based inclusive line range
- `jsonPath` — dot/bracket path into stored JSON (`a.b[3].c`)
- No selector → metadata + first-screen view. Output is always bounded by
`thresholdChars` and reports whether it was truncated.
## Configuration
`config/pi-overflow.json` inside the installed bundle. Every field is
validated independently and falls back to its default; an uncompilable
`signalPattern` falls back to the default pattern (case-insensitive).
## Development
```sh
npm test # node --test test/*.test.ts
npm run check # strip-types syntax check + tests
```
Tests inject a temporary store root — they never write to a real agent
directory.
+8
View File
@@ -0,0 +1,8 @@
# 上游来源
- 本项目完全自维护,无上游;未从任何外部仓库或 npm 包导入源码。
- 实现仅依赖 Pi 扩展宿主 API(`@earendil-works/pi-coding-agent`)与 `typebox`,两者均为组合包 peer 依赖。
## 本地改动记录
- 初始实现:`capture.ts`(首个加载,暂存原始 tool_result 文本)、`index.ts`(最后加载,生成有界视图 + 注册 `overflow_query`)、`src/`(视图、存储、查询、跨扩展暂存区)。
+30
View File
@@ -0,0 +1,30 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { readOverflowSettings } from "./settings.ts";
import { getStash, stashResult } from "./src/stash.ts";
/**
* Pre-bound capture: loaded FIRST in pi.extensions, before RTK or any other
* result-rewriting extension. It only stashes the untouched original text —
* it never modifies the result. The bound stage (`index.ts`, loaded last)
* consumes the stash after all other handlers have run.
*/
export default function overflowCapture(pi: ExtensionAPI): void {
const settings = readOverflowSettings();
pi.on("tool_result", (event) => {
if (!settings.enabled) return undefined;
if (settings.exemptTools.includes(event.toolName)) return undefined;
if (event.parentToolCallId) return undefined;
const rawText = event.content
.filter((part): part is { type: "text"; text: string } => part.type === "text")
.map((part) => part.text)
.join("\n");
const fullOutputPath =
typeof event.details?.fullOutputPath === "string" ? event.details.fullOutputPath : undefined;
if (rawText.length <= settings.thresholdChars && !fullOutputPath) return undefined;
stashResult(getStash(), event.toolCallId, {
rawText,
...(fullOutputPath ? { fullOutputPath } : {}),
});
return undefined;
});
}
+167
View File
@@ -0,0 +1,167 @@
import { getAgentDir, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";
import { stat } from "node:fs/promises";
import { join } from "node:path";
import { readOverflowSettings, type OverflowSettings } from "./settings.ts";
import { getStash } from "./src/stash.ts";
import { buildView, collectViewFromFile, collectViewFromText } from "./src/view.ts";
import { cleanStore, entryId, storeEntry } from "./src/store.ts";
import { runOverflowQuery } from "./src/query.ts";
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
return `${(bytes / 1_048_576).toFixed(1)} MB`;
}
export interface OverflowExtensionOptions {
settings?: OverflowSettings;
storeRoot?: string;
}
/**
* Bound stage: loaded LAST among content extensions (before system-prompt).
* By the time this handler runs, every other tool_result handler — RTK
* included — has already transformed the result, so the bounded view is
* built from the final visible text while the stored original comes from
* capture.ts's stash or bash's fullOutputPath file.
*/
export default function overflowExtension(pi: ExtensionAPI, options: OverflowExtensionOptions = {}): void {
const settings = options.settings ?? readOverflowSettings();
const storeRoot = () => options.storeRoot ?? join(getAgentDir(), "my-pi", "pi-overflow");
pi.on("session_start", () => {
// Retention/size cleanup never throws (cleanStore swallows failures)
// and the returned promise lets tests await it.
return cleanStore(storeRoot(), settings.retentionDays, settings.maxStoreBytes);
});
pi.on("tool_result", async (event, ctx) => {
const stash = getStash();
const staged = stash.get(event.toolCallId);
try {
if (
!settings.enabled ||
settings.exemptTools.includes(event.toolName) ||
event.parentToolCallId
) {
return undefined;
}
const current = event.content
.filter((part): part is { type: "text"; text: string } => part.type === "text")
.map((part) => part.text)
.join("\n");
const fullOutputPath =
typeof event.details?.fullOutputPath === "string"
? event.details.fullOutputPath
: staged?.fullOutputPath;
// Resolve the original: bash's spilled file wins (it is complete even
// when the in-context text was already cut), then the capture stash,
// then the current text itself.
let source: { kind: "text"; text: string } | { kind: "file"; path: string };
let originalSize: number;
if (fullOutputPath) {
try {
const info = await stat(fullOutputPath);
if (!info.isFile()) throw new Error("not a file");
source = { kind: "file", path: fullOutputPath };
originalSize = info.size;
} catch {
const fallback = staged?.rawText ?? current;
source = { kind: "text", text: fallback };
originalSize = fallback.length;
}
} else {
const fallback = staged?.rawText ?? current;
source = { kind: "text", text: fallback };
originalSize = fallback.length;
}
if (current.length <= settings.thresholdChars && originalSize <= settings.thresholdChars) {
return undefined;
}
const sessionId = ctx.sessionManager?.getSessionId?.() ?? "session";
const id = entryId(sessionId, event.toolCallId);
const stored = await storeEntry(
storeRoot(),
id,
source,
{ tool: event.toolName, args: event.input },
settings.maxEntryBytes,
);
const overflow = {
id,
totalLines: stored.totalLines,
totalBytes: stored.totalBytes,
truncated: current.length > settings.thresholdChars,
};
const images = event.content.filter((part) => part.type !== "text");
let text: string;
if (current.length > settings.thresholdChars) {
// The view always describes the stored file — its line numbers and
// totals must match what overflow_query reads, which is not true
// of `current` when RTK rewrote or bash pre-truncated the result.
const viewInput =
stored.storedText !== undefined
? collectViewFromText(stored.storedText, settings)
: await collectViewFromFile(join(storeRoot(), `${id}.txt`), settings);
text = buildView(viewInput, settings, { id, tool: event.toolName });
} else {
text = `${current}\n[pi-overflow id=${id}: full original output stored (${stored.totalLines} lines, ${formatBytes(stored.totalBytes)}); use overflow_query to inspect it]`;
}
return {
content: [{ type: "text", text }, ...images],
structuredContent: event.structuredContent,
details: { ...event.details, overflow },
};
} finally {
stash.delete(event.toolCallId);
}
});
pi.registerTool({
name: "overflow_query",
label: "Overflow Query",
description:
"Inspect the full original of a tool output that pi-overflow stored because it was too long to show " +
"(search with regex, read line ranges, or extract a JSON path).",
promptSnippet: "Inspect a stored oversized tool output by id: regex search, line ranges, or JSON path.",
promptGuidelines: [
"When a tool result carries a [pi-overflow id=…] marker, the model is seeing a bounded view; the complete original is stored on disk.",
"Call overflow_query with the marker's id and a regex, line range, or jsonPath instead of re-running the tool or asking for the full dump.",
],
parameters: Type.Object({
id: Type.String({ description: "12-hex storage id from a [pi-overflow id=…] marker." }),
search: Type.Optional(
Type.String({ description: "Case-insensitive regular expression matched per line." }),
),
context: Type.Optional(
Type.Number({ minimum: 0, maximum: 10, description: "Context lines around each search hit (default 2)." }),
),
lines: Type.Optional(
Type.Object({
start: Type.Number({ minimum: 1, description: "First line to return (1-based, inclusive)." }),
end: Type.Number({ minimum: 1, description: "Last line to return (inclusive)." }),
}),
),
jsonPath: Type.Optional(
Type.String({ description: "Dot/bracket path into the stored JSON, e.g. snapshot.nodes[3].name." }),
),
maxResults: Type.Optional(
Type.Number({ minimum: 1, maximum: 200, description: "Maximum search hits to report (default 50)." }),
),
}),
async execute(_toolCallId, params) {
const result = await runOverflowQuery(storeRoot(), params, settings);
return {
content: [{ type: "text", text: result.text }],
details: { error: result.error === true, truncated: result.truncated },
isError: result.error === true,
};
},
});
}
+49
View File
@@ -0,0 +1,49 @@
{
"name": "pi-overflow",
"version": "0.1.0",
"description": "Bounded views for oversized tool results: store the full original, show head/signals/tail, recover with overflow_query.",
"private": true,
"type": "module",
"license": "MIT",
"main": "./index.ts",
"exports": {
".": "./index.ts"
},
"files": [
"index.ts",
"capture.ts",
"settings.ts",
"src",
"README.md",
"LICENSE"
],
"pi": {
"extensions": [
"./capture.ts",
"./index.ts"
]
},
"scripts": {
"test": "node --test test/*.test.ts",
"check": "node --experimental-strip-types --check index.ts && node --test test/*.test.ts"
},
"peerDependencies": {
"@earendil-works/pi-coding-agent": "*",
"typebox": "*"
},
"peerDependenciesMeta": {
"@earendil-works/pi-coding-agent": {
"optional": true
},
"typebox": {
"optional": true
}
},
"devDependencies": {
"@earendil-works/pi-coding-agent": "latest",
"typebox": "latest"
},
"engines": {
"node": ">=22.19.0"
}
}
+100
View File
@@ -0,0 +1,100 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
export interface OverflowSettings {
enabled: boolean;
thresholdChars: number;
headLines: number;
tailLines: number;
maxSignalLines: number;
signalPattern: string;
signalRegex: RegExp;
maxLineChars: number;
exemptTools: string[];
retentionDays: number;
maxStoreBytes: number;
maxEntryBytes: number;
}
export const DEFAULT_SIGNAL_PATTERN =
"\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b";
export const OVERFLOW_DEFAULTS = {
enabled: true,
thresholdChars: 24_000,
headLines: 40,
tailLines: 120,
maxSignalLines: 40,
signalPattern: DEFAULT_SIGNAL_PATTERN,
maxLineChars: 500,
exemptTools: ["read", "edit", "write", "overflow_query", "ask_user_question"],
retentionDays: 7,
maxStoreBytes: 1_073_741_824,
maxEntryBytes: 67_108_864,
} as const;
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-overflow.json` from inside the installed bundle (resolved
// relative to this package-root source file), never `settings.json`.
export const OVERFLOW_CONFIG_PATH = fileURLToPath(
new URL("../config/pi-overflow.json", import.meta.url),
);
function isObject(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value);
}
function bool(value: unknown, fallback: boolean): boolean {
return typeof value === "boolean" ? value : fallback;
}
function intInRange(value: unknown, fallback: number, min: number, max: number): number {
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max
? value
: fallback;
}
function compileSignalPattern(value: unknown): { pattern: string; regex: RegExp } {
const pattern = typeof value === "string" && value.length > 0 ? value : DEFAULT_SIGNAL_PATTERN;
try {
return { pattern, regex: new RegExp(pattern, "i") };
} catch {
return { pattern: DEFAULT_SIGNAL_PATTERN, regex: new RegExp(DEFAULT_SIGNAL_PATTERN, "i") };
}
}
function stringList(value: unknown, fallback: readonly string[]): string[] {
if (!Array.isArray(value)) return [...fallback];
const list = value.filter((item): item is string => typeof item === "string" && item.length > 0);
return [...new Set(list)];
}
/**
* Read `config/pi-overflow.json`. Every field is validated independently and
* falls back to its bundle default when missing or invalid; an unreadable or
* non-object file yields all defaults. Tests inject an explicit path.
*/
export function readOverflowSettings(configPath: string = OVERFLOW_CONFIG_PATH): OverflowSettings {
let parsed: unknown = {};
try {
parsed = JSON.parse(readFileSync(configPath, "utf8"));
} catch {
parsed = {};
}
const config = isObject(parsed) ? parsed : {};
const signal = compileSignalPattern(config.signalPattern);
return {
enabled: bool(config.enabled, OVERFLOW_DEFAULTS.enabled),
thresholdChars: intInRange(config.thresholdChars, OVERFLOW_DEFAULTS.thresholdChars, 1, 10_000_000),
headLines: intInRange(config.headLines, OVERFLOW_DEFAULTS.headLines, 0, 10_000),
tailLines: intInRange(config.tailLines, OVERFLOW_DEFAULTS.tailLines, 0, 10_000),
maxSignalLines: intInRange(config.maxSignalLines, OVERFLOW_DEFAULTS.maxSignalLines, 0, 1_000),
signalPattern: signal.pattern,
signalRegex: signal.regex,
maxLineChars: intInRange(config.maxLineChars, OVERFLOW_DEFAULTS.maxLineChars, 10, 100_000),
exemptTools: stringList(config.exemptTools, OVERFLOW_DEFAULTS.exemptTools),
retentionDays: intInRange(config.retentionDays, OVERFLOW_DEFAULTS.retentionDays, 0, 3_650),
maxStoreBytes: intInRange(config.maxStoreBytes, OVERFLOW_DEFAULTS.maxStoreBytes, 1_024, 1_099_511_627_776),
maxEntryBytes: intInRange(config.maxEntryBytes, OVERFLOW_DEFAULTS.maxEntryBytes, 1_024, 1_099_511_627_776),
};
}
+188
View File
@@ -0,0 +1,188 @@
import { ID_PATTERN, readStoredEntry, type EntryMeta } from "./store.ts";
import type { OverflowSettings } from "../settings.ts";
export interface QueryParams {
id: string;
search?: string;
context?: number;
lines?: { start: number; end: number };
jsonPath?: string;
maxResults?: number;
}
export interface QueryResult {
text: string;
truncated: boolean;
error?: boolean;
}
function fail(text: string): QueryResult {
return { text, truncated: false, error: true };
}
function boundOutput(text: string, limit: number): QueryResult {
if (text.length <= limit) return { text, truncated: false };
const marker = `\n[output truncated at ${limit} chars; narrow the query (smaller line range, tighter regex, or jsonPath)]`;
const keep = Math.max(0, limit - marker.length);
return { text: text.slice(0, keep) + marker, truncated: true };
}
function metaLine(id: string, meta: EntryMeta | null): string {
if (!meta) return `[pi-overflow id=${id}]`;
return `[pi-overflow id=${id} tool=${meta.tool} ${meta.totalLines} lines / ${meta.totalBytes} bytes]`;
}
/** Regexes only inspect this prefix of each stored line — a model-supplied
* pattern must not backtrack catastrophically across a megabyte-long line. */
const MATCH_SCAN_CHARS = 10_000;
function clipLine(line: string, maxLineChars: number): string {
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
}
function evalJsonPath(root: unknown, path: string): { ok: true; value: unknown } | { ok: false; error: string } {
const tokens: Array<string | number> = [];
const pattern = /(?:^|\.)([A-Za-z_$][A-Za-z0-9_$]*)|\[\s*(\d+)\s*\]|\[\s*["']([^"']+)["']\s*\]/g;
let index = 0;
let match: RegExpExecArray | null;
while ((match = pattern.exec(path)) !== null) {
if (match.index !== index) return { ok: false, error: `cannot parse jsonPath near "${path.slice(index)}"` };
index = pattern.lastIndex;
if (match[1] !== undefined) tokens.push(match[1]);
else if (match[2] !== undefined) tokens.push(Number(match[2]));
else tokens.push(match[3] as string);
}
if (index !== path.length || tokens.length === 0) {
return { ok: false, error: `cannot parse jsonPath "${path}"` };
}
let value: unknown = root;
for (const token of tokens) {
if (value === null || value === undefined) {
return { ok: false, error: `jsonPath stops at null/undefined before "${String(token)}"` };
}
if (typeof token === "number") {
if (!Array.isArray(value) || token >= value.length) {
return { ok: false, error: `index [${token}] out of range` };
}
value = value[token];
} else {
if (typeof value !== "object" || Array.isArray(value)) {
return { ok: false, error: `"${token}" is not a property of this value` };
}
const record = value as Record<string, unknown>;
if (!(token in record)) return { ok: false, error: `property "${token}" not found` };
value = record[token];
}
}
return { ok: true, value };
}
function mergeRanges(hits: number[], context: number): Array<{ start: number; end: number }> {
const ranges: Array<{ start: number; end: number }> = [];
for (const hit of hits) {
const start = Math.max(1, hit - context);
const end = hit + context;
const last = ranges[ranges.length - 1];
if (last && start <= last.end + 1) {
last.end = Math.max(last.end, end);
} else {
ranges.push({ start, end });
}
}
return ranges;
}
/**
* Execute an overflow_query against the store. Never throws on bad input —
* invalid ids, regexes, JSON, and paths all come back as clear error text.
*/
export async function runOverflowQuery(
root: string,
params: QueryParams,
settings: OverflowSettings,
): Promise<QueryResult> {
if (typeof params.id !== "string" || !ID_PATTERN.test(params.id)) {
return fail(`invalid overflow id "${String(params.id)}" — expected 12 lowercase hex chars from a [pi-overflow id=…] marker`);
}
let stored: { text: string; meta: EntryMeta | null };
try {
stored = await readStoredEntry(root, params.id);
} catch {
return fail(`no stored output for id ${params.id} (it may have been cleaned up or never stored)`);
}
const lines = stored.text.split("\n");
const head = metaLine(params.id, stored.meta);
const context = Math.min(Math.max(Math.floor(params.context ?? 2), 0), 10);
const maxResults = Math.min(Math.max(Math.floor(params.maxResults ?? 50), 1), 200);
if (typeof params.search === "string") {
let regex: RegExp;
try {
regex = new RegExp(params.search, "i");
} catch (error) {
return fail(`invalid search regex: ${error instanceof Error ? error.message : String(error)}`);
}
const hits: number[] = [];
for (let index = 0; index < lines.length && hits.length < maxResults; index++) {
regex.lastIndex = 0;
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) hits.push(index + 1);
}
if (hits.length === 0) {
return boundOutput(`${head}\nno lines match /${params.search}/i`, settings.thresholdChars);
}
const parts = [head, `${hits.length}${hits.length >= maxResults ? "+" : ""} matching lines:`];
for (const range of mergeRanges(hits, context)) {
const end = Math.min(range.end, lines.length);
parts.push(`--- lines ${range.start}-${end} ---`);
for (let line = range.start; line <= end; line++) {
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
}
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
if (params.lines && typeof params.lines === "object") {
const start = Math.floor(params.lines.start);
const end = Math.floor(params.lines.end);
if (!Number.isFinite(start) || !Number.isFinite(end) || start < 1 || end < start) {
return fail(`invalid line range {start: ${params.lines.start}, end: ${params.lines.end}} — need 1 <= start <= end`);
}
const clippedEnd = Math.min(end, lines.length);
if (start > lines.length) {
return fail(`start ${start} is beyond the stored ${lines.length} lines`);
}
const parts = [head, `--- lines ${start}-${clippedEnd} ---`];
for (let line = start; line <= clippedEnd; line++) {
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
if (typeof params.jsonPath === "string") {
let parsed: unknown;
try {
parsed = JSON.parse(stored.text);
} catch (error) {
return fail(`stored output is not valid JSON: ${error instanceof Error ? error.message : String(error)}`);
}
const result = evalJsonPath(parsed, params.jsonPath);
if (!result.ok) return fail(`jsonPath error: ${result.error}`);
let rendered: string;
try {
rendered = JSON.stringify(result.value, null, 2) ?? "undefined";
} catch {
rendered = String(result.value);
}
return boundOutput(`${head}\njsonPath ${params.jsonPath}:\n${rendered}`, settings.thresholdChars);
}
// No selector: metadata plus a bounded head/tail preview.
const headCount = Math.min(settings.headLines, lines.length);
const tailCount = Math.min(settings.tailLines, Math.max(0, lines.length - headCount));
const parts = [head, `--- lines 1-${headCount} ---`, ...lines.slice(0, headCount)];
if (tailCount > 0) {
parts.push(`… ${lines.length - headCount - tailCount} lines omitted …`);
parts.push(`--- lines ${lines.length - tailCount + 1}-${lines.length} ---`, ...lines.slice(lines.length - tailCount));
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
+35
View File
@@ -0,0 +1,35 @@
/**
* Cross-extension staging for pre-bound originals. `capture.ts` (loaded
* first) stashes the raw tool_result text; `index.ts` (loaded last) consumes
* it after every other handler — RTK in particular — has run. Sharing goes
* through `globalThis[Symbol.for(...)]`, the same mechanism pi-compaction
* uses for its metrics service.
*/
export interface StashedResult {
rawText: string;
fullOutputPath?: string;
}
const STASH_KEY = Symbol.for("my-pi.pi-overflow.stash.v1");
const STASH_LIMIT = 64;
export function getStash(): Map<string, StashedResult> {
const scope = globalThis as Record<symbol, unknown>;
let stash = scope[STASH_KEY] as Map<string, StashedResult> | undefined;
if (!(stash instanceof Map)) {
stash = new Map<string, StashedResult>();
scope[STASH_KEY] = stash;
}
return stash;
}
export function stashResult(stash: Map<string, StashedResult>, toolCallId: string, entry: StashedResult): void {
stash.delete(toolCallId);
stash.set(toolCallId, entry);
while (stash.size > STASH_LIMIT) {
const oldest = stash.keys().next().value;
if (oldest === undefined) break;
stash.delete(oldest);
}
}
+267
View File
@@ -0,0 +1,267 @@
import { createHash } from "node:crypto";
import { createReadStream, createWriteStream, type WriteStream } from "node:fs";
import { chmod, mkdir, open, readFile, readdir, rename, stat, unlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pipeline } from "node:stream/promises";
export interface EntryMeta {
tool: string;
argsSummary: string;
totalLines: number;
totalBytes: number;
createdAt: string;
omittedMiddle?: boolean;
}
export interface StoredEntry {
id: string;
totalLines: number;
totalBytes: number;
/** Present when the source was text: the exact content written to disk. */
storedText?: string;
}
export const ID_PATTERN = /^[0-9a-f]{12}$/;
const TMP_MAX_AGE_MS = 3_600_000;
export function entryId(sessionId: string, toolCallId: string): string {
return createHash("sha1").update(`${sessionId}\0${toolCallId}`).digest("hex").slice(0, 12);
}
async function ensureDir(dir: string): Promise<void> {
await mkdir(dir, { recursive: true, mode: 0o700 });
try {
await chmod(dir, 0o700);
} catch {
// Best-effort permission tightening only.
}
}
function tmpPath(path: string): string {
return `${path}.tmp-${process.pid}-${Math.random().toString(36).slice(2)}`;
}
async function writeFileAtomic(path: string, content: string | Buffer): Promise<void> {
const tmp = tmpPath(path);
await writeFile(tmp, content, { mode: 0o600 });
await rename(tmp, path);
}
function countNewlines(text: string): number {
let count = 0;
for (let index = 0; index < text.length; index++) {
if (text.charCodeAt(index) === 10) count++;
}
return count;
}
export function lineCount(text: string): number {
if (text.length === 0) return 0;
return countNewlines(text) + (text.endsWith("\n") ? 0 : 1);
}
function truncateSummary(value: string, maxChars = 300): string {
return value.length > maxChars ? `${value.slice(0, maxChars)}…` : value;
}
export function argsSummary(input: unknown): string {
try {
return truncateSummary(JSON.stringify(input) ?? "undefined");
} catch {
return "[unserializable]";
}
}
/**
* Copy `sourcePath` into `<root>/<id>.txt` without holding it all in memory.
* Files larger than `maxEntryBytes` keep the first and last halves with an
* omission marker between them.
*/
async function copyFileBounded(
sourcePath: string,
destPath: string,
maxEntryBytes: number,
): Promise<{ bytes: number; omittedMiddle: boolean }> {
const info = await stat(sourcePath);
if (!info.isFile()) throw new Error("full output path is not a file");
const size = info.size;
if (size <= maxEntryBytes) {
const tmp = tmpPath(destPath);
const out: WriteStream = createWriteStream(tmp, { mode: 0o600 });
await pipeline(createReadStream(sourcePath), out);
await rename(tmp, destPath);
return { bytes: size, omittedMiddle: false };
}
const half = Math.floor(maxEntryBytes / 2);
const omitted = size - half * 2;
const source = await open(sourcePath, "r");
try {
const head = Buffer.alloc(half);
const tail = Buffer.alloc(half);
await source.read(head, 0, half, 0);
await source.read(tail, 0, half, size - half);
const marker = Buffer.from(`\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n`);
await writeFileAtomic(destPath, Buffer.concat([head, marker, tail]));
} finally {
await source.close();
}
return { bytes: size, omittedMiddle: true };
}
async function countFileLines(path: string): Promise<number> {
let count = 0;
let lastByte = -1;
let seen = false;
for await (const chunk of createReadStream(path)) {
seen = true;
const buffer = chunk as Buffer;
for (let index = 0; index < buffer.length; index++) {
if (buffer[index] === 10) count++;
}
if (buffer.length > 0) lastByte = buffer[buffer.length - 1];
}
if (!seen) return 0;
return count + (lastByte === 10 ? 0 : 1);
}
/**
* Persist an original tool output under `<root>/<id>.txt` plus `<id>.json`
* metadata. Directories are 0700, files 0600, writes are atomic. The caller
* chooses `root` (defaults come from getAgentDir() in index.ts; tests inject
* a temp directory).
*/
export async function storeEntry(
root: string,
id: string,
source: { kind: "text"; text: string } | { kind: "file"; path: string },
meta: { tool: string; args: unknown },
maxEntryBytes: number,
): Promise<StoredEntry> {
await ensureDir(root);
const textPath = join(root, `${id}.txt`);
const metaPath = join(root, `${id}.json`);
let totalLines = 0;
let totalBytes = 0;
let omittedMiddle = false;
let storedText: string | undefined;
if (source.kind === "text") {
let text = source.text;
const bytes = Buffer.byteLength(text, "utf8");
totalBytes = bytes;
if (bytes > maxEntryBytes) {
const halfChars = Math.floor(maxEntryBytes / 2);
const omitted = bytes - halfChars * 2;
text = `${text.slice(0, halfChars)}\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n${text.slice(text.length - halfChars)}`;
omittedMiddle = true;
}
totalLines = lineCount(text);
storedText = text;
await writeFileAtomic(textPath, text);
} else {
const copied = await copyFileBounded(source.path, textPath, maxEntryBytes);
totalBytes = copied.bytes;
omittedMiddle = copied.omittedMiddle;
totalLines = await countFileLines(textPath).catch(() => 0);
}
const entry: EntryMeta = {
tool: meta.tool,
argsSummary: argsSummary(meta.args),
totalLines,
totalBytes,
createdAt: new Date().toISOString(),
...(omittedMiddle ? { omittedMiddle: true } : {}),
};
await writeFileAtomic(metaPath, `${JSON.stringify(entry, null, 2)}\n`);
return { id, totalLines, totalBytes, ...(storedText !== undefined ? { storedText } : {}) };
}
export async function readStoredEntry(root: string, id: string): Promise<{ text: string; meta: EntryMeta | null }> {
if (!ID_PATTERN.test(id)) throw new Error(`invalid overflow id: ${id}`);
const textPath = join(root, `${id}.txt`);
const metaPath = join(root, `${id}.json`);
const text = await readFile(textPath, "utf8");
let meta: EntryMeta | null = null;
try {
const parsed = JSON.parse(await readFile(metaPath, "utf8"));
if (parsed && typeof parsed === "object") meta = parsed as EntryMeta;
} catch {
meta = null;
}
return { text, meta };
}
/**
* Delete tmp files left by interrupted writes (older than one hour), entries
* older than `retentionDays`, then evict oldest-first until the store is
* under `maxStoreBytes`. All failures are swallowed — cleanup must never
* break a session.
*/
export async function cleanStore(root: string, retentionDays: number, maxStoreBytes: number): Promise<void> {
try {
const now = Date.now();
const cutoff = now - retentionDays * 86_400_000;
const names = await readdir(root);
// Stale tmp files from interrupted writes never become entries; remove
// them once they are clearly abandoned.
for (const name of names) {
if (!name.includes(".tmp-")) continue;
try {
const info = await stat(join(root, name));
if (info.isFile() && info.mtimeMs < now - TMP_MAX_AGE_MS) {
await unlink(join(root, name));
}
} catch {
// Already gone or unreadable — nothing to do.
}
}
const ids = new Set(
names
.filter((name) => !name.includes(".tmp-"))
.map((name) => name.replace(/\.(txt|json)$/, "")),
);
const entries: Array<{ id: string; bytes: number; mtime: number }> = [];
for (const id of ids) {
let bytes = 0;
let mtime = 0;
let seen = false;
for (const suffix of ["txt", "json"] as const) {
const file = join(root, `${id}.${suffix}`);
try {
const info = await stat(file);
bytes += info.size;
mtime = Math.max(mtime, info.mtimeMs);
seen = true;
} catch {
// Missing half of a pair still counts toward cleanup.
}
}
if (seen) entries.push({ id, bytes, mtime });
}
const remove = async (id: string) => {
for (const suffix of ["txt", "json"] as const) {
await unlink(join(root, `${id}.${suffix}`)).catch(() => {});
}
};
let total = 0;
const survivors: typeof entries = [];
for (const entry of entries) {
if (entry.mtime < cutoff) {
await remove(entry.id);
} else {
total += entry.bytes;
survivors.push(entry);
}
}
if (total > maxStoreBytes) {
survivors.sort((a, b) => a.mtime - b.mtime);
for (const entry of survivors) {
if (total <= maxStoreBytes) break;
await remove(entry.id);
total -= entry.bytes;
}
}
} catch {
// Cleanup failures never propagate.
}
}
+246
View File
@@ -0,0 +1,246 @@
import { createReadStream } from "node:fs";
import { open, stat } from "node:fs/promises";
import type { OverflowSettings } from "../settings.ts";
export interface ViewMeta {
id: string;
tool: string;
}
export interface SignalHit {
line: number;
text: string;
}
/**
* Pre-collected view segments of the STORED original (never the current —
* possibly RTK- or bash-truncated — text). Line numbers here are the stored
* file's line numbers, which is exactly what overflow_query reads.
*/
export interface ViewInput {
head: string[];
signals: SignalHit[];
tail: string[];
totalLines: number;
totalBytes: number;
/** First ~thresholdChars chars of the stored content, for char fallback. */
rawHead: string;
/** Last ~thresholdChars chars of the stored content, for char fallback. */
rawTail: string;
}
/** Regexes only inspect this prefix of each line — catastrophic backtracking
* on a giant single-line blob is not allowed to stall the handler. */
export const MATCH_SCAN_CHARS = 10_000;
function clip(line: string, maxLineChars: number): string {
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
}
/** Slice a text original into the same view segments a file scan produces. */
export function collectViewFromText(text: string, settings: OverflowSettings): ViewInput {
const lines = text.split("\n");
const total = lines.length;
const headCount = Math.min(settings.headLines, total);
const tailCount = Math.min(settings.tailLines, Math.max(0, total - headCount));
const head = lines.slice(0, headCount).map((line) => clip(line, settings.maxLineChars));
const tail = tailCount > 0 ? lines.slice(total - tailCount).map((line) => clip(line, settings.maxLineChars)) : [];
const signals: SignalHit[] = [];
const regex = new RegExp(settings.signalRegex.source, "i");
for (
let index = headCount;
index < total - tailCount && signals.length < settings.maxSignalLines;
index++
) {
regex.lastIndex = 0;
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) {
signals.push({ line: index + 1, text: clip(lines[index], settings.maxLineChars) });
}
}
return {
head,
signals,
tail,
totalLines: total,
totalBytes: Buffer.byteLength(text, "utf8"),
rawHead: text.slice(0, settings.thresholdChars),
rawTail: text.slice(Math.max(0, text.length - settings.thresholdChars)),
};
}
/**
* Stream the stored file line by line — never materialize it. Collects the
* first headLines lines, keeps the last tailLines lines in a ring buffer,
* and records at most maxSignalLines signal hits (line number + clipped
* text); hits landing inside the final tail range are dropped afterwards.
*/
export async function collectViewFromFile(path: string, settings: OverflowSettings): Promise<ViewInput> {
const head: string[] = [];
const tail: string[] = [];
const signals: SignalHit[] = [];
let totalLines = 0;
let remainder = "";
const regex = new RegExp(settings.signalRegex.source, "i");
const handleLine = (line: string) => {
totalLines++;
if (head.length < settings.headLines) {
head.push(clip(line, settings.maxLineChars));
} else if (signals.length < settings.maxSignalLines) {
regex.lastIndex = 0;
if (regex.test(line.slice(0, MATCH_SCAN_CHARS))) {
signals.push({ line: totalLines, text: clip(line, settings.maxLineChars) });
}
}
tail.push(clip(line, settings.maxLineChars));
if (tail.length > settings.tailLines) tail.shift();
};
for await (const chunk of createReadStream(path, { encoding: "utf8" })) {
remainder += chunk;
let index = remainder.indexOf("\n");
while (index >= 0) {
handleLine(remainder.slice(0, index));
remainder = remainder.slice(index + 1);
index = remainder.indexOf("\n");
}
}
if (remainder.length > 0) handleLine(remainder);
const tailStart = totalLines - tail.length + 1;
const middleSignals = signals.filter((hit) => hit.line < tailStart);
const info = await stat(path);
const take = Math.min(settings.thresholdChars, info.size);
let rawHead = "";
let rawTail = "";
if (take > 0) {
const handle = await open(path, "r");
try {
const headBuffer = Buffer.alloc(take);
const tailBuffer = Buffer.alloc(take);
await handle.read(headBuffer, 0, take, 0);
await handle.read(tailBuffer, 0, take, info.size - take);
rawHead = headBuffer.toString("utf8");
rawTail = tailBuffer.toString("utf8");
} finally {
await handle.close();
}
}
return {
head,
signals: middleSignals,
tail,
totalLines,
totalBytes: info.size,
rawHead,
rawTail,
};
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
if (bytes < 1_073_741_824) return `${(bytes / 1_048_576).toFixed(1)} MB`;
return `${(bytes / 1_073_741_824).toFixed(2)} GB`;
}
function signalDescription(pattern: string): string {
const compact = pattern.replace(/\\b/g, "").replace(/[()]/g, "");
return compact.length > 60 ? `${compact.slice(0, 60)}…` : compact;
}
/**
* Deterministic bounded view of the stored original: head lines, signal
* lines from the middle section, and tail lines — all with the stored
* file's line numbers. Output never exceeds settings.thresholdChars and the
* tail of the stored content is always visible.
*
* Shrink order under budget pressure is tail > signals > head: head first
* collapses to min(10, headLines), signals halve stepwise to 10 then 0, then
* head/tail shrink ~35/65 with tail preferred, and char truncation of the
* stored head/tail is the final fallback.
*/
export function buildView(input: ViewInput, settings: OverflowSettings, meta: ViewMeta): string {
const budget = settings.thresholdChars;
const header =
`[pi-overflow id=${meta.id} tool=${meta.tool} ${input.totalLines} lines / ${formatBytes(input.totalBytes)}` +
` — full output stored; use overflow_query to search or read ranges]`;
const render = (headLines: string[], signals: SignalHit[], tailLines: string[]): string => {
const parts: string[] = [header];
if (headLines.length > 0) {
parts.push(`--- lines 1-${headLines.length} ---`, ...headLines);
}
if (signals.length > 0) {
parts.push(
`--- signals (lines matching ${signalDescription(settings.signalPattern)}, up to ${settings.maxSignalLines}) ---`,
...signals.map((hit) => `L${hit.line}: ${hit.text}`),
);
}
if (tailLines.length > 0) {
const first = input.totalLines - tailLines.length + 1;
parts.push(`--- lines ${first}-${input.totalLines} ---`, ...tailLines);
}
return parts.join("\n");
};
const charFallback = (): string => {
const markerReserve = 64;
const bodyBudget = Math.max(64, budget - header.length - markerReserve);
const headChars = Math.floor(bodyBudget * 0.35);
const tailChars = bodyBudget - headChars;
const headText = input.rawHead.slice(0, headChars);
const tailText = input.rawTail.slice(Math.max(0, input.rawTail.length - tailChars));
const omitted = Math.max(0, input.totalBytes - headText.length - tailText.length);
return `${header}\n${headText}\n… ${omitted} chars omitted …\n${tailText}`;
};
let headList = input.head;
let signalList = input.signals;
let tailList = input.tail;
let result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
// 1. Collapse head to its minimum useful window.
headList = headList.slice(0, Math.min(10, settings.headLines));
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
// 2. Halve signals stepwise until 10 remain, then drop them entirely.
while (signalList.length > 10) {
signalList = signalList.slice(0, Math.ceil(signalList.length / 2));
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
}
if (signalList.length > 0) {
signalList = [];
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
}
// When head+tail already span every stored line there is no line-level
// slack left: go straight to char truncation so the ending survives.
if (input.head.length + input.tail.length >= input.totalLines) return charFallback();
// 3. Shrink head/tail ~35/65; tail keeps the larger share and always keeps
// its last line (the stored ending).
while (headList.length + tailList.length > 0) {
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
const headChars = headList.reduce((sum, line) => sum + line.length + 1, 0);
const tailChars = tailList.reduce((sum, line) => sum + line.length + 1, 0);
if (tailList.length > 1 && (tailChars >= headChars * (65 / 35) || headList.length === 0)) {
tailList.shift();
} else if (headList.length > 0) {
headList.pop();
} else {
tailList.shift();
}
}
// 4. Last resort: keep the stored text's own head 35% / tail 65%.
return charFallback();
}
+314
View File
@@ -0,0 +1,314 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, readFile, readdir, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import overflowCapture from "../capture.ts";
import overflowExtension from "../index.ts";
import { getStash } from "../src/stash.ts";
import { readOverflowSettings, OVERFLOW_DEFAULTS, type OverflowSettings } from "../settings.ts";
function harness() {
const handlers = new Map<string, Function[]>();
const tools = new Map<string, any>();
const pi: any = {
on(name: string, handler: Function) {
const list = handlers.get(name) ?? [];
list.push(handler);
handlers.set(name, list);
},
registerTool(tool: any) { tools.set(tool.name, tool); },
};
return { pi, handlers, tools };
}
function ctx() {
return { sessionManager: { getSessionId: () => "session-test" } } as any;
}
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
}
function textEvent(toolCallId: string, text: string, extra: Record<string, unknown> = {}) {
return {
toolCallId,
toolName: "bash",
input: { command: "run" },
content: [{ type: "text", text }],
details: {},
...extra,
};
}
async function runHandlers(handlers: Map<string, Function[]>, event: any, context: any) {
let current = event;
for (const handler of handlers.get("tool_result") ?? []) {
const result = await handler(current, context);
if (result) current = { ...current, ...result };
}
return current;
}
async function makeStore() {
return mkdtemp(join(tmpdir(), "pi-overflow-test-"));
}
test("capture stashes the original before another handler shortens it; bound stage stores original but views current", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = Array.from({ length: 3000 }, (_, i) => `orig ${i}`).join("\n");
const event = textEvent("call-1", big);
const afterCapture = await runHandlers(capture.handlers, event, ctx());
assert.equal(afterCapture, event, "capture must not modify the result");
assert.ok(getStash().has("call-1"), "capture must stash the original");
// Simulate RTK: replace content with a shortened version that is still
// over threshold.
const rtkShortened = Array.from({ length: 900 }, (_, i) => `rtk ${i}`).join("\n") + "\n" + "z".repeat(30_000);
const mid = { ...afterCapture, content: [{ type: "text", text: rtkShortened }] };
const final = await runHandlers(bound.handlers, mid, ctx());
assert.ok(!getStash().has("call-1"), "stash entry must be consumed");
const text = final.content[0].text;
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash/);
assert.ok(text.length <= settings().thresholdChars);
const id = final.details.overflow.id;
const stored = await readFile(join(root, `${id}.txt`), "utf8");
assert.ok(stored.includes("orig 0"), "store must keep the ORIGINAL, not the RTK-shortened text");
assert.ok(stored.includes("orig 2999"));
assert.equal(final.details.overflow.truncated, true);
});
test("details.fullOutputPath file is used as the original", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const spill = join(root, "spill.txt");
await writeFile(spill, Array.from({ length: 5000 }, (_, i) => `spill ${i}`).join("\n"));
const event = textEvent("call-2", "x".repeat(30_000), { details: { fullOutputPath: spill } });
const final = await runHandlers(bound.handlers, event, ctx());
const stored = await readFile(join(root, `${final.details.overflow.id}.txt`), "utf8");
assert.ok(stored.includes("spill 0"));
assert.ok(stored.includes("spill 4999"));
});
test("view line numbers track the STORED file, not the current text", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
// bash spill: 5000 lines on disk, but the in-context result only carries
// the last ~100 lines (still over threshold via padding).
const spill = join(root, "spill-5000.txt");
await writeFile(
spill,
Array.from({ length: 5000 }, (_, i) => `spill line ${i + 1}`).join("\n"),
);
const current = Array.from({ length: 100 }, (_, i) => `spill line ${4901 + i}`).join("\n") + "\n" + "pad".repeat(10_000);
const event = textEvent("call-lines", current, { details: { fullOutputPath: spill } });
const final = await runHandlers(bound.handlers, event, ctx());
const text = final.content[0].text;
const id = final.details.overflow.id;
assert.match(text, /--- lines 4881-5000 ---/, "tail range must come from the stored 5000-line file");
assert.match(text, /spill line 5000/);
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash 5000 lines/);
// overflow_query with the view's tail range returns the same lines.
const tool = bound.tools.get("overflow_query");
const result = await tool.execute("q1", { id, lines: { start: 4881, end: 4883 } });
assert.match(result.content[0].text, /L4881: spill line 4881/);
assert.match(result.content[0].text, /L4883: spill line 4883/);
});
test("RTK scenario: view line numbers come from the stored original text", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const original = Array.from({ length: 3000 }, (_, i) => `orig ${i + 1}`).join("\n");
await runHandlers(capture.handlers, textEvent("call-rtk", original), ctx());
// RTK rewrites the visible text but it stays over threshold.
const mid = {
...textEvent("call-rtk", ""),
content: [{ type: "text", text: "z".repeat(30_000) }],
};
const final = await runHandlers(bound.handlers, mid, ctx());
const text = final.content[0].text;
assert.match(text, /--- lines 2881-3000 ---/, "view tail must be the stored original's tail");
assert.ok(text.includes("orig 3000"));
});
test("file sources over maxEntryBytes keep real head and tail halves", async () => {
const root = await makeStore();
const { storeEntry } = await import("../src/store.ts");
const src = join(root, "big.bin");
const content = "HEAD-".repeat(200) + "MID-".repeat(2000) + "TAIL-".repeat(200);
await writeFile(src, content);
const maxEntryBytes = 2000;
await storeEntry(root, "0123456789ab", { kind: "file", path: src }, { tool: "bash", args: {} }, maxEntryBytes);
const stored = await readFile(join(root, "0123456789ab.txt"), "utf8");
assert.ok(stored.startsWith("HEAD-"));
assert.ok(stored.endsWith("TAIL-"));
assert.match(stored, /bytes omitted/);
// Head and tail halves must equal the source's real bytes.
const half = Math.floor(maxEntryBytes / 2);
assert.equal(stored.slice(0, half), content.slice(0, half));
assert.equal(stored.slice(-half), content.slice(-half));
assert.ok(!stored.includes("\0".repeat(100)), "must not store zeroed content");
});
test("interrupted-write tmp files older than an hour are cleaned", async () => {
const root = await makeStore();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const staleTmp = join(root, "deadbeef.txt.tmp-999-zzz");
const freshTmp = join(root, "livebeef.txt.tmp-111-aaa");
await writeFile(staleTmp, "partial");
await writeFile(freshTmp, "partial");
const old = new Date(Date.now() - 2 * 3_600_000);
const { utimes } = await import("node:fs/promises");
await utimes(staleTmp, old, old);
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const { existsSync } = await import("node:fs");
assert.ok(!existsSync(staleTmp), "stale tmp must be removed");
assert.ok(existsSync(freshTmp), "fresh tmp must survive");
});
test("current under threshold but original over → one-line marker appended", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = "a".repeat(30_000);
const event = textEvent("call-3", big);
await runHandlers(capture.handlers, event, ctx());
const mid = { ...event, content: [{ type: "text", text: "short summary" }] };
const final = await runHandlers(bound.handlers, mid, ctx());
const text = final.content[0].text;
assert.ok(text.startsWith("short summary"));
assert.match(text, /\[pi-overflow id=[0-9a-f]{12}: full original output stored \(\d+ lines, .*; use overflow_query/);
assert.equal(final.details.overflow.truncated, false);
assert.ok(final.details.overflow.id);
});
test("exempt tools and nested calls are ignored", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = "x".repeat(30_000);
const exempt = await runHandlers(bound.handlers, textEvent("c-ex", big, { toolName: "read" }), ctx());
assert.equal(exempt.content[0].text, big, "read results must pass through untouched");
const nested = await runHandlers(bound.handlers, textEvent("c-nest", big, { parentToolCallId: "parent" }), ctx());
assert.equal(nested.content[0].text, big, "nested calls must pass through untouched");
});
test("image parts and structuredContent survive", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const event = textEvent("call-4", "x".repeat(30_000));
event.content.push({ type: "image", data: "b64" } as any);
(event as any).structuredContent = { rows: 3 };
const final = await runHandlers(bound.handlers, event, ctx());
assert.deepEqual(final.structuredContent, { rows: 3 });
assert.equal(final.content.at(-1).type, "image");
assert.equal(final.content.at(-1).data, "b64");
});
test("existing details fields survive the merge", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const event = textEvent("call-5", "x".repeat(30_000), { details: { rtk: { saved: 100 }, fullOutputPath: undefined } });
const final = await runHandlers(bound.handlers, event, ctx());
assert.deepEqual(final.details.rtk, { saved: 100 });
assert.ok(final.details.overflow.id);
});
test("files are 0600 and directories 0700", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const final = await runHandlers(bound.handlers, textEvent("call-6", "x".repeat(30_000)), ctx());
const id = final.details.overflow.id;
const dirMode = (await stat(root)).mode & 0o777;
const fileMode = (await stat(join(root, `${id}.txt`))).mode & 0o777;
const metaMode = (await stat(join(root, `${id}.json`))).mode & 0o777;
assert.equal(dirMode, 0o700);
assert.equal(fileMode, 0o600);
assert.equal(metaMode, 0o600);
});
test("retention and total-size cleanup remove old entries", async () => {
const root = await makeStore();
const bound = harness();
overflowExtension(bound.pi, {
storeRoot: root,
settings: settings({ retentionDays: 1, maxStoreBytes: 100 }),
});
// An old entry (mtime in the past) must be deleted by age.
const oldId = "aaaaaaaaaaaa";
await writeFile(join(root, `${oldId}.txt`), "old");
await writeFile(join(root, `${oldId}.json`), "{}");
const past = new Date(Date.now() - 10 * 86_400_000);
const { utimes } = await import("node:fs/promises");
await utimes(join(root, `${oldId}.txt`), past, past);
await utimes(join(root, `${oldId}.json`), past, past);
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const names = await readdir(root);
assert.ok(!names.some((n) => n.startsWith(oldId)), "old entry must be deleted");
// Fresh entries exceeding maxStoreBytes get evicted oldest-first.
for (const [i, id] of ["bbbbbbbbbbbb", "cccccccccccc", "dddddddddddd"].entries()) {
await writeFile(join(root, `${id}.txt`), "y".repeat(80));
const t = new Date(Date.now() - (3 - i) * 60_000);
await utimes(join(root, `${id}.txt`), t, t);
}
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const after = (await readdir(root)).filter((n) => n.endsWith(".txt"));
assert.ok(after.length <= 2, `expected eviction under maxStoreBytes, got ${after.join(",")}`);
assert.ok(!after.includes("bbbbbbbbbbbb.txt"), "oldest entry must be evicted first");
});
test("disabled setting clears the stash and returns undefined", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings({ enabled: false }) });
getStash().set("c-off", { rawText: "x".repeat(30_000) });
const event = textEvent("c-off", "x".repeat(30_000));
const result = await runHandlers(bound.handlers, event, ctx());
assert.equal(result, event);
assert.ok(!getStash().has("c-off"));
});
+112
View File
@@ -0,0 +1,112 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runOverflowQuery } from "../src/query.ts";
import { storeEntry } from "../src/store.ts";
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
}
async function store(text: string, id = "0123456789ab") {
const root = await mkdtemp(join(tmpdir(), "pi-overflow-query-"));
await storeEntry(root, id, { kind: "text", text }, { tool: "bash", args: { command: "run" } }, 1_000_000);
return root;
}
test("search returns hits with merged context", async () => {
const lines = Array.from({ length: 100 }, (_, i) => (i === 9 || i === 11 || i === 50 ? `line ${i + 1} FAIL` : `line ${i + 1}`));
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 1 }, settings());
assert.ok(!result.error);
assert.match(result.text, /L10: line 10 FAIL/);
assert.match(result.text, /L12: line 12 FAIL/);
assert.match(result.text, /L51: line 51 FAIL/);
// hits at lines 10 and 12 with context 1 merge into one range 9-13
assert.match(result.text, /--- lines 9-13 ---/);
assert.match(result.text, /--- lines 50-52 ---/);
});
test("line-range retrieval returns numbered lines", async () => {
const lines = Array.from({ length: 50 }, (_, i) => `row ${i + 1}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 10, end: 15 } }, settings());
assert.match(result.text, /--- lines 10-15 ---/);
assert.match(result.text, /L10: row 10/);
assert.match(result.text, /L15: row 15/);
assert.ok(!result.text.includes("row 16"));
});
test("jsonPath extracts nested values", async () => {
const doc = JSON.stringify({ snapshot: { nodes: [{ name: "a" }, { name: "b" }, { name: "c" }] } });
const root = await store(doc);
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "snapshot.nodes[2].name" }, settings());
assert.match(result.text, /"c"/);
});
test("invalid id, regex, JSON and path return clear errors without throwing", async () => {
const root = await store('{"a": {"b": 1}}');
assert.ok((await runOverflowQuery(root, { id: "../etc/passwd" }, settings())).error);
assert.ok((await runOverflowQuery(root, { id: "zzzzzzzzzzzz" }, settings())).error);
assert.ok((await runOverflowQuery(root, { id: "0123456789ab", search: "([" }, settings())).error);
const goodPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b" }, settings());
assert.ok(!goodPath.error, "valid path on valid JSON must succeed");
const badPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b.c!!" }, settings());
assert.ok(badPath.error);
const missing = await runOverflowQuery(root, { id: "ffffffffffff" }, settings());
assert.ok(missing.error);
});
test("non-JSON content + jsonPath reports the parse failure", async () => {
const root = await store("not json at all");
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a" }, settings());
assert.ok(result.error);
assert.match(result.text, /not valid JSON/);
});
test("output is bounded by thresholdChars and marked truncated", async () => {
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i + 1} pad ${"p".repeat(40)}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(
root,
{ id: "0123456789ab", lines: { start: 1, end: 5000 } },
settings({ thresholdChars: 4000 }),
);
assert.ok(result.text.length <= 4000);
assert.equal(result.truncated, true);
assert.match(result.text, /truncated/);
});
test("lines and search output clip each line to maxLineChars", async () => {
const long = `row ${"q".repeat(2_000)}`;
const root = await store(`first\n${long}\n${long} ERROR\nlast`);
const s = settings({ maxLineChars: 60 });
const lines = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 2, end: 3 } }, s);
assert.ok(lines.text.includes("q".repeat(56) + "…"));
assert.ok(!lines.text.includes("q".repeat(61)));
const search = await runOverflowQuery(root, { id: "0123456789ab", search: "ERROR", context: 0 }, s);
assert.ok(search.text.includes("q".repeat(56) + "…"));
assert.ok(!search.text.includes("q".repeat(61)));
});
test("search only scans the first 10000 chars of each line", async () => {
const earlyHit = `hit ${"a".repeat(100)} FAIL`;
const lateHit = `${"a".repeat(10_500)} FAIL`;
const root = await store(`${earlyHit}\n${lateHit}\nplain`);
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 0 }, settings());
assert.match(result.text, /L1: hit /);
assert.ok(!result.text.includes("L2:"), "match beyond char 10000 must not be reported");
});
test("default query returns metadata and a first-screen view", async () => {
const lines = Array.from({ length: 300 }, (_, i) => `row ${i + 1}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab" }, settings());
assert.ok(!result.error);
assert.match(result.text, /\[pi-overflow id=0123456789ab tool=bash \d+ lines \/ \d+ bytes\]/);
assert.match(result.text, /--- lines 1-40 ---/);
assert.ok(result.text.includes("row 300"), "tail of the stored file must appear");
});
+86
View File
@@ -0,0 +1,86 @@
import test from "node:test";
import assert from "node:assert/strict";
import { buildView, collectViewFromText } from "../src/view.ts";
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return {
...readOverflowSettings("/nonexistent/pi-overflow.json"),
...overrides,
};
}
const meta = { id: "abc123def456", tool: "bash" };
function view(text: string, s = settings()) {
return buildView(collectViewFromText(text, s), s, meta);
}
test("below threshold remains unchanged", () => {
const out = view("line one\nline two\nline three");
assert.ok(out.includes("line one"));
assert.ok(out.includes("line three"));
});
test("above threshold includes header, line ranges, signal lines and tail", () => {
const lines: string[] = [];
for (let i = 1; i <= 1000; i++) lines.push(`row ${i} ${i === 500 ? "ERROR boom" : "ok"}`);
const out = view(lines.join("\n"));
assert.match(out, /\[pi-overflow id=abc123def456 tool=bash 1000 lines/);
assert.match(out, /--- lines 1-40 ---/);
assert.match(out, /L500: row 500 ERROR boom/);
assert.match(out, /--- lines 881-1000 ---/);
assert.ok(out.includes("row 1000"));
});
test("output never exceeds thresholdChars", () => {
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i} ${"x".repeat(60)}`);
const s = settings({ thresholdChars: 2000 });
const out = view(lines.join("\n"), s);
assert.ok(out.length <= 2000, `view ${out.length} exceeds budget`);
assert.ok(out.includes("line 4999"), "tail must survive");
});
test("few long lines still expose the ending", () => {
const lines = ["start", ...Array.from({ length: 5 }, (_, i) => `middle${i} ${"y".repeat(4000)}`), "THE_END_MARKER"];
const s = settings({ thresholdChars: 3000, headLines: 40, tailLines: 120 });
const out = view(lines.join("\n"), s);
assert.ok(out.includes("THE_END_MARKER"));
assert.ok(out.length <= 3000 + 100, `view ${out.length} far exceeds budget`);
});
test("same input produces identical output", () => {
const lines = Array.from({ length: 800 }, (_, i) => `entry ${i} ${i % 77 === 0 ? "warning" : "fine"}`);
const text = lines.join("\n");
assert.equal(view(text), view(text));
});
test("a single line over maxLineChars is truncated", () => {
const long = "z".repeat(2000);
const out = view(`a\n${long}\nb`, settings({ maxLineChars: 50 }));
assert.ok(out.includes("z".repeat(50) + "…"));
assert.ok(!out.includes("z".repeat(51)));
});
test("signals outrank head and survive when head+tail exceed the budget", () => {
// ~150-char lines: head 40 + tail 120 = 160 lines * 150 = ~24K plus
// overhead, over budget. Shrink order must keep some signals while head
// collapses to 10 lines before the 35/65 phase kicks in.
const lines = Array.from({ length: 400 }, (_, i) => {
const marker = i === 200 ? "ERROR detected here" : "ok";
return `line ${i + 1} ${marker} ${"x".repeat(140)}`;
});
const out = view(lines.join("\n"));
assert.match(out, /--- signals/);
assert.match(out, /L201: line 201 ERROR detected here/);
// head collapsed below the configured 40 lines
assert.match(out, /--- lines 1-(?:[0-9]|10) ---/);
assert.ok(out.includes("line 400"));
});
test("signal hits inside the tail region are not duplicated", () => {
const lines = Array.from({ length: 100 }, (_, i) => `line ${i + 1} ${"y".repeat(40)}`);
lines[99] = "line 100 tail-side ERROR";
const out = view(lines.join("\n"));
assert.ok(!out.includes("L100:"), "tail-region hit must not appear in signals");
});
+1 -1
View File
@@ -8,7 +8,7 @@
- reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。 - reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。
- `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。 - `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。
- `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。 - `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。
- 策略默认允许常规只读与低风险工具;`web_fetch`、`ctx_fetch_and_index`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。 - 策略默认允许常规只读与低风险工具;`web_fetch`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。
- `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。 - `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。
- 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。 - 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。
- reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。 - reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。
+1 -1
View File
@@ -2,7 +2,7 @@
- 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。 - 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。
- 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。 - 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。
- RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合和兜底截断;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。 - RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合;兜底截断默认关闭,新结果的展示截断由 pi-overflow 统一负责;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。
- RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。 - RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。
- FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。 - FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。
- `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。 - `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。
+2 -2
View File
@@ -154,7 +154,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
|--------|------|---------|-------|-------------| |--------|------|---------|-------|-------------|
| `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled | | `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled |
| `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation | | `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation |
| `outputCompaction.truncate.enabled` | boolean | `true` | — | Enable hard character truncation | | `outputCompaction.truncate.enabled` | boolean | `false` | — | Enable hard character truncation (off in this bundle — pi-overflow owns display truncation) |
| `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output | | `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output |
### Source Code Filtering Levels ### Source Code Filtering Levels
@@ -195,7 +195,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
"maxLines": 220 "maxLines": 220
}, },
"truncate": { "truncate": {
"enabled": true, "enabled": false,
"maxChars": 12000 "maxChars": 12000
} }
} }
+1
View File
@@ -13,3 +13,4 @@
- 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。 - 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。
- `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。 - `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。
- `outputCompaction.truncate` 兜底截断默认关闭(组合包 `config/rtk-optimizer.json`),新结果的展示截断由 pi-overflow 统一负责;RTK 只保留语义压缩。
+1 -1
View File
@@ -3,7 +3,7 @@
- 扩展入口是 `pi-tool-search/extensions/index.ts`,组合包权威分组位于 `extensions/bundle-groups.ts`,目录生成/校验/缓存位于 `extensions/catalog.ts`,配置读取与路径解析位于 `extensions/config.ts`(直接读取组合包 `config/tool-search.json`;目录缓存仍在 `<agentDir>/my-pi/tool-search/catalog-v1.json`),测试保留在 `test/`。 - 扩展入口是 `pi-tool-search/extensions/index.ts`,组合包权威分组位于 `extensions/bundle-groups.ts`,目录生成/校验/缓存位于 `extensions/catalog.ts`,配置读取与路径解析位于 `extensions/config.ts`(直接读取组合包 `config/tool-search.json`;目录缓存仍在 `<agentDir>/my-pi/tool-search/catalog-v1.json`),测试保留在 `test/`。
- 同步上游必须记录明确 tag/commit,保留 README、CHANGELOG、文档、作者和许可证说明;不得导入上游 `.pi`、嵌套 `.git`、`node_modules` 或构建产物。 - 同步上游必须记录明确 tag/commit,保留 README、CHANGELOG、文档、作者和许可证说明;不得导入上游 `.pi`、嵌套 `.git`、`node_modules` 或构建产物。
- 根包必须通过 `file:./pi-tool-search` 提供本地依赖并直接加载仓库源码,不得同时安装或加载 npm `pi-tool-search` 实现;同步上游时先核对本地增量动态工具改造,禁止用旧 npm 入口或上游旧宿主实现直接覆盖。 - 根包必须通过 `file:./pi-tool-search` 提供本地依赖并直接加载仓库源码,不得同时安装或加载 npm `pi-tool-search` 实现;同步上游时先核对本地增量动态工具改造,禁止用旧 npm 入口或上游旧宿主实现直接覆盖。
- 默认常驻核心 `read` / `write` / `edit` / `bash` / `grep` / `find`、`codegraph_explore`、`lsp_diagnostics`、`ctx_execute` / `ctx_execute_file` / `ctx_batch_execute`、六个记忆工具与 `context_tree_query`,以及 `tool_search`;Windows 上内置 `powershell` 必须与 `bash` 同属固定本地 Shell 核心工具,非 Windows 平台必须从目录中过滤。配置从已安装组合包内的 `config/tool-search.json` 直接读取,文件缺失或无效时回退 `BUNDLE_TOOL_SEARCH_DEFAULTS`;扩展自身不播种、不迁移,也不再读写 `settings.json` 的 `toolSearch` 段。 - 默认常驻核心 `read` / `write` / `edit` / `bash` / `grep` / `find`、`codegraph_explore`、`lsp_diagnostics`、六个记忆工具与 `context_tree_query`、`overflow_query`,以及 `tool_search`;Windows 上内置 `powershell` 必须与 `bash` 同属固定本地 Shell 核心工具,非 Windows 平台必须从目录中过滤。配置从已安装组合包内的 `config/tool-search.json` 直接读取,文件缺失或无效时回退 `BUNDLE_TOOL_SEARCH_DEFAULTS`;扩展自身不播种、不迁移,也不再读写 `settings.json` 的 `toolSearch` 段。
- 其余组合包工具必须优先使用 `extensions/bundle-groups.ts` 中提交的权威工作流分组;精确已有分组必须直接激活,不得等待无关未知工具的模型增强。默认最多 5 个动态组、模型组最多 8 个工具、动态工具总数最多 28;`groups` 可原子激活最多 5 个精确组,完整请求超限时不得改变当前状态;加载或组内调用更新 LRU,超限先移除最久未用的非请求组。自然语言查询必须优先精确工具名和已提交 workflow alias,弱匹配或近似同分只能返回候选,不得任意激活;`/tool-search-status` 必须显示当前容量、LRU 顺序与最近淘汰原因。 - 其余组合包工具必须优先使用 `extensions/bundle-groups.ts` 中提交的权威工作流分组;精确已有分组必须直接激活,不得等待无关未知工具的模型增强。默认最多 5 个动态组、模型组最多 8 个工具、动态工具总数最多 28;`groups` 可原子激活最多 5 个精确组,完整请求超限时不得改变当前状态;加载或组内调用更新 LRU,超限先移除最久未用的非请求组。自然语言查询必须优先精确工具名和已提交 workflow alias,弱匹配或近似同分只能返回候选,不得任意激活;`/tool-search-status` 必须显示当前容量、LRU 顺序与最近淘汰原因。
- 标准组合包工具全部命中预置目录时不得调用模型、创建用户缓存或发送隐藏 schema。只有存在未识别第三方工具时才允许惰性模型增强;生成结果必须保留全部预置 assignment,模型元数据不得替代真实名称、参数或 schema。缓存必须位于 agent 目录且权限 `600`;模型失败必须继续使用预置目录 + 确定性未知工具分组。`groupOverrides` 优先级高于预置分组。 - 标准组合包工具全部命中预置目录时不得调用模型、创建用户缓存或发送隐藏 schema。只有存在未识别第三方工具时才允许惰性模型增强;生成结果必须保留全部预置 assignment,模型元数据不得替代真实名称、参数或 schema。缓存必须位于 agent 目录且权限 `600`;模型失败必须继续使用预置目录 + 确定性未知工具分组。`groupOverrides` 优先级高于预置分组。
- 必须直接使用当前 `@earendil-works/pi-coding-agent`、`typebox`、`ModelRegistry.complete()` 和 `setActiveTools()`;首次纯增加载使用当前 Pi 的增量结果传播,LRU 替换允许走宿主安全 fallback。不得安装或映射旧 `@mariozechner` runtime,不得恢复 provider payload 改写、代理分发或隐藏 `sendMessage` steer/retry 兼容循环。Tool Search 是唯一可调用 `getActiveTools()` / `setActiveTools()` 管理工具显隐的组合包扩展;`pi-permission-system` 只能在调用时裁决。 - 必须直接使用当前 `@earendil-works/pi-coding-agent`、`typebox`、`ModelRegistry.complete()` 和 `setActiveTools()`;首次纯增加载使用当前 Pi 的增量结果传播,LRU 替换允许走宿主安全 fallback。不得安装或映射旧 `@mariozechner` runtime,不得恢复 provider payload 改写、代理分发或隐藏 `sendMessage` steer/retry 兼容循环。Tool Search 是唯一可调用 `getActiveTools()` / `setActiveTools()` 管理工具显隐的组合包扩展;`pi-permission-system` 只能在调用时裁决。
+2
View File
@@ -3,6 +3,7 @@
## Unreleased — my-pi maintained source ## Unreleased — my-pi maintained source
### Changed ### Changed
- Removed the retired context-mode integration: the `ctx_execute` / `ctx_execute_file` / `ctx_batch_execute` pins left `alwaysEnabled` and the four `context-*` bundle groups were dropped. Large tool results are now bounded by pi-overflow and recoverable through the pinned `overflow_query`.
- Imported upstream `v0.3.6` at `ddfb23646fd3957b791214de278e23aa393c9b13` for direct local maintenance. - Imported upstream `v0.3.6` at `ddfb23646fd3957b791214de278e23aa393c9b13` for direct local maintenance.
- Migrated runtime imports to `@earendil-works/pi-coding-agent` and current `typebox`. - Migrated runtime imports to `@earendil-works/pi-coding-agent` and current `typebox`.
- Replaced the hidden steer/retry compatibility loop with current Pi dynamic-tool propagation. - Replaced the hidden steer/retry compatibility loop with current Pi dynamic-tool propagation.
@@ -12,6 +13,7 @@
- Added checked-in authoritative groups for every tool exposed by the standard my-pi bundle, eliminating first-use model generation and user caching unless unrecognized third-party tools are present. - Added checked-in authoritative groups for every tool exposed by the standard my-pi bundle, eliminating first-use model generation and user caching unless unrecognized third-party tools are present.
- Replaced the Hermes and third-party Hippo assignments with the five tools exposed by the official Hippo Pi extension, grouped into recall and management workflows. - Replaced the Hermes and third-party Hippo assignments with the five tools exposed by the official Hippo Pi extension, grouped into recall and management workflows.
- Replaced the retired Hippo `hippo_*` assignments with the vendored Hermes memory extension's six tools, grouped into `memory-recall` (`memory_search`, `session_search`, `context_tree_query`) and `memory-management` (`memory_add`, `memory_replace`, `memory_remove`, `skill_manage`). - Replaced the retired Hippo `hippo_*` assignments with the vendored Hermes memory extension's six tools, grouped into `memory-recall` (`memory_search`, `session_search`, `context_tree_query`) and `memory-management` (`memory_add`, `memory_replace`, `memory_remove`, `skill_manage`).
- Added `overflow_query` to `memory-recall` and pinned it in `alwaysEnabled` — inspecting a pi-overflow-stored original must not depend on a `tool_search` activation.
- Added the enabled bundle's `Agent`, `get_subagent_result`, and `steer_subagent` tools to a checked-in `subagents` orchestration group. - Added the enabled bundle's `Agent`, `get_subagent_result`, and `steer_subagent` tools to a checked-in `subagents` orchestration group.
- Added authoritative `ssh-connection`, `ssh-remote-files`, `ssh-remote-search`, and `ssh-remote-shell` groups for the maintained `pi-ssh` tools, keeping reviewed connection, bounded discovery, structured file operations, and reviewed shell execution separate; the connection workflow now requires sequential `ssh_connect` to succeed before dependent tools. - Added authoritative `ssh-connection`, `ssh-remote-files`, `ssh-remote-search`, and `ssh-remote-shell` groups for the maintained `pi-ssh` tools, keeping reviewed connection, bounded discovery, structured file operations, and reviewed shell execution separate; the connection workflow now requires sequential `ssh_connect` to succeed before dependent tools.
- Added sequential `ssh_cd` to the authoritative remote-shell workflow and direct the model to wait for a successful workspace change before dependent remote calls, avoiding both the unknown-tools fallback and cwd races. - Added sequential `ssh_cd` to the authoritative remote-shell workflow and direct the model to wait for a successful workspace change before dependent remote calls, avoiding both the unknown-tools fallback and cwd races.
+4 -10
View File
@@ -33,10 +33,6 @@ Run `/tool-search-status` to inspect the current catalog source, pinned count, d
| `ssh-remote-shell` | `ssh_cd`, `ssh_bash` | | `ssh-remote-shell` | `ssh_cd`, `ssh_bash` |
| `code-intelligence` | CodeGraph and all LSP tools | | `code-intelligence` | CodeGraph and all LSP tools |
| `web-search` | `web_search`, `web_fetch` (pi-web-search unified tools) | | `web-search` | `web_search`, `web_fetch` (pi-web-search unified tools) |
| `context-execution` | `ctx_execute`, `ctx_execute_file`, `ctx_batch_execute` |
| `context-knowledge` | Context indexing, searching, and web-doc ingestion |
| `context-observability` | Context stats, doctor, and Insight |
| `context-administration` | Context upgrade and destructive purge |
| `memory-recall` | Hermes memory/session search and condensed-output recovery | | `memory-recall` | Hermes memory/session search and condensed-output recovery |
| `memory-management` | Hermes memory writes and procedural skill management | | `memory-management` | Hermes memory writes and procedural skill management |
| `subagents` | Launch agents, collect background results, and steer running agents | | `subagents` | Launch agents, collect background results, and steer running agents |
@@ -45,7 +41,7 @@ Run `/tool-search-status` to inspect the current catalog source, pinned count, d
| `chrome-interaction` | Chrome click/type/fill/key/hover/drag/tap/scroll input | | `chrome-interaction` | Chrome click/type/fill/key/hover/drag/tap/scroll input |
| `chrome-debugging` | Chrome page evaluation, console/network diagnostics, response retrieval, and file upload | | `chrome-debugging` | Chrome page evaluation, console/network diagnostics, response retrieval, and file upload |
Pinned tools are removed from these dynamic groups at runtime. The defaults pin `codegraph_explore`, `lsp_diagnostics`, the three high-frequency Context Mode execution tools, and all six Hermes memory tools, so none of them consume dynamic group or tool capacity. Pinned tools are removed from these dynamic groups at runtime. The defaults pin `codegraph_explore`, `lsp_diagnostics`, and all six Hermes memory tools, so none of them consume dynamic group or tool capacity.
## Loading and eviction ## Loading and eviction
@@ -78,16 +74,14 @@ Pi reads this configuration directly from the installed bundle's `config/tool-se
"alwaysEnabled": [ "alwaysEnabled": [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
"memory_replace", "memory_replace",
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query" "context_tree_query",
"overflow_query"
], ],
"showToolSearchFooterStatus": false, "showToolSearchFooterStatus": false,
"maxActiveGroups": 5, "maxActiveGroups": 5,
@@ -99,7 +93,7 @@ Pi reads this configuration directly from the installed bundle's `config/tool-se
| Key | Bundle default | Description | | Key | Bundle default | Description |
|---|---|---| |---|---|---|
| `alwaysEnabled` | CodeGraph explore, LSP diagnostics, Context execution trio, all six Hermes memory tools, `context_tree_query` | Exact names pinned alongside Pi core tools | | `alwaysEnabled` | CodeGraph explore, LSP diagnostics, all six Hermes memory tools, `context_tree_query`, `overflow_query` | Exact names pinned alongside Pi core tools |
| `showToolSearchFooterStatus` | `false` | Show active tool/group counts | | `showToolSearchFooterStatus` | `false` | Show active tool/group counts |
| `maxActiveGroups` | `5` | Maximum simultaneously active dynamic groups | | `maxActiveGroups` | `5` | Maximum simultaneously active dynamic groups |
| `maxToolsPerGroup` | `8` | Maximum size accepted from model-generated groups | | `maxToolsPerGroup` | `8` | Maximum size accepted from model-generated groups |
+2 -2
View File
@@ -2,7 +2,7 @@
`pi-tool-search` keeps full low-frequency schemas hidden behind a compact, validated group catalog. `pi-tool-search` keeps full low-frequency schemas hidden behind a compact, validated group catalog.
1. `session_start` activates `tool_search`, the six cross-platform Pi core tools, and configured `alwaysEnabled` names. The my-pi defaults also pin CodeGraph exploration, LSP diagnostics, the three high-frequency Context Mode execution tools, and the six Hermes memory tools. On Windows, `powershell` is pinned beside `bash`; on non-Windows hosts it is excluded. Pinned tools do not consume the dynamic-group or dynamic-tool budget. 1. `session_start` activates `tool_search`, the six cross-platform Pi core tools, and configured `alwaysEnabled` names. The my-pi defaults also pin CodeGraph exploration, LSP diagnostics, and the six Hermes memory tools. On Windows, `powershell` is pinned beside `bash`; on non-Windows hosts it is excluded. Pinned tools do not consume the dynamic-group or dynamic-tool budget.
2. Hidden definitions are hashed with grouping constraints and matched against `extensions/bundle-groups.ts`. Available standard tools receive checked-in groups; unavailable optional tools are omitted. 2. Hidden definitions are hashed with grouping constraints and matched against `extensions/bundle-groups.ts`. Available standard tools receive checked-in groups; unavailable optional tools are omitted.
3. If every hidden tool is recognized, the bundle catalog is used directly. No model call or per-user cache is required. 3. If every hidden tool is recognized, the bundle catalog is used directly. No model call or per-user cache is required.
4. Additional tools receive immediate deterministic prefix/source groups. A matching model-enriched cache is reused only if it preserves all checked-in bundle assignments. 4. Additional tools receive immediate deterministic prefix/source groups. A matching model-enriched cache is reused only if it preserves all checked-in bundle assignments.
@@ -13,7 +13,7 @@
9. Natural-language lookup gives exact tool names and workflow aliases more weight than descriptive overlap. Weak or closely tied matches return candidates without activation. 9. Natural-language lookup gives exact tool names and workflow aliases more weight than descriptive overlap. Weak or closely tied matches return candidates without activation.
10. `turn_start` notices tool or grouping-configuration changes by hash, resets stale dynamic groups, and reconstructs the checked-in/hybrid catalog. 10. `turn_start` notices tool or grouping-configuration changes by hash, resets stale dynamic groups, and reconstructs the checked-in/hybrid catalog.
The standard bundle assigns the Hermes memory extension's six tools to recall and management groups: `memory_search` and `session_search` join `context_tree_query` in `memory-recall`, while `memory_add`, `memory_replace`, `memory_remove`, and `skill_manage` form `memory-management`. All six are pinned by default, because the injected `<memory-policy>` tells the model to consult and write memory during normal work, and a schema the model cannot see would silently disable that loop. Third-party `tff-memory_*` names and the retired Hippo `hippo_*` names are not seeded. It also assigns `Agent`, `get_subagent_result`, and `steer_subagent` to the `subagents` orchestration group. The standard bundle assigns the Hermes memory extension's six tools to recall and management groups: `memory_search`, `session_search`, and `overflow_query` join `context_tree_query` in `memory-recall`, while `memory_add`, `memory_replace`, `memory_remove`, and `skill_manage` form `memory-management`. All six are pinned by default, because the injected `<memory-policy>` tells the model to consult and write memory during normal work, and a schema the model cannot see would silently disable that loop. Third-party `tff-memory_*` names and the retired Hippo `hippo_*` names are not seeded. It also assigns `Agent`, `get_subagent_result`, and `steer_subagent` to the `subagents` orchestration group.
The maintained `pi-ssh` tools are also split by workflow: sequential reviewed agent-controlled connection lives in `ssh-connection`, structured `ssh_read`/`ssh_write`/`ssh_edit` file operations live in `ssh-remote-files`, bounded `ssh_find`/`ssh_grep` discovery lives in `ssh-remote-search`, and sequential reviewed workspace changes plus reviewed command execution live in `ssh-remote-shell`. The workflows tell the model to call `ssh_connect` or `ssh_cd` separately and wait for success before issuing remote calls that depend on the new connection or cwd. The maintained `pi-ssh` tools are also split by workflow: sequential reviewed agent-controlled connection lives in `ssh-connection`, structured `ssh_read`/`ssh_write`/`ssh_edit` file operations live in `ssh-remote-files`, bounded `ssh_find`/`ssh_grep` discovery lives in `ssh-remote-search`, and sequential reviewed workspace changes plus reviewed command execution live in `ssh-remote-shell`. The workflows tell the model to call `ssh_connect` or `ssh_cd` separately and wait for success before issuing remote calls that depend on the new connection or cwd.
The maintained `ask_user_question` tool has its own `user-interaction` group so the model can load the complete interactive schema only when a missing decision or preference must come from the user. The maintained `ask_user_question` tool has its own `user-interaction` group so the model can load the complete interactive schema only when a missing decision or preference must come from the user.
The pinned `pi-chrome` bridge registers its 21 `chrome_*` tools only after explicit per-session authorization. Once registered, catalog refresh assigns them to `chrome-navigation`, `chrome-interaction`, and `chrome-debugging`; the 28-tool default allows all three groups to coexist. Revocation remains enforced inside every upstream tool execution even if a schema is later reactivated by Tool Search. The pinned `pi-chrome` bridge registers its 21 `chrome_*` tools only after explicit per-session authorization. Once registered, catalog refresh assigns them to `chrome-navigation`, `chrome-interaction`, and `chrome-debugging`; the 28-tool default allows all three groups to coexist. Revocation remains enforced inside every upstream tool execution even if a schema is later reactivated by Tool Search.
+3 -39
View File
@@ -85,50 +85,14 @@ export const BUNDLE_GROUP_DEFINITIONS: BundleGroupDefinition[] = [
keywords: ["web", "search", "fetch", "news", "docs", "网页", "搜索", "新闻", "文档"], keywords: ["web", "search", "fetch", "news", "docs", "网页", "搜索", "新闻", "文档"],
tools: ["web_search", "web_fetch"], tools: ["web_search", "web_fetch"],
}, },
{
id: "context-execution",
title: "Context-isolated execution",
summary: "Run commands or analyze large files in a sandbox while returning only compact derived output.",
useWhen: ["Logs, tests, builds, generated data, commands, or workspace files may produce large output"],
avoidWhen: ["Exact source lines are needed for an anchored edit"],
keywords: ["execute", "large file", "logs", "tests", "build", "大文件", "日志", "测试", "构建"],
tools: ["ctx_execute", "ctx_execute_file", "ctx_batch_execute"],
},
{
id: "context-knowledge",
title: "Context knowledge base",
summary: "Index local or web documentation and retrieve focused passages from the persistent context-mode knowledge base.",
useWhen: ["Documentation or session knowledge should be stored and queried without rereading raw content"],
avoidWhen: ["A one-shot small source can be read directly"],
keywords: ["index", "search", "knowledge base", "fetch docs", "索引", "知识库", "文档检索"],
tools: ["ctx_index", "ctx_search", "ctx_fetch_and_index"],
},
{
id: "context-observability",
title: "Context observability",
summary: "Inspect context-mode health, savings statistics, and the hosted Insight dashboard.",
useWhen: ["Diagnosing context-mode or reviewing context savings and analytics"],
avoidWhen: ["The task is normal code or data processing"],
keywords: ["stats", "doctor", "insight", "health", "统计", "诊断", "上下文节省"],
tools: ["ctx_stats", "ctx_doctor", "ctx_insight"],
},
{
id: "context-administration",
title: "Context administration",
summary: "Upgrade context-mode or destructively purge indexed context data.",
useWhen: ["The user explicitly asks to upgrade context-mode or purge a named scope"],
avoidWhen: ["Routine searching, indexing, or performance inspection"],
keywords: ["upgrade", "purge", "delete", "升级", "清除", "删除知识库"],
tools: ["ctx_upgrade", "ctx_purge"],
},
{ {
id: "memory-recall", id: "memory-recall",
title: "Persistent memory recall", title: "Persistent memory recall",
summary: "Search durable user, global, project, and failure memories, search indexed past conversations, or recover pruned tool-call output.", summary: "Search durable user, global, project, and failure memories, search indexed past conversations, or recover pruned or overflow-stored tool-call output.",
useWhen: ["The task may depend on durable context from previous sessions: preferences, project conventions, prior decisions, known failures, corrections, or condensed outputs"], useWhen: ["The task may depend on durable context from previous sessions: preferences, project conventions, prior decisions, known failures, corrections, or condensed outputs"],
avoidWhen: ["The required evidence is already in the current visible context, or the question is a generic one-off that durable memory would not help"], avoidWhen: ["The required evidence is already in the current visible context, or the question is a generic one-off that durable memory would not help"],
keywords: ["memory", "recall", "search", "session", "history", "recover", "记忆", "召回", "检索", "历史会话", "恢复输出"], keywords: ["memory", "recall", "search", "session", "history", "recover", "overflow", "记忆", "召回", "检索", "历史会话", "恢复输出"],
tools: ["memory_search", "session_search", "context_tree_query"], tools: ["memory_search", "session_search", "context_tree_query", "overflow_query"],
}, },
{ {
id: "memory-management", id: "memory-management",
-1
View File
@@ -335,7 +335,6 @@ export function writeCachedCatalog(cachePath: string, catalog: ToolCatalog): voi
} }
const GROUP_QUERY_ALIASES: Record<string, string[]> = { const GROUP_QUERY_ALIASES: Record<string, string[]> = {
"context-execution": ["context mode", "run command", "test build logs", "执行命令", "运行测试", "分析日志"],
"ssh-connection": ["ssh connect", "remote server connection", "连接服务器", "远程连接"], "ssh-connection": ["ssh connect", "remote server connection", "连接服务器", "远程连接"],
"ssh-remote-shell": ["ssh shell", "remote command", "remote terminal", "远程命令", "远程终端"], "ssh-remote-shell": ["ssh shell", "remote command", "remote terminal", "远程命令", "远程终端"],
"ssh-remote-files": ["ssh file", "remote read write edit", "远程文件", "远程读写"], "ssh-remote-files": ["ssh file", "remote read write edit", "远程文件", "远程读写"],
+1 -3
View File
@@ -25,9 +25,6 @@ export const BUNDLE_TOOL_SEARCH_DEFAULTS = {
alwaysEnabled: [ alwaysEnabled: [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
@@ -35,6 +32,7 @@ export const BUNDLE_TOOL_SEARCH_DEFAULTS = {
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query", "context_tree_query",
"overflow_query",
], ],
showToolSearchFooterStatus: false, showToolSearchFooterStatus: false,
maxActiveGroups: 5, maxActiveGroups: 5,
+1 -1
View File
@@ -25,7 +25,7 @@ test("bundle seed names are unique and cover the declared my-pi tools", () => {
assert.deepEqual(result.unknownTools, []); assert.deepEqual(result.unknownTools, []);
assert.equal(result.coveredNames.size, names.length); assert.equal(result.coveredNames.size, names.length);
assert.equal(result.catalog.tools.length, names.length); assert.equal(result.catalog.tools.length, names.length);
assert.equal(result.catalog.tools.find((card) => card.name === "ctx_purge")?.primaryGroup, "context-administration"); assert.equal(result.catalog.tools.find((card) => card.name === "overflow_query")?.primaryGroup, "memory-recall");
assert.equal(result.catalog.tools.find((card) => card.name === "web_fetch")?.primaryGroup, "web-search"); assert.equal(result.catalog.tools.find((card) => card.name === "web_fetch")?.primaryGroup, "web-search");
assert.equal(result.catalog.tools.find((card) => card.name === "memory_search")?.primaryGroup, "memory-recall"); assert.equal(result.catalog.tools.find((card) => card.name === "memory_search")?.primaryGroup, "memory-recall");
assert.equal(result.catalog.tools.find((card) => card.name === "session_search")?.primaryGroup, "memory-recall"); assert.equal(result.catalog.tools.find((card) => card.name === "session_search")?.primaryGroup, "memory-recall");
+1 -3
View File
@@ -33,9 +33,6 @@ test("bundle defaults keep the committed tool list", () => {
assert.deepEqual(BUNDLE_TOOL_SEARCH_DEFAULTS.alwaysEnabled, [ assert.deepEqual(BUNDLE_TOOL_SEARCH_DEFAULTS.alwaysEnabled, [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
@@ -43,6 +40,7 @@ test("bundle defaults keep the committed tool list", () => {
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query", "context_tree_query",
"overflow_query",
]); ]);
assert.deepEqual(bundleFallbackConfig(), { assert.deepEqual(bundleFallbackConfig(), {
alwaysEnabled: [...BUNDLE_TOOL_SEARCH_DEFAULTS.alwaysEnabled], alwaysEnabled: [...BUNDLE_TOOL_SEARCH_DEFAULTS.alwaysEnabled],
+4 -4
View File
@@ -254,9 +254,9 @@ test("uses the precomputed bundle groups without calling a model", async () => {
sourceTool("read", "Read a file"), sourceTool("read", "Read a file"),
sourceTool("web_search", "Search broadly for current web sources"), sourceTool("web_search", "Search broadly for current web sources"),
sourceTool("web_fetch", "Fetch a selected web source"), sourceTool("web_fetch", "Fetch a selected web source"),
sourceTool("ctx_execute", "Run commands in a context sandbox"), sourceTool("context_tree_query", "Inspect the context tree ledger"),
sourceTool("ctx_execute_file", "Analyze a file in a context sandbox"), sourceTool("overflow_query", "Inspect a stored oversized tool result"),
sourceTool("ctx_batch_execute", "Run a bounded command batch"), sourceTool("memory_search", "Search durable memories"),
]; ];
const api = { const api = {
getAllTools: () => sourceTools, getAllTools: () => sourceTools,
@@ -280,7 +280,7 @@ test("uses the precomputed bundle groups without calling a model", async () => {
} as unknown as ExtensionContext["modelRegistry"], } as unknown as ExtensionContext["modelRegistry"],
}); });
handlers.get("session_start")?.({}, ctx); handlers.get("session_start")?.({}, ctx);
assert.ok(["ctx_execute", "ctx_execute_file", "ctx_batch_execute"].every((name) => activeCalls.at(-1)?.includes(name))); assert.ok(["context_tree_query", "overflow_query", "memory_search"].every((name) => activeCalls.at(-1)?.includes(name)));
const ambiguous = await registered.get("tool_search")?.execute( const ambiguous = await registered.get("tool_search")?.execute(
"call-1", "call-1",
+4 -1
View File
@@ -6,9 +6,12 @@ Unified web search and fetch for Pi. The main model only ever sees two tools —
- `web_search(query, max_results?, site?, published_after?, published_before?, depth?)` - `web_search(query, max_results?, site?, published_after?, published_before?, depth?)`
routes each query to Tavily, Exa, or Keenable using the session's current main routes each query to Tavily, Exa, or Keenable using the session's current main
model (`ctx.model` + `ctx.modelRegistry.complete`), guided by per-provider model (`ctx.model` + `ctx.modelRegistry.complete`), guided by per-provider
rules. Results include `details.provider` / `details.routedBy` for debugging. rules. `max_results` defaults to 8 and is clamped to 1–20. Results include
`details.provider` / `details.routedBy` for debugging.
- `web_fetch(urls, max_characters?)` always reads pages via Exa - `web_fetch(urls, max_characters?)` always reads pages via Exa
(`web_fetch_exa`), degrading silently to Keenable then Tavily. (`web_fetch_exa`), degrading silently to Keenable then Tavily.
`max_characters` defaults to 20,000 and is clamped to 1–100,000 per page;
oversized fetch output is further bounded by pi-overflow downstream.
## Providers (`clients/`) ## Providers (`clients/`)
+17 -6
View File
@@ -21,8 +21,15 @@ import {
} from "./router.ts"; } from "./router.ts";
const DEFAULT_MAX_RESULTS = 8; const DEFAULT_MAX_RESULTS = 8;
const MAX_RESULTS_LIMIT = 20;
const MAX_FETCH_URLS = 10; const MAX_FETCH_URLS = 10;
const DEFAULT_FETCH_MAX_CHARACTERS = 3000; const DEFAULT_FETCH_MAX_CHARACTERS = 20000;
const FETCH_MAX_CHARACTERS_LIMIT = 100000;
function clampInt(value: number | undefined, fallback: number, min: number, max: number): number {
if (typeof value !== "number" || !Number.isFinite(value)) return fallback;
return Math.min(Math.max(Math.floor(value), min), max);
}
export interface WebSearchParams { export interface WebSearchParams {
query: string; query: string;
@@ -48,7 +55,7 @@ function formatSearchPayload(payload: unknown): string {
export function toTavilyParams(params: WebSearchParams) { export function toTavilyParams(params: WebSearchParams) {
return { return {
query: params.query, query: params.query,
max_results: params.max_results ?? DEFAULT_MAX_RESULTS, max_results: clampInt(params.max_results, DEFAULT_MAX_RESULTS, 1, MAX_RESULTS_LIMIT),
search_depth: (params.depth ?? "basic") as "basic" | "advanced", search_depth: (params.depth ?? "basic") as "basic" | "advanced",
include_domains: params.site ? [params.site] : undefined, include_domains: params.site ? [params.site] : undefined,
start_date: params.published_after, start_date: params.published_after,
@@ -59,7 +66,7 @@ export function toTavilyParams(params: WebSearchParams) {
export function toExaParams(params: WebSearchParams) { export function toExaParams(params: WebSearchParams) {
return { return {
query: params.query, query: params.query,
numResults: params.max_results ?? DEFAULT_MAX_RESULTS, numResults: clampInt(params.max_results, DEFAULT_MAX_RESULTS, 1, MAX_RESULTS_LIMIT),
includeDomains: params.site ? [params.site] : undefined, includeDomains: params.site ? [params.site] : undefined,
startPublishedDate: params.published_after, startPublishedDate: params.published_after,
endPublishedDate: params.published_before, endPublishedDate: params.published_before,
@@ -171,7 +178,7 @@ export async function dispatchFetch(
if (urls.length > MAX_FETCH_URLS) { if (urls.length > MAX_FETCH_URLS) {
return toResult(`web_fetch accepts at most ${MAX_FETCH_URLS} URLs per call; received ${urls.length}.`, { error: true }); return toResult(`web_fetch accepts at most ${MAX_FETCH_URLS} URLs per call; received ${urls.length}.`, { error: true });
} }
const maxCharacters = params.max_characters ?? DEFAULT_FETCH_MAX_CHARACTERS; const maxCharacters = clampInt(params.max_characters, DEFAULT_FETCH_MAX_CHARACTERS, 1, FETCH_MAX_CHARACTERS_LIMIT);
const order: SearchProvider[] = ["exa", "keenable"]; const order: SearchProvider[] = ["exa", "keenable"];
if (tavilyAvailable(resolve)) order.push("tavily"); if (tavilyAvailable(resolve)) order.push("tavily");
@@ -213,7 +220,9 @@ export default function webSearchExtension(pi: ExtensionAPI): void {
}, },
max_results: { max_results: {
type: "number", type: "number",
description: `Maximum number of results to return (default ${DEFAULT_MAX_RESULTS}).`, minimum: 1,
maximum: MAX_RESULTS_LIMIT,
description: `Maximum number of results to return (default ${DEFAULT_MAX_RESULTS}, max ${MAX_RESULTS_LIMIT}).`,
}, },
site: { site: {
type: "string", type: "string",
@@ -268,7 +277,9 @@ export default function webSearchExtension(pi: ExtensionAPI): void {
}, },
max_characters: { max_characters: {
type: "number", type: "number",
description: `Maximum characters to extract per page (default ${DEFAULT_FETCH_MAX_CHARACTERS}).`, minimum: 1,
maximum: FETCH_MAX_CHARACTERS_LIMIT,
description: `Maximum characters to extract per page (default ${DEFAULT_FETCH_MAX_CHARACTERS}, max ${FETCH_MAX_CHARACTERS_LIMIT}).`,
}, },
}, },
required: ["urls"], required: ["urls"],
+51 -28
View File
@@ -10,13 +10,16 @@ import test from "node:test";
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), ".."); const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const hasPi = spawnSync("pi", ["--version"], { encoding: "utf8" }).status === 0; const hasPi = spawnSync("pi", ["--version"], { encoding: "utf8" }).status === 0;
const filesIncludePackageDir = (files: string[], packageDir: string): boolean =>
files.some((entry) => entry === packageDir || entry.startsWith(`${packageDir}/`));
test("enabled subagents wrapper loads with its bundle-owned mandatory permission entry", { skip: !hasPi }, async () => { test("enabled subagents wrapper loads with its bundle-owned mandatory permission entry", { skip: !hasPi }, async () => {
const packageJson = JSON.parse(await readFile(join(repositoryRoot, "package.json"), "utf8")) as { const packageJson = JSON.parse(await readFile(join(repositoryRoot, "package.json"), "utf8")) as {
dependencies: Record<string, string>; dependencies: Record<string, string>;
files: string[]; files: string[];
pi: { extensions: string[] }; pi: { extensions: string[] };
}; };
assert.ok(packageJson.files.includes("pi-subagents")); assert.ok(filesIncludePackageDir(packageJson.files, "pi-subagents"));
assert.equal(packageJson.dependencies["@tintinweb/pi-subagents"], "file:./pi-subagents"); assert.equal(packageJson.dependencies["@tintinweb/pi-subagents"], "file:./pi-subagents");
assert.ok( assert.ok(
packageJson.pi.extensions.includes("./extensions/subagents.ts"), packageJson.pi.extensions.includes("./extensions/subagents.ts"),
@@ -48,8 +51,8 @@ test("all package extensions load together without global registration conflicts
const packageJson = JSON.parse(await readFile(join(repositoryRoot, "package.json"), "utf8")) as { const packageJson = JSON.parse(await readFile(join(repositoryRoot, "package.json"), "utf8")) as {
dependencies: Record<string, string>; dependencies: Record<string, string>;
files: string[]; files: string[];
pi: { extensions: string[] }; pi: { extensions: string[]; skills?: string[] };
allowScripts: Record<string, boolean>; allowScripts?: Record<string, boolean>;
}; };
assert.ok( assert.ok(
packageJson.pi.extensions.includes("./extensions/codegraph.ts"), packageJson.pi.extensions.includes("./extensions/codegraph.ts"),
@@ -74,7 +77,7 @@ test("all package extensions load together without global registration conflicts
"the bundle must load the locally maintained minimal footer", "the bundle must load the locally maintained minimal footer",
); );
assert.equal(packageJson.dependencies["pi-compaction"], "file:./pi-compaction"); assert.equal(packageJson.dependencies["pi-compaction"], "file:./pi-compaction");
assert.ok(packageJson.files.includes("pi-compaction"), "the packed bundle must include maintained pi-compaction source"); assert.ok(filesIncludePackageDir(packageJson.files, "pi-compaction"), "the packed bundle must include maintained pi-compaction source");
assert.ok( assert.ok(
packageJson.pi.extensions.includes("./pi-compaction/index.ts"), packageJson.pi.extensions.includes("./pi-compaction/index.ts"),
"the bundle must load the unified pi-compaction authority", "the bundle must load the unified pi-compaction authority",
@@ -97,20 +100,20 @@ test("all package extensions load together without global registration conflicts
assert.equal(packageJson.dependencies["pi-ssh"], "file:./pi-ssh"); assert.equal(packageJson.dependencies["pi-ssh"], "file:./pi-ssh");
assert.match(packageJson.dependencies.ssh2 ?? "", /^\d+\.\d+\.\d+$/, "the packed bundle must install pi-ssh's runtime transport at an exact version"); assert.match(packageJson.dependencies.ssh2 ?? "", /^\d+\.\d+\.\d+$/, "the packed bundle must install pi-ssh's runtime transport at an exact version");
assert.match(packageJson.dependencies.jiti ?? "", /^\d+\.\d+\.\d+$/, "the packed helper must load local TypeScript outside Pi at an exact version"); assert.match(packageJson.dependencies.jiti ?? "", /^\d+\.\d+\.\d+$/, "the packed helper must load local TypeScript outside Pi at an exact version");
assert.ok(packageJson.files.includes("pi-ssh"), "the packed bundle must include the locally maintained SSH source"); assert.ok(filesIncludePackageDir(packageJson.files, "pi-ssh"), "the packed bundle must include the locally maintained SSH source");
assert.ok(packageJson.files.includes("configure.sh"), "the packed bundle must include the search/SSH config helper"); assert.ok(packageJson.files.includes("configure.sh"), "the packed bundle must include the search/SSH config helper");
assert.ok(packageJson.pi.extensions.includes("./pi-ask-user/index.ts"), "the bundle must load pi-ask-user"); assert.ok(packageJson.pi.extensions.includes("./pi-ask-user/index.ts"), "the bundle must load pi-ask-user");
assert.equal(packageJson.dependencies["pi-ask-user"], "file:./pi-ask-user"); assert.equal(packageJson.dependencies["pi-ask-user"], "file:./pi-ask-user");
assert.ok(packageJson.files.includes("pi-ask-user"), "the packed bundle must include pi-ask-user source"); assert.ok(filesIncludePackageDir(packageJson.files, "pi-ask-user"), "the packed bundle must include pi-ask-user source");
assert.ok( assert.ok(
packageJson.pi.extensions.includes("./pi-worktree/index.ts"), packageJson.pi.extensions.includes("./pi-worktree/index.ts"),
"the bundle must load the locally maintained worktree commands", "the bundle must load the locally maintained worktree commands",
); );
assert.equal(packageJson.dependencies["pi-worktree"], "file:./pi-worktree"); assert.equal(packageJson.dependencies["pi-worktree"], "file:./pi-worktree");
assert.ok(packageJson.files.includes("pi-worktree"), "the packed bundle must include pi-worktree source"); assert.ok(filesIncludePackageDir(packageJson.files, "pi-worktree"), "the packed bundle must include pi-worktree source");
const chromeExtension = "./pi-chrome/extensions/chrome-profile-bridge/index.ts"; const chromeExtension = "./pi-chrome/extensions/chrome-profile-bridge/index.ts";
assert.equal(packageJson.dependencies["pi-chrome"], "file:./pi-chrome", "the browser bridge must use the locally maintained source"); assert.equal(packageJson.dependencies["pi-chrome"], "file:./pi-chrome", "the browser bridge must use the locally maintained source");
assert.ok(packageJson.files.includes("pi-chrome"), "the packed bundle must include the locally maintained pi-chrome source"); assert.ok(filesIncludePackageDir(packageJson.files, "pi-chrome"), "the packed bundle must include the locally maintained pi-chrome source");
assert.ok(packageJson.pi.extensions.includes(chromeExtension), "the bundle must load the local Chrome profile bridge directly"); assert.ok(packageJson.pi.extensions.includes(chromeExtension), "the bundle must load the local Chrome profile bridge directly");
assert.ok( assert.ok(
packageJson.pi.extensions.indexOf(chromeExtension) < packageJson.pi.extensions.indexOf(chromeExtension) <
@@ -167,7 +170,7 @@ test("all package extensions load together without global registration conflicts
); );
assert.equal(packageJson.dependencies["@smoose/pi-notify"], "file:./pi-notify"); assert.equal(packageJson.dependencies["@smoose/pi-notify"], "file:./pi-notify");
assert.ok( assert.ok(
packageJson.files.includes("pi-notify"), filesIncludePackageDir(packageJson.files, "pi-notify"),
"the packed bundle must include the locally maintained notification source", "the packed bundle must include the locally maintained notification source",
); );
assert.equal(packageJson.dependencies["@ogulcancelik/pi-minimal-footer"], "file:./pi-minimal-footer"); assert.equal(packageJson.dependencies["@ogulcancelik/pi-minimal-footer"], "file:./pi-minimal-footer");
@@ -195,14 +198,34 @@ test("all package extensions load together without global registration conflicts
"./extensions/system-prompt.ts", "./extensions/system-prompt.ts",
"the single system-prompt owner must be the last loaded extension", "the single system-prompt owner must be the last loaded extension",
); );
assert.ok( // pi-overflow has two entry points with strict ordering: capture.ts must
packageJson.pi.extensions.includes("./extensions/context-mode.ts"), // run first so it sees results before RTK rewrites them, and index.ts must
"Context Mode must load through the bundle's prompt-guarded wrapper", // run last (before system-prompt) so it bounds the final visible text.
assert.equal(
packageJson.pi.extensions.at(0),
"./pi-overflow/capture.ts",
"pi-overflow capture must be the first loaded extension",
);
assert.equal(packageJson.dependencies["pi-overflow"], "file:./pi-overflow");
assert.ok(filesIncludePackageDir(packageJson.files, "pi-overflow"), "the packed bundle must include pi-overflow source");
assert.equal(
packageJson.pi.extensions.at(-2),
"./pi-overflow/index.ts",
"pi-overflow bound stage must load after all other content extensions and before system-prompt",
); );
assert.equal( assert.equal(
packageJson.pi.extensions.some((entry) => entry.includes("context-mode/build/")), packageJson.dependencies["context-mode"],
undefined,
"context-mode was retired; new tool-result bounding is owned by pi-overflow",
);
assert.equal(
packageJson.pi.extensions.some((entry) => entry.includes("context-mode")),
false, false,
"the raw Context Mode adapter must not be loaded alongside the wrapper", "no context-mode entry point may be loaded",
);
assert.ok(
!Array.isArray(packageJson.pi.skills) || packageJson.pi.skills.length === 0,
"pi.skills must be absent or empty now that the context-mode skills are gone",
); );
assert.equal( assert.equal(
packageJson.pi.extensions.includes("./extensions/system-prompt-cover.ts"), packageJson.pi.extensions.includes("./extensions/system-prompt-cover.ts"),
@@ -227,7 +250,7 @@ test("all package extensions load together without global registration conflicts
} }
assert.equal(packageJson.dependencies["pi-hermes-memory"], "file:./pi-hermes-memory"); assert.equal(packageJson.dependencies["pi-hermes-memory"], "file:./pi-hermes-memory");
assert.ok( assert.ok(
packageJson.files.includes("pi-hermes-memory"), filesIncludePackageDir(packageJson.files, "pi-hermes-memory"),
"the packed bundle must include the locally maintained Hermes memory source", "the packed bundle must include the locally maintained Hermes memory source",
); );
assert.ok( assert.ok(
@@ -242,28 +265,28 @@ test("all package extensions load together without global registration conflicts
); );
assert.equal(packageJson.dependencies["@the-forge-flow/hippo-memory-pi"], undefined); assert.equal(packageJson.dependencies["@the-forge-flow/hippo-memory-pi"], undefined);
assert.equal(packageJson.dependencies["better-sqlite3"], undefined); assert.equal(packageJson.dependencies["better-sqlite3"], undefined);
// allowScripts must approve exactly the root lockfile's better-sqlite3 // better-sqlite3 was only present as context-mode's transitive build
// build — and nothing else. pi-hermes-memory's own lockfile carries a // dependency; with context-mode retired the root lockfile must not carry a
// nested major that ships N-API prebuilds and needs no approval entry. // root-level better-sqlite3 and allowScripts must not approve one.
// pi-hermes-memory's own lockfile carries a nested major that ships N-API
// prebuilds and needs no approval entry.
const rootLock = JSON.parse(await readFile(join(repositoryRoot, "package-lock.json"), "utf8")) as { const rootLock = JSON.parse(await readFile(join(repositoryRoot, "package-lock.json"), "utf8")) as {
packages: Record<string, { version?: string }>; packages: Record<string, { version?: string }>;
}; };
const hermesLock = JSON.parse(await readFile(join(repositoryRoot, "pi-hermes-memory", "package-lock.json"), "utf8")) as { const hermesLock = JSON.parse(await readFile(join(repositoryRoot, "pi-hermes-memory", "package-lock.json"), "utf8")) as {
packages: Record<string, { version?: string }>; packages: Record<string, { version?: string }>;
}; };
const rootSqliteVersion = rootLock.packages["node_modules/better-sqlite3"]?.version;
const hermesSqliteVersion = hermesLock.packages["node_modules/better-sqlite3"]?.version; const hermesSqliteVersion = hermesLock.packages["node_modules/better-sqlite3"]?.version;
assert.match(rootSqliteVersion ?? "", /^\d+\.\d+\.\d+$/, "the root lockfile must pin a root-level better-sqlite3"); assert.equal(
rootLock.packages["node_modules/better-sqlite3"],
undefined,
"the root lockfile must not pin a root-level better-sqlite3 now that context-mode is gone",
);
assert.match(hermesSqliteVersion ?? "", /^\d+\.\d+\.\d+$/, "the Hermes lockfile must pin its own better-sqlite3"); assert.match(hermesSqliteVersion ?? "", /^\d+\.\d+\.\d+$/, "the Hermes lockfile must pin its own better-sqlite3");
assert.deepEqual( assert.deepEqual(
Object.keys(packageJson.allowScripts).filter((key) => key.startsWith("better-sqlite3@")), Object.keys(packageJson.allowScripts ?? {}).filter((key) => key.startsWith("better-sqlite3@")),
[`better-sqlite3@${rootSqliteVersion}`], [],
"allowScripts must approve exactly the root lockfile's better-sqlite3 build", "allowScripts must not approve a better-sqlite3 build without a root-level package",
);
assert.equal(
packageJson.allowScripts[`better-sqlite3@${hermesSqliteVersion}`],
undefined,
`Hermes uses better-sqlite3 ${hermesSqliteVersion} N-API prebuilds, so no extra install-script approval is required`,
); );
assert.equal(packageJson.dependencies["@sinclair/typebox"], undefined); assert.equal(packageJson.dependencies["@sinclair/typebox"], undefined);
const home = await mkdtemp(join(tmpdir(), "my-pi-extension-load-")); const home = await mkdtemp(join(tmpdir(), "my-pi-extension-load-"));
+94
View File
@@ -0,0 +1,94 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { readFileSync } from "node:fs";
import { dirname, join, posix, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import test from "node:test";
const repositoryRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const packageJson = JSON.parse(readFileSync(join(repositoryRoot, "package.json"), "utf8")) as {
dependencies: Record<string, string>;
pi: { extensions: string[] };
};
const packOutput = execFileSync("npm", ["pack", "--dry-run", "--json", "--ignore-scripts"], {
cwd: repositoryRoot,
encoding: "utf8",
});
const packResult = JSON.parse(packOutput.slice(packOutput.indexOf("["))) as Array<{
files: Array<{ path: string }>;
}>;
const packedFiles = new Set(packResult[0].files.map((entry) => entry.path));
test("the packed bundle ships no nested node_modules or build output", () => {
assert.ok(packedFiles.size > 0, "npm pack --dry-run must report a non-empty file list");
for (const filePath of packedFiles) {
const segments = filePath.split("/");
assert.equal(segments.includes("node_modules"), false, `${filePath} must not ship a nested node_modules tree`);
assert.equal(segments.includes("dist"), false, `${filePath} must not ship build output`);
}
});
test("every declared extension entry and file: dependency manifest is inside the packed bundle", () => {
for (const entry of packageJson.pi.extensions) {
const bundled = entry.replace(/^\.\//u, "");
assert.ok(packedFiles.has(bundled), `extension entry ${entry} must be inside the packed bundle`);
}
for (const [name, specifier] of Object.entries(packageJson.dependencies)) {
const match = /^file:\.\/(.+)$/u.exec(specifier);
if (!match) continue;
const manifest = `${match[1]}/package.json`;
assert.ok(packedFiles.has(manifest), `${name} is a file: dependency — ${manifest} must be packed`);
}
});
test("relative import specifiers inside packed sources resolve to packed files", () => {
const specifierPatterns = [
/\bfrom\s+["'](\.{1,2}\/[^"']+)["']/gu,
/(?<![\w$.])import\s*\(\s*["'](\.{1,2}\/[^"']+)["']\s*\)/gu,
/\bnew\s+URL\s*\(\s*["'](\.{1,2}\/[^"']+)["']\s*,\s*import\.meta\.url\s*\)/gu,
];
const isPacked = (resolved: string): boolean => {
if (packedFiles.has(resolved)) return true;
const candidates = [`${resolved}.ts`, `${resolved}.js`, `${resolved}.mjs`, `${resolved}/index.ts`];
if (candidates.some((candidate) => packedFiles.has(candidate))) return true;
// TypeScript sources import sibling ".js"/".mjs" specifiers that map to ".ts" files.
const swapped = resolved.replace(/\.(js|mjs|cjs)$/u, ".ts");
if (swapped !== resolved && packedFiles.has(swapped)) return true;
// new URL("../config/prompt/", import.meta.url) style directory references.
for (const filePath of packedFiles) {
if (filePath.startsWith(`${resolved}/`)) return true;
}
return false;
};
const unresolved: string[] = [];
for (const filePath of packedFiles) {
if (!/\.(ts|js|mjs)$/u.test(filePath)) continue;
const segments = filePath.split("/");
if (/\.test\.(ts|js|mjs)$/u.test(filePath) || segments.includes("test") || segments.includes("tests")) continue;
const source = readFileSync(join(repositoryRoot, filePath), "utf8");
for (const pattern of specifierPatterns) {
pattern.lastIndex = 0;
for (const match of source.matchAll(pattern)) {
const resolved = posix.normalize(posix.join(posix.dirname(filePath), match[1])).replace(/\/$/u, "");
if (resolved === ".." || resolved.startsWith("../")) continue;
if (resolved.split("/").includes("node_modules")) continue;
if (!isPacked(resolved)) unresolved.push(`${filePath} -> ${match[1]}`);
}
}
}
assert.deepEqual(unresolved, [], "packed sources reference files missing from the bundle");
});
test("prompt config and lifecycle scripts are inside the packed bundle", () => {
const tracked = execFileSync("git", ["ls-files", "config/prompt"], {
cwd: repositoryRoot,
encoding: "utf8",
})
.split("\n")
.filter(Boolean);
for (const filePath of [...tracked, "install.sh", "update.sh", "uninstall.sh", "configure.sh", "scripts/lib.sh"]) {
assert.ok(packedFiles.has(filePath), `${filePath} must be inside the packed bundle`);
}
});
+3 -69
View File
@@ -29,10 +29,6 @@ import systemPromptExtension, {
getSystemPromptDumpPath, getSystemPromptDumpPath,
} from "../extensions/system-prompt.ts"; } from "../extensions/system-prompt.ts";
import { guardExtensionApi } from "../extensions/prompt-guard.ts"; import { guardExtensionApi } from "../extensions/prompt-guard.ts";
import {
CONTEXT_MODE_ANCHOR_PREFIX,
filterContextModeMessages,
} from "../extensions/context-mode-filter.ts";
import { BUNDLE_GROUP_DEFINITIONS } from "../pi-tool-search/extensions/bundle-groups.ts"; import { BUNDLE_GROUP_DEFINITIONS } from "../pi-tool-search/extensions/bundle-groups.ts";
import { BUNDLE_TOOL_SEARCH_DEFAULTS } from "../pi-tool-search/extensions/config.ts"; import { BUNDLE_TOOL_SEARCH_DEFAULTS } from "../pi-tool-search/extensions/config.ts";
@@ -164,7 +160,6 @@ test("prompt config carries no retired guidance phrasing", () => {
"read/grep", "read/grep",
"replace_text", "replace_text",
"Read/edit files", "Read/edit files",
"ctx_fetch_and_index then ctx_search",
]; ];
for (const file of ["preamble.md", "tools.jsonc", "routing.jsonc"]) { for (const file of ["preamble.md", "tools.jsonc", "routing.jsonc"]) {
const source = readFileSync(join(promptConfigDir, file), "utf8"); const source = readFileSync(join(promptConfigDir, file), "utf8");
@@ -267,8 +262,8 @@ test("bundled routing rules select the same guidance the retired tool-routing ex
assert.match(ssh, /only when the user explicitly names an imported host/); assert.match(ssh, /only when the user explicitly names an imported host/);
assert.match(ssh, /command-local directory change/); assert.match(ssh, /command-local directory change/);
const chrome = renderRouting(routing.rules, ["chrome_snapshot", "ctx_execute_file"]); const chrome = renderRouting(routing.rules, ["chrome_snapshot", "overflow_query"]);
assert.match(chrome, /immediately analyze the returned workspace-relative capture path/); assert.match(chrome, /return the page snapshot JSON directly/);
const minimal = renderRouting(routing.rules, ["read"]); const minimal = renderRouting(routing.rules, ["read"]);
assert.doesNotMatch(minimal, /codegraph_explore first/); assert.doesNotMatch(minimal, /codegraph_explore first/);
@@ -646,7 +641,7 @@ test("prompt guard runs context results through the provided contextFilter", asy
test("prompt guard returns the handler result untouched when the filter changes nothing", async () => { test("prompt guard returns the handler result untouched when the filter changes nothing", async () => {
const { pi, handlers } = makeRecordingPi(); const { pi, handlers } = makeRecordingPi();
const guarded = guardExtensionApi(pi as unknown as ExtensionAPI, { const guarded = guardExtensionApi(pi as unknown as ExtensionAPI, {
contextFilter: filterContextModeMessages, contextFilter: (_before, after) => after,
}); });
const expected = { messages: [{ role: "user", content: "plain" }] }; const expected = { messages: [{ role: "user", content: "plain" }] };
@@ -662,67 +657,6 @@ test("prompt guard passes unrelated API members through to the real pi object",
assert.equal((guarded as unknown as { someOtherApi: () => string }).someOtherApi(), "passthrough"); assert.equal((guarded as unknown as { someOtherApi: () => string }).someOtherApi(), "passthrough");
}); });
// ── context-mode filter ─────────────────────────────────────────────────────
test("context-mode filter drops only the routing-anchor part", () => {
const before = [{ role: "user", content: "real" }];
const anchor = `${CONTEXT_MODE_ANCHOR_PREFIX} ctx_batch_execute > ctx_execute`;
const after = [
...before,
{ role: "user", content: `${anchor}\n\n<active_memory>\nx\n</active_memory>` },
];
const filtered = filterContextModeMessages(before, after);
assert.equal(filtered.length, 2);
assert.equal(filtered[1].content, "<active_memory>\nx\n</active_memory>");
});
test("context-mode filter removes the appended message when only the anchor remains", () => {
const before = [{ role: "user", content: "real" }];
const after = [
...before,
{ role: "user", content: `${CONTEXT_MODE_ANCHOR_PREFIX} only anchor` },
];
assert.deepEqual(filterContextModeMessages(before, after), before);
});
test("context-mode filter returns the same reference when nothing changed", () => {
const before = [{ role: "user", content: "real" }];
const after = [...before, { role: "user", content: "no anchor here" }];
assert.equal(filterContextModeMessages(before, after), after);
assert.equal(filterContextModeMessages(after, after), after);
});
test("context-mode filter ignores appended non-user or non-string messages", () => {
const before = [{ role: "user", content: "real" }];
const after = [
...before,
{ role: "assistant", content: `${CONTEXT_MODE_ANCHOR_PREFIX} not ours` },
{ role: "user", content: [{ type: "text", text: "structured" }] },
];
assert.equal(filterContextModeMessages(before, after), after);
});
// ── Pinned upstream text ────────────────────────────────────────────────────
test("context-mode upstream still appends the single anchor-bearing user message", async () => {
const packageJson = JSON.parse(
await readFile(join(repositoryRoot, "package.json"), "utf8"),
) as { dependencies: Record<string, string> };
assert.match(
packageJson.dependencies["context-mode"] ?? "",
/^\d+\.\d+\.\d+$/,
"the context-filter pin requires an exact upstream version",
);
const upstream = await readFile(
join(repositoryRoot, "node_modules/context-mode/build/adapters/pi/extension.js"),
"utf8",
);
assert.ok(upstream.includes('parts.push("context-mode active. Hierarchy:'));
assert.ok(upstream.includes('_pendingContext = extraParts.join("\\n\\n")'));
assert.ok(upstream.includes("event.messages.push({"));
assert.ok(upstream.includes('role: "user"'));
});
// ── /dump-system-prompt ───────────────────────────────────────────────────── // ── /dump-system-prompt ─────────────────────────────────────────────────────
test("dump-system-prompt writes the last composed prompt under .pi-debug", async () => { test("dump-system-prompt writes the last composed prompt under .pi-debug", async () => {
+6 -3
View File
@@ -5,9 +5,12 @@ import test from "node:test";
const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")) as { const packageJson = JSON.parse(await readFile(new URL("../package.json", import.meta.url), "utf8")) as {
dependencies: Record<string, string>; dependencies: Record<string, string>;
files: string[]; files: string[];
pi: { extensions: string[]; skills: string[] }; pi: { extensions: string[]; skills?: string[] };
}; };
const filesIncludePackageDir = (files: string[], packageDir: string): boolean =>
files.some((entry) => entry === packageDir || entry.startsWith(`${packageDir}/`));
test("pi-web-search is the single search surface and reads credentials from auth.json", () => { test("pi-web-search is the single search surface and reads credentials from auth.json", () => {
const entries = packageJson.pi.extensions; const entries = packageJson.pi.extensions;
assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search"); assert.ok(entries.includes("./pi-web-search/index.ts"), "the bundle must load pi-web-search");
@@ -27,7 +30,7 @@ test("pi-web-search is the single search surface and reads credentials from auth
test("pi-web-search ships as a maintained file: dependency without upstream search packages", () => { test("pi-web-search ships as a maintained file: dependency without upstream search packages", () => {
assert.equal(packageJson.dependencies["pi-web-search"], "file:./pi-web-search"); assert.equal(packageJson.dependencies["pi-web-search"], "file:./pi-web-search");
assert.ok(packageJson.files.includes("pi-web-search")); assert.ok(filesIncludePackageDir(packageJson.files, "pi-web-search"));
assert.equal(packageJson.dependencies["@tavily/pi-extension"], undefined); assert.equal(packageJson.dependencies["@tavily/pi-extension"], undefined);
assert.equal(packageJson.dependencies["@keenable/pi-search"], undefined); assert.equal(packageJson.dependencies["@keenable/pi-search"], undefined);
assert.equal(packageJson.dependencies["pi-mcp-adapter"], undefined, "Exa is a direct client now — no MCP adapter"); assert.equal(packageJson.dependencies["pi-mcp-adapter"], undefined, "Exa is a direct client now — no MCP adapter");
@@ -42,7 +45,7 @@ test("no extension in the bundle loads the MCP adapter or upstream search extens
}); });
test("the keenable skill path is gone with the upstream package", () => { test("the keenable skill path is gone with the upstream package", () => {
for (const skillPath of packageJson.pi.skills) { for (const skillPath of packageJson.pi.skills ?? []) {
assert.equal(skillPath.includes("@keenable"), false); assert.equal(skillPath.includes("@keenable"), false);
} }
}); });