Compare commits

..
10 Commits
Author SHA1 Message Date
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 87940da51a feat(prompt): 重新注册 bash,把"不要用 bash 搜索"写进工具描述
真实会话里模型仍习惯性先调 bash grep,被拒后才改用 grep 工具;系统提示词里已有 4 处
提醒,但模型下调用决定时看的是工具 schema 描述,而 Pi 内置 bash 描述对此只字未提。

- 新增 extensions/bash-override.ts:用 Pi 的 createBashToolDefinition 重新注册 bash,
  执行、输出结构与 codemode 说明完全沿用上游;shellPath/commandPrefix 按 ctx.cwd
  经 SettingsManager 惰性解析,与内置一致
- tools.jsonc 新增 descriptionAppend 字段,由 prompt-guard 注册时拼接到上游描述后
- bash 增加命令→工具对照规则(grep -n / grep -rl / find -name),并提示
  文件清单用 $(git ls-files) 传参、不要接 xargs(会触发权限审核)

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 01:20:52 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 2b21163a18 fix(pi-permission-system): bash 命令枚举下降进入控制流与赋值载荷
for/while/if/case/{ }/函数体此前只按整段文本匹配,
"for d in ...; do n=$(find ...); done" 中的 find 绕过了 find* deny(真实会话复现);
x=$(grep ...)、export X=$(find ...)、! grep ... 同样绕过。

控制流语句仍整段判定,并逐条枚举条件与体内命令;do/else/case 分支容器只下降不单独成单元;
其余语句兜底额外枚举其中的 $(…)。结果只会更严,hard guard 共享同一枚举。

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 00:14:49 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> dae1a01f23 feat(prompt): 搜索引导改为意图→工具的正向指引,multi_grep 按上游保持关闭
真实会话中模型仍用 bash grep -o | sort -u 做抽取,被拒后直接放弃核对;
原提示词只说"不要",没告诉它抽取/计数、给程序传文件清单、过滤命令输出时该用什么。

- routing:按已知信息选 grep/find(多变体用 grep 正则 alternation),
  codemode + tools.grep 做聚合,bash 文件清单用 git ls-files,
  overflow_query 过滤输出,被拒后改用替代方案继续
- 权限策略:rg/grep/git grep/find/fd 的 deny 附带指向替代工具的 reason
- tools.jsonc:grep/find 新 snippet,补上 codegraph_explore snippet
- FFF 上游标注 multi_grep 有害:wrapper 删除 PI_FFF_MULTIGREP,清理全部引用
- JSONC 测试改用字符串感知的注释剥离(reason 中含 'src/**/*.ts')

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 00:14:49 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 833b9514d4 feat(update): update.sh 先同步 Pi 最新 release 再升级组合包
pi update <source> 只更新单个包,Pi 本体不会随之升级;改为先执行
pi update --self(版本比较由 Pi 自身完成,已是最新时跳过,失败记录后继续),
再升级组合包。

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 23:26:43 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> f9a7f743dd feat: 接入 Pi 内置 codemode,并修复全面审查发现的问题
- codemode:Tool Search 常驻(alwaysEnabled)、权限 allow(嵌套调用仍逐个过权限门)、
  tools.jsonc 登记使用规则;RTK 跳过 parentToolCallId 嵌套调用,避免丢失 bash
  structuredContent;Tool Search 目录过滤 hidden / codemode(script-only)exposure;
  扩展加载测试加载 builtin:codemode 并断言其常驻激活
- 搜索 key 不再经命令行参数传递:search-auth.mjs 改为 --set-stdin(先读校验再加锁),
  configure.sh 交互输入经管道、非交互用 --<provider> -
- CodeGraph 客户端监听子进程 stdin error,避免 EPIPE 未处理导致崩溃
- 移除 PI_TOOL_SEARCH/HERMES_MEMORY/RTK_CONFIG_PATH 环境变量入口,测试改用工厂 configPath 注入
- pi-subagents 固定 vitest / coverage-istanbul 4.1.10,修复 npm ls 版本冲突
- 补 pi-web-search / pi-worktree UPSTREAM.md 与 pi-minimal-footer AGENTS.md,
  files 白名单补齐并加测试;.codegraph/.gitignore 移出跟踪
- detect_kitty 优先识别官方安装目录

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 23:18:34 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 56cbc83296 refactor: 移除 context-mode,Chrome 快照改为直接返回
context-mode 的 ctx_execute 系工具以真实 shell 执行且权限为 allow,绕过了
bash 策略与 protectedWrites;其每轮注入、独立 MCP 子进程与 skills 也与组合包
提示词单一所有权冲突。大输出职责已由 pi-overflow(入口截断)与
pi-compaction(压力遮蔽)承担:
- 删除 context-mode 依赖、wrapper/过滤器、skills,以及 ctx_* 的权限、
  tools.jsonc、路由规则、tool-search 常驻与 4 个分组
- better-sqlite3@12 仅为 context-mode 传递依赖,随之移除根 allowScripts
- pi-chrome:按用户决定移除快照隐私规则与 .pi/chrome-context 临时文件机制,
  快照 JSON 直接作为工具结果返回、details 只留小型元数据,超长由 pi-overflow
  截断并用 overflow_query 提取;授权生命周期不变

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:45:33 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> 2a6bf8c929 feat(pi-overflow): 新增统一的新工具结果展示截断扩展
此前截断分散在多层且互相冲突:Pi bash 保留尾部 50KB,RTK 兜底截断又只留
开头 12000 字符(砍掉报错与完整输出路径),pi-chrome 截开头 30K,web_fetch
各 provider 不一致。新扩展统一负责"新结果给模型看多少":
- capture.ts 最先加载暂存原文,index.ts 最后加载(system-prompt 之前)
- 超过 24K 字符时原文落盘 <agentDir>/my-pi/pi-overflow/(700/600、原子写),
  优先取 bash fullOutputPath 完整文件;模型看到 头 + 中段信号行 + 尾 的有界
  视图,行号基于落盘原文、结尾永远可见
- overflow_query:按 id 正则搜索、行区间、jsonPath 只读查询,id 校验防穿越
- 收编:RTK truncate 默认关闭、删除 pi-chrome truncateText、web_fetch 默认
  每页 20000(上限 100000)、web_search max_results 限定 1–20
- pi-compaction 遮蔽带 details.overflow.id 的结果时在占位符提示 overflow_query
- 登记 tools.jsonc / routing / 权限 allow / tool-search 常驻与分组

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:25:44 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> a6d8416c76 refactor(pi-compaction): 改为压力触发的观察遮蔽与空闲阈值整段摘要
原实现对所有工具结果(含 read)在下一次请求即替换为约 4K 的首尾存根,
不看上下文压力,导致模型看不到大文件中段;后台逐条语义摘要又反复改写
历史、打碎 prompt 缓存。参照 Anthropic 上下文编辑、Manus 与 JetBrains
观察遮蔽研究重构为:
- L2:上下文达到阈值时在 turn_end 批量提交 Pi 原生 context_edit,把最近
  keepRecentTurns 轮之外的旧 toolResult 换成确定性占位符;trigger/target
  回差与 minClearTokens 保证少触发、一次清够;原始条目保留,可经
  context_tree_query 取回
- L3:空闲 agent_settled 且超过绝对阈值时调用一次 ctx.compact
- 阈值按绝对 token 计并受窗口比例约束,默认针对 1M 窗口(DeepSeek V4.1
  Flash):遮蔽 200K→100K、整段摘要 400K
- 删除逐条存根、语义 worker、soft/hard/target 比例与自行取消 Pi 压缩的门控;
  Pi 原生阈值压缩保留为最终安全网

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:51:36 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> a254d4fd68 fix: 根 files 改为子包运行时白名单,并删除过期提示词快照
整目录列入子包时,pi-subagents 上游 .npmignore 覆盖 .gitignore,导致
嵌套 node_modules(含第二套 Pi runtime)与 dist 被打包:64.9MB/19,570
文件降至 1.2MB/471 文件。新增 tests/package-files.test.ts 守护:无嵌套
node_modules/dist、扩展入口与 file: 依赖清单在包内、包内源码相对引用闭包。
同时删除无引用的 pi-default-system-prompt.md 旧快照(可用 /dump-system-prompt)。

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:16:47 -07:00
云服务部-叶林立andDevin <158243242+devin-ai-integration[bot]@users.noreply.github.com> e996f59469 refactor: notify/footer 持久配置收编到 config/,chrome bridge 地址固定
仓库规则要求 config/ 为唯一配置来源、不以环境变量作持久默认值:
- pi-notify 改读 config/pi-notify.json(逐字段校验回退,复用原 parsers 钳制)
- pi-minimal-footer 改读 config/pi-minimal-footer.json
- pi-chrome 移除 PI_CHROME_BRIDGE_HOST/PORT,固定 127.0.0.1:17318
  (浏览器端 service_worker.js 硬编码同一地址,单侧可配只会断连)

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 19:16:47 -07:00
135 changed files with 5060 additions and 3403 deletions
-5
View File
@@ -1,5 +0,0 @@
# CodeGraph data files — local to each machine, not for committing.
# Ignore everything in .codegraph/ except this file itself, so transient
# files (the database, daemon.pid, sockets, logs) never show up in git.
*
!.gitignore
+7 -6
View File
@@ -18,10 +18,11 @@
| `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` | | `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` |
| `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 | | `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 |
| `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 | | `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 |
| `pi-compaction/` | 增量 Session segment、确定性压缩、后台语义摘要、effective token、native compaction | | `pi-compaction/` | 压力触发的观察遮蔽(Pi context_edit)、阈值整段摘要、effective token、native compaction |
| `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep`/`multi_grep` 与 `@` 补全 | | `pi-overflow/` | 新工具结果统一截断:超阈值原文落盘 + 有界视图 + `overflow_query` |
| `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep` 与 `@` 补全,`multi_grep` 按上游保持关闭 |
| `extensions/bash-override.ts` | 用 Pi 的 `createBashToolDefinition` 重新注册 `bash`,仅替换模型可见描述(追加 `tools.jsonc` `descriptionAppend`);执行逻辑、输出结构、codemode 说明不变 |
| `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) | | `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) |
| `extensions/context-mode.ts` | 经 `prompt-guard.ts` 加载 context-mode adapter 与 skills(`context-mode-filter.ts` 过滤冲突锚点) |
| `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) | | `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) |
| `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` | | `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` |
| `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) | | `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) |
@@ -35,13 +36,13 @@
## 全局硬规则 ## 全局硬规则
- 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。 - 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。
- 加载顺序:`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。 - 加载顺序:`pi-overflow/capture.ts` 必须排在最前(先于 RTK 等结果改写扩展暂存原文),`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `pi-overflow/index.ts` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。
- 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前仅 `better-sqlite3@12.11.1`;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不新增放行)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。 - 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前无放行项;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不经根 `allowScripts`)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。根 `files` 对子包按运行时子路径白名单列出(源码、package.json、README/LICENSE/CHANGELOG/UPSTREAM),不得整目录列入,以免打入嵌套 node_modules、dist 和测试;由 `tests/package-files.test.ts` 守护。
- 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。 - 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。
- 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。 - 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。
- 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。 - 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。
- 脚本:`install.sh`、`update.sh`、`uninstall.sh`、`configure.sh` 与 `scripts/lib.sh` 保持 POSIX `sh` 兼容并留在 `files` 中,行为变化同步 README;安装默认来源 `git:git@bitbucket.org:siakitem/my-pi.git`,可由 `PI_PACKAGE_SOURCE` 覆盖(update/uninstall 同规则);组合包安装失败立即停止,机器级依赖失败继续检查其余项并最终返回非零。 - 脚本:`install.sh`、`update.sh`、`uninstall.sh`、`configure.sh` 与 `scripts/lib.sh` 保持 POSIX `sh` 兼容并留在 `files` 中,行为变化同步 README;安装默认来源 `git:git@bitbucket.org:siakitem/my-pi.git`,可由 `PI_PACKAGE_SOURCE` 覆盖(update/uninstall 同规则);组合包安装失败立即停止,机器级依赖失败继续检查其余项并最终返回非零。
- 安装/升级纪律:机器级依赖与终端配置逐项询问、默认拒绝;`update.sh` 只升级已安装项、缺失项跳过,组件升级先比较本地与远端版本、一致不重装,成功升级后把 git 来源的受管 checkout `git reset --hard` 还原到已发布版本(本地路径来源不 reset、不 `git clean`);`pi update` 失败 fail-fast——Pi 1.3.0 会据 `.pi-update-incomplete` 标记在下次 update 自愈,不做预清理或人工恢复。`pi update` 只在有新版本时整体覆盖已安装副本(含 `config/`),不逐字段迁移本机改动;需长期保留的自定义必须改仓库 `config/` 源文件。`.zshrc` 只改 `# >>> my-pi:<id> >>>` 受管块并先建时间戳备份,不覆盖用户 `~/.p10k.zsh`;卸载不得删除用户记忆数据(`.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/`)或共享机器级工具。 - 安装/升级纪律:机器级依赖与终端配置逐项询问、默认拒绝;`update.sh` 只升级已安装项、缺失项跳过,组件升级先比较本地与远端版本、一致不重装,成功升级后把 git 来源的受管 checkout `git reset --hard` 还原到已发布版本(本地路径来源不 reset、不 `git clean`);`update.sh` 先 `pi update --self` 同步 Pi 最新 release(版本比较由 Pi 自身完成,失败记录后继续),再升级组合包;组合包 `pi update` 失败 fail-fast——Pi 1.3.0 会据 `.pi-update-incomplete` 标记在下次 update 自愈,不做预清理或人工恢复。`pi update` 只在有新版本时整体覆盖已安装副本(含 `config/`),不逐字段迁移本机改动;需长期保留的自定义必须改仓库 `config/` 源文件。`.zshrc` 只改 `# >>> my-pi:<id> >>>` 受管块并先建时间戳备份,不覆盖用户 `~/.p10k.zsh`;卸载不得删除用户记忆数据(`.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/`)或共享机器级工具。
- 未经明确要求不执行发布、提交、推送、运行安装/卸载脚本、安装到用户 Pi 目录等外部写操作;凭据不进入命令参数、日志、会话或仓库(`auth.json` 权限 `600`、不提交)。 - 未经明确要求不执行发布、提交、推送、运行安装/卸载脚本、安装到用户 Pi 目录等外部写操作;凭据不进入命令参数、日志、会话或仓库(`auth.json` 权限 `600`、不提交)。
- 优先在目标扩展目录内完成改动;保留原项目 LICENSE 与版权;不让一个扩展依赖另一个扩展的未公开内部实现。 - 优先在目标扩展目录内完成改动;保留原项目 LICENSE 与版权;不让一个扩展依赖另一个扩展的未公开内部实现。
+45 -34
View File
@@ -2,9 +2,8 @@
个人维护的 Pi 扩展组合包。安装一次即可加载并配置: 个人维护的 Pi 扩展组合包。安装一次即可加载并配置:
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。 - `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find` 和 `grep`;上游默认关闭的 `multi_grep` 保持关闭。
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。 - 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
- `context-mode@1.0.169`:通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。
- 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。 - 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。 - `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。 - `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
@@ -12,11 +11,12 @@
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。 - `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
- 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。 - 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。
- 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。 - 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。
- 本仓库维护的 `pi-compaction`:以增量投影统一确定性工具结果压缩、后台语义摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。 - 本仓库维护的 `pi-compaction`:压力触发时以 Pi `context_edit` 批量遮蔽旧工具结果、空闲时按阈值触发整段摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。
- 本仓库自维护的 `pi-overflow`:统一的新工具结果展示截断——超阈值原文落盘、向模型展示「头 + 信号行 + 尾」的有界视图,并提供 `overflow_query` 按 id 搜索、行区间或 JSON 路径取回原文。
- 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。 - 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。
- 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。 - 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。
- 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。 - 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。
- 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;大体积页面快照写入 Session 临时文件并交给 Context Mode 隔离分析,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。 - 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;页面快照 JSON 直接作为工具结果返回,超长时由 pi-overflow 截断并用 `overflow_query` 查询,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。
- 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。 - 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。
- 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。 - 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。
- 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。 - 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。
@@ -68,13 +68,14 @@ Powerlevel10k 主题/配置和各 Zsh 插件使用 `# >>> my-pi:<id> >>>` 到
./update.sh ./update.sh
``` ```
升级脚本先执行: 升级脚本先把 Pi 本体同步到最新发布版本,再升级组合包:
```bash ```bash
pi update --self
pi update git:git@bitbucket.org:siakitem/my-pi.git pi update git:git@bitbucket.org:siakitem/my-pi.git
``` ```
脚本不会在升级前破坏性预清理安装缓存:`pi update` 失败时立即停止后续升级——Pi 1.3.0 会在下一次 `pi update` 时依据自己的 `.pi-update-incomplete` 标记自动重新清理并恢复未完成的升级,重新运行 `update.sh` 即可,脚本不做人工恢复。 `pi update --self` 由 Pi 自己比较本地与最新 release,已是最新时直接跳过;它失败只记录为升级失败,不阻断组合包升级。脚本不会在升级前破坏性预清理安装缓存:组合包的 `pi update` 失败时立即停止后续升级——Pi 1.3.0 会在下一次 `pi update` 时依据自己的 `.pi-update-incomplete` 标记自动重新清理并恢复未完成的升级,重新运行 `update.sh` 即可,脚本不做人工恢复。
随后只处理已经安装的项目,未安装项直接跳过且不会询问安装:Kitty 及已启用的 随后只处理已经安装的项目,未安装项直接跳过且不会询问安装:Kitty 及已启用的
Solarized Dark、Oh My Zsh、Powerlevel10k、两个第三方 Zsh 插件、CodeGraph,以及由 Solarized Dark、Oh My Zsh、Powerlevel10k、两个第三方 Zsh 插件、CodeGraph,以及由
@@ -137,11 +138,11 @@ CodeGraph 的更新或遥测设置。
### 工具与搜索路由 ### 工具与搜索路由
`config/prompt/routing.jsonc` 定义跨工具路由规则,由 `extensions/system-prompt.ts` 在每轮开始前按当前激活工具渲染为 `<tool_routing>` 分区。代码结构、调用关系和待修改 symbol 优先使用 CodeGraph,定义、引用、类型和修改后诊断优先使用 LSP;仓库字面搜索先用 FFF `find` 缩小文件或目录范围,再在已收敛的路径中用 `grep`/`multi_grep` 获取行号。宽泛搜索命中大量结果、发生截断或达到上限时,应继续缩小路径、glob 或 pattern,而不是提高 limit 或输出全部结果。 `config/prompt/routing.jsonc` 定义跨工具路由规则,由 `extensions/system-prompt.ts` 在每轮开始前按当前激活工具渲染为 `<tool_routing>` 分区。代码结构、调用关系和待修改 symbol 优先使用 CodeGraph,定义、引用、类型和修改后诊断优先使用 LSP;本地字面搜索按已知信息直接选工具:确切标识符或文本用 `grep` 字面模式并限定 path/exclude,多个命名变体合并为一次 `grep` 正则 alternation,文件名、路径片段或功能概念用 FFF `find` 模糊全路径匹配,然后只 `read` 命中区域。对搜索结果做提取、去重、计数或聚合时,在 codemode 脚本中调用 `tools.grep` / `tools.find` 并用 JavaScript 后处理;bash 中不得运行 `grep`/`rg`/`find`/`fd`/`git grep`(策略 deny 的 reason 指明替代工具),程序需要文件清单时传 `$(git ls-files '<glob>')`。宽泛搜索命中大量结果、发生截断或达到上限时,应继续缩小路径、glob 或 pattern,而不是提高 limit 或输出全部结果。
在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。 在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。
日志、测试/构建输出、工作区内大文件的探索/分析/总结以及不可预测的大输出优先交给 Context Mode。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。 日志、测试/构建输出以及不可预测的大输出由 pi-overflow 截断展示,原文可用 `overflow_query` 按 id 检索。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。
使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。 使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。
@@ -149,7 +150,7 @@ CodeGraph 的更新或遥测设置。
`extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。 `extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。
其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP、Context Mode)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。 其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。
`config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。 `config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。
@@ -159,7 +160,7 @@ CodeGraph 的更新或遥测设置。
授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。 授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。
`chrome_snapshot`、`chrome_find` 和带 `includeSnapshot=true` 的交互不会再把完整页面结构直接塞进模型上下文:完整 JSON 只写入当前项目 `.pi/chrome-context/` 下按 Session/目标隔离的临时文件,工具结果仅返回相对路径和小型摘要。模型应立即用 `ctx_execute_file` 从该文件提取下一步所需的 uid、文本或状态,不能用 `read` / `cat` 重新读入全文。每个目标只保留最新快照;页面操作后旧快照失效,Session 结束、`/chrome revoke` 或授权过期时清理。 `chrome_snapshot` 和带 `includeSnapshot=true` 的交互把页面快照 JSON 直接作为工具结果返回,`details` 只保留元素数、匹配数等小型元数据;`chrome_find` 只返回紧凑的排序匹配。快照超长时结果会带 `[pi-overflow id=…]` 标记,模型用 `overflow_query` 的 `jsonPath`(如 `elements[0].uid`)或 `search` 提取所需的 uid、文本或状态,而不是重复抓取快照。页面交互后旧快照即失效,需要重新抓取;早期版本写入 Session 临时文件的快照机制已移除。
### 按需加载工具 schema ### 按需加载工具 schema
@@ -167,7 +168,7 @@ CodeGraph 的更新或遥测设置。
本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。 本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。
新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、`ctx_execute`、`ctx_execute_file`、`ctx_batch_execute` 和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Context Mode 执行/知识库/观测/管理、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。 新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、六个记忆工具、`context_tree_query`、`overflow_query`、Pi 内置 `codemode`(exposure `model-only`)和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。exposure 为 `hidden` 或 `codemode`(script-only)的工具一律不进入目录。`codemode` 在权限策略中为 `allow`:脚本本身无副作用,脚本发起的每个嵌套工具调用仍携带 `parentToolCallId` 经过同一权限门裁决;RTK 跳过所有嵌套调用,保证 bash 的 `structuredContent` 原样传给脚本。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。
标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。 标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。
@@ -180,16 +181,15 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
"alwaysEnabled": [ "alwaysEnabled": [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
"memory_replace", "memory_replace",
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query" "context_tree_query",
"overflow_query",
"codemode"
], ],
"showToolSearchFooterStatus": false, "showToolSearchFooterStatus": false,
"maxActiveGroups": 5, "maxActiveGroups": 5,
@@ -205,29 +205,32 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。 组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。
Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位、再用 Context Mode 隔离分析,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。 Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。
### 搜索归 FFF ### 搜索归 FFF
`extensions/fff-override.ts` 强制设置 FFF 官方的 `PI_FFF_MODE=override`(显式传入 `extensions/fff-override.ts` 强制设置 FFF 官方的 `PI_FFF_MODE=override`(显式传入
`--fff-mode` 仍按 FFF 官方优先级覆盖它)。FFF 替换 `--fff-mode` 仍按 FFF 官方优先级覆盖它)。FFF 替换
Pi 的 `find` 和 `grep`,并提供 `multi_grep`;RTK 对所有 `grep` 工具结果直接跳过, Pi 的 `find` 和 `grep`;`multi_grep` 上游标注为有害、默认关闭,wrapper 在加载前
删除 `PI_FFF_MULTIGREP` 环境变量,使其永远无法注册。RTK 对所有 `grep` 工具结果直接跳过,
也默认关闭 Bash 命令改写,因此不会把 `rg`、`grep`、`find` 或 `fd` 改写到 RTK。 也默认关闭 Bash 命令改写,因此不会把 `rg`、`grep`、`find` 或 `fd` 改写到 RTK。
`extensions/bash-override.ts` 用 Pi 的 `createBashToolDefinition` 重新注册内置 `bash`,
把「不要用 bash 搜索」的警告经 `tools.jsonc` 的 `descriptionAppend` 写进工具 schema,
让模型在调用时就看到替代工具;执行逻辑、输出结构和 codemode 的 `tools.bash` 说明完全沿用上游定义。
### 输出压缩归 RTK ### 输出压缩归 RTK
RTK 保留 Bash、read、build、test、lint 和 Git 输出压缩。默认 `read` 的有损压缩 RTK 保留 Bash、read、build、test、lint 和 Git 输出压缩。默认 `read` 的有损压缩
仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有 仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有
`commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。 `commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。
### 大输出归 Context Mode ### 大输出归 pi-overflow
Context Mode 直接加载 npm 包内置的 Pi adapter 和 skills,不需要额外写入 新工具结果的展示截断统一由 `pi-overflow` 负责:超阈值的原文落盘、模型只看到
`~/.pi/agent/mcp.json`,也不需要全局安装 `context-mode` 命令。它提供 `ctx_*` 工具, 「头 + 信号行 + 尾」的有界视图,需要原文时用 `overflow_query` 检索(详见下文
将批量读取、命令研究和网页内容先在隔离进程中处理,再把收敛后的结果送入上下文。 「Pi Overflow」小节)。FFF 仍负责精确字面搜索;RTK 继续对 Bash、build、test、lint
和 Git 输出做语义压缩,其兜底硬截断已关闭。
FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash、build、test、lint
和 Git 输出。Context Mode 不替换这两个扩展的现有配置。
### SSH 服务器配置 ### SSH 服务器配置
@@ -245,10 +248,10 @@ FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash
### 在线搜索服务 ### 在线搜索服务
运行 `./configure.sh search` 可用隐藏输入依次配置 Tavily、Exa 和 Keenable;也可一次无交互配置: 运行 `./configure.sh search` 可用隐藏输入依次配置 Tavily、Exa 和 Keenable;key 不得出现在命令行参数中,非交互方式从标准输入读取(每次调用最多一个 provider):
```bash ```bash
./configure.sh search --tavily '...' --exa '...' --keenable '...' printf '%s' "$TAVILY_API_KEY" | ./configure.sh search --tavily -
``` ```
只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的 只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的
@@ -366,23 +369,31 @@ compaction 和排队 follow-up 完成前不会提前通知;同时监听 `pi-pe
`permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的 `permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的
决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`, 决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`,
并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty 并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty
返回原始来源;仍可通过环境变量改用 `unfocused` 或 `invisible`。 返回原始来源;仍可通过 `visibility` 改用 `unfocused` 或 `invisible`。
默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制 默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制
当前 Session 的两类通知;持久默认通过 `PI_NOTIFY_ENABLED`、`PI_NOTIFY_MIN_SECONDS`、 当前 Session 的两类通知;持久默认通过组合包 `config/pi-notify.json` 的 `enabled`、`minSeconds`、
`PI_NOTIFY_VISIBILITY`、`PI_NOTIFY_ACTION`、`PI_NOTIFY_MESSAGE_SOURCE` 等环境变量配置。 `visibility`、`action`、`messageSource` 等字段配置。`messageSource: "none"`
`PI_NOTIFY_MESSAGE_SOURCE=none` 会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要 会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要
`set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见 `set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见
[`pi-notify/README.md`](pi-notify/README.md)。 [`pi-notify/README.md`](pi-notify/README.md)。
### Pi Compaction ### Pi Compaction
组合包只加载仓库内维护的 `pi-compaction/`。它以增量 Segment Index 管理已经进入 Session 的历史:已闭合工具结果先同步替换为有界的确定性证据 stub,大于阈值的结果再由单 Session 有界后台 worker 生成语义摘要;Provider 请求不会等待后台摘要。原始工具证据不重复写入 custom entry,可通过 `context_tree_query` 使用 source key 或 tool call ID 有界恢复。 组合包只加载仓库内维护的 `pi-compaction/`。它管理已经进入 Session 的历史投影:上下文压力达到阈值时,在 `turn_end` 批量提交 Pi 原生 `context_edit`,把最近 `keepRecentTurns` 轮之外的大型工具结果遮蔽为指向 `context_tree_query` 的单行占位符;遮蔽成批发生、受 trigger/target 回差与 `minClearTokens` 约束。原始工具结果保留在 Session 中(UI、导出不变),可通过 `context_tree_query` 按 tool call ID 有界恢复。
普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。Pi Core 的 raw threshold 只负责唤醒:只有 effective token 下界达到默认 90% 才执行 threshold compaction;manual 与 overflow 始终允许继续并在 native 失败时回退 Pi 默认 compaction。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。 普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。整段摘要只在空闲 `agent_settled` 且达到绝对 token 阈值(默认 400K,受窗口比例约束,272K 窗口时约为 90%)时经 `ctx.compact` 触发一次;Pi 自身的阈值压缩作为最终安全网始终有效,`session_before_compact` 不再按扩展自身阈值取消。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。
唯一配置位于已安装组合包的 `config/pi-compaction.json`,默认 soft/hard/target 比例为 72%/90%/55%,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和后台任务。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。 唯一配置位于已安装组合包的 `config/pi-compaction.json`:`masking` 段为遮蔽触发/目标阈值、`keepRecentTurns`、`minEntryTokens` 与 `excludeTools`,`summary` 段为整段摘要阈值与 thinking 档位,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和已遮蔽结果数。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。
原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。 原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。
本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 Context Mode 职责互补:Context Mode 防止大型原始内容进入会话,Pi Compaction 管理已经进入会话的长期历史。 本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 pi-overflow 职责互补:pi-overflow 在新工具结果进入会话时即截断展示,Pi Compaction 管理已经进入会话的长期历史。
### Pi Overflow
组合包自维护的 `pi-overflow/` 统一负责「每个新工具结果给模型看多少」。它有两个入口:`capture.ts` 排在 `pi.extensions` 最前,只把原始结果文本暂存到进程内 Map(容量 64),绝不修改结果;`index.ts` 排在 `system-prompt.ts` 之前、所有其他扩展之后,此时 RTK 等 handler 已经完成变换——它用最终可见文本判断是否超阈值,落盘的原文取 bash 的 `details.fullOutputPath` 文件、暂存原文或当前文本三者中的最完整版本,视图与其中的行号则一律基于落盘原文生成,保证与 `overflow_query` 读取的内容一致。
当最终文本仍超过 `thresholdChars`(默认 24000)时,模型看到的是确定性的「头部 40 行 + 中段匹配 error/warn 等信号行(至多 40 条)+ 尾部 120 行」视图,并写入 `details.overflow.id`;当上游已把结果压短但原文超阈值时,只追加一行 `[pi-overflow id=…]` 恢复标记。原文以 `sha1(sessionId+toolCallId)` 前 12 位 hex 为 id 存入 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),session 启动时按 `retentionDays`/`maxStoreBytes` 清理。模型可用 `overflow_query` 按 id 正则搜索(带上下文合并)、按行区间或 `jsonPath` 读取;`read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用永远豁免。配置为组合包内 `config/pi-overflow.json`,详见 [`pi-overflow/README.md`](pi-overflow/README.md)。
RTK 的兜底截断与 pi-chrome 的 30K 字符截断已收编进本扩展:`config/rtk-optimizer.json` 中 `outputCompaction.truncate.enabled` 默认关闭,chrome_* 工具返回完整文本;pi-web-search 的 `web_fetch` 默认 `max_characters` 提高到 20000(上限 100000),`web_search` 的 `max_results` 约束在 1–20。
+13 -13
View File
@@ -1,20 +1,20 @@
{ {
"enabled": true, "enabled": true,
"softRatio": 0.72, "masking": {
"hardRatio": 0.9,
"targetRatio": 0.55,
"deterministic": {
"enabled": true, "enabled": true,
"maxStubTokens": 1024 "triggerTokens": 200000,
"triggerRatio": 0.6,
"targetTokens": 100000,
"targetRatio": 0.35,
"minClearTokens": 50000,
"keepRecentTurns": 10,
"minEntryTokens": 256,
"excludeTools": ["ask_user_question"]
}, },
"semantic": { "summary": {
"enabled": true, "triggerTokens": 400000,
"model": "default", "triggerRatio": 0.9,
"thinking": "low", "thinking": "low"
"concurrency": 1,
"minInputChars": 4000,
"maxInputChars": 24000,
"maxRetries": 2
}, },
"native": { "native": {
"enabled": true "enabled": true
+4
View File
@@ -0,0 +1,4 @@
{
"showCwd": true,
"showBranch": true
}
+11
View File
@@ -0,0 +1,11 @@
{
"enabled": true,
"title": "π - {project}",
"body": "{message}",
"messageSource": "assistant",
"messageMax": 80,
"minSeconds": 3,
"visibility": "always",
"action": "focus",
"soundCommand": ""
}
+13
View File
@@ -0,0 +1,13 @@
{
"enabled": true,
"thresholdChars": 24000,
"headLines": 40,
"tailLines": 120,
"maxSignalLines": 40,
"signalPattern": "\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b",
"maxLineChars": 500,
"exemptTools": ["read", "edit", "write", "overflow_query", "ask_user_question"],
"retentionDays": 7,
"maxStoreBytes": 1073741824,
"maxEntryBytes": 67108864
}
+11 -14
View File
@@ -37,7 +37,6 @@
"edit": "allow", "edit": "allow",
"grep": "allow", "grep": "allow",
"find": "allow", "find": "allow",
"multi_grep": "allow",
"ls": "allow", "ls": "allow",
"codegraph_explore": "allow", "codegraph_explore": "allow",
"lsp_diagnostics": "allow", "lsp_diagnostics": "allow",
@@ -45,13 +44,6 @@
"lsp_references": "allow", "lsp_references": "allow",
"lsp_hover": "allow", "lsp_hover": "allow",
"lsp_symbols": "allow", "lsp_symbols": "allow",
"ctx_execute": "allow",
"ctx_execute_file": "allow",
"ctx_batch_execute": "allow",
"ctx_search": "allow",
"ctx_index": "allow",
"ctx_stats": "allow",
"ctx_doctor": "allow",
"tool_search": "allow", "tool_search": "allow",
"ask_user_question": "allow", "ask_user_question": "allow",
// Launching, steering or reading a subagent has no side effect itself; // Launching, steering or reading a subagent has no side effect itself;
@@ -86,10 +78,15 @@
// index and never touches the workspace. It is pinned in Tool Search's // index and never touches the workspace. It is pinned in Tool Search's
// alwaysEnabled set, so it must not pay an AutoReview round trip per turn. // alwaysEnabled set, so it must not pay an AutoReview round trip per turn.
"context_tree_query": "allow", "context_tree_query": "allow",
// pi-overflow's inspector reads only the extension's own stored output
// directory (id-validated); pinned in Tool Search's alwaysEnabled set.
"overflow_query": "allow",
// A codemode script does nothing by itself: every nested tool call it makes
// runs through this same gate (Pi emits tool_call with parentToolCallId).
"codemode": "allow",
// Model-chosen URLs can exfiltrate data — same reason curl/wget ask. // Model-chosen URLs can exfiltrate data — same reason curl/wget ask.
"web_fetch": "ask", "web_fetch": "ask",
"ctx_fetch_and_index": "ask",
"ssh_connect": "ask", "ssh_connect": "ask",
"ssh_cd": "ask", "ssh_cd": "ask",
@@ -105,10 +102,10 @@
}, },
"bash": { "bash": {
"*": "allow", "*": "allow",
"rg*": "deny", "rg*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
"grep*": "deny", "grep*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
"find*": "deny", "find*": { "action": "deny", "reason": "Bash file search is disabled. Locally use the find tool (fuzzy path, or a glob in path such as 'src/**/*.ts'); to pass a file list to a program use $(git ls-files '<glob>'). Over SSH use ssh_find." },
"fd*": "deny", "fd*": { "action": "deny", "reason": "Bash file search is disabled. Locally use the find tool (fuzzy path, or a glob in path such as 'src/**/*.ts'); to pass a file list to a program use $(git ls-files '<glob>'). Over SSH use ssh_find." },
"git *": "ask", "git *": "ask",
"git status*": "allow", "git status*": "allow",
"git diff*": "allow", "git diff*": "allow",
@@ -116,7 +113,7 @@
"git show*": "allow", "git show*": "allow",
"git rev-parse*": "allow", "git rev-parse*": "allow",
"git ls-files*": "allow", "git ls-files*": "allow",
"git grep*": "deny", "git grep*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
"git branch": "allow", "git branch": "allow",
"git branch --show-current*": "allow", "git branch --show-current*": "allow",
"git remote -v": "allow", "git remote -v": "allow",
+21 -17
View File
@@ -16,16 +16,25 @@
"text": "Use LSP for symbol definitions, references, hover/type information, and post-edit diagnostics when the language server supports the file." "text": "Use LSP for symbol definitions, references, hover/type information, and post-edit diagnostics when the language server supports the file."
}, },
{ {
"whenAny": ["find", "grep", "multi_grep"], "whenAny": ["find", "grep"],
"text": "For literal search, narrow in stages instead of requesting a repository-wide dump:\n 1. Use find to identify likely files or directories. Constrain path/glob and keep the result limit small.\n 2. Use grep or multi_grep only inside the narrowed path or file to obtain matching line numbers. Start with a specific or literal pattern, a small context window, and a bounded limit.\n 3. Use read with offset/limit only for the exact matching region when full source is still needed." "text": "Pick the local search tool by what you already know:\n - Exact identifier or text: grep with a literal pattern, constrained by path/exclude. Do not run find first just to locate it.\n - Several identifiers or naming variants: one grep with a regex alternation (e.g. user_id|userId|UserId) instead of several greps.\n - A filename, path fragment, or feature concept: find (fuzzy whole-path match; use a glob in path for exact names).\n - Then read only the matching region with offset/limit."
}, },
{ {
"whenAny": ["find", "grep", "multi_grep"], "whenAny": ["find", "grep"],
"text": "Do not run a broad search expected to return hundreds or thousands of files or matches. If results truncate or hit a limit, narrow the path, glob, or pattern instead of increasing the limit or dumping all results." "text": "Do not run a broad search expected to return hundreds or thousands of files or matches. If results truncate or hit a limit, narrow the path, glob, or pattern instead of increasing the limit or dumping all results."
}, },
{
"whenAll": ["codemode"],
"whenAny": ["find", "grep"],
"text": "To extract, deduplicate, count, or aggregate across search matches (what grep -o | sort -u | wc would do), call tools.grep / tools.find inside one codemode script and post-process the results in JavaScript."
},
{ {
"whenAny": ["bash"], "whenAny": ["bash"],
"text": "Use bash for builds, tests, git, package scripts, and other programs. Never search files by running grep, find, rg, or fd through bash; use the find/grep tools." "text": "Use bash for builds, tests, git, package scripts, and other programs. Never run grep, rg, find, fd, or git grep in bash — standalone, piped, inside a loop, or in $(…). Search files with the grep/find tools; when a program needs a file list (wc, node, tsc), pass $(git ls-files '<glob>') as arguments rather than piping into xargs."
},
{
"whenAll": ["bash", "overflow_query"],
"text": "To filter a command's output, run it without | grep; if the result carries a [pi-overflow id=…] marker, search it with overflow_query instead of re-running."
}, },
{ {
"whenAny": ["ssh_connect"], "whenAny": ["ssh_connect"],
@@ -50,29 +59,24 @@
"text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches." "text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches."
}, },
{ {
"whenAll": ["web_fetch", "ctx_fetch_and_index"], "whenAll": ["chrome_snapshot", "overflow_query"],
"text": "Read a web page with web_fetch. Use ctx_fetch_and_index only when the content should be indexed into the context-mode knowledge base for repeated ctx_search queries." "text": "chrome_snapshot (and Chrome actions with includeSnapshot=true) return the page snapshot JSON directly. When it is large the result carries a [pi-overflow id=…] marker: extract only the uids, text, or state you need with overflow_query (jsonPath or search) instead of taking another snapshot, and take a fresh snapshot after any page interaction because the previous one is stale."
}, },
{ {
"whenAny": ["ctx_execute", "ctx_execute_file", "ctx_batch_execute"], "whenAny": ["overflow_query"],
"text": "Use Context Mode for logs, test/build output, generated data, large-file exploration/analysis/summarization, and any command whose output may be large or unpredictable. Prefer one ctx_batch_execute over several ctx_execute calls for multi-command research, and print only the derived answer needed for the task." "text": "When a tool result starts with or ends with a [pi-overflow id=…] marker, the full original output is stored; inspect it with overflow_query (search, line ranges, or jsonPath) instead of re-running the command or dumping it again."
},
{
"whenAny": ["ctx_execute_file"],
"text": "ctx_execute_file analyzes a file and never edits it; make changes with read + edit. It is confined to the current project root: do not call it for absolute paths outside the workspace, ../ traversal, or symlinks that resolve outside the workspace; host permission approval does not bypass this boundary. Use another explicitly authorized tool for those files."
},
{
"whenAll": ["chrome_snapshot", "ctx_execute_file"],
"text": "After chrome_snapshot or a Chrome action with includeSnapshot=true, immediately analyze the returned workspace-relative capture path with ctx_execute_file; never use read or cat for the raw JSON. Print only the uids, text, or state needed for the next action, and take a fresh snapshot after any interaction because the previous capture may be stale."
}, },
{ {
"whenAny": ["read"], "whenAny": ["read"],
"text": "Use read only when exact source is needed or the file is small. Before editing, request the smallest useful offset/limit range; do not read a complete large file merely to copy, compare, hash, count, explore, or summarize it." "text": "Use read only when exact source is needed or the file is small. Before editing, request the smallest useful offset/limit range; do not read a complete large source file merely to copy, compare, hash, count, or explore it. Reading project overview docs such as a README in full is fine."
}, },
{ {
"whenAll": ["read", "edit"], "whenAll": ["read", "edit"],
"text": "For existing-file changes, obtain fresh LINE#HASH anchors with a small read (find/grep output has no anchors), then use anchored edit operations. Reuse fresh anchors returned by successful edits for chained changes; re-read only when anchors are stale or the next target region was not shown." "text": "For existing-file changes, obtain fresh LINE#HASH anchors with a small read (find/grep output has no anchors), then use anchored edit operations. Reuse fresh anchors returned by successful edits for chained changes; re-read only when anchors are stale or the next target region was not shown."
}, },
{
"text": "When a tool call is denied by policy, use the alternative named in the denial and continue the subtask; do not drop it or present unverified facts as checked."
},
{ {
"text": "Stop searching once the current evidence is sufficient. Avoid repeating the same lookup through CodeGraph, find, grep, and read without a concrete information gap." "text": "Stop searching once the current evidence is sufficient. Avoid repeating the same lookup through CodeGraph, find, grep, and read without a concrete information gap."
} }
+1 -1
View File
@@ -14,7 +14,7 @@ You locate code and answer "where is X / what references Y" questions for the de
## Hard limits ## Hard limits
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp). - Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state. - Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
- Never search through bash (grep, rg, find, fd); use the find and grep tools. - Never search through bash (grep, rg, find, fd); use the find and grep tools. For several names use one grep with a regex alternation (a|b); when a program needs a file list, pass $(git ls-files '<glob>').
## Approach ## Approach
- Match the requested breadth: "quick" is one targeted lookup, "medium" covers a few related locations, "very thorough" searches multiple locations and naming conventions. - Match the requested breadth: "quick" is one targeted lookup, "medium" covers a few related locations, "very thorough" searches multiple locations and naming conventions.
+1 -1
View File
@@ -14,7 +14,7 @@ You explore the codebase and return an implementation plan to the delegating age
## Hard limits ## Hard limits
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp). - Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state. - Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
- Never search through bash (grep, rg, find, fd); use the find and grep tools. - Never search through bash (grep, rg, find, fd); use the find and grep tools. For several names use one grep with a regex alternation (a|b); when a program needs a file list, pass $(git ls-files '<glob>').
## Process ## Process
1. Restate the goal and the constraints you were given. Record anything ambiguous as an open question instead of guessing. 1. Restate the goal and the constraints you were given. Record anything ambiguous as an open question instead of guessing.
+27 -23
View File
@@ -2,23 +2,32 @@
// Bundle-authoritative tool guidance, owned by extensions/system-prompt.ts. // Bundle-authoritative tool guidance, owned by extensions/system-prompt.ts.
// //
// Every tool the bundle can expose must have an entry. Per field: // Every tool the bundle can expose must have an entry. Per field:
// "snippet" — replaces the tool's one-line <tools> entry (null hides it) // "snippet" — replaces the tool's one-line <tools> entry (null hides it)
// "guidelines" — replaces the tool's <rules> lines // "guidelines" — replaces the tool's <rules> lines
// "description" — replaces the tool description at registration; only // "description" — replaces the tool description at registration
// applied to third-party tools loaded through a bundle // "descriptionAppend" — appended to the description at registration (on top
// adapter (in-repo tools edit their own source instead) // of "description" when both are set)
// "description"/"descriptionAppend" only apply to tools registered through a
// bundle adapter (prompt-guard), which includes the bash override and the
// third-party wrappers; other in-repo tools edit their own source instead.
// An absent field keeps the upstream text. "inherit": true means the // An absent field keeps the upstream text. "inherit": true means the
// upstream text was reviewed against this bundle and is kept as-is. // upstream text was reviewed against this bundle and is kept as-is.
"version": 1, "version": 1,
"tools": { "tools": {
"bash": { "bash": {
"snippet": "Run shell commands for builds, tests, git, package scripts, and other programs; search files with the find/grep tools, not bash" "snippet": "Run shell commands for builds, tests, git, package scripts, and other programs; search files with the find/grep tools, not bash",
"descriptionAppend": "Not for searching files: use the grep tool for file contents and the find tool for paths (e.g. `grep -n X src/*.ts` → grep {pattern: \"X\", path: \"src/*.ts\"}); grep, rg, find, fd, and git grep are denied here. When a program needs many files, pass $(git ls-files '<glob>') as arguments instead of piping into xargs, which triggers a permission review.",
"guidelines": [
"You can inspect PI_* environment variables for current model and session details.",
"Search commands are denied in bash; call the tools instead: `grep -n X src/*.ts` → grep {pattern: \"X\", path: \"src/*.ts\"}; `grep -rl X src` → grep {pattern: \"X\", path: \"src/\"} and take the file names; `find . -name '*.ts'` → find {pattern: \"\", path: \"**/*.ts\"}.",
"When a program needs many files (wc, node, tsc), pass $(git ls-files '<glob>') as arguments; piping into xargs triggers a permission review."
]
}, },
"read": { "inherit": true }, "read": { "inherit": true },
"edit": { "edit": {
"guidelines": [ "guidelines": [
"Use edit with LINE#HASH anchors from the latest read or edit result for all file changes; batch every change to one file into a single edit call.", "Use edit with LINE#HASH anchors from the latest read or edit result for all file changes; batch every change to one file into a single edit call.",
"find/grep/multi_grep results carry plain line numbers, not LINE#HASH anchors; read the exact target range before editing.", "find/grep results carry plain line numbers, not LINE#HASH anchors; read the exact target range before editing.",
"Never reuse anchors from an earlier read or edit response once a newer one exists for the same file; treat only the most recent response as carrying valid anchors.", "Never reuse anchors from an earlier read or edit response once a newer one exists for the same file; treat only the most recent response as carrying valid anchors.",
"After a successful edit, the returned --- Anchors --- block replaces a re-read for nearby follow-up edits.", "After a successful edit, the returned --- Anchors --- block replaces a re-read for nearby follow-up edits.",
"On [E_STALE_ANCHOR], the error lists the stale refs and any content-matched candidate anchors; re-read the file to get current anchors before retrying." "On [E_STALE_ANCHOR], the error lists the stale refs and any content-matched candidate anchors; re-read the file to get current anchors before retrying."
@@ -28,12 +37,18 @@
"write": { "inherit": true }, "write": { "inherit": true },
"ls": { "inherit": true }, "ls": { "inherit": true },
"powershell": { "inherit": true }, "powershell": { "inherit": true },
"grep": { "inherit": true }, "grep": { "snippet": "Search file contents via the FFF index (git-aware, smart-case, frecency-ranked, regex or literal); replaces bash grep/rg" },
"find": { "inherit": true }, "find": { "snippet": "Find files by fuzzy path or glob via the FFF index; replaces bash find/fd" },
"multi_grep": { "inherit": true },
"tool_search": { "inherit": true }, "tool_search": { "inherit": true },
"codemode": {
"guidelines": [
"Use codemode to batch two or more independent tool calls (Promise.allSettled), chain dependent lookups, or filter large output down to what you need; call a single tool directly instead of wrapping it in a script.",
"Call edit, write, ssh_connect, ssh_cd, ask_user_question, and Agent directly, never from a codemode script.",
"Inside scripts, search with tools.grep or tools.find and do extraction, dedup, and counting in JavaScript; never run grep, find, rg, or fd through tools.bash."
]
},
"codegraph_explore": { "inherit": true }, "codegraph_explore": { "snippet": "Explore code structure from the CodeGraph index: symbols, relationships, call flows, and their source" },
"lsp_diagnostics": { "inherit": true }, "lsp_diagnostics": { "inherit": true },
"lsp_definition": { "inherit": true }, "lsp_definition": { "inherit": true },
"lsp_references": { "inherit": true }, "lsp_references": { "inherit": true },
@@ -43,18 +58,6 @@
"web_search": { "inherit": true }, "web_search": { "inherit": true },
"web_fetch": { "inherit": true }, "web_fetch": { "inherit": true },
"ctx_execute": { "inherit": true },
"ctx_execute_file": { "inherit": true },
"ctx_batch_execute": { "inherit": true },
"ctx_index": { "inherit": true },
"ctx_search": { "inherit": true },
"ctx_fetch_and_index": { "inherit": true },
"ctx_stats": { "inherit": true },
"ctx_doctor": { "inherit": true },
"ctx_insight": { "inherit": true },
"ctx_upgrade": { "inherit": true },
"ctx_purge": { "inherit": true },
"memory_search": { "inherit": true }, "memory_search": { "inherit": true },
"session_search": { "inherit": true }, "session_search": { "inherit": true },
"memory_add": { "inherit": true }, "memory_add": { "inherit": true },
@@ -62,6 +65,7 @@
"memory_remove": { "inherit": true }, "memory_remove": { "inherit": true },
"skill_manage": { "inherit": true }, "skill_manage": { "inherit": true },
"context_tree_query": { "inherit": true }, "context_tree_query": { "inherit": true },
"overflow_query": { "inherit": true },
"ask_user_question": { "inherit": true }, "ask_user_question": { "inherit": true },
+1 -1
View File
@@ -11,7 +11,7 @@
"enabled": false "enabled": false
}, },
"truncate": { "truncate": {
"enabled": true, "enabled": false,
"maxChars": 12000 "maxChars": 12000
}, },
"sourceCodeFilteringEnabled": false, "sourceCodeFilteringEnabled": false,
+3 -4
View File
@@ -2,16 +2,15 @@
"alwaysEnabled": [ "alwaysEnabled": [
"codegraph_explore", "codegraph_explore",
"lsp_diagnostics", "lsp_diagnostics",
"ctx_execute",
"ctx_execute_file",
"ctx_batch_execute",
"memory_search", "memory_search",
"session_search", "session_search",
"memory_add", "memory_add",
"memory_replace", "memory_replace",
"memory_remove", "memory_remove",
"skill_manage", "skill_manage",
"context_tree_query" "context_tree_query",
"overflow_query",
"codemode"
], ],
"showToolSearchFooterStatus": false, "showToolSearchFooterStatus": false,
"maxActiveGroups": 5, "maxActiveGroups": 5,
+21 -13
View File
@@ -24,24 +24,25 @@ search_usage() {
cat <<'EOF' cat <<'EOF'
用法: 用法:
./configure.sh search ./configure.sh search
./configure.sh search --tavily KEY --exa KEY --keenable KEY ./configure.sh search --tavily - 从标准输入读取 key
./configure.sh search --migrate [search.env 路径] ./configure.sh search --migrate [search.env 路径]
./configure.sh search --list ./configure.sh search --list
选项: 选项:
--tavily KEY 配置 tavily api_key 凭据 --tavily - 从标准输入读取 tavily api_key 凭据
--exa KEY 配置 exa api_key 凭据 --exa - 从标准输入读取 exa api_key 凭据
--keenable KEY 配置 keenable api_key 凭据 --keenable - 从标准输入读取 keenable api_key 凭据
--migrate 从旧 search.env 导入缺失的 key --migrate 从旧 search.env 导入缺失的 key
--list 显示各 provider 配置状态(不输出 key) --list 显示各 provider 配置状态(不输出 key)
--auth FILE 覆盖 auth.json 路径 --auth FILE 覆盖 auth.json 路径
-h, --help 显示帮助 -h, --help 显示帮助
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互, key 不得出现在命令行参数中:非交互方式用 `--<provider> -` 从标准输入
未指定的 provider 保留已有值。凭据写入 Pi 的 auth.json 读取,每次调用最多一个 provider 可经标准输入配置;不传参数时进入
($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,权限 600)。 隐藏输入的交互模式,未指定的 provider 保留已有值。凭据写入 Pi 的
key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR} 插值、 auth.json($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,
以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。 权限 600)。key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR}
插值、以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。
EOF EOF
} }
@@ -52,6 +53,7 @@ configure_search() {
SEARCH_AUTH_FILE= SEARCH_AUTH_FILE=
search_mode=interactive search_mode=interactive
search_did_work=0 search_did_work=0
search_stdin_provider=
run_search_auth() { run_search_auth() {
if [ -n "$SEARCH_AUTH_FILE" ]; then if [ -n "$SEARCH_AUTH_FILE" ]; then
@@ -69,7 +71,7 @@ configure_search() {
prompt_label=$1 prompt_label=$1
prompt_configured=$2 prompt_configured=$2
if [ ! -t 0 ]; then if [ ! -t 0 ]; then
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。" warn "交互模式需要终端;请改用 '--<provider> -' 从标准输入提供 key。"
return 2 return 2
fi fi
if [ "$prompt_configured" -eq 1 ]; then if [ "$prompt_configured" -eq 1 ]; then
@@ -88,9 +90,15 @@ configure_search() {
while [ "$#" -gt 0 ]; do while [ "$#" -gt 0 ]; do
case $1 in case $1 in
--tavily|--exa|--keenable) --tavily|--exa|--keenable)
[ "$#" -ge 2 ] || { warn "$1 缺少 KEY"; exit 2; } [ "$#" -ge 2 ] || { warn "$1 缺少 KEY 或 -"; exit 2; }
if [ "$2" != "-" ]; then
warn "$1 不接受明文 key:key 不得出现在命令行参数中;请用交互模式或 '$1 -' 从标准输入提供。"
exit 2
fi
[ -z "$search_stdin_provider" ] || { warn "每次调用最多一个 provider 可经标准输入读取 key。"; exit 2; }
search_stdin_provider=${1#--}
search_mode=noninteractive; search_did_work=1 search_mode=noninteractive; search_did_work=1
run_search_auth --set "${1#--}=$2" || exit $? run_search_auth --set-stdin "$search_stdin_provider" || exit $?
shift 2 ;; shift 2 ;;
--migrate) --migrate)
search_mode=noninteractive; search_did_work=1 search_mode=noninteractive; search_did_work=1
@@ -121,7 +129,7 @@ configure_search() {
if search_has_key "$provider"; then configured=1; else configured=0; fi if search_has_key "$provider"; then configured=1; else configured=0; fi
value=$(search_prompt_key "$label" "$configured") || exit $? value=$(search_prompt_key "$label" "$configured") || exit $?
if [ -n "$value" ]; then if [ -n "$value" ]; then
run_search_auth --set "$provider=$value" || exit $? printf '%s' "$value" | run_search_auth --set-stdin "$provider" || exit $?
search_did_work=1 search_did_work=1
fi fi
done done
+2 -2
View File
@@ -24,10 +24,10 @@
## 各 wrapper 约定 ## 各 wrapper 约定
- `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 `@` 补全;FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。 - `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep` 和 `@` 补全;`multi_grep` 按上游保持关闭(加载前删除 `PI_FFF_MULTIGREP` 环境变量)。FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。
- `bash-override.ts`:用 `createBashToolDefinition` 重新注册内置 `bash`,目的只是让 `tools.jsonc` 的 `descriptionAppend` 落到模型可见描述里(prompt-guard 在注册时拼接)。ExtensionContext 不暴露 session settings,因此 `shellPath`/`commandPrefix` 在首次 execute 时按 `ctx.cwd` 经 `SettingsManager.create(ctx.cwd, getAgentDir())` 惰性解析并按 cwd 缓存,metadata 来自 `process.cwd()` 的定义;执行、输出结构与 codemode 说明完全委托上游定义,不得改动上游执行逻辑,也不得用 prepareLoadout 或 provider-payload 改写实现同样效果。
- `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。 - `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。
- CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。 - CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。
- `context-mode.ts` + `context-mode-filter.ts`:经 `prompt-guard.ts` 代理加载上游 `context-mode` Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性;Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文。filter 只丢弃其 context 注入中与本组合包路由重复且冲突的锚点段。
- `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。 - `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
- `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。 - `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。
- `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。 - `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。
+38
View File
@@ -0,0 +1,38 @@
import {
createBashToolDefinition,
type ExtensionAPI,
getAgentDir,
SettingsManager,
} from "@earendil-works/pi-coding-agent";
import { guardExtensionApi } from "./prompt-guard.ts";
type BashDefinition = ReturnType<typeof createBashToolDefinition>;
/** Re-register Pi's bash so tools.jsonc can append to its model-facing description; execution stays upstream. */
export default function bashOverrideExtension(pi: ExtensionAPI): void {
const sessionDefinitions = new Map<string, BashDefinition>();
const sessionDefinition = (cwd: string): BashDefinition => {
let definition = sessionDefinitions.get(cwd);
if (definition === undefined) {
const settings = SettingsManager.create(cwd, getAgentDir());
definition = createBashToolDefinition(cwd, {
shellPath: settings.getShellPath(),
commandPrefix: settings.getShellCommandPrefix(),
});
sessionDefinitions.set(cwd, definition);
}
return definition;
};
guardExtensionApi(pi).registerTool({
...sessionDefinition(process.cwd()),
execute: (toolCallId, params, signal, onUpdate, ctx) =>
sessionDefinition(ctx?.cwd ?? process.cwd()).execute(
toolCallId,
params,
signal,
onUpdate,
ctx,
),
});
}
+2
View File
@@ -222,6 +222,8 @@ export class CodegraphRpcClient {
// ── internals ──────────────────────────────────────────────────────────── // ── internals ────────────────────────────────────────────────────────────
private attach(child: ChildProcess): void { private attach(child: ChildProcess): void {
// EPIPE on a write after the child died is reported via exit/close.
child.stdin?.on("error", () => {});
child.stdout?.setEncoding("utf8"); child.stdout?.setEncoding("utf8");
child.stdout?.on("data", (chunk: string) => { child.stdout?.on("data", (chunk: string) => {
this.stdoutBuffer += chunk; this.stdoutBuffer += chunk;
-36
View File
@@ -1,36 +0,0 @@
/**
* Pure filter applied after context-mode's `context` handler: its upstream
* build appends one `{ role: "user", content }` message whose parts are
* `"\n\n"`-joined and whose first part is a routing anchor that duplicates —
* and contradicts — the bundle-owned `config/prompt/routing.jsonc`. The anchor
* part is stripped; remaining parts (active memory, resume snapshot) stay.
* No other message is touched; when nothing changes the input array is
* returned unchanged (same reference).
*/
export const CONTEXT_MODE_ANCHOR_PREFIX = "context-mode active. Hierarchy:";
type MessageLike = { role?: unknown; content?: unknown } & Record<string, unknown>;
export function filterContextModeMessages<T extends MessageLike>(
before: readonly T[],
after: readonly T[],
): T[] {
if (after.length <= before.length) return after as T[];
let result: T[] | null = null;
for (let i = after.length - 1; i >= before.length; i--) {
const message = after[i];
if (message === null || typeof message !== "object") continue;
if (message.role !== "user" || typeof message.content !== "string") continue;
const parts = message.content.split("\n\n");
const kept = parts.filter((part) => !part.startsWith(CONTEXT_MODE_ANCHOR_PREFIX));
if (kept.length === parts.length) continue;
result ??= after.slice();
if (kept.length === 0) {
result.splice(i, 1);
} else {
result[i] = { ...message, content: kept.join("\n\n") };
}
}
return result ?? (after as T[]);
}
-18
View File
@@ -1,18 +0,0 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
// The adapter is a prebuilt .js bundle, not a TypeScript source.
import piExtension from "../node_modules/context-mode/build/adapters/pi/extension.js";
import { filterContextModeMessages } from "./context-mode-filter.ts";
import { guardExtensionApi } from "./prompt-guard.ts";
/**
* Load context-mode through the prompt guard: the bundle keeps sole ownership
* of the system prompt, so the upstream routing anchor appended to the
* `context` message list is filtered out (bundle routing lives in
* `config/prompt/routing.jsonc`), while upstream active-memory/resume parts
* pass through untouched. The upstream code this filter depends on is pinned
* by `tests/system-prompt.test.ts` against the exact `context-mode` version.
*/
export default function bundledContextModeExtension(pi: ExtensionAPI): void {
piExtension(guardExtensionApi(pi, { contextFilter: filterContextModeMessages }));
}
+1 -1
View File
@@ -17,7 +17,7 @@ export default function contextView(pi: ExtensionAPI): void {
`Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`, `Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`,
`Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`, `Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`,
`Segments: ${formatNumber(metrics.segmentCount)}`, `Segments: ${formatNumber(metrics.segmentCount)}`,
`Semantic jobs: ${formatNumber(metrics.semanticJobs)}`, `Masked tool results: ${formatNumber(metrics.maskedResults)}`,
`Generation: ${formatNumber(metrics.generation)}`, `Generation: ${formatNumber(metrics.generation)}`,
].join("\n") ].join("\n")
: "No pi-compaction projection is available yet. Send one message to build the first snapshot."; : "No pi-compaction projection is available yet. Send one message to build the first snapshot.";
+1
View File
@@ -5,5 +5,6 @@ import { guardExtensionApi } from "./prompt-guard.ts";
/** Load FFF as the bundle's canonical `find` and `grep` implementation. */ /** Load FFF as the bundle's canonical `find` and `grep` implementation. */
export default function fffOverrideExtension(pi: ExtensionAPI): void { export default function fffOverrideExtension(pi: ExtensionAPI): void {
process.env.PI_FFF_MODE = "override"; process.env.PI_FFF_MODE = "override";
delete process.env.PI_FFF_MULTIGREP;
fffExtension(guardExtensionApi(pi)); fffExtension(guardExtensionApi(pi));
} }
+28 -7
View File
@@ -1,11 +1,28 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { loadToolCatalog, type ToolCatalog } from "./system-prompt-config.ts"; import {
loadToolCatalog,
type ToolCatalog,
type ToolCatalogEntry,
} from "./system-prompt-config.ts";
type AnyMessage = { role?: string; content?: unknown } & Record<string, unknown>; type AnyMessage = { role?: string; content?: unknown } & Record<string, unknown>;
type AnyHandler = (event: never, ctx: never) => unknown; type AnyHandler = (event: never, ctx: never) => unknown;
/**
* The description a tool registers with under its catalog entry: a `description`
* override replaces the upstream text, a `descriptionAppend` is joined onto
* whichever base wins (override first, upstream otherwise) with a blank line.
*/
export function catalogDescription(description: string, entry: ToolCatalogEntry): string {
if ("inherit" in entry) return description;
const base = typeof entry.description === "string" ? entry.description : description;
return typeof entry.descriptionAppend === "string"
? `${base.trimEnd()}\n\n${entry.descriptionAppend}`
: base;
}
export interface PromptGuardOptions { export interface PromptGuardOptions {
/** /**
* Post-process the messages a wrapped `context` handler leaves behind. * Post-process the messages a wrapped `context` handler leaves behind.
@@ -19,9 +36,11 @@ export interface PromptGuardOptions {
* Proxy an `ExtensionAPI` for third-party extensions so they cannot bypass the * Proxy an `ExtensionAPI` for third-party extensions so they cannot bypass the
* bundle's single system-prompt owner (`extensions/system-prompt.ts`): * bundle's single system-prompt owner (`extensions/system-prompt.ts`):
* *
* - `registerTool` applies the tool's `description` override from * - `registerTool` applies the tool's `description` override and
* `config/prompt/tools.jsonc` when the catalog carries one (lazily loaded; * `descriptionAppend` suffix from `config/prompt/tools.jsonc` when the
* a catalog error keeps the upstream description). * catalog carries them (lazily loaded; a catalog error keeps the upstream
* description). An append composes onto the override when both are set, or
* onto the upstream description when only the append is set.
* - `before_agent_start` handlers keep every returned field except * - `before_agent_start` handlers keep every returned field except
* `systemPrompt`, which is dropped. * `systemPrompt`, which is dropped.
* - `context` handlers are post-processed through `opts.contextFilter`. * - `context` handlers are post-processed through `opts.contextFilter`.
@@ -42,10 +61,12 @@ export function guardExtensionApi(pi: ExtensionAPI, opts: PromptGuardOptions = {
const registerTool: ExtensionAPI["registerTool"] = (tool) => { const registerTool: ExtensionAPI["registerTool"] = (tool) => {
const entry = catalog()?.tools[tool.name]; const entry = catalog()?.tools[tool.name];
if (entry !== undefined && !("inherit" in entry) && typeof entry.description === "string") { const description =
pi.registerTool({ ...tool, description: entry.description }); entry === undefined ? tool.description : catalogDescription(tool.description, entry);
} else { if (description === tool.description) {
pi.registerTool(tool); pi.registerTool(tool);
} else {
pi.registerTool({ ...tool, description });
} }
}; };
+15 -7
View File
@@ -10,7 +10,12 @@ import { fileURLToPath } from "node:url";
/** Entry in `config/prompt/tools.jsonc`. */ /** Entry in `config/prompt/tools.jsonc`. */
export type ToolCatalogEntry = export type ToolCatalogEntry =
| { inherit: true } | { inherit: true }
| { snippet?: string | null; guidelines?: string[]; description?: string }; | {
snippet?: string | null;
guidelines?: string[];
description?: string;
descriptionAppend?: string;
};
export interface ToolCatalog { export interface ToolCatalog {
version: 1; version: 1;
@@ -152,7 +157,7 @@ function isStringArray(value: unknown): value is string[] {
return Array.isArray(value) && value.every((entry) => typeof entry === "string"); return Array.isArray(value) && value.every((entry) => typeof entry === "string");
} }
const TOOL_ENTRY_KEYS = new Set(["inherit", "snippet", "guidelines", "description"]); const TOOL_ENTRY_KEYS = new Set(["inherit", "snippet", "guidelines", "description", "descriptionAppend"]);
function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry { function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
if (!isPlainObject(raw)) { if (!isPlainObject(raw)) {
@@ -170,9 +175,9 @@ function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
return { inherit: true }; return { inherit: true };
} }
const hasField = const hasField =
"snippet" in raw || "guidelines" in raw || "description" in raw; "snippet" in raw || "guidelines" in raw || "description" in raw || "descriptionAppend" in raw;
if (!hasField) { if (!hasField) {
fail("tools.jsonc", `tool "${name}" needs "inherit": true or at least one of snippet/guidelines/description`); fail("tools.jsonc", `tool "${name}" needs "inherit": true or at least one of snippet/guidelines/description/descriptionAppend`);
} }
if ("snippet" in raw && raw.snippet !== null && typeof raw.snippet !== "string") { if ("snippet" in raw && raw.snippet !== null && typeof raw.snippet !== "string") {
fail("tools.jsonc", `tool "${name}".snippet must be a string or null`); fail("tools.jsonc", `tool "${name}".snippet must be a string or null`);
@@ -183,10 +188,13 @@ function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
if ("description" in raw && typeof raw.description !== "string") { if ("description" in raw && typeof raw.description !== "string") {
fail("tools.jsonc", `tool "${name}".description must be a string`); fail("tools.jsonc", `tool "${name}".description must be a string`);
} }
if ("descriptionAppend" in raw && typeof raw.descriptionAppend !== "string") {
fail("tools.jsonc", `tool "${name}".descriptionAppend must be a string`);
}
return raw as ToolCatalogEntry; return raw as ToolCatalogEntry;
} }
function validateToolCatalog(raw: unknown): ToolCatalog { export function validateToolCatalog(raw: unknown): ToolCatalog {
if (!isPlainObject(raw) || raw.version !== 1 || !isPlainObject(raw.tools)) { if (!isPlainObject(raw) || raw.version !== 1 || !isPlainObject(raw.tools)) {
fail("tools.jsonc", 'expected { "version": 1, "tools": { ... } }'); fail("tools.jsonc", 'expected { "version": 1, "tools": { ... } }');
} }
@@ -321,8 +329,8 @@ export function isSubagentCustomPrompt(customPrompt: unknown): boolean {
* Apply the tool catalog to the mutable prompt options: a `snippet` string * Apply the tool catalog to the mutable prompt options: a `snippet` string
* replaces the tool's `<tools>` line (`null` hides it), `guidelines` replaces * replaces the tool's `<tools>` line (`null` hides it), `guidelines` replaces
* the tool's `<rules>` lines; `{inherit:true}` entries are left untouched. * the tool's `<rules>` lines; `{inherit:true}` entries are left untouched.
* `description` is not applied here — it is a registration-time concern handled * `description` and `descriptionAppend` are not applied here — they are
* by `extensions/prompt-guard.ts`. * registration-time concerns handled by `extensions/prompt-guard.ts`.
* *
* Returns the selected tool names that have no catalog entry at all. * Returns the selected tool names that have no catalog entry at all.
*/ */
+66 -1553
View File
File diff suppressed because it is too large Load Diff
+89 -19
View File
@@ -10,15 +10,76 @@
], ],
"files": [ "files": [
"extensions", "extensions",
"pi-ask-user", "pi-ask-user/index.ts",
"pi-compaction", "pi-ask-user/src",
"pi-chrome", "pi-ask-user/package.json",
"pi-hermes-memory", "pi-ask-user/README.md",
"pi-rtk-optimizer", "pi-ask-user/LICENSE",
"pi-minimal-footer", "pi-ask-user/UPSTREAM.md",
"pi-subagents", "pi-compaction/index.ts",
"pi-notify", "pi-compaction/src",
"pi-ssh", "pi-compaction/package.json",
"pi-compaction/README.md",
"pi-compaction/CHANGELOG.md",
"pi-compaction/LICENSES",
"pi-compaction/UPSTREAM.md",
"pi-chrome/extensions",
"pi-chrome/package.json",
"pi-chrome/README.md",
"pi-chrome/CHANGELOG.md",
"pi-chrome/SECURITY.md",
"pi-chrome/LICENSE",
"pi-chrome/UPSTREAM.md",
"pi-hermes-memory/src",
"pi-hermes-memory/package.json",
"pi-hermes-memory/README.md",
"pi-hermes-memory/CHANGELOG.md",
"pi-hermes-memory/LICENSE",
"pi-hermes-memory/UPSTREAM.md",
"pi-rtk-optimizer/index.ts",
"pi-rtk-optimizer/src",
"pi-rtk-optimizer/package.json",
"pi-rtk-optimizer/README.md",
"pi-rtk-optimizer/CHANGELOG.md",
"pi-rtk-optimizer/LICENSE",
"pi-rtk-optimizer/UPSTREAM.md",
"pi-minimal-footer/index.ts",
"pi-minimal-footer/status.ts",
"pi-minimal-footer/settings.ts",
"pi-minimal-footer/assets",
"pi-minimal-footer/package.json",
"pi-minimal-footer/README.md",
"pi-minimal-footer/LICENSE",
"pi-minimal-footer/UPSTREAM.md",
"pi-subagents/src",
"pi-subagents/package.json",
"pi-subagents/README.md",
"pi-subagents/CHANGELOG.md",
"pi-subagents/SECURITY.md",
"pi-subagents/LICENSE",
"pi-subagents/UPSTREAM.md",
"pi-overflow/index.ts",
"pi-overflow/capture.ts",
"pi-overflow/settings.ts",
"pi-overflow/src",
"pi-overflow/package.json",
"pi-overflow/README.md",
"pi-overflow/LICENSE",
"pi-overflow/UPSTREAM.md",
"pi-notify/src",
"pi-notify/package.json",
"pi-notify/README.md",
"pi-notify/LICENSE",
"pi-notify/UPSTREAM.md",
"pi-ssh/index.ts",
"pi-ssh/permission-integration.ts",
"pi-ssh/src",
"pi-ssh/scripts",
"pi-ssh/package.json",
"pi-ssh/README.md",
"pi-ssh/CHANGELOG.md",
"pi-ssh/LICENSE",
"pi-ssh/UPSTREAM.md",
"pi-tool-search/extensions", "pi-tool-search/extensions",
"pi-tool-search/docs", "pi-tool-search/docs",
"pi-tool-search/package.json", "pi-tool-search/package.json",
@@ -26,8 +87,20 @@
"pi-tool-search/CHANGELOG.md", "pi-tool-search/CHANGELOG.md",
"pi-tool-search/LICENSE", "pi-tool-search/LICENSE",
"pi-tool-search/UPSTREAM.md", "pi-tool-search/UPSTREAM.md",
"pi-web-search", "pi-web-search/index.ts",
"pi-worktree", "pi-web-search/auth.ts",
"pi-web-search/router.ts",
"pi-web-search/clients",
"pi-web-search/package.json",
"pi-web-search/README.md",
"pi-web-search/LICENSE",
"pi-web-search/UPSTREAM.md",
"pi-worktree/index.ts",
"pi-worktree/src",
"pi-worktree/package.json",
"pi-worktree/README.md",
"pi-worktree/LICENSE",
"pi-worktree/UPSTREAM.md",
"pi-permission-system/LICENSES", "pi-permission-system/LICENSES",
"pi-permission-system/docs/auto-review", "pi-permission-system/docs/auto-review",
"pi-permission-system/src", "pi-permission-system/src",
@@ -36,6 +109,7 @@
"pi-permission-system/README.md", "pi-permission-system/README.md",
"pi-permission-system/CHANGELOG.md", "pi-permission-system/CHANGELOG.md",
"pi-permission-system/LICENSE", "pi-permission-system/LICENSE",
"pi-permission-system/UPSTREAM.md",
"config", "config",
"scripts", "scripts",
"install.sh", "install.sh",
@@ -47,12 +121,13 @@
], ],
"pi": { "pi": {
"extensions": [ "extensions": [
"./pi-overflow/capture.ts",
"./extensions/fff-override.ts", "./extensions/fff-override.ts",
"./extensions/bash-override.ts",
"./pi-web-search/index.ts", "./pi-web-search/index.ts",
"./extensions/codegraph.ts", "./extensions/codegraph.ts",
"./pi-rtk-optimizer/index.ts", "./pi-rtk-optimizer/index.ts",
"./extensions/hashline.ts", "./extensions/hashline.ts",
"./extensions/context-mode.ts",
"./extensions/lsp.ts", "./extensions/lsp.ts",
"./pi-hermes-memory/src/index.ts", "./pi-hermes-memory/src/index.ts",
"./pi-compaction/index.ts", "./pi-compaction/index.ts",
@@ -66,10 +141,8 @@
"./pi-worktree/index.ts", "./pi-worktree/index.ts",
"./pi-chrome/extensions/chrome-profile-bridge/index.ts", "./pi-chrome/extensions/chrome-profile-bridge/index.ts",
"./pi-tool-search/extensions/index.ts", "./pi-tool-search/extensions/index.ts",
"./pi-overflow/index.ts",
"./extensions/system-prompt.ts" "./extensions/system-prompt.ts"
],
"skills": [
"./node_modules/context-mode/skills"
] ]
}, },
"dependencies": { "dependencies": {
@@ -78,13 +151,13 @@
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer", "@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
"@smoose/pi-notify": "file:./pi-notify", "@smoose/pi-notify": "file:./pi-notify",
"@tintinweb/pi-subagents": "file:./pi-subagents", "@tintinweb/pi-subagents": "file:./pi-subagents",
"context-mode": "1.0.169",
"jiti": "2.7.0", "jiti": "2.7.0",
"pi-ask-user": "file:./pi-ask-user", "pi-ask-user": "file:./pi-ask-user",
"pi-chrome": "file:./pi-chrome", "pi-chrome": "file:./pi-chrome",
"pi-compaction": "file:./pi-compaction", "pi-compaction": "file:./pi-compaction",
"pi-hashline-edit": "0.8.3", "pi-hashline-edit": "0.8.3",
"pi-hermes-memory": "file:./pi-hermes-memory", "pi-hermes-memory": "file:./pi-hermes-memory",
"pi-overflow": "file:./pi-overflow",
"pi-lsp": "0.1.7", "pi-lsp": "0.1.7",
"pi-ssh": "file:./pi-ssh", "pi-ssh": "file:./pi-ssh",
"pi-tool-search": "file:./pi-tool-search", "pi-tool-search": "file:./pi-tool-search",
@@ -117,8 +190,5 @@
}, },
"engines": { "engines": {
"node": ">=22.19.0" "node": ">=22.19.0"
},
"allowScripts": {
"better-sqlite3@12.11.1": true
} }
} }
+2 -3
View File
@@ -1,9 +1,8 @@
# my-pi 维护约定(pi-chrome) # my-pi 维护约定(pi-chrome)
- 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,Session 快照存储位于 `src/session-snapshot-store.ts`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。 - 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。
- 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。 - 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。
- 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。 - 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。
- 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。 - 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
- 完整 Chrome snapshot 不得进入普通工具文本或 `details`;只能原子写入当前项目 `.pi/chrome-context/<session>/<target>/snapshot.json`,使用哈希目录、POSIX `700` 目录 / `600` 文件,并只返回工作区相对路径和有界元数据。快照不得跨 Session 持久化,Session 启动/关闭、撤销和授权过期必须清理。 - `chrome_snapshot` 与带 `includeSnapshot=true` 的动作把快照 JSON 直接放进工具结果文本返回;超长结果的展示截断由 pi-overflow 统一负责(原文落盘,模型用 `overflow_query` 的 jsonPath/search 提取)。`details` 只放元素数、匹配数等小型元数据,不得把完整快照写进 `details`(会进入会话文件),也不再写 Session 临时快照文件。`chrome_find` 只返回有界匹配摘要,不保存文件。
- `chrome_snapshot` 和带 `includeSnapshot=true` 的动作必须引导模型立即使用 `ctx_execute_file`,不得使用 `read` / `cat` 把原始 JSON 重新塞入上下文;任何可能改变页面或目标的成功操作必须使旧快照失效,`chrome_find` 只可额外返回有界匹配摘要。
- 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。 - 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。
+4 -10
View File
@@ -6,7 +6,7 @@
```text ```text
You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI." You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
Agent: chrome_tab(list) → chrome_snapshot(...) → ctx_execute_file(capture) → chrome_screenshot(...) Agent: chrome_tab(list) → chrome_snapshot(...) → overflow_query(jsonPath) → chrome_screenshot(...)
✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png ✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
You: [keeps coding — agent never asked you to log in] You: [keeps coding — agent never asked you to log in]
``` ```
@@ -74,17 +74,11 @@ Second doctor run should show all checks passing.
Tool parameters and gotchas are documented inline in Pi. Tool parameters and gotchas are documented inline in Pi.
### Session-local snapshots and Context Mode ### Snapshots and pi-overflow
In this maintained build, `chrome_snapshot`, `chrome_find`, and actions with `includeSnapshot=true` keep the full structured page snapshot out of the normal tool result. They atomically save only the latest capture for each target under: In this maintained build, `chrome_snapshot` and actions with `includeSnapshot=true` return the full structured page snapshot as JSON directly in the tool result. `chrome_find` returns only up to 20 compact ranked matches.
```text Large snapshots are bounded by the bundle's `pi-overflow` extension: the result keeps a bounded view and the full original is stored, marked with `[pi-overflow id=…]`. Extract the uids, text, or state you need with `overflow_query` — a `jsonPath` such as `elements[0].uid` or `summary.focused`, or a regex `search` — instead of taking another snapshot. Any page interaction makes earlier snapshots stale, so take a fresh one (or pass `includeSnapshot=true` on the action) when the page has changed.
.pi/chrome-context/<session-hash>/<target-hash>/snapshot.json
```
The tool returns a workspace-relative path plus bounded metadata. Analyze that path immediately with `ctx_execute_file` and print only the uids, text, or state needed for the next browser action. Do **not** use `read` or `cat` on the capture, because that would put the raw page back into model context.
Captures are deliberately ephemeral: a successful page interaction invalidates the old target capture, and session shutdown, `/chrome revoke`, or authorization expiry removes the session files. `chrome_find` still returns up to 20 compact ranked matches for direct use while storing the full capture privately.
--- ---
+4 -5
View File
@@ -15,10 +15,9 @@
导入后的本地改动: 导入后的本地改动:
- Chrome 快照以原子方式写入当前项目 `.pi/chrome-context/` 目录下的 owner-only、Session 临时文件,而不是把原始页面结构返回给模型上下文。 - Chrome 快照与 `includeSnapshot=true` 的动作结果把快照 JSON 直接放入工具文本返回,`details` 只保留元素数/匹配数等小型元数据;展示截断由组合包 pi-overflow 统一负责(原文落盘 + `overflow_query` 查询)。`chrome_find` 只返回紧凑排序匹配,不保存文件。早期版本曾把快照原子写入 `.pi/chrome-context/` 临时文件并引导 `ctx_execute_file` 分析;该 Session 临时文件机制已随 context-mode 一起移除,各项目中残留的 `.pi/chrome-context/` 目录不再创建,可手动删除。
- 每个 Session/目标只保留最新快照;浏览器交互会使陈旧捕获失效,Session 关闭时移除该 Session 目录。
- `chrome_find` 返回紧凑的排序匹配,同时把完整结构化快照保留在 Session 本地捕获中,供可选的 `ctx_execute_file` 分析使用。
- click/type/fill/key 上的 `includeSnapshot=true` 会保存动作后的快照并返回其捕获路径,而不是把快照嵌入工具结果。
- 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。 - 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。
- 移除 `PI_CHROME_BRIDGE_HOST`/`PI_CHROME_BRIDGE_PORT` 环境变量,host/port 固定为 `127.0.0.1:17318`(Companion 扩展 `service_worker.js` 同样硬编码该地址,单侧可配只会断连且非回环 host 有安全风险)。
- 移除 `MAX_TEXT_CHARS`/`truncateText` 的 30K 字符文本截断:chrome_* 工具返回完整文本,超长结果的展示截断由 pi-overflow 统一负责;`MAX_ELEMENTS` 等条数上限保留不变。
同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期、隐私和 Context Mode 集成。 同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期和组合包集成改动。
@@ -4,11 +4,6 @@ import { existsSync, readFileSync, statSync } from "node:fs";
import { mkdir, writeFile } from "node:fs/promises"; import { mkdir, writeFile } from "node:fs/promises";
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
import { dirname, join, resolve } from "node:path"; import { dirname, join, resolve } from "node:path";
import {
SessionSnapshotStore,
type SnapshotCaptureMetadata,
type SnapshotTarget,
} from "../../src/session-snapshot-store.ts";
/** /**
* Existing-profile Chrome bridge for pi. * Existing-profile Chrome bridge for pi.
@@ -64,17 +59,11 @@ const PI_CHROME_GLOBAL_KEY = "__piChromeProfileBridgeLoaded__";
// reload) so a /reload — which tears down and re-evaluates the module — does not silently drop // reload) so a /reload — which tears down and re-evaluates the module — does not silently drop
// an active /chrome authorize grant. // an active /chrome authorize grant.
const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__"; const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__";
const DEFAULT_HOST = process.env.PI_CHROME_BRIDGE_HOST ?? "127.0.0.1"; const DEFAULT_HOST = "127.0.0.1";
const DEFAULT_PORT = Number(process.env.PI_CHROME_BRIDGE_PORT ?? "17318"); const DEFAULT_PORT = 17318;
const DEFAULT_TIMEOUT_MS = 30_000; const DEFAULT_TIMEOUT_MS = 30_000;
const MAX_TEXT_CHARS = 30_000;
const MAX_ELEMENTS = 80; const MAX_ELEMENTS = 80;
function truncateText(text: string, maxChars = MAX_TEXT_CHARS): string {
if (text.length <= maxChars) return text;
return `${text.slice(0, maxChars)}\n\n[truncated ${text.length - maxChars} characters]`;
}
function safeJson(value: unknown): string { function safeJson(value: unknown): string {
return JSON.stringify(value, null, 2); return JSON.stringify(value, null, 2);
} }
@@ -95,13 +84,28 @@ function snapshotFromIncludedResult(raw: unknown): unknown {
return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined; return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined;
} }
export function formatSnapshotCapture(metadata: SnapshotCaptureMetadata): string { /**
const kib = Math.max(1, Math.ceil(metadata.bytes / 1024)); * Small, session-file-safe metadata describing a returned snapshot. The full
return [ * snapshot goes to the model as text — details must stay small because Pi
`Chrome snapshot saved to ${metadata.path} (${kib} KiB, ${metadata.elementCount} elements${metadata.matchCount ? `, ${metadata.matchCount} matches` : ""}).`, * persists them in the session file.
"Analyze it immediately with ctx_execute_file; FILE_CONTENT is JSON and the page data is under .snapshot.", */
"This capture is replaced by the next snapshot for the same target and becomes stale after page interaction.", export function snapshotStats(snapshot: unknown): { elementCount: number; matchCount: number } {
].join("\n"); const object = snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
? snapshot as { elements?: unknown; matches?: unknown }
: {};
return {
elementCount: Array.isArray(object.elements) ? object.elements.length : 0,
matchCount: Array.isArray(object.matches) ? object.matches.length : 0,
};
}
/**
* Action summary followed by the inline snapshot JSON. Oversized results are
* bounded by pi-overflow and recoverable through overflow_query, so no local
* capture file is written.
*/
export function formatActionResultText(text: string, snapshot: unknown | undefined): string {
return snapshot === undefined ? text : `${text}\n\n${safeJson(snapshot)}`;
} }
export function formatChromeFindMatches(snapshot: any): string { export function formatChromeFindMatches(snapshot: any): string {
@@ -142,7 +146,7 @@ function formatChromeInspect(inspect: any): string {
lines.push("\n## Ancestors"); lines.push("\n## Ancestors");
for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`); for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`);
} }
return truncateText(lines.join("\n")); return lines.join("\n");
} }
function extensionRoot(): string { function extensionRoot(): string {
@@ -648,7 +652,6 @@ export default function (pi: ExtensionAPI): void {
let countdownInterval: NodeJS.Timeout | undefined; let countdownInterval: NodeJS.Timeout | undefined;
// Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy. // Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy.
let sessionCtx: ExtensionContext | undefined; let sessionCtx: ExtensionContext | undefined;
let snapshotStore: SessionSnapshotStore | undefined;
const clearAuthExpiryTimer = (): void => { const clearAuthExpiryTimer = (): void => {
if (!authExpiryTimer) return; if (!authExpiryTimer) return;
@@ -722,7 +725,6 @@ export default function (pi: ExtensionAPI): void {
persistAuth(); persistAuth();
// Revoking control ends pi-chrome's automation for this session; tidy up the target we own. // Revoking control ends pi-chrome's automation for this session; tidy up the target we own.
cleanupAutomationTargetBestEffort(); cleanupAutomationTargetBestEffort();
void snapshotStore?.clear().catch(() => undefined);
}; };
const authSummary = (): string => { const authSummary = (): string => {
@@ -853,45 +855,22 @@ export default function (pi: ExtensionAPI): void {
return { ...params, foreground: !background } as T; return { ...params, foreground: !background } as T;
}; };
const snapshotTargetFrom = (params: Record<string, unknown>): SnapshotTarget => ({ const formatActionResult = (raw: unknown, params: Record<string, unknown>, text: string): ToolTextResult => {
targetId: typeof params.targetId === "string" ? params.targetId : undefined,
urlIncludes: typeof params.urlIncludes === "string" ? params.urlIncludes : undefined,
titleIncludes: typeof params.titleIncludes === "string" ? params.titleIncludes : undefined,
});
const saveSnapshotCapture = async (snapshot: unknown, params: Record<string, unknown>): Promise<SnapshotCaptureMetadata> => {
if (!snapshotStore) throw new Error("Chrome snapshot storage is unavailable before session_start.");
const stored = await snapshotStore.save(snapshot, snapshotTargetFrom(params));
return stored.metadata;
};
const invalidateSnapshotCapture = async (params?: Record<string, unknown>): Promise<void> => {
if (!snapshotStore) return;
if (!params) await snapshotStore.clear();
else await snapshotStore.invalidate(snapshotTargetFrom(params));
};
const formatActionWithCapture = async (raw: unknown, params: Record<string, unknown>, text: string): Promise<ToolTextResult> => {
const snapshot = snapshotFromIncludedResult(raw); const snapshot = snapshotFromIncludedResult(raw);
const result = params.includeSnapshot && raw && typeof raw === "object" const result = params.includeSnapshot && raw && typeof raw === "object"
? (raw as { result?: unknown }).result ? (raw as { result?: unknown }).result
: raw; : raw;
if (!snapshot) { if (!snapshot) {
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text }], details: { result: result as Json } }; return { content: [{ type: "text", text }], details: { result: result as Json } };
} }
const capture = await saveSnapshotCapture(snapshot, params);
return { return {
content: [{ type: "text", text: `${text}\n\n${formatSnapshotCapture(capture)}` }], content: [{ type: "text", text: formatActionResultText(text, snapshot) }],
details: { result: result as Json, capture }, details: { result: result as Json, snapshot: snapshotStats(snapshot) },
}; };
}; };
pi.on("session_start", async (_event, ctx) => { pi.on("session_start", async (_event, ctx) => {
sessionCtx = ctx; sessionCtx = ctx;
const sessionId = ctx.sessionManager.getSessionId?.() || "unknown-session";
snapshotStore = new SessionSnapshotStore(workspaceCwd(ctx), sessionId);
await snapshotStore.clear();
await bridge.start(); await bridge.start();
// Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis. // Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis.
if (chromeControlAuthorized()) { if (chromeControlAuthorized()) {
@@ -909,10 +888,6 @@ export default function (pi: ExtensionAPI): void {
pi.on("session_shutdown", async (event) => { pi.on("session_shutdown", async (event) => {
clearAuthExpiryTimer(); clearAuthExpiryTimer();
clearCountdownInterval(); clearCountdownInterval();
// Snapshot files are intentionally ephemeral. A reload must take a fresh observation too,
// because the browser page may have changed while the extension was reloading.
await snapshotStore?.clear().catch(() => undefined);
snapshotStore = undefined;
// Keep the owned automation target across /reload, but clean it up on real session end. // Keep the owned automation target across /reload, but clean it up on real session end.
if (event?.reason !== "reload") cleanupAutomationTargetBestEffort(); if (event?.reason !== "reload") cleanupAutomationTargetBestEffort();
bridge.stop(); bridge.stop();
@@ -939,12 +914,12 @@ Capability model (important):
- Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates. - Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates.
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached. - Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP. - \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP.
- \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` are written to a private session-local JSON capture; raw page data is never returned inline. The tool result gives the path for immediate \`ctx_execute_file\` analysis. - \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` return the page snapshot as JSON directly in the tool result. When the result is large it carries a \`[pi-overflow id=…]\` marker — extract only the uids, text, or state you need with \`overflow_query\` (jsonPath such as \`elements[0].uid\` or \`summary.focused\`, or a \`search\`) instead of taking another snapshot.
Usage rules: Usage rules:
1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying. 1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying.
2. Call \`chrome_snapshot\`, then immediately call \`ctx_execute_file\` on the returned path and print only the uids/state needed for the next action. Never use read/cat on the full capture. 2. Call \`chrome_snapshot\` and read the returned JSON (or query a pi-overflow marker with \`overflow_query\`); keep only the uids/state needed for the next action.
3. Prefer \`uid\` over \`selector\`. After any page interaction, treat the old capture as stale. Use \`includeSnapshot=true\` when a fresh post-action capture is needed. 3. Prefer \`uid\` over \`selector\`. After any page interaction, earlier snapshots are stale — take a fresh one (or pass \`includeSnapshot=true\` on the action) instead of reusing old uids.
4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm. 4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm.
5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now). 5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now).
6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work. 6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work.
@@ -1337,7 +1312,6 @@ Usage rules:
const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs."; const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs.";
return { content: [{ type: "text", text }], details: { tabs } }; return { content: [{ type: "text", text }], details: { tabs } };
} }
if (params.action !== "version") await invalidateSnapshotCapture();
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1346,8 +1320,8 @@ Usage rules:
name: "chrome_snapshot", name: "chrome_snapshot",
label: "Chrome Snapshot", label: "Chrome Snapshot",
description: description:
"Capture the current Chrome page into a private, session-local JSON file instead of placing the raw DOM snapshot in model context. The result returns a path for immediate ctx_execute_file analysis. Each target keeps only its latest capture; page interaction makes it stale. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.", "Capture the current Chrome page and return the snapshot as JSON directly in the result. When it is large the result carries a [pi-overflow id=…] marker — extract only the uids, text, or state you need with overflow_query (jsonPath or search). Page interaction makes a snapshot stale; take a fresh one after navigating or acting. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.",
promptSnippet: "Save the current Chrome page to a session-local JSON capture, then analyze that path with ctx_execute_file for only the required uids and state.", promptSnippet: "Capture the current Chrome page and return its JSON snapshot; query large results via overflow_query.",
parameters: Type.Object({ parameters: Type.Object({
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1372,8 +1346,7 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params); return { content: [{ type: "text", text: safeJson(snapshot) }], details: { snapshot: snapshotStats(snapshot) } };
return { content: [{ type: "text", text: formatSnapshotCapture(capture) }], details: { capture } };
}, },
}); });
@@ -1381,8 +1354,8 @@ Usage rules:
name: "chrome_find", name: "chrome_find",
label: "Chrome Find", label: "Chrome Find",
description: description:
"Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates; the full structured snapshot is saved to the same private session-local file used by chrome_snapshot for optional ctx_execute_file analysis.", "Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates.",
promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches while keeping the full snapshot out of model context.", promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches.",
parameters: Type.Object({ parameters: Type.Object({
query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }), query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }),
mode: Type.Optional(StringEnum(snapshotModeValues)), mode: Type.Optional(StringEnum(snapshotModeValues)),
@@ -1403,10 +1376,9 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params);
return { return {
content: [{ type: "text", text: `${formatChromeFindMatches(snapshot)}\n\n${formatSnapshotCapture(capture)}` }], content: [{ type: "text", text: formatChromeFindMatches(snapshot) }],
details: { capture }, details: { snapshot: snapshotStats(snapshot) },
}; };
}, },
}); });
@@ -1452,9 +1424,8 @@ Usage rules:
DEFAULT_TIMEOUT_MS, DEFAULT_TIMEOUT_MS,
signal, signal,
); );
const capture = await saveSnapshotCapture(snapshot, params); const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${safeJson(snapshot)}`;
const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${formatSnapshotCapture(capture)}`; return { content: [{ type: "text", text }], details: { snapshot: snapshotStats(snapshot), fallback: "page.snapshot" } };
return { content: [{ type: "text", text }], details: { capture, fallback: "page.snapshot" } };
} }
}, },
}); });
@@ -1481,7 +1452,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal); const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } };
}, },
}); });
@@ -1506,13 +1476,12 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
const text = value === undefined const text = value === undefined
? "undefined" ? "undefined"
: typeof value === "string" : typeof value === "string"
? value ? value
: safeJson(value) ?? "undefined"; : safeJson(value) ?? "undefined";
return { content: [{ type: "text", text: truncateText(text) }], details: { value: value as Json } }; return { content: [{ type: "text", text }], details: { value: value as Json } };
}, },
}); });
@@ -1520,7 +1489,7 @@ Usage rules:
name: "chrome_click", name: "chrome_click",
label: "Chrome Click", label: "Chrome Click",
description: description:
"Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-click snapshot to the session-local file and return its ctx_execute_file path, never the raw snapshot.", "Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. Earlier snapshots become stale after the click. Pass includeSnapshot=true to get a fresh post-click snapshot JSON inline.",
promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.", promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.",
parameters: Type.Object({ parameters: Type.Object({
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })), uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })),
@@ -1528,7 +1497,7 @@ Usage rules:
x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })), x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })),
y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })), y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })),
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })), domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-click snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-click snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1545,7 +1514,7 @@ Usage rules:
const summary = summarizeActionResult(result); const summary = summarizeActionResult(result);
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`; const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`; const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1553,13 +1522,13 @@ Usage rules:
name: "chrome_type", name: "chrome_type",
label: "Chrome Type", label: "Chrome Type",
description: description:
"Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-type snapshot and return its ctx_execute_file path.", "Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. Earlier snapshots become stale after typing. Pass includeSnapshot=true to get a fresh post-type snapshot JSON inline.",
promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.", promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.",
parameters: Type.Object({ parameters: Type.Object({
text: Type.String(), text: Type.String(),
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })), uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })), selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-type snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-type snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
pressEnter: Type.Optional(Type.Boolean()), pressEnter: Type.Optional(Type.Boolean()),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
@@ -1578,7 +1547,7 @@ Usage rules:
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : ""; const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
const base = `Typed ${params.text.length} character(s)${into}.`; const base = `Typed ${params.text.length} character(s)${into}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1586,7 +1555,7 @@ Usage rules:
name: "chrome_fill", name: "chrome_fill",
label: "Chrome Fill", label: "Chrome Fill",
description: description:
"Set the full value of a text input, textarea, or contenteditable element using Chrome input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-fill snapshot and return its ctx_execute_file path.", "Set the full value of a text input, textarea, or contenteditable element using Chrome input. Earlier snapshots become stale after filling. Pass includeSnapshot=true to get a fresh post-fill snapshot JSON inline.",
promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.", promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.",
parameters: Type.Object({ parameters: Type.Object({
text: Type.String(), text: Type.String(),
@@ -1594,7 +1563,7 @@ Usage rules:
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })), selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })), submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })), domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-fill snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-fill snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1612,7 +1581,7 @@ Usage rules:
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : ""; const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
const base = `Filled ${params.text.length} character(s)${into}.`; const base = `Filled ${params.text.length} character(s)${into}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1620,7 +1589,7 @@ Usage rules:
name: "chrome_key", name: "chrome_key",
label: "Chrome Key", label: "Chrome Key",
description: description:
"Send a keyboard key to an existing Chrome tab. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-key snapshot and return its ctx_execute_file path. Runs in the background by default; pass background=false so the user can watch.", "Send a keyboard key to an existing Chrome tab. Earlier snapshots become stale after the key press. Pass includeSnapshot=true to get a fresh post-key snapshot JSON inline. Runs in the background by default; pass background=false so the user can watch.",
promptSnippet: "Press keys in Chrome through the companion extension.", promptSnippet: "Press keys in Chrome through the companion extension.",
parameters: Type.Object({ parameters: Type.Object({
key: Type.String(), key: Type.String(),
@@ -1630,7 +1599,7 @@ Usage rules:
altKey: Type.Optional(Type.Boolean()), altKey: Type.Optional(Type.Boolean()),
metaKey: Type.Optional(Type.Boolean()), metaKey: Type.Optional(Type.Boolean()),
}, { description: "Modifier keys to hold while pressing the key (chord)." })), }, { description: "Modifier keys to hold while pressing the key (chord)." })),
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-key snapshot to the session-local capture file and return its path." })), includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-key snapshot JSON inline after the action summary." })),
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })), maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
targetId: Type.Optional(Type.String()), targetId: Type.Optional(Type.String()),
urlIncludes: Type.Optional(Type.String()), urlIncludes: Type.Optional(Type.String()),
@@ -1647,7 +1616,7 @@ Usage rules:
const summary = summarizeActionResult(result); const summary = summarizeActionResult(result);
const base = `Pressed ${params.key}.`; const base = `Pressed ${params.key}.`;
const text = summary ? `${base} (${summary})` : base; const text = summary ? `${base} (${summary})` : base;
return formatActionWithCapture(raw, params, text); return formatActionResult(raw, params, text);
}, },
}); });
@@ -1669,7 +1638,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal); const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } };
}, },
}); });
@@ -1691,7 +1659,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1713,7 +1681,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1733,7 +1701,7 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } }; return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
}, },
}); });
@@ -1813,7 +1781,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } };
}, },
}); });
@@ -1840,7 +1807,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } };
}, },
}); });
@@ -1863,7 +1829,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`; const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } };
}, },
@@ -1887,7 +1852,6 @@ Usage rules:
}), }),
async execute(_id, params, signal): Promise<ToolTextResult> { async execute(_id, params, signal): Promise<ToolTextResult> {
const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } };
}, },
}); });
@@ -1910,7 +1874,6 @@ Usage rules:
const cwd = workspaceCwd(ctx); const cwd = workspaceCwd(ctx);
const paths = params.paths.map((p) => resolve(cwd, p)); const paths = params.paths.map((p) => resolve(cwd, p));
const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal); const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal);
await invalidateSnapshotCapture(params);
return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } }; return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } };
}, },
}); });
+1 -2
View File
@@ -2,7 +2,7 @@
"name": "pi-chrome", "name": "pi-chrome",
"version": "0.15.46", "version": "0.15.46",
"scripts": { "scripts": {
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/session-snapshot-store.test.ts", "test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/snapshot-result.test.ts",
"version": "node scripts/sync-manifest-version.js", "version": "node scripts/sync-manifest-version.js",
"prepublishOnly": "node scripts/sync-manifest-version.js" "prepublishOnly": "node scripts/sync-manifest-version.js"
}, },
@@ -52,7 +52,6 @@
"type": "commonjs", "type": "commonjs",
"files": [ "files": [
"extensions", "extensions",
"src",
"test", "test",
"docs", "docs",
"test-suite", "test-suite",
-140
View File
@@ -1,140 +0,0 @@
import { createHash, randomUUID } from "node:crypto";
import { chmod, mkdir, rename, rm, writeFile } from "node:fs/promises";
import { join, relative, resolve, sep } from "node:path";
type SnapshotObject = Record<string, unknown>;
export type SnapshotTarget = {
targetId?: string;
urlIncludes?: string;
titleIncludes?: string;
};
export type SnapshotCaptureMetadata = {
version: 1;
captureId: string;
capturedAt: string;
path: string;
bytes: number;
targetKey: string;
targetId?: string | number;
mode?: string;
elementCount: number;
matchCount: number;
staleAfterInteraction: true;
};
export type StoredSnapshotCapture = {
metadata: SnapshotCaptureMetadata;
absolutePath: string;
};
export type SnapshotCaptureFile = {
version: 1;
captureId: string;
capturedAt: string;
targetKey: string;
snapshot: unknown;
};
function hashSegment(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
function normalizeRelativePath(path: string): string {
return path.split(sep).join("/");
}
function snapshotObject(snapshot: unknown): SnapshotObject | undefined {
return snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
? snapshot as SnapshotObject
: undefined;
}
export function snapshotTargetKey(target: SnapshotTarget = {}): string {
if (target.targetId) return `target:${target.targetId}`;
if (target.urlIncludes) return `url:${target.urlIncludes}`;
if (target.titleIncludes) return `title:${target.titleIncludes}`;
return "default";
}
export class SessionSnapshotStore {
readonly workspaceRoot: string;
readonly sessionRoot: string;
readonly sessionHash: string;
constructor(workspaceRoot: string, sessionId: string) {
this.workspaceRoot = resolve(workspaceRoot);
this.sessionHash = hashSegment(sessionId || "unknown-session");
this.sessionRoot = join(this.workspaceRoot, ".pi", "chrome-context", this.sessionHash);
}
async save(snapshot: unknown, target: SnapshotTarget = {}): Promise<StoredSnapshotCapture> {
const targetKey = snapshotTargetKey(target);
const targetDirectory = join(this.sessionRoot, hashSegment(targetKey));
const absolutePath = join(targetDirectory, "snapshot.json");
const temporaryPath = join(targetDirectory, `.snapshot.${process.pid}.${randomUUID()}.tmp`);
const captureId = randomUUID();
const capturedAt = new Date().toISOString();
const capture: SnapshotCaptureFile = {
version: 1,
captureId,
capturedAt,
targetKey,
snapshot,
};
const serialized = `${JSON.stringify(capture, null, 2)}\n`;
await mkdir(targetDirectory, { recursive: true, mode: 0o700 });
await chmod(this.sessionRoot, 0o700).catch(() => undefined);
await chmod(targetDirectory, 0o700).catch(() => undefined);
try {
await writeFile(temporaryPath, serialized, { encoding: "utf8", mode: 0o600, flag: "wx" });
await chmod(temporaryPath, 0o600).catch(() => undefined);
await rename(temporaryPath, absolutePath);
await chmod(absolutePath, 0o600).catch(() => undefined);
} catch (error) {
await rm(temporaryPath, { force: true }).catch(() => undefined);
throw error;
}
const object = snapshotObject(snapshot);
const elements = Array.isArray(object?.elements) ? object.elements : [];
const matches = Array.isArray(object?.matches) ? object.matches : [];
const targetId = typeof object?.targetId === "string" || typeof object?.targetId === "number"
? object.targetId
: undefined;
const mode = typeof object?.mode === "string" ? object.mode : undefined;
const path = normalizeRelativePath(relative(this.workspaceRoot, absolutePath));
return {
absolutePath,
metadata: {
version: 1,
captureId,
capturedAt,
path,
bytes: Buffer.byteLength(serialized),
targetKey,
targetId,
mode,
elementCount: elements.length,
matchCount: matches.length,
staleAfterInteraction: true,
},
};
}
async invalidate(target?: SnapshotTarget): Promise<void> {
if (!target) {
await this.clear();
return;
}
const targetDirectory = join(this.sessionRoot, hashSegment(snapshotTargetKey(target)));
await rm(targetDirectory, { recursive: true, force: true });
}
async clear(): Promise<void> {
await rm(this.sessionRoot, { recursive: true, force: true });
}
}
@@ -1,100 +0,0 @@
import assert from "node:assert/strict";
import { mkdtemp, readFile, stat } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { SessionSnapshotStore, snapshotTargetKey } from "../src/session-snapshot-store.ts";
import { CHROME_TOOL_NAMES, formatChromeFindMatches, formatSnapshotCapture } from "../extensions/chrome-profile-bridge/index.ts";
test("snapshotTargetKey prefers explicit target selectors", () => {
assert.equal(snapshotTargetKey(), "default");
assert.equal(snapshotTargetKey({ titleIncludes: "Inbox" }), "title:Inbox");
assert.equal(snapshotTargetKey({ urlIncludes: "example.com" }), "url:example.com");
assert.equal(snapshotTargetKey({ targetId: "42", urlIncludes: "ignored" }), "target:42");
});
test("SessionSnapshotStore atomically keeps only the latest capture per target", async () => {
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-snapshot-"));
const store = new SessionSnapshotStore(workspace, "session/private/id");
const first = await store.save({ mode: "interactive", targetId: 42, elements: [{ uid: "el-1" }], matches: [] });
const second = await store.save({ mode: "changes", targetId: 42, elements: [{ uid: "el-2" }], matches: [{ uid: "el-2" }] });
assert.equal(first.metadata.path, second.metadata.path);
assert.notEqual(first.metadata.captureId, second.metadata.captureId);
assert.equal(second.metadata.elementCount, 1);
assert.equal(second.metadata.matchCount, 1);
assert.equal(second.metadata.targetId, 42);
assert.ok(!second.metadata.path.includes("private"));
const stored = JSON.parse(await readFile(second.absolutePath, "utf8"));
assert.equal(stored.version, 1);
assert.equal(stored.captureId, second.metadata.captureId);
assert.equal(stored.snapshot.elements[0].uid, "el-2");
if (process.platform !== "win32") {
assert.equal((await stat(store.sessionRoot)).mode & 0o777, 0o700);
assert.equal((await stat(second.absolutePath)).mode & 0o777, 0o600);
}
await store.clear();
await assert.rejects(stat(second.absolutePath), { code: "ENOENT" });
});
test("target invalidation and session cleanup do not affect another session", async () => {
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-isolation-"));
const firstSession = new SessionSnapshotStore(workspace, "session-a");
const secondSession = new SessionSnapshotStore(workspace, "session-b");
const firstTarget = { targetId: "101" };
const secondTarget = { targetId: "202" };
const first = await firstSession.save({ elements: [] }, firstTarget);
const otherTarget = await firstSession.save({ elements: [] }, secondTarget);
const otherSession = await secondSession.save({ elements: [] }, firstTarget);
await firstSession.invalidate(firstTarget);
await assert.rejects(stat(first.absolutePath), { code: "ENOENT" });
await stat(otherTarget.absolutePath);
await stat(otherSession.absolutePath);
await firstSession.clear();
await assert.rejects(stat(otherTarget.absolutePath), { code: "ENOENT" });
await stat(otherSession.absolutePath);
await secondSession.clear();
});
test("snapshot result formatting stays bounded and points to Context Mode", () => {
const text = formatSnapshotCapture({
version: 1,
captureId: "capture-1",
capturedAt: "2026-08-28T00:00:00.000Z",
path: ".pi/chrome-context/session/target/snapshot.json",
bytes: 80_000,
targetKey: "target:42",
targetId: 42,
mode: "interactive",
elementCount: 80,
matchCount: 3,
staleAfterInteraction: true,
});
assert.match(text, /ctx_execute_file/);
assert.match(text, /\.pi\/chrome-context\/session\/target\/snapshot\.json/);
assert.doesNotMatch(text, /raw-secret-page-content/);
});
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
const matches = Array.from({ length: 25 }, (_, index) => ({
kind: "element",
uid: `el-${index}`,
role: "button",
label: `Button ${index}`,
rect: { x: index, y: index, width: 10, height: 10 },
}));
const text = formatChromeFindMatches({ query: "button", matches });
assert.match(text, /el-0/);
assert.match(text, /el-19/);
assert.doesNotMatch(text, /el-20/);
assert.equal(CHROME_TOOL_NAMES.length, 21);
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
});
+48
View File
@@ -0,0 +1,48 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
CHROME_TOOL_NAMES,
formatActionResultText,
formatChromeFindMatches,
snapshotStats,
} from "../extensions/chrome-profile-bridge/index.ts";
test("formatActionResultText appends the inline snapshot JSON", () => {
const snapshot = { elements: [{ uid: "el-1", role: "button", label: "Save" }], matches: [] };
const text = formatActionResultText("Clicked el-1", snapshot);
const [summary, json] = text.split("\n\n");
assert.equal(summary, "Clicked el-1");
assert.deepEqual(JSON.parse(json), snapshot);
});
test("formatActionResultText returns the summary alone without a snapshot", () => {
assert.equal(formatActionResultText("Clicked el-1", undefined), "Clicked el-1");
});
test("snapshotStats stays small: only element and match counts", () => {
assert.deepEqual(
snapshotStats({ elements: [{ uid: "a" }, { uid: "b" }], matches: [{ uid: "a" }] }),
{ elementCount: 2, matchCount: 1 },
);
assert.deepEqual(snapshotStats({ elements: "nope" }), { elementCount: 0, matchCount: 0 });
assert.deepEqual(snapshotStats(undefined), { elementCount: 0, matchCount: 0 });
assert.deepEqual(snapshotStats([1, 2, 3]), { elementCount: 0, matchCount: 0 });
});
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
const matches = Array.from({ length: 25 }, (_, index) => ({
kind: "element",
uid: `el-${index}`,
role: "button",
label: `Button ${index}`,
rect: { x: index, y: index, width: 10, height: 10 },
}));
const text = formatChromeFindMatches({ query: "button", matches });
assert.match(text, /el-0/);
assert.match(text, /el-19/);
assert.doesNotMatch(text, /el-20/);
assert.equal(CHROME_TOOL_NAMES.length, 21);
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
});
+13 -9
View File
@@ -1,19 +1,23 @@
# my-pi 维护约定(pi-compaction) # my-pi 维护约定(pi-compaction)
- 扩展入口是 `pi-compaction/index.ts`,增量索引、投影、后台摘要、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。 - 扩展入口是 `pi-compaction/index.ts`,遮蔽规划、投影索引、遮蔽台账、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。
- 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。 - 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。
- `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune` 或 `openai-codex-native-compaction` 状态;切换后应使用新 Session。配置不写回 `settings.json`,默认 soft/hard/target 比例为 72%/90%/55%。 - `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune`、`openai-codex-native-compaction` 或旧 `segment-summary` 状态;切换后应使用新 Session。配置不写回 `settings.json`。
- Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage;background worker 不得在 Provider 请求中被 await。 - Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。 - `pi-compaction` 负责已经进入 Session 的历史投影,与 pi-overflow 的新结果入口截断职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。 - L2 遮蔽只用 Pi 原生 `context_edit`(`{ type: "context_edit", targetId, replacement }`)处理保护窗口外的旧 toolResult:成批提交、受 trigger/target 回差与 `minClearTokens` 约束,不逐条即时替换;不为遮蔽写 `segment-summary` 等 custom entry,原始条目、UI 与导出保持不变。
- Segment custom entry 只能保存摘要、hash 和 source reference,不得复制完整原始工具输出。`context_tree_query` 恢复结果必须有界,native checkpoint 的 portable summary 也必须有界。 - 不得改写最近 `keepRecentTurns` 轮;不得遮蔽 user/assistant 消息、`excludeTools` 中的工具、`SKILL.md` 的 `read` 结果、已遮蔽或已省略的条目、以及小于 `minEntryTokens` 的结果。
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。Pi Core raw threshold 只负责唤醒,只有 effective lower bound 达到 `hardRatio` 才执行 threshold compaction;manual 与 overflow 不得被阻断。 - 遮蔽占位符必须确定性、单行、以 `MASK_PREFIX`(`"[pi-compaction] Earlier "`)开头,并在文本中指向 `context_tree_query` 恢复;`context_tree_query` 与 native checkpoint 的 portable summary 都必须有界。被遮蔽的 toolResult 若带 `details.overflow.id`(pi-overflow 已存原文),占位符末尾追加 `overflow_query` 提示——只读该公开字段,不得 import pi-overflow 内部实现。
- L3 整段摘要只能从空闲的 `agent_settled` 通过 `ctx.compact` 触发(`ctx.compact` 会中止当前运行,非空闲时调用会破坏运行);同一 leaf 不重复触发。阈值以绝对 token 计并受窗口比例约束(`effectiveThreshold = min(tokens, floor(ratio × contextWindow))`)。
- `context` 钩子不得改写消息——非 checkpoint 路径只更新 snapshot 并返回 undefined;唯一例外是 native checkpoint 路径:checkpoint 之后的条目在投影时必须应用各 target 最新的 `context_edit`(null 省略、字符串规范成 text block),因为 `sessionEntryToContextMessages` 不处理 edits。
- `session_before_compact` 只负责生成 portable/native 摘要,不得按自身阈值取消 Pi 的压缩(Pi 的 `contextTokens > window - reserveTokens` 是最终安全网)。
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。遮蔽台账在 `session_start` / `session_tree` / `session_compact` 从当前 branch 的 `context_edit` 条目重建,指标暴露 `maskedResults` 与 `reclaimedTokens`。
- Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。 - Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。
- 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认在 turn boundary 达到 effective 下界 90% 时触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。 - 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认由 L3 阈值或 Pi 自身阈值触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。
- SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。 - SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。
- 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。 - 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。
## 验证 ## 验证
- 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。 - 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。
- 修改 projection、token accounting、threshold、checkpoint、模型切换、后台并发或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。 - 修改 projection、token accounting、遮蔽规划/台账、threshold、checkpoint、模型切换或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。
+36 -31
View File
@@ -1,19 +1,22 @@
# pi-compaction # pi-compaction
High-performance unified context projection and compaction for Pi. Pressure-aware context masking and threshold compaction for Pi.
## Responsibilities ## Responsibilities
- incrementally index completed tool calls; Pi context management is layered; this extension owns the middle two layers:
- replace large tool results with deterministic bounded evidence stubs;
- pass small tool results through unchanged whenever a stub would not be shorter;
- refine large stubs with non-blocking semantic summaries;
- expose one effective context projection to normal provider requests and Codex/SubAPI native compaction;
- cancel Pi's raw threshold compaction until the effective lower-bound reaches the configured threshold;
- retain original tool evidence through `context_tree_query`;
- publish read-only effective metrics for the footer and context observer.
Context Mode remains responsible for preventing large external data from entering the session. RTK remains responsible for ordinary tool-output cleanup. - **L1 — entry truncation** (future `pi-overflow`): truncate new tool results as they are written. Not implemented here.
- **L2 — observation masking** (this extension): under real context pressure, replace old tool results outside a protected recent window with deterministic one-line placeholders via Pi `context_edit` entries. Originals stay in the session untouched and are recoverable with `context_tree_query`.
- **L3 — whole-context summary** (this extension): when the session idles (`agent_settled`) above the summary threshold, trigger `ctx.compact` once per leaf.
- **Pi native threshold compaction** is the final safety net and is never blocked; this extension no longer cancels `session_before_compact` based on its own thresholds.
Additionally it:
- publishes one effective-context snapshot for normal provider requests and Codex/SubAPI native compaction;
- maintains a masked-results ledger so `/context`, `/compaction-status`, the footer, and AutoReview see the same placeholders the model sees.
pi-overflow bounds new tool results as they arrive. RTK remains responsible for ordinary tool-output cleanup.
## Configuration ## Configuration
@@ -22,21 +25,21 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
```json ```json
{ {
"enabled": true, "enabled": true,
"softRatio": 0.72, "masking": {
"hardRatio": 0.9,
"targetRatio": 0.55,
"deterministic": {
"enabled": true, "enabled": true,
"maxStubTokens": 1024 "triggerTokens": 200000,
"triggerRatio": 0.6,
"targetTokens": 100000,
"targetRatio": 0.35,
"minClearTokens": 50000,
"keepRecentTurns": 10,
"minEntryTokens": 256,
"excludeTools": ["ask_user_question"]
}, },
"semantic": { "summary": {
"enabled": true, "triggerTokens": 400000,
"model": "default", "triggerRatio": 0.9,
"thinking": "low", "thinking": "low"
"concurrency": 1,
"minInputChars": 4000,
"maxInputChars": 24000,
"maxRetries": 2
}, },
"native": { "native": {
"enabled": true "enabled": true
@@ -44,7 +47,9 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
} }
``` ```
Invalid or missing fields are normalized in memory. All thresholds are absolute token counts constrained by a context-window ratio: `effectiveThreshold = min(tokens, floor(ratio × contextWindow))`. Masking fires when `currentTokens ≥ triggerEff` and clears the oldest eligible results until `currentTokens - cleared ≤ targetEff`; if the clearable volume stays under `min(minClearTokens, triggerEff - targetEff)` nothing is masked, so masking is rare, batched, and hysteretic. `masking.target` is normalized below `masking.trigger` and `summary.trigger` above `masking.trigger` when a config would violate that ordering. Invalid or missing fields fall back to defaults in memory.
Masking never touches the most recent `keepRecentTurns` assistant turns, user/assistant messages, `excludeTools`, `SKILL.md` reads, entries already masked, or results smaller than `minEntryTokens`.
## Native providers ## Native providers
@@ -67,17 +72,17 @@ The gateway must have a compatible deployed version, model and available upstrea
## Session format ## Session format
The only custom entry type is `pi-compaction`, version 1. Segment summary entries contain hashes, summaries, and source references; they never duplicate complete raw tool output. Native checkpoints use Pi compaction-entry details with the same kind/version. The extension writes no `segment-summary` or other custom entries for masking: L2 uses Pi's native `context_edit` entries (`{ type: "context_edit", targetId, replacement }`), which alter only future model context. The raw session entries — and therefore UI, exports, and the evidence ledger — are unchanged; every masked result remains recoverable through `context_tree_query`. Masking placeholders are deterministic single lines starting with `[pi-compaction] Earlier `.
Projection replaces a tool result only when the resulting summary plus its `Source:` recovery line is strictly shorter than the original text content. Small tool results are therefore passed through byte-for-byte and keep their original content parts, including any non-text parts. Because a stub is only larger than the text it would replace when the original is short, this comparison is the sole gate; no separate size option exists. Every indexed result remains recoverable through `context_tree_query` whether or not the message was replaced. Compaction entries still carry `pi-compaction` version 1 details; native checkpoints use the same kind/version. This implementation intentionally does not read legacy `context-prune-*`, `openai-codex-native-compaction`, or older `segment-summary` entries. Start a new session after switching to this extension.
This implementation intentionally does not read legacy `context-prune-*` or `openai-codex-native-compaction` entries. Start a new session after switching to this extension.
## Interfaces ## Interfaces
- `context_tree_query`: recover original bounded tool evidence by source key or tool call ID. - `context_tree_query`: recover the original output of a tool result that pi-compaction cleared from context, by tool call ID (1–20 ids per call, bounded output). Use it when a `[pi-compaction] Earlier` placeholder lacks evidence needed for the current task.
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, and pending semantic jobs. - pi-overflow linkage: when the masked tool result carries `details.overflow.id` (pi-overflow stored the oversized original), the placeholder ends with `Full original stored: overflow_query {"id":"…"}`. pi-compaction only reads that public field and never imports pi-overflow internals.
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, masked results, and generation.
- Service `my-pi.pi-compaction.service.v1`: `getSnapshot`, `getMetrics`, and `query(sessionId, ids)` — `query` returns a placeholder record only for tool call IDs that are currently masked, so consumers such as the permission system's AutoReview see the same evidence the model does.
## Performance model ## Performance model
The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots. The `context` hook performs no model call and no session write. Semantic summaries run in a bounded background worker and become visible only at later safe event boundaries. The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots; the `context` hook performs no model call, no session write, and returns the message list unchanged except on the native-checkpoint path, where post-checkpoint `context_edit` entries are applied during projection. Masking drafts are produced in `turn_end` in one batch and only when both the trigger and the `minClear` hysteresis pass, so the hot path stays allocation-free for unprompted turns.
+5
View File
@@ -29,3 +29,8 @@
- 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。 - 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。
- 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。 - 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。
- 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。 - 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。
## 本地改动记录
- 上下文管理重构为 L2/L3:L2 在上下文压力下经 `turn_end` 提交 Pi 原生 `context_edit`,把保护窗口外的旧 toolResult 批量遮蔽为指向 `context_tree_query` 的确定性占位符(原始条目保留、不写 custom entry);L3 只在空闲 `agent_settled` 且达到绝对 token 阈值(受窗口比例约束)时调用一次 `ctx.compact`。删除逐条确定性存根、后台语义 worker、`softRatio`/`hardRatio`/`targetRatio` 配置与 `session_before_compact` 的自行阈值取消;Pi 原生阈值压缩保留为最终安全网。
- L2 遮蔽占位符联动 pi-overflow:被遮蔽的 toolResult 若携带 `details.overflow.id`,占位符末尾追加 `overflow_query {"id":"…"}` 恢复提示。
+174 -76
View File
@@ -14,6 +14,12 @@ import {
type SessionEntry, type SessionEntry,
} from "@earendil-works/pi-coding-agent"; } from "@earendil-works/pi-coding-agent";
import { DEFAULT_CONFIG, loadConfig } from "./src/config.js"; import { DEFAULT_CONFIG, loadConfig } from "./src/config.js";
import {
applyContextEdits,
planMasking,
rebuildMaskedLedger,
type MaskedLedgerEntry,
} from "./src/masking.js";
import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js"; import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js";
import { import {
buildCompactionRequestBody, buildCompactionRequestBody,
@@ -32,27 +38,28 @@ import {
type NativeCompactionDetails, type NativeCompactionDetails,
type ResponseItem, type ResponseItem,
} from "./src/native-compaction.js"; } from "./src/native-compaction.js";
import { SegmentIndex } from "./src/segment-index.js"; import { contentText, SegmentIndex } from "./src/segment-index.js";
import { SemanticWorker } from "./src/semantic-worker.js";
import { shouldCompact } from "./src/token-accounting.js";
import { import {
CompactionServiceRegistry, CompactionServiceRegistry,
publishCompactionService, publishCompactionService,
unpublishCompactionService, unpublishCompactionService,
} from "./src/service.js"; } from "./src/service.js";
import { effectiveThreshold } from "./src/token-accounting.js";
import { import {
COMPACTION_CUSTOM_TYPE, COMPACTION_CUSTOM_TYPE,
COMPACTION_VERSION, COMPACTION_VERSION,
QUERY_TOOL_NAME, QUERY_TOOL_NAME,
type MaskedResultRecord,
type PiCompactionConfig, type PiCompactionConfig,
type ProjectionSnapshot, type ProjectionSnapshot,
type ToolHistoryRecord,
} from "./src/types.js"; } from "./src/types.js";
interface SessionRuntime { interface SessionRuntime {
index: SegmentIndex; index: SegmentIndex;
config: PiCompactionConfig; config: PiCompactionConfig;
codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] }; codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] };
masked: Map<string, MaskedLedgerEntry>;
lastAutoCompactLeafId: string | null;
} }
interface CachedPayload { interface CachedPayload {
@@ -64,10 +71,6 @@ function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error); return error instanceof Error ? error.message : String(error);
} }
function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
interface EnvironmentValue { interface EnvironmentValue {
hash: string; hash: string;
tokens: number; tokens: number;
@@ -81,6 +84,10 @@ interface EnvironmentCacheEntry extends EnvironmentValue {
tools: unknown[]; tools: unknown[];
} }
function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20);
}
function environment( function environment(
pi: ExtensionAPI, pi: ExtensionAPI,
ctx: ExtensionContext, ctx: ExtensionContext,
@@ -182,34 +189,46 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const runtimes = new Map<string, SessionRuntime>(); const runtimes = new Map<string, SessionRuntime>();
const payloads = new Map<string, CachedPayload>(); const payloads = new Map<string, CachedPayload>();
const environments = new Map<string, EnvironmentCacheEntry>(); const environments = new Map<string, EnvironmentCacheEntry>();
const worker = new SemanticWorker();
const service = new CompactionServiceRegistry(); const service = new CompactionServiceRegistry();
publishCompactionService(service); publishCompactionService(service);
const maskedRecords = (sessionId: string, ids: string[]): MaskedResultRecord[] => {
const masked = runtimes.get(sessionId)?.masked;
if (!masked) return [];
return ids.flatMap((id) => {
const entry = masked.get(id);
return entry
? [{
toolCallId: id,
resultTimestamp: entry.resultTimestamp,
deterministicSummary: entry.placeholder,
}]
: [];
});
};
const runtimeFor = (ctx: ExtensionContext): SessionRuntime => { const runtimeFor = (ctx: ExtensionContext): SessionRuntime => {
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
let runtime = runtimes.get(sessionId); let runtime = runtimes.get(sessionId);
if (!runtime) { if (!runtime) {
const index = new SegmentIndex(sessionId); runtime = {
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], DEFAULT_CONFIG); index: new SegmentIndex(sessionId),
runtime = { index, config: structuredClone(DEFAULT_CONFIG) }; config: structuredClone(DEFAULT_CONFIG),
masked: rebuildLedger(ctx),
lastAutoCompactLeafId: null,
};
runtimes.set(sessionId, runtime); runtimes.set(sessionId, runtime);
service.setSession(sessionId, (ids) => index.query(ids)); service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
} }
return runtime; return runtime;
}; };
const persist = (record: unknown): void => { const ledgerStats = (runtime: SessionRuntime): { maskedResults: number; reclaimedTokens: number } => {
pi.appendEntry(COMPACTION_CUSTOM_TYPE, record); let reclaimedTokens = 0;
}; for (const entry of runtime.masked.values()) {
reclaimedTokens += Math.max(0, entry.originalTokens - entry.placeholderTokens);
const drainSemantic = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
const sessionId = ctx.sessionManager.getSessionId();
for (const result of worker.drain(sessionId)) {
const record = runtime.index.applySemantic(result.sourceKey, result.sourceHash, result.summary);
if (record) persist(record);
} }
runtime.index.setSemanticJobs(worker.count(sessionId)); return { maskedResults: runtime.masked.size, reclaimedTokens };
}; };
const projectMessages = ( const projectMessages = (
@@ -218,16 +237,32 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
messages: readonly any[], messages: readonly any[],
): ProjectionSnapshot => { ): ProjectionSnapshot => {
const env = environment(pi, ctx, environments); const env = environment(pi, ctx, environments);
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId())); const stats = ledgerStats(runtime);
const snapshot = runtime.index.project(messages, { const snapshot = runtime.index.project(messages, {
environmentHash: env.hash, environmentHash: env.hash,
environmentTokens: env.tokens, environmentTokens: env.tokens,
contextWindow: env.contextWindow, contextWindow: env.contextWindow,
maskedResults: stats.maskedResults,
reclaimedTokens: stats.reclaimedTokens,
}); });
service.setSnapshot(snapshot); service.setSnapshot(snapshot);
return snapshot; return snapshot;
}; };
const rebuildLedger = (ctx: ExtensionContext): Map<string, MaskedLedgerEntry> =>
rebuildMaskedLedger(
ctx.sessionManager.getBranch(),
ctx.sessionManager.getLeafId(),
(message) => estimateTokens(message as any),
);
const hydrateRuntime = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
runtime.index.hydrate();
runtime.masked = rebuildLedger(ctx);
runtime.lastAutoCompactLeafId = null;
runtime.codexCache = undefined;
};
const branchMessages = ( const branchMessages = (
ctx: ExtensionContext, ctx: ExtensionContext,
runtime: SessionRuntime, runtime: SessionRuntime,
@@ -237,9 +272,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const checkpoint = findNativeCheckpoint(branch); const checkpoint = findNativeCheckpoint(branch);
let messages: any[]; let messages: any[];
if (checkpoint.status === "valid") { if (checkpoint.status === "valid") {
messages = branch // sessionEntryToContextMessages sees only raw entries; applyContextEdits
.slice(checkpoint.checkpoint.entryIndex + 1) // additionally replays the latest context_edit per target (omissions and
.flatMap((entry) => sessionEntryToContextMessages(entry)); // placeholder replacements included).
messages = applyContextEdits(branch.slice(checkpoint.checkpoint.entryIndex + 1));
if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) { if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) {
messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages]; messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages];
} }
@@ -278,15 +314,22 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
publishCompactionService(service); publishCompactionService(service);
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
const config = await loadConfig(); const config = await loadConfig();
const index = new SegmentIndex(sessionId); const runtime = runtimes.get(sessionId) ?? {
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], config); index: new SegmentIndex(sessionId),
runtimes.set(sessionId, { index, config }); config,
service.setSession(sessionId, (ids) => index.query(ids)); masked: new Map<string, MaskedLedgerEntry>(),
lastAutoCompactLeafId: null,
};
runtime.config = config;
runtime.index.hydrate();
runtime.masked = rebuildLedger(ctx);
runtime.lastAutoCompactLeafId = null;
runtimes.set(sessionId, runtime);
service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
}); });
pi.on("session_shutdown", (_event, ctx) => { pi.on("session_shutdown", (_event, ctx) => {
const sessionId = ctx.sessionManager.getSessionId(); const sessionId = ctx.sessionManager.getSessionId();
worker.cancelSession(sessionId);
runtimes.delete(sessionId); runtimes.delete(sessionId);
payloads.delete(sessionId); payloads.delete(sessionId);
environments.delete(sessionId); environments.delete(sessionId);
@@ -296,15 +339,13 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("session_tree", (_event, ctx) => { pi.on("session_tree", (_event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config); hydrateRuntime(ctx, runtime);
runtime.codexCache = undefined;
service.clearSnapshot(ctx.sessionManager.getSessionId()); service.clearSnapshot(ctx.sessionManager.getSessionId());
}); });
pi.on("session_compact", (_event, ctx) => { pi.on("session_compact", (_event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config); hydrateRuntime(ctx, runtime);
runtime.codexCache = undefined;
service.clearSnapshot(ctx.sessionManager.getSessionId()); service.clearSnapshot(ctx.sessionManager.getSessionId());
payloads.delete(ctx.sessionManager.getSessionId()); payloads.delete(ctx.sessionManager.getSessionId());
}); });
@@ -319,21 +360,9 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("turn_end", (event, ctx) => { pi.on("turn_end", (event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
drainSemantic(ctx, runtime); if (!runtime.config.enabled) return undefined;
const previous = runtime.index.getSnapshot();
const captured = runtime.index.captureTurn(event.message, event.toolResults, runtime.config);
for (const record of captured) persist(runtime.index.toPersisted(record));
const pressure = previous && previous.contextWindow > 0
? previous.upperBound / previous.contextWindow >= runtime.config.softRatio
: false;
const semanticCandidates = pressure
? runtime.index.allRecords()
: captured.filter((record) => record.rawText.length >= runtime.config.semantic.maxInputChars);
for (const record of semanticCandidates) {
worker.enqueue(ctx.sessionManager.getSessionId(), record, ctx, runtime.config);
}
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId()));
const previous = runtime.index.getSnapshot();
const usage = ctx.getContextUsage(); const usage = ctx.getContextUsage();
if (usage?.tokens && previous) { if (usage?.tokens && previous) {
const env = environment(pi, ctx, environments); const env = environment(pi, ctx, environments);
@@ -343,6 +372,50 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
env.hash, env.hash,
); );
} }
const currentTokens = usage?.tokens
?? (event.context?.contextMessages ?? []).reduce((sum, message) => sum + estimateTokens(message as any), 0)
+ environment(pi, ctx, environments).tokens;
const plan = planMasking({
contextEntries: event.context?.contextEntries ?? [],
currentTokens,
contextWindow: ctx.model?.contextWindow ?? 0,
masking: runtime.config.masking,
estimate: (message) => estimateTokens(message as any),
});
if (!plan.drafts.length) return undefined;
for (const result of plan.masked) {
runtime.masked.set(result.toolCallId, {
placeholder: result.placeholder,
originalTokens: result.originalTokens,
placeholderTokens: result.placeholderTokens,
resultTimestamp: result.resultTimestamp,
});
}
return { entries: [...(event.entries ?? []), ...plan.drafts] };
});
pi.on("agent_settled", (_event, ctx) => {
const runtime = runtimeFor(ctx);
if (!runtime.config.enabled || ctx.isIdle?.() === false) return;
const snapshot = runtime.index.getSnapshot();
const tokens = ctx.getContextUsage()?.tokens ?? snapshot?.effectiveTokens;
if (tokens == null) return;
const contextWindow = ctx.model?.contextWindow ?? snapshot?.contextWindow ?? 0;
const threshold = effectiveThreshold(
runtime.config.summary.triggerTokens,
runtime.config.summary.triggerRatio,
contextWindow,
);
if (tokens < threshold) return;
const leafId = ctx.sessionManager.getLeafId();
if (leafId !== null && runtime.lastAutoCompactLeafId === leafId) return;
runtime.lastAutoCompactLeafId = leafId;
ctx.compact({
onError: (error) => {
if (ctx.hasUI) ctx.ui.notify(`pi-compaction auto summary failed: ${errorMessage(error)}`, "warning");
},
});
}); });
pi.on("context", (event, ctx) => { pi.on("context", (event, ctx) => {
@@ -358,7 +431,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const projected = branchMessages(ctx, runtime, branch); const projected = branchMessages(ctx, runtime, branch);
return { messages: projected.snapshot.messages as any }; return { messages: projected.snapshot.messages as any };
} }
return { messages: projectMessages(ctx, runtime, event.messages).messages as any }; // Non-checkpoint path only refreshes the shared snapshot; the message list
// is returned to Pi unchanged — masking is append-only via context_edit.
projectMessages(ctx, runtime, event.messages);
return undefined;
}); });
pi.on("before_provider_headers", (event, ctx) => { pi.on("before_provider_headers", (event, ctx) => {
@@ -387,7 +463,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.on("session_before_compact", async (event, ctx) => { pi.on("session_before_compact", async (event, ctx) => {
const runtime = runtimeFor(ctx); const runtime = runtimeFor(ctx);
if (!runtime.config.enabled) return undefined; if (!runtime.config.enabled) return undefined;
drainSemantic(ctx, runtime);
const { snapshot, checkpoint } = branchMessages( const { snapshot, checkpoint } = branchMessages(
ctx, ctx,
runtime, runtime,
@@ -395,8 +470,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.reason === "overflow" && event.willRetry, event.reason === "overflow" && event.willRetry,
); );
if (!shouldCompact(event.reason, snapshot, runtime.config)) return { cancel: true };
const model = ctx.model; const model = ctx.model;
if (!model) return undefined; if (!model) return undefined;
if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) { if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) {
@@ -412,7 +485,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.signal, event.signal,
event.customInstructions, event.customInstructions,
undefined, undefined,
runtime.config.semantic.thinking, runtime.config.summary.thinking,
); );
return { return {
compaction: { compaction: {
@@ -463,7 +536,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
replacementHistory: buildReplacementHistory( replacementHistory: buildReplacementHistory(
input, input,
remote.compactionItem, remote.compactionItem,
Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * Math.max(0.05, runtime.config.targetRatio - 0.3)))), Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * 0.25))),
), ),
portableSummary: portableSummary(snapshot.messages), portableSummary: portableSummary(snapshot.messages),
sourceFingerprint: snapshot.fingerprint, sourceFingerprint: snapshot.fingerprint,
@@ -504,7 +577,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
event.signal, event.signal,
event.preparation.customInstructions, event.preparation.customInstructions,
undefined, undefined,
runtime.config.semantic.thinking, runtime.config.summary.thinking,
undefined, undefined,
undefined, undefined,
undefined, undefined,
@@ -531,38 +604,64 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
pi.registerTool({ pi.registerTool({
name: QUERY_TOOL_NAME, name: QUERY_TOOL_NAME,
label: "Query Compacted Tool History", label: "Query Compacted Tool History",
description: "Recover original tool outputs by source key or tool call id after pi-compaction replaced them with bounded summaries.", description: "Recover the original output of a tool result that pi-compaction cleared from context, by tool call id.",
promptSnippet: "Recover original compacted tool evidence", promptSnippet: "Recover original tool output cleared from context",
promptGuidelines: [ promptGuidelines: [
"Use context_tree_query only when a pi-compaction summary lacks evidence needed for the current task.", "Use context_tree_query when you see a \"[pi-compaction] Earlier\" placeholder and still need the original tool output.",
], ],
parameters: Type.Object({ parameters: Type.Object({
toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }), toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }),
}), }),
async execute(_id, params, _signal, _update, ctx) { async execute(_id, params, _signal, _update, ctx) {
const records = service.query(ctx.sessionManager.getSessionId(), params.toolCallIds); const branch = ctx.sessionManager.getBranch() as SessionEntry[];
const blocks = records.map((record: ToolHistoryRecord) => { const calls = new Map<string, { toolName: string; args: unknown }>();
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, records.length))); const results = new Map<string, any>();
const args = truncateHead(JSON.stringify(record.args), { maxLines: 20, maxBytes: 2_048 }).content; for (const entry of branch) {
const truncated = truncateHead(record.rawText, { const message = (entry as any)?.message;
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, records.length))), if ((entry as any)?.type !== "message" || !message) continue;
if (message.role === "assistant" && Array.isArray(message.content)) {
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
calls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.arguments ?? block.input ?? block.args ?? {},
});
}
}
} else if (message.role === "toolResult" && typeof message.toolCallId === "string") {
results.set(message.toolCallId, message);
}
}
const found: string[] = [];
const blocks: string[] = [];
for (const id of params.toolCallIds) {
const result = results.get(id);
if (!result) continue;
found.push(id);
const call = calls.get(id) ?? { toolName: String(result.toolName ?? "unknown"), args: {} };
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, params.toolCallIds.length)));
const args = truncateHead(JSON.stringify(call.args ?? {}), { maxLines: 20, maxBytes: 2_048 }).content;
const rawText = contentText(result.content);
const truncated = truncateHead(rawText, {
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, params.toolCallIds.length))),
maxBytes: perResultBytes, maxBytes: perResultBytes,
}); });
return [ blocks.push([
`## ${record.sourceKey}`, `## ${id}`,
`Tool: ${record.toolName}`, `Tool: ${call.toolName}`,
`Args: ${args}`, `Args: ${args}`,
`Status: ${record.isError ? "ERROR" : "OK"}`, `Status: ${result.isError ? "ERROR" : "OK"}`,
"", "",
truncated.content, truncated.content,
...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []), ...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []),
].join("\n"); ].join("\n"));
}); }
const missing = params.toolCallIds.filter((id: string) => !records.some((record) => record.sourceKey === id || record.toolCallId === id)); const missing = params.toolCallIds.filter((id: string) => !results.has(id));
for (const id of missing) blocks.push(`## ${id}\n(not found)`); for (const id of missing) blocks.push(`## ${id}\n(not found)`);
return { return {
content: [{ type: "text", text: blocks.join("\n\n---\n\n") }], content: [{ type: "text", text: blocks.join("\n\n---\n\n") }],
details: { sourceKeys: records.map((record) => record.sourceKey), missing }, details: { toolCallIds: found, missing },
}; };
}, },
}); });
@@ -576,12 +675,11 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
`Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`, `Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`,
`Raw: ${metrics.rawTokens}/${metrics.contextWindow}`, `Raw: ${metrics.rawTokens}/${metrics.contextWindow}`,
`Reclaimed: ${metrics.reclaimedTokens}`, `Reclaimed: ${metrics.reclaimedTokens}`,
`Segments: ${metrics.segmentCount}; semantic jobs: ${metrics.semanticJobs}; generation: ${metrics.generation}`, `Segments: ${metrics.segmentCount}; masked tool results: ${metrics.maskedResults}; generation: ${metrics.generation}`,
].join("\n") ].join("\n")
: "pi-compaction has no projection snapshot yet."; : "pi-compaction has no projection snapshot yet.";
if (ctx.hasUI) ctx.ui.notify(text, "info"); if (ctx.hasUI) ctx.ui.notify(text, "info");
else process.stderr.write(`${text}\n`); else process.stderr.write(`${text}\n`);
}, },
}); });
} }
+58 -35
View File
@@ -12,27 +12,29 @@ export const CONFIG_PATH = fileURLToPath(
export const DEFAULT_CONFIG: PiCompactionConfig = { export const DEFAULT_CONFIG: PiCompactionConfig = {
enabled: true, enabled: true,
softRatio: 0.72, masking: {
hardRatio: 0.9,
targetRatio: 0.55,
deterministic: {
enabled: true, enabled: true,
maxStubTokens: 1_024, triggerTokens: 200_000,
triggerRatio: 0.6,
targetTokens: 100_000,
targetRatio: 0.35,
minClearTokens: 50_000,
keepRecentTurns: 10,
minEntryTokens: 256,
excludeTools: ["ask_user_question"],
}, },
semantic: { summary: {
enabled: true, triggerTokens: 400_000,
model: "default", triggerRatio: 0.9,
thinking: "low", thinking: "low",
concurrency: 1,
minInputChars: 4_000,
maxInputChars: 24_000,
maxRetries: 2,
}, },
native: { native: {
enabled: true, enabled: true,
}, },
}; };
const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh"];
function ratio(value: unknown, fallback: number): number { function ratio(value: unknown, fallback: number): number {
return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback; return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback;
} }
@@ -41,46 +43,67 @@ function integer(value: unknown, fallback: number, min = 0, max = Number.MAX_SAF
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback; return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback;
} }
function positiveInteger(value: unknown, fallback: number): number {
return integer(value, fallback, 1);
}
function object(value: unknown): Record<string, unknown> { function object(value: unknown): Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value) return value !== null && typeof value === "object" && !Array.isArray(value)
? value as Record<string, unknown> ? value as Record<string, unknown>
: {}; : {};
} }
function stringArray(value: unknown, fallback: string[]): string[] {
if (!Array.isArray(value) || !value.every((entry) => typeof entry === "string")) return fallback;
return value.map((entry) => entry.trim()).filter((entry) => entry.length > 0);
}
export function normalizeConfig(value: unknown): PiCompactionConfig { export function normalizeConfig(value: unknown): PiCompactionConfig {
const input = object(value); const input = object(value);
const deterministic = object(input.deterministic); const masking = object(input.masking);
const semantic = object(input.semantic); const summary = object(input.summary);
const native = object(input.native); const native = object(input.native);
const softRatio = ratio(input.softRatio, DEFAULT_CONFIG.softRatio);
const hardRatio = Math.max(softRatio + 0.01, ratio(input.hardRatio, DEFAULT_CONFIG.hardRatio));
const targetRatio = Math.min(softRatio - 0.01, ratio(input.targetRatio, DEFAULT_CONFIG.targetRatio));
const thinking = ["off", "minimal", "low", "medium", "high", "xhigh"].includes(String(semantic.thinking))
? semantic.thinking as PiCompactionConfig["semantic"]["thinking"]
: DEFAULT_CONFIG.semantic.thinking;
return { const normalized: PiCompactionConfig = {
enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled, enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled,
softRatio, masking: {
hardRatio: Math.min(0.99, hardRatio), enabled: typeof masking.enabled === "boolean" ? masking.enabled : DEFAULT_CONFIG.masking.enabled,
targetRatio: Math.max(0.1, targetRatio), triggerTokens: positiveInteger(masking.triggerTokens, DEFAULT_CONFIG.masking.triggerTokens),
deterministic: { triggerRatio: ratio(masking.triggerRatio, DEFAULT_CONFIG.masking.triggerRatio),
enabled: typeof deterministic.enabled === "boolean" ? deterministic.enabled : DEFAULT_CONFIG.deterministic.enabled, targetTokens: positiveInteger(masking.targetTokens, DEFAULT_CONFIG.masking.targetTokens),
maxStubTokens: integer(deterministic.maxStubTokens, DEFAULT_CONFIG.deterministic.maxStubTokens, 32, 4_096), targetRatio: ratio(masking.targetRatio, DEFAULT_CONFIG.masking.targetRatio),
minClearTokens: positiveInteger(masking.minClearTokens, DEFAULT_CONFIG.masking.minClearTokens),
keepRecentTurns: integer(masking.keepRecentTurns, DEFAULT_CONFIG.masking.keepRecentTurns, 1, 100),
minEntryTokens: integer(masking.minEntryTokens, DEFAULT_CONFIG.masking.minEntryTokens, 0, 100_000),
excludeTools: stringArray(masking.excludeTools, DEFAULT_CONFIG.masking.excludeTools),
}, },
semantic: { summary: {
enabled: typeof semantic.enabled === "boolean" ? semantic.enabled : DEFAULT_CONFIG.semantic.enabled, triggerTokens: positiveInteger(summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens),
model: typeof semantic.model === "string" && semantic.model.trim() ? semantic.model.trim() : DEFAULT_CONFIG.semantic.model, triggerRatio: ratio(summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio),
thinking, thinking: THINKING_LEVELS.includes(String(summary.thinking))
concurrency: integer(semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency, 1, 4), ? summary.thinking as PiCompactionConfig["summary"]["thinking"]
minInputChars: integer(semantic.minInputChars, DEFAULT_CONFIG.semantic.minInputChars, 0, 1_000_000), : DEFAULT_CONFIG.summary.thinking,
maxInputChars: integer(semantic.maxInputChars, DEFAULT_CONFIG.semantic.maxInputChars, 1_000, 1_000_000),
maxRetries: integer(semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries, 0, 5),
}, },
native: { native: {
enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled, enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled,
}, },
}; };
// Trigger/target must keep their hysteresis ordering; fix up rather than
// letting a degenerate pair mask-or-compact every turn.
if (normalized.masking.targetTokens >= normalized.masking.triggerTokens) {
normalized.masking.targetTokens = Math.floor(normalized.masking.triggerTokens / 2);
}
if (normalized.masking.targetRatio >= normalized.masking.triggerRatio) {
normalized.masking.targetRatio = normalized.masking.triggerRatio / 2;
}
if (normalized.summary.triggerTokens <= normalized.masking.triggerTokens) {
normalized.summary.triggerTokens = normalized.masking.triggerTokens * 2;
}
if (normalized.summary.triggerRatio <= normalized.masking.triggerRatio) {
normalized.summary.triggerRatio = Math.min(0.99, normalized.masking.triggerRatio + 0.01);
}
return normalized;
} }
export async function loadConfig( export async function loadConfig(
+257
View File
@@ -0,0 +1,257 @@
import {
buildContextEntries,
sessionEntryToContextMessages,
type ContextEditEntryDraft,
} from "@earendil-works/pi-coding-agent";
import { contentText, timestampOf } from "./segment-index.js";
import { effectiveThreshold } from "./token-accounting.js";
import type { MaskingConfig } from "./types.js";
// Every placeholder written through a context_edit starts with this prefix so
// re-masking, ledger rebuilds, and the model-facing recovery hint all key off
// one marker.
export const MASK_PREFIX = "[pi-compaction] Earlier ";
const MAX_PLACEHOLDER_ARGS_CHARS = 300;
const EDITABLE_ROLES = new Set(["user", "assistant", "toolResult", "custom"]);
export interface MaskingProjectedEntry {
sourceEntry: { id?: string; type?: string; message?: any };
messages: any[];
}
export interface MaskedResult {
toolCallId: string;
entryId: string;
placeholder: string;
originalTokens: number;
placeholderTokens: number;
resultTimestamp: number;
}
export interface MaskingPlan {
drafts: ContextEditEntryDraft[];
clearedTokens: number;
masked: MaskedResult[];
}
export interface MaskedLedgerEntry {
placeholder: string;
originalTokens: number;
placeholderTokens: number;
resultTimestamp: number;
}
const EMPTY_PLAN: MaskingPlan = { drafts: [], clearedTokens: 0, masked: [] };
function json(value: unknown): string {
try {
return JSON.stringify(value) ?? "undefined";
} catch {
return "[unserializable]";
}
}
function maskPlaceholder(
toolName: string,
args: unknown,
toolCallId: string,
originalTokens: number,
overflowId?: string,
): string {
let argsText = json(args);
if (argsText.length > MAX_PLACEHOLDER_ARGS_CHARS) {
argsText = `${argsText.slice(0, MAX_PLACEHOLDER_ARGS_CHARS)}…`;
}
// `details.overflow.id` is pi-overflow's public contract — read the field
// only, never import its implementation.
const overflowHint = typeof overflowId === "string" && overflowId.length > 0
? ` Full original stored: overflow_query {"id":"${overflowId}"}.`
: "";
return `${MASK_PREFIX}${toolName} result cleared to save context (~${originalTokens} tokens). ` +
`Call args: ${argsText}. Re-run the call if still needed, or recover the original with ` +
`context_tree_query {"toolCallIds":["${toolCallId}"]}.${overflowHint}`;
}
function isSkillRead(toolName: string, args: unknown): boolean {
if (toolName !== "read" || args === null || typeof args !== "object") return false;
const record = args as Record<string, unknown>;
const raw = record.path ?? record.filePath ?? record.file;
if (typeof raw !== "string") return false;
return (raw.split(/[\\/]/).pop() ?? raw) === "SKILL.md";
}
function toolResultCalls(entries: readonly MaskingProjectedEntry[]): Map<string, { toolName: string; args: unknown }> {
const calls = new Map<string, { toolName: string; args: unknown }>();
for (const entry of entries) {
for (const message of entry.messages ?? []) {
if (message?.role !== "assistant" || !Array.isArray(message.content)) continue;
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
calls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.arguments ?? block.input ?? block.args ?? {},
});
}
}
}
}
return calls;
}
export function planMasking(params: {
contextEntries: readonly MaskingProjectedEntry[];
currentTokens: number;
contextWindow: number;
masking: MaskingConfig;
estimate: (message: any) => number;
}): MaskingPlan {
const { masking } = params;
const triggerEff = effectiveThreshold(masking.triggerTokens, masking.triggerRatio, params.contextWindow);
if (!masking.enabled || params.currentTokens < triggerEff) return EMPTY_PLAN;
const targetEff = effectiveThreshold(masking.targetTokens, masking.targetRatio, params.contextWindow);
const minClear = Math.min(masking.minClearTokens, Math.max(0, triggerEff - targetEff));
const entries = params.contextEntries;
// The last keepRecentTurns assistant messages and everything after them stay
// untouched; when fewer turns exist, the whole context is protected.
let protectedIndex = 0;
let turns = 0;
for (let index = entries.length - 1; index >= 0; index--) {
const entry = entries[index];
if (entry?.sourceEntry?.type === "message" && (entry.messages ?? []).some((m) => m?.role === "assistant")) {
turns++;
if (turns >= masking.keepRecentTurns) {
protectedIndex = index;
break;
}
}
}
const calls = toolResultCalls(entries);
const needed = params.currentTokens - targetEff;
const masked: MaskedResult[] = [];
let clearedTokens = 0;
for (let index = 0; index < protectedIndex && clearedTokens < needed; index++) {
const entry = entries[index];
const entryId = entry?.sourceEntry?.id;
if (entry?.sourceEntry?.type !== "message" || typeof entryId !== "string") continue;
for (const message of entry.messages ?? []) {
if (clearedTokens >= needed) break;
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
const call = calls.get(message.toolCallId);
const toolName = call?.toolName ?? String(message.toolName ?? "unknown");
if (masking.excludeTools.includes(toolName)) continue;
const args = call?.args ?? {};
if (isSkillRead(toolName, args)) continue;
if (contentText(message.content).startsWith(MASK_PREFIX)) continue;
const originalTokens = params.estimate(message);
if (originalTokens < masking.minEntryTokens) continue;
const overflowId = (message.details as { overflow?: { id?: unknown } } | undefined)?.overflow?.id;
const placeholder = maskPlaceholder(
toolName,
args,
message.toolCallId,
originalTokens,
typeof overflowId === "string" ? overflowId : undefined,
);
const placeholderTokens = params.estimate({ ...message, content: [{ type: "text", text: placeholder }] });
clearedTokens += originalTokens - placeholderTokens;
masked.push({
toolCallId: message.toolCallId,
entryId,
placeholder,
originalTokens,
placeholderTokens,
resultTimestamp: timestampOf(message.timestamp),
});
}
}
if (clearedTokens < minClear) return EMPTY_PLAN;
return {
drafts: masked.map((result) => ({
type: "context_edit",
targetId: result.entryId,
replacement: { content: result.placeholder },
})),
clearedTokens,
masked,
};
}
/**
* Rebuild the masked-results ledger from a session branch after hydration:
* the latest context_edit per target wins, only edits carrying this
* extension's placeholder prefix count as masked, and only targets that are
* still part of the active context (post-compaction kept set) are counted.
*/
export function rebuildMaskedLedger(
branch: readonly any[],
leafId: string | null | undefined,
estimate: (message: any) => number,
): Map<string, MaskedLedgerEntry> {
const latestEdits = new Map<string, any>();
const targets = new Map<string, any>();
for (const entry of branch) {
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
latestEdits.set(entry.targetId, entry);
} else if (entry?.type === "message" && typeof entry.id === "string") {
targets.set(entry.id, entry);
}
}
const contextIds = new Set(
buildContextEntries(branch as any, leafId).map((entry) => entry.id),
);
const masked = new Map<string, MaskedLedgerEntry>();
for (const [targetId, edit] of latestEdits) {
if (!contextIds.has(targetId)) continue;
const text = contentText(edit.replacement?.content);
if (!text.startsWith(MASK_PREFIX)) continue;
const message = targets.get(targetId)?.message;
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
masked.set(message.toolCallId, {
placeholder: text,
originalTokens: estimate(message),
placeholderTokens: estimate({ ...message, content: [{ type: "text", text }] }),
resultTimestamp: timestampOf(message.timestamp),
});
}
return masked;
}
/**
* Apply the latest context_edit per target the way buildSessionProjection does
* for message entries `sessionEntryToContextMessages` alone would miss: null
* omits the entry, a string replacement becomes one text block for
* assistant/toolResult roles.
*/
export function applyContextEdits(entries: readonly any[]): any[] {
const edits = new Map<string, { content: unknown } | null>();
for (const entry of entries) {
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
edits.set(entry.targetId, entry.replacement ?? null);
}
}
const projected: any[] = [];
for (const entry of entries) {
const messages = sessionEntryToContextMessages(entry);
const edit = edits.get(entry?.id);
if (edit === undefined) {
projected.push(...messages);
continue;
}
if (edit === null) continue;
for (const message of messages) {
if (!EDITABLE_ROLES.has(message?.role)) {
projected.push(message);
continue;
}
const content = (message.role === "assistant" || message.role === "toolResult")
&& typeof edit.content === "string"
? [{ type: "text", text: edit.content }]
: edit.content;
projected.push({ ...message, content });
}
}
return projected;
}
+31 -246
View File
@@ -1,16 +1,8 @@
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { estimateTokens } from "@earendil-works/pi-coding-agent"; import { estimateTokens } from "@earendil-works/pi-coding-agent";
import { import type { ProjectionSnapshot, UsageBaseline } from "./types.js";
COMPACTION_CUSTOM_TYPE,
COMPACTION_VERSION,
type PiCompactionConfig,
type ProjectionSnapshot,
type SegmentSummaryRecord,
type ToolHistoryRecord,
type UsageBaseline,
} from "./types.js";
function timestampOf(value: unknown): number { export function timestampOf(value: unknown): number {
if (typeof value === "number" && Number.isFinite(value)) return value; if (typeof value === "number" && Number.isFinite(value)) return value;
if (typeof value === "string") { if (typeof value === "string") {
const parsed = Date.parse(value); const parsed = Date.parse(value);
@@ -19,10 +11,6 @@ function timestampOf(value: unknown): number {
return 0; return 0;
} }
export function sourceKey(toolCallId: string, timestamp: unknown): string {
return `${toolCallId}@${timestampOf(timestamp)}`;
}
export function contentText(content: unknown): string { export function contentText(content: unknown): string {
if (typeof content === "string") return content; if (typeof content === "string") return content;
if (!Array.isArray(content)) return ""; if (!Array.isArray(content)) return "";
@@ -37,42 +25,6 @@ function hash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 20); return createHash("sha256").update(value).digest("hex").slice(0, 20);
} }
function json(value: unknown): string {
try {
return JSON.stringify(value) ?? "undefined";
} catch {
return "[unserializable]";
}
}
function boundedEvidence(text: string, maxChars: number): string {
const normalized = text.replace(/\u001b\[[0-9;]*m/g, "").trim();
if (normalized.length <= maxChars) return normalized;
const marker = `\n… ${normalized.length - maxChars} chars omitted …\n`;
const available = Math.max(0, maxChars - marker.length);
const head = Math.ceil(available * 0.7);
const tail = Math.max(0, available - head);
return `${normalized.slice(0, head)}${marker}${tail ? normalized.slice(-tail) : ""}`;
}
export function deterministicSummary(params: {
toolName: string;
args: unknown;
rawText: string;
isError: boolean;
maxStubTokens: number;
}): string {
const maxChars = params.maxStubTokens * 4;
const argsText = boundedEvidence(json(params.args), Math.min(600, Math.floor(maxChars * 0.25)));
const evidenceBudget = Math.max(128, maxChars - argsText.length - 120);
const evidence = boundedEvidence(params.rawText || "(no output)", evidenceBudget);
return [
`[pi-compaction tool=${params.toolName} status=${params.isError ? "error" : "ok"}]`,
`Args: ${argsText}`,
`Evidence:\n${evidence}`,
].join("\n");
}
function messageKey(message: any): string { function messageKey(message: any): string {
const role = String(message?.role ?? "unknown"); const role = String(message?.role ?? "unknown");
const timestamp = timestampOf(message?.timestamp); const timestamp = timestampOf(message?.timestamp);
@@ -85,24 +37,14 @@ function messageKey(message: any): string {
return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`; return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`;
} }
function isSummaryRecord(value: unknown): value is SegmentSummaryRecord { /**
if (!value || typeof value !== "object") return false; * Incremental context projection: tracks the branch's message list, estimates
const record = value as Partial<SegmentSummaryRecord>; * tokens, and maintains the usage-backed effective-token baseline. Projection
return record.version === COMPACTION_VERSION * is identity — pi-compaction never rewrites messages on this path; context
&& record.kind === "segment-summary" * pressure reduction happens through append-only context_edit entries.
&& typeof record.sourceKey === "string" */
&& typeof record.toolCallId === "string"
&& typeof record.resultTimestamp === "number"
&& typeof record.toolName === "string"
&& typeof record.sourceHash === "string"
&& typeof record.deterministicSummary === "string";
}
export class SegmentIndex { export class SegmentIndex {
private readonly records = new Map<string, ToolHistoryRecord>();
private readonly ids = new Map<string, string[]>();
private generation = 0; private generation = 0;
private semanticJobs = 0;
private usageBaseline?: UsageBaseline; private usageBaseline?: UsageBaseline;
private sourceMessages: any[] = []; private sourceMessages: any[] = [];
private sourceKeys: string[] = []; private sourceKeys: string[] = [];
@@ -117,67 +59,13 @@ export class SegmentIndex {
this.sessionId = sessionId; this.sessionId = sessionId;
} }
hydrate(branch: readonly any[], config: PiCompactionConfig): void { hydrate(): void {
this.reset(); this.reset();
if (!config.enabled) return;
const summaries = new Map<string, SegmentSummaryRecord>();
for (const entry of branch) {
if (entry?.type === "custom" && entry.customType === COMPACTION_CUSTOM_TYPE && isSummaryRecord(entry.data)) {
summaries.set(entry.data.sourceKey, entry.data);
}
}
const pendingCalls = new Map<string, { toolName: string; args: unknown }>();
for (const entry of branch) {
if (entry?.type !== "message") continue;
const message = entry.message;
if (message?.role === "assistant" && Array.isArray(message.content)) {
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") {
pendingCalls.set(block.id, {
toolName: String(block.name ?? "unknown"),
args: block.input ?? block.arguments ?? block.args ?? {},
});
}
}
} else if (message?.role === "toolResult" && typeof message.toolCallId === "string") {
const call = pendingCalls.get(message.toolCallId) ?? {
toolName: String(message.toolName ?? "unknown"),
args: {},
};
const key = sourceKey(message.toolCallId, message.timestamp);
const rawText = contentText(message.content);
const persisted = summaries.get(key);
const record: ToolHistoryRecord = {
sourceKey: key,
toolCallId: message.toolCallId,
resultTimestamp: timestampOf(message.timestamp),
toolName: call.toolName,
args: call.args,
isError: Boolean(message.isError),
rawText,
sourceHash: hash(`${call.toolName}\0${json(call.args)}\0${rawText}`),
deterministicSummary: persisted?.deterministicSummary ?? deterministicSummary({
toolName: call.toolName,
args: call.args,
rawText,
isError: Boolean(message.isError),
maxStubTokens: config.deterministic.maxStubTokens,
}),
semanticSummary: persisted?.semanticSummary,
};
this.put(record);
pendingCalls.delete(message.toolCallId);
}
}
this.generation++; this.generation++;
} }
reset(): void { reset(): void {
this.records.clear();
this.ids.clear();
this.generation = 0; this.generation = 0;
this.semanticJobs = 0;
this.usageBaseline = undefined; this.usageBaseline = undefined;
this.sourceMessages = []; this.sourceMessages = [];
this.sourceKeys = []; this.sourceKeys = [];
@@ -187,86 +75,6 @@ export class SegmentIndex {
this.cached = undefined; this.cached = undefined;
} }
private put(record: ToolHistoryRecord): void {
this.records.set(record.sourceKey, record);
const keys = this.ids.get(record.toolCallId) ?? [];
if (!keys.includes(record.sourceKey)) keys.push(record.sourceKey);
this.ids.set(record.toolCallId, keys);
}
captureTurn(message: any, toolResults: any[], config: PiCompactionConfig): ToolHistoryRecord[] {
if (!config.enabled || !config.deterministic.enabled || !Array.isArray(message?.content)) return [];
const calls = new Map<string, any>();
for (const block of message.content) {
if (block?.type === "toolCall" && typeof block.id === "string") calls.set(block.id, block);
}
const captured: ToolHistoryRecord[] = [];
for (const result of toolResults ?? []) {
if (!result || typeof result.toolCallId !== "string") continue;
const call = calls.get(result.toolCallId);
if (!call) continue;
const rawText = contentText(result.content);
const toolName = String(call.name ?? result.toolName ?? "unknown");
const args = call.input ?? call.arguments ?? call.args ?? {};
const key = sourceKey(result.toolCallId, result.timestamp);
const record: ToolHistoryRecord = {
sourceKey: key,
toolCallId: result.toolCallId,
resultTimestamp: timestampOf(result.timestamp),
toolName,
args,
isError: Boolean(result.isError),
rawText,
sourceHash: hash(`${toolName}\0${json(args)}\0${rawText}`),
deterministicSummary: deterministicSummary({
toolName,
args,
rawText,
isError: Boolean(result.isError),
maxStubTokens: config.deterministic.maxStubTokens,
}),
};
this.put(record);
captured.push(record);
}
if (captured.length) {
this.generation++;
this.cached = undefined;
}
return captured;
}
applySemantic(sourceKeyValue: string, sourceHash: string, summary: string): SegmentSummaryRecord | undefined {
const record = this.records.get(sourceKeyValue);
if (!record || record.sourceHash !== sourceHash || !summary.trim()) return undefined;
record.semanticSummary = summary.trim();
this.generation++;
this.cached = undefined;
return this.toPersisted(record);
}
toPersisted(record: ToolHistoryRecord): SegmentSummaryRecord {
const effective = record.semanticSummary ?? record.deterministicSummary;
return {
version: COMPACTION_VERSION,
kind: "segment-summary",
sourceKey: record.sourceKey,
toolCallId: record.toolCallId,
resultTimestamp: record.resultTimestamp,
toolName: record.toolName,
sourceHash: record.sourceHash,
deterministicSummary: record.deterministicSummary,
...(record.semanticSummary ? { semanticSummary: record.semanticSummary } : {}),
rawChars: record.rawText.length,
effectiveChars: effective.length,
};
}
setSemanticJobs(count: number): void {
this.semanticJobs = Math.max(0, count);
if (this.cached) this.cached = { ...this.cached, semanticJobs: this.semanticJobs };
}
recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void { recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void {
if (!Number.isFinite(tokens) || tokens <= 0) return; if (!Number.isFinite(tokens) || tokens <= 0) return;
this.usageBaseline = { tokens, messageEstimate, environmentHash }; this.usageBaseline = { tokens, messageEstimate, environmentHash };
@@ -277,11 +85,27 @@ export class SegmentIndex {
environmentHash: string; environmentHash: string;
environmentTokens: number; environmentTokens: number;
contextWindow: number; contextWindow: number;
maskedResults: number;
reclaimedTokens: number;
}): ProjectionSnapshot { }): ProjectionSnapshot {
const sameTail = messages.length === this.sourceMessages.length const sameTail = messages.length === this.sourceMessages.length
&& messages.length > 0 && messages.length > 0
&& messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1]; && messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1];
if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) return this.cached; if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) {
if (this.cached.maskedResults === params.maskedResults
&& this.cached.reclaimedTokens === params.reclaimedTokens) {
return this.cached;
}
// Masking stats changed but the message list did not: publish a new
// immutable snapshot object instead of re-projecting or mutating.
this.cached = {
...this.cached,
maskedResults: params.maskedResults,
reclaimedTokens: params.reclaimedTokens,
rawTokens: this.cached.effectiveTokens + params.reclaimedTokens,
};
return this.cached;
}
const appendOnly = this.sourceMessages.length > 0 const appendOnly = this.sourceMessages.length > 0
&& messages.length >= this.sourceMessages.length && messages.length >= this.sourceMessages.length
@@ -301,17 +125,14 @@ export class SegmentIndex {
for (let index = start; index < messages.length; index++) { for (let index = start; index < messages.length; index++) {
const message = messages[index]; const message = messages[index];
const projected = this.projectMessage(message);
this.sourceMessages.push(message); this.sourceMessages.push(message);
this.sourceKeys.push(messageKey(message)); this.sourceKeys.push(messageKey(message));
this.projectedMessages.push(projected); this.projectedMessages.push(message);
this.rawEstimates.push(estimateTokens(message as any)); this.rawEstimates.push(estimateTokens(message as any));
this.effectiveEstimates.push(estimateTokens(projected as any)); this.effectiveEstimates.push(estimateTokens(message as any));
} }
const rawMessageTokens = this.rawEstimates.reduce((sum, value) => sum + value, 0);
const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0); const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0);
const rawTokens = rawMessageTokens + params.environmentTokens;
const baseline = this.usageBaseline; const baseline = this.usageBaseline;
const usageBacked = baseline?.environmentHash === params.environmentHash; const usageBacked = baseline?.environmentHash === params.environmentHash;
const effectiveTokens = usageBacked const effectiveTokens = usageBacked
@@ -329,55 +150,19 @@ export class SegmentIndex {
environmentHash: params.environmentHash, environmentHash: params.environmentHash,
messages: this.projectedMessages, messages: this.projectedMessages,
messageEstimate: effectiveMessageTokens, messageEstimate: effectiveMessageTokens,
rawTokens, rawTokens: effectiveTokens + params.reclaimedTokens,
effectiveTokens, effectiveTokens,
reclaimedTokens: Math.max(0, rawTokens - effectiveTokens), reclaimedTokens: params.reclaimedTokens,
contextWindow: params.contextWindow, contextWindow: params.contextWindow,
confidence, confidence,
lowerBound, lowerBound,
upperBound, upperBound,
segmentCount: this.sourceMessages.length, segmentCount: this.sourceMessages.length,
semanticJobs: this.semanticJobs, maskedResults: params.maskedResults,
}; };
return this.cached; return this.cached;
} }
private projectMessage(message: any): any {
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") return message;
const record = this.records.get(sourceKey(message.toolCallId, message.timestamp));
if (!record) return message;
const original = contentText(message.content);
const summary = record.semanticSummary ?? record.deterministicSummary;
const projected = `${summary}\nSource: ${record.sourceKey}; recover with context_tree_query.`;
if (projected.length >= original.length) return message;
return {
...message,
content: [{ type: "text", text: projected }],
};
}
query(ids: string[]): ToolHistoryRecord[] {
const found: ToolHistoryRecord[] = [];
const seen = new Set<string>();
for (const id of ids) {
const direct = this.records.get(id);
const keys = direct ? [id] : this.ids.get(id) ?? [];
for (const key of keys) {
if (seen.has(key)) continue;
const record = this.records.get(key);
if (record) {
found.push(record);
seen.add(key);
}
}
}
return found;
}
allRecords(): ToolHistoryRecord[] {
return [...this.records.values()];
}
getSnapshot(): ProjectionSnapshot | undefined { getSnapshot(): ProjectionSnapshot | undefined {
return this.cached; return this.cached;
} }
-189
View File
@@ -1,189 +0,0 @@
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import type { PiCompactionConfig, ToolHistoryRecord } from "./types.js";
interface Job {
sessionId: string;
sourceKey: string;
sourceHash: string;
toolName: string;
args: unknown;
isError: boolean;
rawText: string;
ctx: ExtensionContext;
config: PiCompactionConfig;
controller: AbortController;
}
export interface SemanticResult {
sessionId: string;
sourceKey: string;
sourceHash: string;
summary: string;
}
function modelFor(config: PiCompactionConfig, ctx: ExtensionContext): any {
if (config.semantic.model === "default") return ctx.model;
const slash = config.semantic.model.indexOf("/");
if (slash <= 0) return ctx.model;
return ctx.modelRegistry.find(
config.semantic.model.slice(0, slash),
config.semantic.model.slice(slash + 1),
) ?? ctx.model;
}
function serialize(job: Job): string {
const args = (() => {
try { return JSON.stringify(job.args); } catch { return "[unserializable]"; }
})();
const max = job.config.semantic.maxInputChars;
const raw = job.rawText.length <= max
? job.rawText
: `${job.rawText.slice(0, Math.floor(max * 0.7))}\n… omitted …\n${job.rawText.slice(-Math.floor(max * 0.3))}`;
return `Summarize this completed coding-agent tool call for future context.\nKeep concrete outcomes, paths, identifiers, values, and exact errors.\nUse at most 6 concise bullets. Do not give advice and do not call tools.\n\nTool: ${job.toolName}\nStatus: ${job.isError ? "ERROR" : "OK"}\nArgs: ${args}\nOutput:\n${raw}`;
}
async function run(job: Job): Promise<string> {
const model = modelFor(job.config, job.ctx);
if (!model) throw new Error("No summarizer model selected");
const message = await job.ctx.modelRegistry.complete(model, {
messages: [{
role: "user",
content: [{ type: "text", text: serialize(job) }],
timestamp: Date.now(),
}],
} as any, {
signal: job.controller.signal,
reasoningEffort: job.config.semantic.thinking === "off" ? undefined : job.config.semantic.thinking,
} as any);
if (message.stopReason === "length" || message.stopReason === "error" || message.stopReason === "aborted") {
throw new Error(message.errorMessage ?? `Semantic compaction stopped: ${message.stopReason}`);
}
const text = message.content
.flatMap((part: any) => part.type === "text" && typeof part.text === "string" ? [part.text] : [])
.join("\n")
.trim();
if (!text) throw new Error("Semantic compaction returned an empty summary");
return text;
}
function abortableDelay(ms: number, signal: AbortSignal): Promise<void> {
if (signal.aborted) return Promise.reject(new Error("Semantic compaction aborted"));
return new Promise((resolve, reject) => {
const timer = setTimeout(done, ms);
function done(): void {
signal.removeEventListener("abort", aborted);
resolve();
}
function aborted(): void {
clearTimeout(timer);
signal.removeEventListener("abort", aborted);
reject(new Error("Semantic compaction aborted"));
}
signal.addEventListener("abort", aborted, { once: true });
});
}
export class SemanticWorker {
private readonly queue: Job[] = [];
private readonly active = new Map<string, Job>();
private readonly ready: SemanticResult[] = [];
private readonly queuedKeys = new Set<string>();
enqueue(sessionId: string, record: ToolHistoryRecord, ctx: ExtensionContext, config: PiCompactionConfig): void {
if (!config.semantic.enabled || record.semanticSummary || record.rawText.length < config.semantic.minInputChars) return;
const key = `${sessionId}:${record.sourceKey}:${record.sourceHash}`;
if (this.queuedKeys.has(key)) return;
this.queuedKeys.add(key);
this.queue.push({
sessionId,
sourceKey: record.sourceKey,
sourceHash: record.sourceHash,
toolName: record.toolName,
args: record.args,
isError: record.isError,
rawText: record.rawText,
ctx,
config,
controller: new AbortController(),
});
this.pump(config.semantic.concurrency);
}
private pump(limit: number): void {
while (this.active.size < limit && this.queue.length > 0) {
const job = this.queue.shift()!;
const key = `${job.sessionId}:${job.sourceKey}:${job.sourceHash}`;
this.active.set(key, job);
void this.execute(key, job, limit);
}
}
private async execute(key: string, job: Job, limit: number): Promise<void> {
let lastError: unknown;
try {
for (let attempt = 0; attempt <= job.config.semantic.maxRetries; attempt++) {
try {
const summary = await run(job);
if (!job.controller.signal.aborted) {
this.ready.push({
sessionId: job.sessionId,
sourceKey: job.sourceKey,
sourceHash: job.sourceHash,
summary,
});
}
return;
} catch (error) {
lastError = error;
if (job.controller.signal.aborted || attempt === job.config.semantic.maxRetries) break;
await abortableDelay(Math.min(8_000, 1_000 * 2 ** attempt), job.controller.signal);
}
}
if (!job.controller.signal.aborted) {
console.error(`pi-compaction: semantic summary failed for ${job.sourceKey}:`, lastError);
}
} finally {
this.active.delete(key);
this.queuedKeys.delete(key);
this.pump(limit);
}
}
drain(sessionId: string): SemanticResult[] {
const drained: SemanticResult[] = [];
for (let index = this.ready.length - 1; index >= 0; index--) {
if (this.ready[index]!.sessionId !== sessionId) continue;
drained.push(this.ready[index]!);
this.ready.splice(index, 1);
}
return drained.reverse();
}
count(sessionId: string): number {
return this.queue.filter((job) => job.sessionId === sessionId).length
+ [...this.active.values()].filter((job) => job.sessionId === sessionId).length;
}
cancelSession(sessionId: string): void {
for (const job of this.active.values()) {
if (job.sessionId === sessionId) job.controller.abort();
}
for (let index = this.queue.length - 1; index >= 0; index--) {
if (this.queue[index]!.sessionId === sessionId) {
const job = this.queue[index]!;
this.queuedKeys.delete(`${job.sessionId}:${job.sourceKey}:${job.sourceHash}`);
this.queue.splice(index, 1);
}
}
for (let index = this.ready.length - 1; index >= 0; index--) {
if (this.ready[index]!.sessionId === sessionId) this.ready.splice(index, 1);
}
}
cancelAll(): void {
for (const job of this.active.values()) job.controller.abort();
this.queue.length = 0;
this.ready.length = 0;
this.queuedKeys.clear();
}
}
+5 -5
View File
@@ -2,8 +2,8 @@ import {
SERVICE_SYMBOL, SERVICE_SYMBOL,
type CompactionMetrics, type CompactionMetrics,
type CompactionService, type CompactionService,
type MaskedResultRecord,
type ProjectionSnapshot, type ProjectionSnapshot,
type ToolHistoryRecord,
} from "./types.js"; } from "./types.js";
interface GlobalWithCompactionService { interface GlobalWithCompactionService {
@@ -14,11 +14,11 @@ const root = globalThis as GlobalWithCompactionService;
export class CompactionServiceRegistry implements CompactionService { export class CompactionServiceRegistry implements CompactionService {
private readonly snapshots = new Map<string, ProjectionSnapshot>(); private readonly snapshots = new Map<string, ProjectionSnapshot>();
private readonly queryHandlers = new Map<string, (ids: string[]) => ToolHistoryRecord[]>(); private readonly queryHandlers = new Map<string, (ids: string[]) => MaskedResultRecord[]>();
setSession( setSession(
sessionId: string, sessionId: string,
query: (ids: string[]) => ToolHistoryRecord[], query: (ids: string[]) => MaskedResultRecord[],
): void { ): void {
this.queryHandlers.set(sessionId, query); this.queryHandlers.set(sessionId, query);
} }
@@ -58,11 +58,11 @@ export class CompactionServiceRegistry implements CompactionService {
percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0, percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0,
confidence: snapshot.confidence, confidence: snapshot.confidence,
segmentCount: snapshot.segmentCount, segmentCount: snapshot.segmentCount,
semanticJobs: snapshot.semanticJobs, maskedResults: snapshot.maskedResults,
}; };
} }
query(sessionId: string, ids: string[]): ToolHistoryRecord[] { query(sessionId: string, ids: string[]): MaskedResultRecord[] {
return this.queryHandlers.get(sessionId)?.(ids) ?? []; return this.queryHandlers.get(sessionId)?.(ids) ?? [];
} }
} }
+4 -12
View File
@@ -1,13 +1,5 @@
import type { PiCompactionConfig, ProjectionSnapshot } from "./types.js"; // Effective trigger for one configured (tokens, ratio) pair: the absolute
// token figure wins on large windows, the ratio caps it on small ones.
export type CompactionReason = "manual" | "threshold" | "overflow"; export function effectiveThreshold(tokens: number, ratio: number, contextWindow: number): number {
return contextWindow > 0 ? Math.min(tokens, Math.floor(ratio * contextWindow)) : tokens;
export function shouldCompact(
reason: CompactionReason,
snapshot: Pick<ProjectionSnapshot, "lowerBound" | "contextWindow">,
config: Pick<PiCompactionConfig, "hardRatio">,
): boolean {
if (reason === "manual" || reason === "overflow") return true;
if (snapshot.contextWindow <= 0) return true;
return snapshot.lowerBound / snapshot.contextWindow >= config.hardRatio;
} }
+30 -48
View File
@@ -5,58 +5,33 @@ export const COMPACTION_VERSION = 1 as const;
export const QUERY_TOOL_NAME = "context_tree_query"; export const QUERY_TOOL_NAME = "context_tree_query";
export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1"); export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1");
export interface MaskingConfig {
enabled: boolean;
triggerTokens: number;
triggerRatio: number;
targetTokens: number;
targetRatio: number;
minClearTokens: number;
keepRecentTurns: number;
minEntryTokens: number;
excludeTools: string[];
}
export interface SummaryConfig {
triggerTokens: number;
triggerRatio: number;
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
}
export interface PiCompactionConfig { export interface PiCompactionConfig {
enabled: boolean; enabled: boolean;
softRatio: number; masking: MaskingConfig;
hardRatio: number; summary: SummaryConfig;
targetRatio: number;
deterministic: {
enabled: boolean;
maxStubTokens: number;
};
semantic: {
enabled: boolean;
model: string;
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
concurrency: number;
minInputChars: number;
maxInputChars: number;
maxRetries: number;
};
native: { native: {
enabled: boolean; enabled: boolean;
}; };
} }
export interface SegmentSummaryRecord {
version: typeof COMPACTION_VERSION;
kind: "segment-summary";
sourceKey: string;
toolCallId: string;
resultTimestamp: number;
toolName: string;
sourceHash: string;
deterministicSummary: string;
semanticSummary?: string;
rawChars: number;
effectiveChars: number;
}
export type PiCompactionRecord = SegmentSummaryRecord;
export interface ToolHistoryRecord {
sourceKey: string;
toolCallId: string;
resultTimestamp: number;
toolName: string;
args: unknown;
isError: boolean;
rawText: string;
sourceHash: string;
deterministicSummary: string;
semanticSummary?: string;
}
export interface UsageBaseline { export interface UsageBaseline {
tokens: number; tokens: number;
messageEstimate: number; messageEstimate: number;
@@ -78,7 +53,7 @@ export interface ProjectionSnapshot {
lowerBound: number; lowerBound: number;
upperBound: number; upperBound: number;
segmentCount: number; segmentCount: number;
semanticJobs: number; maskedResults: number;
codexItems?: ResponseItem[]; codexItems?: ResponseItem[];
} }
@@ -99,6 +74,13 @@ export interface PortableCheckpointDetails {
sourceFingerprint: string; sourceFingerprint: string;
} }
/** What the service returns for one masked tool result — the same placeholder the model sees. */
export interface MaskedResultRecord {
toolCallId: string;
resultTimestamp: number;
deterministicSummary: string;
}
export interface CompactionMetrics { export interface CompactionMetrics {
sessionId: string; sessionId: string;
generation: number; generation: number;
@@ -109,11 +91,11 @@ export interface CompactionMetrics {
percent: number; percent: number;
confidence: ProjectionSnapshot["confidence"]; confidence: ProjectionSnapshot["confidence"];
segmentCount: number; segmentCount: number;
semanticJobs: number; maskedResults: number;
} }
export interface CompactionService { export interface CompactionService {
getSnapshot(sessionId: string): ProjectionSnapshot | undefined; getSnapshot(sessionId: string): ProjectionSnapshot | undefined;
getMetrics(sessionId: string): CompactionMetrics | undefined; getMetrics(sessionId: string): CompactionMetrics | undefined;
query(sessionId: string, ids: string[]): ToolHistoryRecord[]; query(sessionId: string, ids: string[]): MaskedResultRecord[];
} }
+68 -20
View File
@@ -2,34 +2,82 @@ import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts"; import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts";
test("normalizes one greenfield piCompaction config without legacy aliases", () => { test("normalizes a full pi-compaction config", () => {
const config = normalizeConfig({ const config = normalizeConfig({
enabled: false, enabled: false,
softRatio: 0.7, masking: {
hardRatio: 0.91, enabled: false,
targetRatio: 0.5, triggerTokens: 150_000,
deterministic: { enabled: false, maxStubTokens: 512 }, triggerRatio: 0.5,
semantic: { enabled: false, concurrency: 2, maxRetries: 0 }, targetTokens: 60_000,
targetRatio: 0.2,
minClearTokens: 30_000,
keepRecentTurns: 5,
minEntryTokens: 512,
excludeTools: ["bash", "read"],
},
summary: { triggerTokens: 300_000, triggerRatio: 0.8, thinking: "medium" },
native: { enabled: false }, native: { enabled: false },
}); });
assert.equal(config.enabled, false); assert.equal(config.enabled, false);
assert.equal(config.deterministic.enabled, false); assert.deepEqual(config.masking, {
assert.equal(config.deterministic.maxStubTokens, 512); enabled: false,
assert.equal(config.semantic.enabled, false); triggerTokens: 150_000,
assert.equal(config.semantic.maxRetries, 0); triggerRatio: 0.5,
targetTokens: 60_000,
targetRatio: 0.2,
minClearTokens: 30_000,
keepRecentTurns: 5,
minEntryTokens: 512,
excludeTools: ["bash", "read"],
});
assert.equal(config.summary.triggerTokens, 300_000);
assert.equal(config.summary.triggerRatio, 0.8);
assert.equal(config.summary.thinking, "medium");
assert.equal(config.native.enabled, false); assert.equal(config.native.enabled, false);
}); });
test("invalid ratios and bounds fail safe to bounded defaults", () => { test("invalid fields fail safe to bounded defaults", () => {
const config = normalizeConfig({ const config = normalizeConfig({
softRatio: 2, enabled: "yes",
hardRatio: -1, masking: {
targetRatio: 0.99, triggerTokens: -1,
semantic: { concurrency: 50, maxRetries: -1 }, triggerRatio: 2,
targetTokens: "a lot",
targetRatio: 0,
minClearTokens: 0,
keepRecentTurns: 500,
minEntryTokens: -3,
excludeTools: "read",
},
summary: { triggerTokens: Number.NaN, triggerRatio: 1.5, thinking: "insane" },
native: { enabled: 1 },
}); });
assert.equal(config.softRatio, DEFAULT_CONFIG.softRatio); assert.equal(config.enabled, DEFAULT_CONFIG.enabled);
assert.ok(config.targetRatio < config.softRatio); assert.deepEqual(config.masking, DEFAULT_CONFIG.masking);
assert.ok(config.hardRatio > config.softRatio); assert.equal(config.summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens);
assert.equal(config.semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency); assert.equal(config.summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio);
assert.equal(config.semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries); assert.equal(config.summary.thinking, DEFAULT_CONFIG.summary.thinking);
assert.equal(config.native.enabled, DEFAULT_CONFIG.native.enabled);
});
test("masking target is corrected below trigger and summary above masking", () => {
const config = normalizeConfig({
masking: { triggerTokens: 80_000, triggerRatio: 0.5, targetTokens: 120_000, targetRatio: 0.9 },
summary: { triggerTokens: 50_000, triggerRatio: 0.4 },
});
assert.equal(config.masking.targetTokens, 40_000);
assert.equal(config.masking.targetRatio, 0.25);
assert.equal(config.summary.triggerTokens, 160_000);
assert.ok(config.summary.triggerRatio > config.masking.triggerRatio);
});
test("minClearTokens and excludeTools accept valid overrides", () => {
const config = normalizeConfig({
masking: { minClearTokens: 1, keepRecentTurns: 1, minEntryTokens: 0, excludeTools: [] },
});
assert.equal(config.masking.minClearTokens, 1);
assert.equal(config.masking.keepRecentTurns, 1);
assert.equal(config.masking.minEntryTokens, 0);
assert.deepEqual(config.masking.excludeTools, []);
}); });
+225 -61
View File
@@ -1,7 +1,8 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { performance } from "node:perf_hooks";
import piCompaction from "../index.ts"; import piCompaction from "../index.ts";
import { getCompactionService } from "../src/service.ts";
import { MASK_PREFIX } from "../src/masking.ts";
function harness() { function harness() {
const handlers = new Map<string, Function[]>(); const handlers = new Map<string, Function[]>();
@@ -24,90 +25,253 @@ function harness() {
return { pi, handlers, tools, commands, entries }; return { pi, handlers, tools, commands, entries };
} }
function context(branch: any[] = []) { function context(branch: any[] = [], tokens = 1_000) {
return { const compactCalls: any[] = [];
const sessionId = `session-${Math.random().toString(36).slice(2)}`;
const ctx = {
sessionManager: { sessionManager: {
getSessionId: () => "session-test", getSessionId: () => sessionId,
getBranch: () => branch, getBranch: () => branch,
getLeafId: () => "leaf-1",
}, },
model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 }, model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 },
modelRegistry: {}, modelRegistry: {},
getSystemPrompt: () => "system", getSystemPrompt: () => "system",
getContextUsage: () => ({ tokens: 1_000, contextWindow: 100_000, percent: 1 }), getContextUsage: () => ({ tokens, contextWindow: 100_000, percent: 1 }),
compact(options: unknown) { compactCalls.push(options); },
isIdle: () => true,
hasUI: false, hasUI: false,
abort() {}, abort() {},
} as any; } as any;
return { ctx, compactCalls };
} }
test("one runtime projects tool evidence and recovers it through context_tree_query", async () => { function projectedEntry(id: string, message: any) {
return { sourceEntry: { id, type: "message", message }, messages: [message] };
}
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
return {
role: "assistant",
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
timestamp: id,
};
}
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
return {
role: "toolResult",
toolCallId,
toolName,
content: [{ type: "text", text }],
timestamp: id,
};
}
// Default config: masking trigger = min(200K, 60% of window); at the harness
// 100K window that is 60K tokens and the target is 35K, so 60K used needs
// ~25K tokens of maskable results outside the 10-turn protected window.
function pressurizedContextEntries() {
const entries = [
projectedEntry("a1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }])),
projectedEntry("r1", toolResultMessage("r1", "c1", `failure evidence\n${"x".repeat(120_000)}\nstack tail`)),
];
for (let index = 0; index < 10; index++) {
entries.push(projectedEntry(`keep-${index}`, assistantMessage(`keep-${index}`, [])));
}
return entries;
}
test("turn_end masks old tool results under pressure and preserves prior drafts", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context([], 60_000);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const assistant = { const prior = { type: "custom", customType: "other-extension", data: {} };
role: "assistant", const result = await h.handlers.get("turn_end")![0]!({
content: [{ type: "toolCall", id: "call-1", name: "bash", arguments: { command: "npm test" } }], entries: [prior],
timestamp: 10, message: assistantMessage("a1", []),
}; toolResults: [],
const result = { context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
role: "toolResult", }, ctx);
toolCallId: "call-1",
toolName: "bash",
content: [{ type: "text", text: `failure evidence\n${"x".repeat(12_000)}\nstack tail` }],
isError: true,
timestamp: 11,
};
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx);
assert.match(projected.messages[1].content[0].text, /pi-compaction tool=bash/); assert.ok(result?.entries);
const cacheStart = performance.now(); assert.strictEqual(result.entries[0], prior);
for (let iteration = 0; iteration < 1_000; iteration++) { const edits = result.entries.slice(1);
await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx); assert.equal(edits.length, 1);
} assert.equal(edits[0]!.type, "context_edit");
const cacheAverageMs = (performance.now() - cacheStart) / 1_000; assert.equal(edits[0]!.targetId, "r1");
assert.ok(cacheAverageMs < 2, `full context handler cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`); assert.ok(edits[0]!.replacement.content.startsWith(MASK_PREFIX));
assert.equal(h.entries.length, 1); // Masking is append-only context_edit; nothing is written as custom entries.
assert.equal(h.entries[0]!.customType, "pi-compaction"); assert.equal(h.entries.length, 0);
const query = h.tools.get("context_tree_query");
const recovered = await query.execute("q", { toolCallIds: ["call-1"] }, new AbortController().signal, undefined, ctx);
assert.match(recovered.content[0].text, /failure evidence/);
await h.handlers.get("session_shutdown")![0]!({}, ctx); await h.handlers.get("session_shutdown")![0]!({}, ctx);
}); });
test("small tool results are passed through without a stub wrapper", async () => { test("turn_end below the masking trigger returns undefined", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context([], 40_000);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const assistant = { const result = await h.handlers.get("turn_end")![0]!({
role: "assistant", entries: [],
content: [{ type: "toolCall", id: "call-small", name: "bash", arguments: { command: "ls" } }], message: assistantMessage("a1", []),
timestamp: 20, toolResults: [],
}; context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
const result = { }, ctx);
role: "toolResult", assert.equal(result, undefined);
toolCallId: "call-small", await h.handlers.get("session_shutdown")![0]!({}, ctx);
toolName: "bash", });
content: [{ type: "text", text: "ok\nfile-a.ts\nfile-b.ts" }],
timestamp: 21,
};
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx); test("context hook returns messages untouched on the non-checkpoint path", async () => {
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx); const h = harness();
piCompaction(h.pi);
const { ctx } = context();
await h.handlers.get("session_start")![0]!({}, ctx);
assert.strictEqual(projected.messages[1].content[0].text, "ok\nfile-a.ts\nfile-b.ts"); const messages = [assistantMessage("a1", []), toolResultMessage("r1", "c1", "visible\nfile-a.ts")];
assert.doesNotMatch(projected.messages[1].content[0].text, /pi-compaction tool=/); const projected = await h.handlers.get("context")![0]!({ messages }, ctx);
assert.equal(projected, undefined);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("agent_settled compacts once above the summary threshold and never repeats on the same leaf", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx, compactCalls } = context([], 95_000);
await h.handlers.get("session_start")![0]!({}, ctx);
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 1);
assert.equal(typeof compactCalls[0].onError, "function");
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 1);
ctx.sessionManager.getLeafId = () => "leaf-2";
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 2);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("agent_settled stays idle below the summary threshold", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx, compactCalls } = context([], 50_000);
await h.handlers.get("session_start")![0]!({}, ctx);
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
assert.equal(compactCalls.length, 0);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("context_tree_query recovers the original tool output from the branch", async () => {
const h = harness();
piCompaction(h.pi);
const branch = [
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "failure evidence\nstack tail") },
];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
// Original evidence stays recoverable even when the message is passed through.
const query = h.tools.get("context_tree_query")!; const query = h.tools.get("context_tree_query")!;
const recovered = await query.execute("q", { toolCallIds: ["call-small"] }, new AbortController().signal, undefined, ctx); const recovered = await query.execute("q", { toolCallIds: ["c1"] }, new AbortController().signal, undefined, ctx);
assert.match(recovered.content[0].text, /file-a\.ts/); assert.match(recovered.content[0].text, /failure evidence/);
assert.match(recovered.content[0].text, /stack tail/);
assert.deepEqual(recovered.details.missing, []);
const missing = await query.execute("q", { toolCallIds: ["nope"] }, new AbortController().signal, undefined, ctx);
assert.match(missing.content[0].text, /not found/);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("service query returns placeholders only for masked tool call ids", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx } = context([], 60_000);
await h.handlers.get("session_start")![0]!({}, ctx);
const sessionId = ctx.sessionManager.getSessionId();
await h.handlers.get("turn_end")![0]!({
entries: [],
message: assistantMessage("a1", []),
toolResults: [],
context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
}, ctx);
const records = getCompactionService()!.query(sessionId, ["c1", "never-masked"]);
assert.equal(records.length, 1);
assert.equal(records[0]!.toolCallId, "c1");
assert.ok(records[0]!.deterministicSummary.startsWith(MASK_PREFIX));
assert.equal(typeof records[0]!.resultTimestamp, "number");
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("hydrating a branch with this extension's context_edit rebuilds masking metrics", async () => {
const h = harness();
piCompaction(h.pi);
const placeholder = `${MASK_PREFIX}bash result cleared to save context (~1_000 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["c1"]}.`;
const branch = [
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "x".repeat(8_000)) },
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: placeholder } },
];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
const sessionId = ctx.sessionManager.getSessionId();
await h.handlers.get("context")![0]!({ messages: [toolResultMessage("r1", "c1", "x".repeat(8_000))] }, ctx);
const metrics = getCompactionService()!.getMetrics(sessionId);
assert.equal(metrics!.maskedResults, 1);
assert.ok(metrics!.reclaimedTokens > 0);
assert.ok(metrics!.rawTokens > metrics!.effectiveTokens);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("session_before_compact no longer cancels threshold compactions", async () => {
const h = harness();
piCompaction(h.pi);
const { ctx } = context();
await h.handlers.get("session_start")![0]!({}, ctx);
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: false, error: "no credentials" });
const result = await h.handlers.get("session_before_compact")![0]!({
reason: "threshold", branchEntries: [], preparation: { firstKeptEntryId: "u1" },
signal: new AbortController().signal,
}, ctx);
assert.equal(result, undefined);
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
test("native checkpoint projection applies context edits made after the checkpoint", async () => {
const h = harness();
piCompaction(h.pi);
const details = {
kind: "pi-compaction",
version: 1,
checkpoint: "native",
modelKey: "test:test:model",
replacementHistory: [{ type: "compaction", encrypted_content: "opaque" }],
portableSummary: "checkpoint summary",
sourceFingerprint: "fp",
};
const branch = [
{ type: "compaction", id: "c1", parentId: null, timestamp: "t", summary: "marker", firstKeptEntryId: "r1", tokensBefore: 1_000, details },
{ type: "message", id: "r1", parentId: "c1", timestamp: "t", message: toolResultMessage("r1", "c1", "ORIGINAL-CONTENT-HERE") },
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: `${MASK_PREFIX}bash result cleared to save context (~5 tokens). Call args: {}.` } },
{ type: "message", id: "u1", parentId: "e1", timestamp: "t", message: { role: "user", content: "next question", timestamp: 30 } },
] as any[];
const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
const serialized = JSON.stringify(projected.messages);
assert.doesNotMatch(serialized, /ORIGINAL-CONTENT-HERE/);
assert.match(serialized, /result cleared to save context/);
assert.match(serialized, /next question/);
await h.handlers.get("session_shutdown")![0]!({}, ctx); await h.handlers.get("session_shutdown")![0]!({}, ctx);
}); });
@@ -127,7 +291,7 @@ test("model switching after a native checkpoint projects the portable checkpoint
{ type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details }, { type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details },
{ type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } }, { type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } },
] as any[]; ] as any[];
const ctx = context(branch); const { ctx } = context(branch);
await h.handlers.get("session_start")![0]!({}, ctx); await h.handlers.get("session_start")![0]!({}, ctx);
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx); const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
assert.match(projected.messages[0].content[0].text, /decisions and files/); assert.match(projected.messages[0].content[0].text, /decisions and files/);
@@ -143,7 +307,7 @@ for (const provider of ["sub-api", "openai-codex"]) {
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Remember the test decision.", timestamp: 1 }, message: { role: "user", content: "Remember the test decision.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
ctx.model = { ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses", provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
id: "selected-model", input: ["text"], contextWindow: 100_000, id: "selected-model", input: ["text"], contextWindow: 100_000,
@@ -231,7 +395,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Test only.", timestamp: 1 }, message: { role: "user", content: "Test only.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" }; ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" };
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" }); ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" });
const responses = [ const responses = [
@@ -266,7 +430,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => { test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => {
const h = harness(); const h = harness();
piCompaction(h.pi); piCompaction(h.pi);
const ctx = context(); const { ctx } = context();
try { try {
for (const model of [ for (const model of [
{ provider: "other", api: "openai-responses", id: "test" }, { provider: "other", api: "openai-responses", id: "test" },
@@ -302,7 +466,7 @@ test("native runtime delegates authentication to each provider without blocking
type: "message", id: "u1", parentId: null, type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 }, message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 },
}]; }];
const ctx = context(branch); const { ctx } = context(branch);
const provider = scenario.provider ?? "sub-api"; const provider = scenario.provider ?? "sub-api";
ctx.model = { ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses", provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
+209
View File
@@ -0,0 +1,209 @@
import test from "node:test";
import assert from "node:assert/strict";
import { MASK_PREFIX, planMasking, rebuildMaskedLedger } from "../src/masking.ts";
import { effectiveThreshold } from "../src/token-accounting.ts";
import type { MaskingConfig } from "../src/types.ts";
const MASKING: MaskingConfig = {
enabled: true,
triggerTokens: 1_000,
triggerRatio: 0.9,
targetTokens: 400,
targetRatio: 0.35,
minClearTokens: 200,
keepRecentTurns: 2,
minEntryTokens: 10,
excludeTools: ["ask_user_question"],
};
const estimate = (message: any): number => Math.ceil(JSON.stringify(message).length / 4);
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
return {
role: "assistant",
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
timestamp: id,
};
}
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
return {
role: "toolResult",
toolCallId,
toolName,
content: [{ type: "text", text }],
timestamp: id,
};
}
function entry(id: string, message: any, messages = [message]) {
return { sourceEntry: { id, type: "message", message }, messages };
}
function protect(turns = MASKING.keepRecentTurns) {
return Array.from({ length: turns }, (_unused, index) =>
entry(`recent-${index}`, { role: "assistant", content: [{ type: "text", text: "ok" }], timestamp: `recent-${index}` }));
}
test("below the effective trigger plans no drafts", () => {
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "ls" } }])),
entry("e2", toolResultMessage("r1", "c1", "x".repeat(8_000))),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 999, contextWindow: 0, masking: MASKING, estimate });
assert.equal(plan.drafts.length, 0);
assert.equal(plan.masked.length, 0);
});
test("above the trigger masks oldest first until context fits under target", () => {
const entries = [
entry("e1", assistantMessage("a1", [
{ id: "c1", name: "bash", args: { command: "one" } },
{ id: "c2", name: "bash", args: { command: "two" } },
{ id: "c3", name: "bash", args: { command: "three" } },
])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000))),
entry("e4", toolResultMessage("r3", "c3", "c".repeat(4_000))),
entry("e5", toolResultMessage("r4", "c4", "recent result".repeat(200))),
...protect(),
];
// needed = 2_000 - 400 = 1_600; each result clears ~980 tokens.
const plan = planMasking({ contextEntries: entries, currentTokens: 2_000, contextWindow: 0, masking: MASKING, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e2", "e3"]);
});
test("results inside the recent keepRecentTurns window are never masked", () => {
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(4_000))),
entry("recent-a", assistantMessage("ra", [{ id: "rc", name: "bash", args: {} }])),
entry("recent-r", toolResultMessage("rr", "rc", "b".repeat(4_000))),
entry("recent-b", assistantMessage("rb", [])),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e1"]);
});
test("excluded tools, SKILL.md reads, already-masked and small results are skipped", () => {
const entries = [
entry("e1", assistantMessage("a1", [
{ id: "c1", name: "ask_user_question", args: { question: "x" } },
{ id: "c2", name: "read", args: { path: "/repo/skills/foo/SKILL.md" } },
{ id: "c3", name: "bash", args: { command: "old" } },
{ id: "c4", name: "bash", args: { command: "tiny" } },
])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000), "ask_user_question")),
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000), "read")),
entry("e4", toolResultMessage("r3", "c3", `${MASK_PREFIX}bash result cleared to save context (~900 tokens).`), ),
entry("e5", toolResultMessage("r4", "c4", "tiny")),
entry("e6", toolResultMessage("r5", "c5", "d".repeat(4_000))),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 100, minEntryTokens: 100 };
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e6"]);
assert.equal(plan.masked[0]!.toolCallId, "c5");
});
test("when clearable volume stays under minClear nothing is masked", () => {
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(600))),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
// ~130 tokens clearable, below minClearTokens (200) → not worth invalidating cache.
assert.equal(plan.drafts.length, 0);
});
test("placeholder is deterministic, names the tool and call id, and truncates args at 300 chars", () => {
const longArgs = { command: `run ${"v".repeat(600)}` };
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: longArgs }])),
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 1 };
const first = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
const second = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.equal(first.drafts.length, 1);
const placeholder = first.masked[0]!.placeholder;
assert.equal(placeholder, second.masked[0]!.placeholder);
assert.ok(placeholder.startsWith(MASK_PREFIX));
assert.match(placeholder, /bash result cleared/);
assert.match(placeholder, /"toolCallIds":\["c1"\]/);
const argsText = placeholder.match(/Call args: (.*)\. Re-run/)?.[1] ?? "";
assert.ok(argsText.length <= 301, `args excerpt ${argsText.length} exceeded 300+ellipsis`);
assert.ok(placeholder.includes("…"));
});
test("a masked result carrying details.overflow.id gets an overflow_query hint in the placeholder", () => {
const withOverflow = {
...toolResultMessage("r1", "c1", "a".repeat(4_000)),
details: { overflow: { id: "0123456789ab", totalLines: 5, totalBytes: 4000, truncated: true } },
};
const withoutOverflow = toolResultMessage("r2", "c2", "b".repeat(4_000));
const entries = [
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash" }, { id: "c2", name: "bash" }])),
entry("e2", withOverflow),
entry("e3", withoutOverflow),
...protect(),
];
const masking = { ...MASKING, minClearTokens: 1 };
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
assert.equal(plan.masked.length, 2);
const first = plan.masked.find((m) => m.toolCallId === "c1")!.placeholder;
const second = plan.masked.find((m) => m.toolCallId === "c2")!.placeholder;
assert.ok(first.includes(` Full original stored: overflow_query {"id":"0123456789ab"}.`));
assert.ok(!second.includes("overflow_query"));
});
test("the small-window ratio caps both trigger and target", () => {
const masking: MaskingConfig = {
...MASKING,
triggerTokens: 200_000,
triggerRatio: 0.6,
targetTokens: 100_000,
targetRatio: 0.35,
minClearTokens: 50_000,
};
assert.equal(effectiveThreshold(masking.triggerTokens, masking.triggerRatio, 272_000), 163_200);
assert.equal(effectiveThreshold(masking.targetTokens, masking.targetRatio, 272_000), 95_200);
const entries = [
entry("e1", toolResultMessage("r1", "c1", "a".repeat(300_000))),
...protect(),
];
const below = planMasking({ contextEntries: entries, currentTokens: 163_199, contextWindow: 272_000, masking, estimate });
assert.equal(below.drafts.length, 0);
const above = planMasking({ contextEntries: entries, currentTokens: 163_200, contextWindow: 272_000, masking, estimate });
assert.equal(above.drafts.length, 1);
});
test("ledger rebuild counts only masked targets still inside the active context", () => {
const placeholder = (id: string) =>
`${MASK_PREFIX}bash result cleared to save context (~100 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["${id}"]}.`;
const branch = [
{ type: "message", id: "u1", parentId: null, timestamp: "t", message: { role: "user", content: "start", timestamp: 1 } },
{ type: "message", id: "r-old", parentId: "u1", timestamp: "t", message: toolResultMessage("m-old", "c-old", "o".repeat(2_000)) },
{ type: "context_edit", id: "e-old", parentId: "r-old", timestamp: "t", targetId: "r-old", replacement: { content: placeholder("c-old") } },
{ type: "message", id: "m-keep", parentId: "e-old", timestamp: "t", message: { role: "user", content: "kept question", timestamp: 2 } },
{ type: "compaction", id: "c1", parentId: "m-keep", timestamp: "t", summary: "summary", firstKeptEntryId: "m-keep", tokensBefore: 9_999 },
{ type: "message", id: "a-keep", parentId: "c1", timestamp: "t", message: assistantMessage("a-keep", []) },
{ type: "message", id: "r-new", parentId: "a-keep", timestamp: "t", message: toolResultMessage("m-new", "c-new", "n".repeat(2_000)) },
{ type: "context_edit", id: "e-new", parentId: "r-new", timestamp: "t", targetId: "r-new", replacement: { content: placeholder("c-new") } },
];
const ledger = rebuildMaskedLedger(branch, "e-new", estimate);
assert.equal(ledger.size, 1);
assert.ok(ledger.has("c-new"));
assert.ok(!ledger.has("c-old"));
});
test("user and assistant messages are never masked", () => {
const entries = [
entry("e1", { role: "user", content: "u".repeat(8_000), timestamp: "u1" }),
entry("e2", { role: "assistant", content: [{ type: "text", text: "a".repeat(8_000) }], timestamp: "a1" }),
...protect(),
];
const plan = planMasking({ contextEntries: entries, currentTokens: 9_000, contextWindow: 0, masking: { ...MASKING, keepRecentTurns: 1, minClearTokens: 1 }, estimate });
assert.equal(plan.drafts.length, 0);
});
+3 -2
View File
@@ -10,9 +10,9 @@ function message(index: number) {
test("cached projection and append-only delta stay inside the hot-path budget", () => { test("cached projection and append-only delta stay inside the hot-path budget", () => {
const index = new SegmentIndex("perf"); const index = new SegmentIndex("perf");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const messages = Array.from({ length: 10_000 }, (_, value) => message(value)); const messages = Array.from({ length: 10_000 }, (_, value) => message(value));
const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000 }; const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
index.project(messages, params); index.project(messages, params);
const cacheStart = performance.now(); const cacheStart = performance.now();
@@ -24,6 +24,7 @@ test("cached projection and append-only delta stay inside the hot-path budget",
index.project(appended, params); index.project(appended, params);
const deltaMs = performance.now() - deltaStart; const deltaMs = performance.now() - deltaStart;
console.log(`cache-hit average ${cacheAverageMs.toFixed(3)}ms; delta projection ${deltaMs.toFixed(3)}ms`);
assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`); assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`);
assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`); assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`);
}); });
+69 -40
View File
@@ -1,7 +1,6 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { DEFAULT_CONFIG } from "../src/config.ts"; import { SegmentIndex } from "../src/segment-index.ts";
import { SegmentIndex, deterministicSummary } from "../src/segment-index.ts";
const assistant = { const assistant = {
role: "assistant", role: "assistant",
@@ -17,66 +16,96 @@ const result = {
timestamp: 11, timestamp: 11,
}; };
test("deterministic summary is bounded and preserves head/tail evidence", () => { // Regression: a ~14.5K-char read result projected under a 1M window must reach
const summary = deterministicSummary({ // the model byte-for-byte unchanged — masking only happens via context_edit
toolName: "bash", // drafts under real pressure, never on the hot projection path.
args: { command: "npm test" }, test("projection is byte-for-byte identity for a 300-line read result under a 1M window", () => {
rawText: result.content[0]!.text, const readBody = Array.from({ length: 300 }, (_unused, index) => `line ${index + 1}: ${"y".repeat(38)}`).join("\n");
isError: true, const readResult = {
maxStubTokens: 256, role: "toolResult",
toolCallId: "call-read",
toolName: "read",
content: [{ type: "text", text: readBody }],
timestamp: 12,
};
assert.ok(readBody.length > 14_000 && readBody.length < 15_000, `fixture ${readBody.length} chars`);
const index = new SegmentIndex("session-1");
index.hydrate();
const snapshot = index.project([assistant, readResult], {
environmentHash: "env",
environmentTokens: 100,
contextWindow: 1_000_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
assert.match(summary, /tool=bash status=error/);
assert.match(summary, /failure/); assert.strictEqual(snapshot.messages[1], readResult);
assert.match(summary, /stack tail/); assert.strictEqual(snapshot.messages[1]!.content[0].text, readBody);
assert.ok(summary.length <= 1_100); assert.ok(snapshot.messages[1]!.content[0].text.includes("line 150:"));
assert.equal(snapshot.reclaimedTokens, 0);
assert.equal(snapshot.maskedResults, 0);
}); });
test("projection incrementally replaces tool results and caches identical input", () => { test("projection caches identical input and handles append-only deltas", () => {
const index = new SegmentIndex("session-1"); const index = new SegmentIndex("session-2");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const records = index.captureTurn(assistant, [result], DEFAULT_CONFIG);
assert.equal(records.length, 1);
const messages = [assistant, result]; const messages = [assistant, result];
const first = index.project(messages, { const params = { environmentHash: "env", environmentTokens: 100, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
environmentHash: "env", const first = index.project(messages, params);
environmentTokens: 100, const second = index.project(messages, params);
contextWindow: 300_000,
});
const second = index.project(messages, {
environmentHash: "env",
environmentTokens: 100,
contextWindow: 300_000,
});
assert.strictEqual(first, second); assert.strictEqual(first, second);
assert.ok(first.effectiveTokens < first.rawTokens); assert.ok(first.rawTokens > 0);
assert.match(first.messages[1]!.content[0].text, /context_tree_query/);
const record = records[0]!; const appended = index.project([...messages, { role: "user", content: "next", timestamp: 13 }], params);
assert.ok(index.applySemantic(record.sourceKey, record.sourceHash, "- npm test failed\n- stack tail")); assert.notStrictEqual(appended, first);
const semantic = index.project(messages, { assert.equal(appended.messages.length, 3);
assert.strictEqual(appended.messages[1], result);
});
test("changed masking stats publish a new snapshot without mutating the old one", () => {
const index = new SegmentIndex("session-stats");
index.hydrate();
const params = {
environmentHash: "env", environmentHash: "env",
environmentTokens: 100, environmentTokens: 100,
contextWindow: 300_000, contextWindow: 300_000,
}); maskedResults: 0,
assert.notStrictEqual(semantic, first); reclaimedTokens: 0,
assert.match(semantic.messages[1]!.content[0].text, /npm test failed/); };
const first = index.project([assistant, result], params);
const same = index.project([assistant, result], params);
assert.strictEqual(same, first);
const updated = index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 });
assert.notStrictEqual(updated, first);
assert.equal(updated.maskedResults, 1);
assert.equal(updated.reclaimedTokens, 500);
assert.equal(updated.rawTokens, updated.effectiveTokens + 500);
// The previously published object stays immutable.
assert.equal(first.maskedResults, 0);
assert.equal(first.reclaimedTokens, 0);
// Unchanged stats keep returning the same cached object.
assert.strictEqual(index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 }), updated);
}); });
test("usage-backed baseline is adjusted only by projected delta", () => { test("usage-backed baseline is adjusted only by projected delta", () => {
const index = new SegmentIndex("session-2"); const index = new SegmentIndex("session-3");
index.hydrate([], DEFAULT_CONFIG); index.hydrate();
const initial = index.project([assistant], { const initial = index.project([assistant], {
environmentHash: "env", environmentHash: "env",
environmentTokens: 50, environmentTokens: 50,
contextWindow: 100_000, contextWindow: 100_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
index.recordUsage(1_000, initial.messageEstimate, "env"); index.recordUsage(1_000, initial.messageEstimate, "env");
const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], { const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], {
environmentHash: "env", environmentHash: "env",
environmentTokens: 50, environmentTokens: 50,
contextWindow: 100_000, contextWindow: 100_000,
maskedResults: 0,
reclaimedTokens: 0,
}); });
assert.equal(next.confidence, "usage-backed"); assert.equal(next.confidence, "usage-backed");
assert.ok(next.effectiveTokens >= 1_000); assert.ok(next.effectiveTokens >= 1_000);
+11 -13
View File
@@ -1,19 +1,17 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { shouldCompact } from "../src/token-accounting.ts"; import { effectiveThreshold } from "../src/token-accounting.ts";
test("raw pressure cannot trigger compaction while effective lower-bound stays below 90%", () => { test("absolute token threshold wins on large windows", () => {
const config = { hardRatio: 0.9 }; assert.equal(effectiveThreshold(200_000, 0.6, 1_000_000), 200_000);
const snapshot = { lowerBound: 231_000, contextWindow: 300_000 };
assert.equal(shouldCompact("threshold", snapshot, config), false);
assert.equal(shouldCompact("manual", snapshot, config), true);
assert.equal(shouldCompact("overflow", snapshot, config), true);
}); });
test("effective lower-bound at the configured threshold compacts", () => { test("window ratio caps the threshold on small windows", () => {
assert.equal(shouldCompact( assert.equal(effectiveThreshold(200_000, 0.6, 272_000), 163_200);
"threshold", assert.equal(effectiveThreshold(100_000, 0.35, 272_000), 95_200);
{ lowerBound: 270_000, contextWindow: 300_000 }, });
{ hardRatio: 0.9 },
), true); test("a missing context window falls back to the absolute threshold", () => {
assert.equal(effectiveThreshold(200_000, 0.6, 0), 200_000);
assert.equal(effectiveThreshold(200_000, 0.6, -1), 200_000);
}); });
-31
View File
@@ -1,31 +0,0 @@
You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.
Available tools:
- read: Read file contents
- bash: Execute bash commands (ls, grep, find, etc.)
- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call
- write: Create or overwrite files
In addition to the tools above, you may have access to other custom tools depending on the project.
Guidelines:
- Use bash for file operations like ls, rg, find
- Use read to examine files instead of cat or sed.
- You can inspect PI_* environment variables for current model and session details.
- Use edit for precise changes (edits[].oldText must match exactly)
- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls
- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.
- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.
- Use write only for new files or complete rewrites.
- Be concise in your responses
- Show file paths clearly when working with files
Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):
- Main documentation: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/README.md
- Additional docs: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/docs
- Examples: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/examples (extensions, custom tools, SDK)
- When reading pi docs or examples, resolve docs/... under Additional docs and examples/... under Examples, not the current working directory
- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md), environment variables (docs/environment-variables.md)
- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing
- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)
Current working directory: /Users/sia/code/my-pi
+1 -1
View File
@@ -19,7 +19,7 @@
### 依赖 ### 依赖
- `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,与 Context Mode 使用的根 `better-sqlite3@12.11.1` 并存;因此根 `allowScripts` 无需新增条目,也不得删除既有的 `better-sqlite3@12.11.1` 放行。 - `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,根 `allowScripts` 无需任何 `better-sqlite3` 放行条目(context-mode 移除后根依赖已不再包含 better-sqlite3@12)。
- `strip-ansi@7.2.0`:解析到根 `node_modules`。 - `strip-ansi@7.2.0`:解析到根 `node_modules`。
### 本地改动 ### 本地改动
@@ -24,10 +24,10 @@ const emit = async (event, data = {}) => {
try { try {
process.env.PI_CODING_AGENT_DIR = root; process.env.PI_CODING_AGENT_DIR = root;
process.env.PI_HERMES_MEMORY_CONFIG_PATH = join(root, "hermes-memory-config.json"); const configPath = join(root, "hermes-memory-config.json");
await mkdir(globalDir); await mkdir(globalDir);
await mkdir(ctx.cwd); await mkdir(ctx.cwd);
await writeFile(process.env.PI_HERMES_MEMORY_CONFIG_PATH, JSON.stringify({ await writeFile(configPath, JSON.stringify({
lazyInitialization: lazy, memoryMode: "policy-only", lazyInitialization: lazy, memoryMode: "policy-only",
reviewEnabled: false, correctionDetection: false, reviewEnabled: false, correctionDetection: false,
flushOnShutdown: false, flushOnCompact: false, flushOnShutdown: false, flushOnCompact: false,
@@ -40,7 +40,10 @@ try {
} }
const started = performance.now(); const started = performance.now();
const { default: register } = await import(pathToFileURL(resolve(entry)).href); const module = await import(pathToFileURL(resolve(entry)).href);
const register = module.createHermesMemoryExtension
? module.createHermesMemoryExtension({ configPath })
: module.default;
const imported = performance.now(); const imported = performance.now();
register({ register({
on(event, handler) { on(event, handler) {
+1 -10
View File
@@ -89,16 +89,7 @@ export const DEFAULT_CONFIG_PATH = fileURLToPath(
new URL("../../config/pi-hermes-memory.json", import.meta.url), new URL("../../config/pi-hermes-memory.json", import.meta.url),
); );
/** export function loadConfig(configPath = DEFAULT_CONFIG_PATH): MemoryConfig {
* Resolve the effective config path. `PI_HERMES_MEMORY_CONFIG_PATH` relocates
* the file (tests point it at a temp file; it is never a persisted config
* value).
*/
export function resolveConfigPath(): string {
return process.env.PI_HERMES_MEMORY_CONFIG_PATH ?? DEFAULT_CONFIG_PATH;
}
export function loadConfig(configPath = resolveConfigPath()): MemoryConfig {
try { try {
if (fs.existsSync(configPath)) { if (fs.existsSync(configPath)) {
const raw = fs.readFileSync(configPath, "utf-8"); const raw = fs.readFileSync(configPath, "utf-8");
+16 -3
View File
@@ -52,7 +52,7 @@ import { registerPreviewContextCommand } from "./handlers/preview-context.js";
import { registerStandingPinCommand } from "./handlers/standing-pin.js"; import { registerStandingPinCommand } from "./handlers/standing-pin.js";
import { StandingInstructions } from "./store/standing-instructions.js"; import { StandingInstructions } from "./store/standing-instructions.js";
import { STANDING_FILE } from "./constants.js"; import { STANDING_FILE } from "./constants.js";
import { loadConfig } from "./config.js"; import { DEFAULT_CONFIG_PATH, loadConfig } from "./config.js";
import { shouldWarnAutoConsolidationFailure } from "./auto-consolidation-warning.js"; import { shouldWarnAutoConsolidationFailure } from "./auto-consolidation-warning.js";
import { detectProject, detectProjectSkills } from "./project.js"; import { detectProject, detectProjectSkills } from "./project.js";
import { buildPromptContext } from "./prompt-context.js"; import { buildPromptContext } from "./prompt-context.js";
@@ -90,8 +90,12 @@ export function registerProjectSkillDiscoveryHandler(
}); });
} }
export default function (pi: ExtensionAPI) { export interface HermesMemoryExtensionOptions {
const config = loadConfig(); configPath?: string;
}
function hermesMemoryExtension(pi: ExtensionAPI, configPath: string) {
const config = loadConfig(configPath);
const lazy = config.lazyInitialization === true && config.memoryMode === "policy-only"; const lazy = config.lazyInitialization === true && config.memoryMode === "policy-only";
let sessionContext: ExtensionContext | undefined; let sessionContext: ExtensionContext | undefined;
@@ -480,3 +484,12 @@ export default function (pi: ExtensionAPI) {
} }
}); });
} }
export function createHermesMemoryExtension(
options?: HermesMemoryExtensionOptions,
): (pi: ExtensionAPI) => void {
const configPath = options?.configPath ?? DEFAULT_CONFIG_PATH;
return (pi) => hermesMemoryExtension(pi, configPath);
}
export default createHermesMemoryExtension();
+4 -6
View File
@@ -8,10 +8,10 @@ import { after, beforeEach, afterEach, describe, it } from "node:test";
// Bind the entry point to a disposable agent root, never the user's memory. // Bind the entry point to a disposable agent root, never the user's memory.
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-lazy-startup-")); const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-lazy-startup-"));
const previousRoot = process.env.PI_CODING_AGENT_DIR; const previousRoot = process.env.PI_CODING_AGENT_DIR;
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH;
process.env.PI_CODING_AGENT_DIR = root; process.env.PI_CODING_AGENT_DIR = root;
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(root, "hermes-memory-config.json"); const configPath = path.join(root, "hermes-memory-config.json");
const { default: registerExtension } = await import("../src/index.js"); const { createHermesMemoryExtension } = await import("../src/index.js");
const registerExtension = createHermesMemoryExtension({ configPath });
const { MemoryStore } = await import("../src/store/memory-store.js"); const { MemoryStore } = await import("../src/store/memory-store.js");
const { DatabaseManager } = await import("../src/store/db.js"); const { DatabaseManager } = await import("../src/store/db.js");
const globalDir = path.join(root, "pi-hermes-memory"); const globalDir = path.join(root, "pi-hermes-memory");
@@ -24,7 +24,7 @@ let notifications: string[];
let ctx: any; let ctx: any;
async function configure(overrides: Record<string, unknown> = {}) { async function configure(overrides: Record<string, unknown> = {}) {
await fs.writeFile(path.join(root, "hermes-memory-config.json"), JSON.stringify({ await fs.writeFile(configPath, JSON.stringify({
lazyInitialization: true, lazyInitialization: true,
reviewEnabled: false, correctionDetection: false, reviewEnabled: false, correctionDetection: false,
flushOnCompact: false, flushOnShutdown: false, flushOnCompact: false, flushOnShutdown: false,
@@ -87,8 +87,6 @@ afterEach(async () => {
after(async () => { after(async () => {
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR; if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
else process.env.PI_CODING_AGENT_DIR = previousRoot; else process.env.PI_CODING_AGENT_DIR = previousRoot;
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
await fs.rm(root, { recursive: true, force: true }); await fs.rm(root, { recursive: true, force: true });
}); });
@@ -18,11 +18,11 @@ describe("session project memory rebinding", () => {
const launchDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-launch-")); const launchDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-launch-"));
const targetDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-target-")); const targetDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-target-"));
const previousAgentRoot = process.env.PI_CODING_AGENT_DIR; const previousAgentRoot = process.env.PI_CODING_AGENT_DIR;
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH; const configPath = path.join(agentRoot, "hermes-memory-config.json");
const previousCwd = process.cwd(); const previousCwd = process.cwd();
try { try {
await fs.writeFile( await fs.writeFile(
path.join(agentRoot, "hermes-memory-config.json"), configPath,
JSON.stringify({ JSON.stringify({
memoryMode: "legacy-inject", memoryMode: "legacy-inject",
reviewEnabled: false, reviewEnabled: false,
@@ -47,10 +47,10 @@ describe("session project memory rebinding", () => {
); );
process.env.PI_CODING_AGENT_DIR = agentRoot; process.env.PI_CODING_AGENT_DIR = agentRoot;
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(agentRoot, "hermes-memory-config.json");
process.chdir(launchDir); process.chdir(launchDir);
// Import after setting PI_CODING_AGENT_DIR so AGENT_ROOT is test-local. // Import after setting PI_CODING_AGENT_DIR so AGENT_ROOT is test-local.
const { default: registerExtension } = await import("../src/index.js"); const { createHermesMemoryExtension } = await import("../src/index.js");
const registerExtension = createHermesMemoryExtension({ configPath });
const mockPi: MockPi = { const mockPi: MockPi = {
handlers: {}, handlers: {},
tools: {}, tools: {},
@@ -103,8 +103,6 @@ describe("session project memory rebinding", () => {
process.chdir(previousCwd); process.chdir(previousCwd);
if (previousAgentRoot === undefined) delete process.env.PI_CODING_AGENT_DIR; if (previousAgentRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
else process.env.PI_CODING_AGENT_DIR = previousAgentRoot; else process.env.PI_CODING_AGENT_DIR = previousAgentRoot;
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
await fs.rm(agentRoot, { recursive: true, force: true }); await fs.rm(agentRoot, { recursive: true, force: true });
await fs.rm(launchDir, { recursive: true, force: true }); await fs.rm(launchDir, { recursive: true, force: true });
await fs.rm(targetDir, { recursive: true, force: true }); await fs.rm(targetDir, { recursive: true, force: true });
@@ -24,10 +24,10 @@ import { after, beforeEach, describe, it } from "node:test";
// Bind the entry point to a disposable agent root, never the user's memory. // Bind the entry point to a disposable agent root, never the user's memory.
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-prompt-injection-")); const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-prompt-injection-"));
const previousRoot = process.env.PI_CODING_AGENT_DIR; const previousRoot = process.env.PI_CODING_AGENT_DIR;
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH;
process.env.PI_CODING_AGENT_DIR = root; process.env.PI_CODING_AGENT_DIR = root;
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(root, "hermes-memory-config.json"); const configPath = path.join(root, "hermes-memory-config.json");
const { default: registerExtension } = await import("../src/index.js"); const { createHermesMemoryExtension } = await import("../src/index.js");
const registerExtension = createHermesMemoryExtension({ configPath });
const globalDir = path.join(root, "pi-hermes-memory"); const globalDir = path.join(root, "pi-hermes-memory");
const cwd = path.join(root, "workspace"); const cwd = path.join(root, "workspace");
@@ -68,7 +68,7 @@ let handlers: Record<string, Array<(event: any, ctx: any) => any>>;
let ctx: any; let ctx: any;
async function configure(overrides: Record<string, unknown> = {}) { async function configure(overrides: Record<string, unknown> = {}) {
await fs.writeFile(path.join(root, "hermes-memory-config.json"), JSON.stringify({ await fs.writeFile(configPath, JSON.stringify({
memoryMode: "policy-only", memoryMode: "policy-only",
memoryPolicyStyle: "full", memoryPolicyStyle: "full",
lazyInitialization: true, lazyInitialization: true,
@@ -115,8 +115,6 @@ beforeEach(async () => {
after(async () => { after(async () => {
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR; if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
else process.env.PI_CODING_AGENT_DIR = previousRoot; else process.env.PI_CODING_AGENT_DIR = previousRoot;
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
try { try {
await fs.rm(root, { recursive: true, force: true }); await fs.rm(root, { recursive: true, force: true });
} catch { } catch {
+10
View File
@@ -0,0 +1,10 @@
# my-pi 维护约定(pi-minimal-footer)
- 扩展入口是 `pi-minimal-footer/index.ts`(`package.json` 的 `pi.extensions` 指向 `./index.ts`);`status.ts` 收集 `ctx.ui.setStatus()` 发布的非空扩展状态并渲染为 footer 片段,`settings.ts` 读取 footer 配置,`assets/` 为 provider 图标,测试在 `test/`。
- 持久配置只读组合包 `config/pi-minimal-footer.json`(`settings.ts` 的 `readFooterSettings` 解析 `showCwd` / `showBranch`,缺失或非法时回退默认 `true`);不读写 `settings.json`,不以环境变量作持久默认。
- 上下文仪表优先显示 pi-compaction 发布的 effective usage 快照(`globalThis[Symbol.for("my-pi.pi-compaction.service.v1")].getMetrics(sessionId)` 的 `percent` / `effectiveTokens`);无快照时回退宿主用量(`buildSessionContext` 取最后一条非 aborted assistant 消息的 `input + output + cacheRead + cacheWrite`)。
- 来源记录在 `UPSTREAM.md`:从 `ogulcancelik/pi-extensions` 的 `pi-minimal-footer@0.1.10`(commit `77bd1e1`)导入,保留上游 LICENSE 与作者;本地改动为扩展状态渲染与 bundle 配置文件读取。
## 验证
- 在本目录内运行 `npm test`(`node --experimental-strip-types --test test/*.test.ts`)与 `npm run check`(`node --experimental-strip-types --check index.ts`)。
+6 -6
View File
@@ -34,14 +34,14 @@ pi install npm:@ogulcancelik/pi-minimal-footer
## Configuration ## Configuration
Environment variables (all optional): Persistent defaults are read from `config/pi-minimal-footer.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
| Variable | Description | Default | | Field | Description | Default |
| ------------------------------- | -------------------------------------------------------- | ------- | | ------------ | -------------------------------------------------------- | ------- |
| `PI_MINIMAL_FOOTER_SHOW_CWD` | Show current working directory in footer status line | `1` | | `showCwd` | Show current working directory in footer status line | `true` |
| `PI_MINIMAL_FOOTER_SHOW_BRANCH` | Show git branch/dirty/ahead/behind in footer status line | `1` | | `showBranch` | Show git branch/dirty/ahead/behind in footer status line | `true` |
Accepted false values: `0`, `false`, `no`, `off` (case-insensitive). Only `true`/`false` values are honoured; anything else falls back to the default for that field.
## How it works ## How it works
+1 -1
View File
@@ -5,4 +5,4 @@
- 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10` - 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10`
- 导入日期:2026-08-19 - 导入日期:2026-08-19
本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段)。 本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段),并把 showCwd/showBranch 持久配置由 `PI_MINIMAL_FOOTER_*` 环境变量改为组合包 `config/pi-minimal-footer.json`。
+3 -16
View File
@@ -15,6 +15,7 @@ import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path"; import { join } from "node:path";
import { homedir } from "node:os"; import { homedir } from "node:os";
import { collectFooterStatusItems } from "./status.ts"; import { collectFooterStatusItems } from "./status.ts";
import { readFooterSettings } from "./settings.ts";
// ============ Types ============ // ============ Types ============
@@ -43,19 +44,6 @@ interface GitCache {
const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes
const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider
// ============ Env Flags ============
function parseBooleanEnv(value: string | undefined, fallback: boolean): boolean {
if (value === undefined) return fallback;
const normalized = value.trim().toLowerCase();
if (!normalized) return fallback;
if (["1", "true", "yes", "on"].includes(normalized)) return true;
if (["0", "false", "no", "off"].includes(normalized)) return false;
return fallback;
}
// ============ Identity Cache ============ // ============ Identity Cache ============
let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved
@@ -754,9 +742,8 @@ export default function (pi: ExtensionAPI) {
const BAR_FILLED = "━"; const BAR_FILLED = "━";
const BAR_EMPTY = "─"; const BAR_EMPTY = "─";
// Optional visibility toggles (default: enabled) // Optional visibility toggles from bundle config (default: enabled)
const showCwd = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_CWD, true); const { showCwd, showBranch } = readFooterSettings();
const showBranch = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_BRANCH, true);
function formatTokenCount(tokens: number): string { function formatTokenCount(tokens: number): string {
if (tokens >= 1_000_000) { if (tokens >= 1_000_000) {
+30
View File
@@ -0,0 +1,30 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
export interface FooterSettings {
showCwd: boolean;
showBranch: boolean;
}
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-minimal-footer.json` from inside the installed bundle (resolved
// relative to this source file), never environment variables.
export const FOOTER_CONFIG_PATH = fileURLToPath(
new URL("../config/pi-minimal-footer.json", import.meta.url),
);
export function readFooterSettings(configPath: string = FOOTER_CONFIG_PATH): FooterSettings {
try {
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
const raw =
parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: {};
return {
showCwd: typeof raw.showCwd === "boolean" ? raw.showCwd : true,
showBranch: typeof raw.showBranch === "boolean" ? raw.showBranch : true,
};
} catch {
return { showCwd: true, showBranch: true };
}
}
+51
View File
@@ -0,0 +1,51 @@
import assert from "node:assert/strict";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, it } from "node:test";
import { readFooterSettings } from "../settings.ts";
let tempDir: string | undefined;
afterEach(() => {
if (tempDir) {
rmSync(tempDir, { recursive: true, force: true });
tempDir = undefined;
}
});
function configPath(contents: string): string {
tempDir = mkdtempSync(join(tmpdir(), "pi-minimal-footer-settings-"));
const path = join(tempDir, "pi-minimal-footer.json");
writeFileSync(path, contents);
return path;
}
const expectedDefaults = { showCwd: true, showBranch: true };
describe("readFooterSettings", () => {
it("returns all defaults when the config file is missing", () => {
assert.deepEqual(readFooterSettings("/does/not/exist/pi-minimal-footer.json"), expectedDefaults);
});
it("returns all defaults for malformed or non-object JSON", () => {
for (const contents of ["not json", "[1,2]", "42"]) {
assert.deepEqual(readFooterSettings(configPath(contents)), expectedDefaults, contents);
}
});
it("adopts explicit boolean values", () => {
assert.deepEqual(
readFooterSettings(configPath(JSON.stringify({ showCwd: false, showBranch: false }))),
{ showCwd: false, showBranch: false },
);
});
it("falls back per-field for non-boolean values", () => {
assert.deepEqual(
readFooterSettings(configPath(JSON.stringify({ showCwd: "no", showBranch: 0 }))),
expectedDefaults,
);
});
});
+1 -1
View File
@@ -5,7 +5,7 @@
- Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。 - Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。
- 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。 - 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。
- 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。 - 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。
- `/notify on|off` 只覆盖当前 Session;持久默认来自 `PI_NOTIFY_*` 环境变量。`PI_NOTIFY_MESSAGE_SOURCE=none` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。 - `/notify on|off` 只覆盖当前 Session;持久默认来自组合包 `config/pi-notify.json`(`src/settings.ts` 读取,文件缺失或字段非法逐字段回退默认)。`messageSource: "none"` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。
- 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。 - 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。
## 验证 ## 验证
+19 -15
View File
@@ -28,19 +28,21 @@ Do not separately install upstream when using the bundle, or notifications will
- `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus. - `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus.
- `/notify status`: show the active backend, visibility policy, and duration threshold. - `/notify status`: show the active backend, visibility policy, and duration threshold.
Command overrides are session-local. Use environment variables for persistent defaults. Command overrides are session-local. Persistent defaults come from the bundle's `config/pi-notify.json`.
## Configuration ## Configuration
- `PI_NOTIFY_ENABLED`: `1`/`true` or `0`/`false`; default enabled. Persistent defaults are read once per session from `config/pi-notify.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
- `PI_NOTIFY_TITLE`: title template shared by completion and permission notifications; default `π - {project}`.
- `PI_NOTIFY_BODY`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary. - `enabled`: `true`/`false`; default `true`.
- `PI_NOTIFY_MESSAGE_SOURCE`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details. - `title`: title template shared by completion and permission notifications; default `π - {project}`.
- `PI_NOTIFY_MESSAGE_MAX`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`. - `body`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary.
- `PI_NOTIFY_MIN_SECONDS`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts. - `messageSource`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details.
- `PI_NOTIFY_VISIBILITY`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`. - `messageMax`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`.
- `PI_NOTIFY_ACTION`: Kitty click action `focus` or `none`; default `focus`. - `minSeconds`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts.
- `PI_NOTIFY_SOUND_CMD`: optional shell command run after the notification. This is trusted local configuration and executes through a shell. - `visibility`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`.
- `action`: Kitty click action `focus` or `none`; default `focus`.
- `soundCommand`: optional shell command run after the notification. This is trusted local configuration and executes through a shell.
Visibility policies: Visibility policies:
@@ -70,12 +72,14 @@ The active tmux client type is checked so a stale `KITTY_WINDOW_ID` cannot force
- `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict. - `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict.
- `{tool_error_count}`: observed tool error count. - `{tool_error_count}`: observed tool error count.
Example for multiple projects without exposing assistant text on the lock screen: Example for multiple projects without exposing assistant text on the lock screen (`config/pi-notify.json`):
```bash ```json
export PI_NOTIFY_TITLE='π {status_icon} · {project}' {
export PI_NOTIFY_MESSAGE_SOURCE=none "title": "π {status_icon} · {project}",
export PI_NOTIFY_VISIBILITY=unfocused "messageSource": "none",
"visibility": "unfocused"
}
``` ```
Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications. Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications.
+1
View File
@@ -18,5 +18,6 @@
- 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。 - 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。
- 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。 - 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。
- 新增生命周期、多窗口元数据、回退、解析器和配置测试。 - 新增生命周期、多窗口元数据、回退、解析器和配置测试。
- 持久配置由 `PI_NOTIFY_*` 环境变量改为组合包 `config/pi-notify.json`(`src/settings.ts` 读取并逐字段回退默认值)。
上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。 上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。
+11 -17
View File
@@ -1,13 +1,6 @@
import type { ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import { extractMessageText, truncateText } from "./messages.ts"; import { extractMessageText, truncateText } from "./messages.ts";
import { import { readNotifySettings, type NotifySettings } from "./settings.ts";
parseEnabled,
parseKittyVisibility,
parseMessageMax,
parseMessageSource,
parseMinDurationMs,
parseNotificationAction,
} from "./parsers.ts";
import { renderTemplate } from "./template.ts"; import { renderTemplate } from "./template.ts";
import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts"; import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts";
@@ -20,12 +13,13 @@ export function getConfig(
messages: unknown[] | undefined, messages: unknown[] | undefined,
runtime: NotificationRuntimeValues, runtime: NotificationRuntimeValues,
now = Date.now(), now = Date.now(),
settings: NotifySettings = readNotifySettings(),
): NotifyConfig { ): NotifyConfig {
const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt); const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt);
const messageMax = parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX); const messageMax = settings.messageMax;
const userMessage = truncateText(extractMessageText(messages, "user"), messageMax); const userMessage = truncateText(extractMessageText(messages, "user"), messageMax);
const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax); const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax);
const messageSource = parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE); const messageSource = settings.messageSource;
const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : ""; const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : "";
const message = selectedMessage || "Ready for input"; const message = selectedMessage || "Ready for input";
const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0); const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0);
@@ -39,15 +33,15 @@ export function getConfig(
}; };
return { return {
enabled: parseEnabled(process.env.PI_NOTIFY_ENABLED), enabled: settings.enabled,
title: renderTemplate(process.env.PI_NOTIFY_TITLE || DEFAULT_TITLE, ctx, durationMs, values), title: renderTemplate(settings.title || DEFAULT_TITLE, ctx, durationMs, values),
body: renderTemplate(process.env.PI_NOTIFY_BODY || DEFAULT_BODY, ctx, durationMs, values), body: renderTemplate(settings.body || DEFAULT_BODY, ctx, durationMs, values),
durationMs, durationMs,
minDurationMs: parseMinDurationMs(process.env.PI_NOTIFY_MIN_SECONDS), minDurationMs: settings.minDurationMs,
notificationId: runtime.notificationId, notificationId: runtime.notificationId,
visibility: parseKittyVisibility(process.env.PI_NOTIFY_VISIBILITY), visibility: settings.visibility,
action: parseNotificationAction(process.env.PI_NOTIFY_ACTION), action: settings.action,
soundCommand: process.env.PI_NOTIFY_SOUND_CMD?.trim() || undefined, soundCommand: settings.soundCommand,
}; };
} }
+10 -7
View File
@@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { getConfig, shouldNotify } from "./config.ts"; import { getConfig, shouldNotify } from "./config.ts";
import { truncateText } from "./messages.ts"; import { truncateText } from "./messages.ts";
import { parseMessageMax, parseMessageSource } from "./parsers.ts"; import { readNotifySettings, type NotifySettings } from "./settings.ts";
import { getNotificationBackend, sendNotification } from "./terminal.ts"; import { getNotificationBackend, sendNotification } from "./terminal.ts";
import type { NotifyConfig } from "./types.ts"; import type { NotifyConfig } from "./types.ts";
@@ -11,13 +11,14 @@ type ExtensionDependencies = {
createNotificationId: () => string; createNotificationId: () => string;
notify: (config: NotifyConfig) => void; notify: (config: NotifyConfig) => void;
getBackend: typeof getNotificationBackend; getBackend: typeof getNotificationBackend;
settings: NotifySettings;
}; };
const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt"; const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt";
function formatPermissionPromptBody(raw: unknown): string | undefined { function formatPermissionPromptBody(raw: unknown, settings: NotifySettings): string | undefined {
if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined; if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined;
if (parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE) === "none") { if (settings.messageSource === "none") {
return "Permission confirmation required"; return "Permission confirmation required";
} }
@@ -29,7 +30,7 @@ function formatPermissionPromptBody(raw: unknown): string | undefined {
if (!value) return labels.join(" · "); if (!value) return labels.join(" · ");
const normalizedValue = value.replace(/\s+/g, " ").trim(); const normalizedValue = value.replace(/\s+/g, " ").trim();
return `${labels.join(" · ")}\n${truncateText(normalizedValue, parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX))}`; return `${labels.join(" · ")}\n${truncateText(normalizedValue, settings.messageMax)}`;
} }
function readNonEmptyString(value: unknown): string | undefined { function readNonEmptyString(value: unknown): string | undefined {
@@ -47,6 +48,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`); const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`);
const notify = dependencies.notify ?? sendNotification; const notify = dependencies.notify ?? sendNotification;
const getBackend = dependencies.getBackend ?? getNotificationBackend; const getBackend = dependencies.getBackend ?? getNotificationBackend;
const settings = dependencies.settings ?? readNotifySettings();
return function piNotify(pi: ExtensionAPI): void { return function piNotify(pi: ExtensionAPI): void {
const notificationId = createNotificationId(); const notificationId = createNotificationId();
@@ -81,7 +83,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
pi.on("agent_settled", (_event, ctx) => { pi.on("agent_settled", (_event, ctx) => {
try { try {
const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now()); const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now(), settings);
if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config); if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config);
} finally { } finally {
resetRun(); resetRun();
@@ -89,7 +91,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
}); });
const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => { const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => {
const body = formatPermissionPromptBody(raw); const body = formatPermissionPromptBody(raw, settings);
if (!body || currentCtx?.mode !== "tui") return; if (!body || currentCtx?.mode !== "tui") return;
const config = getConfig( const config = getConfig(
@@ -98,6 +100,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
undefined, undefined,
{ notificationId, toolErrorCount }, { notificationId, toolErrorCount },
now(), now(),
settings,
); );
const permissionConfig = { ...config, body, minDurationMs: 0 }; const permissionConfig = { ...config, body, minDurationMs: 0 };
if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig); if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig);
@@ -119,7 +122,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
return; return;
} }
const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now()); const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now(), settings);
if (action === "test") { if (action === "test") {
if (ctx.mode !== "tui") { if (ctx.mode !== "tui") {
ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning"); ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning");
+63
View File
@@ -0,0 +1,63 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import {
parseEnabled,
parseKittyVisibility,
parseMessageMax,
parseMessageSource,
parseMinDurationMs,
parseNotificationAction,
} from "./parsers.ts";
import type { KittyVisibility, MessageSource, NotificationAction } from "./types.ts";
export interface NotifySettings {
enabled: boolean;
title: string;
body: string;
messageSource: MessageSource;
messageMax: number;
minDurationMs: number;
visibility: KittyVisibility;
action: NotificationAction;
soundCommand?: string;
}
const DEFAULT_TITLE = "π - {project}";
const DEFAULT_BODY = "{message}";
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-notify.json` from inside the installed bundle (resolved relative
// to this source file), never environment variables.
export const NOTIFY_CONFIG_PATH = fileURLToPath(new URL("../../config/pi-notify.json", import.meta.url));
function readConfigFile(configPath: string): Record<string, unknown> {
try {
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
return parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: {};
} catch {
return {};
}
}
function optionalString(value: unknown): string | undefined {
if (typeof value === "string") return value;
if (typeof value === "number" || typeof value === "boolean") return String(value);
return undefined;
}
export function readNotifySettings(configPath: string = NOTIFY_CONFIG_PATH): NotifySettings {
const raw = readConfigFile(configPath);
return {
enabled: parseEnabled(optionalString(raw.enabled)),
title: typeof raw.title === "string" && raw.title ? raw.title : DEFAULT_TITLE,
body: typeof raw.body === "string" && raw.body ? raw.body : DEFAULT_BODY,
messageSource: parseMessageSource(optionalString(raw.messageSource)),
messageMax: parseMessageMax(optionalString(raw.messageMax)),
minDurationMs: parseMinDurationMs(optionalString(raw.minSeconds)),
visibility: parseKittyVisibility(optionalString(raw.visibility)),
action: parseNotificationAction(optionalString(raw.action)),
soundCommand: typeof raw.soundCommand === "string" ? raw.soundCommand.trim() || undefined : undefined,
};
}
+24 -34
View File
@@ -1,28 +1,18 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { afterEach, describe, it } from "node:test"; import { describe, it } from "node:test";
import type { ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
import { getConfig, shouldNotify } from "../src/config.ts"; import { getConfig, shouldNotify } from "../src/config.ts";
import type { NotifySettings } from "../src/settings.ts";
const envKeys = [ const defaultSettings = (): NotifySettings => ({
"PI_NOTIFY_ENABLED", enabled: true,
"PI_NOTIFY_TITLE", title: "π - {project}",
"PI_NOTIFY_BODY", body: "{message}",
"PI_NOTIFY_MESSAGE_SOURCE", messageSource: "assistant",
"PI_NOTIFY_MESSAGE_MAX", messageMax: 80,
"PI_NOTIFY_MIN_SECONDS", minDurationMs: 3_000,
"PI_NOTIFY_VISIBILITY", visibility: "always",
"PI_NOTIFY_ACTION", action: "focus",
"PI_NOTIFY_SOUND_CMD",
] as const;
const savedEnv = new Map(envKeys.map((key) => [key, process.env[key]]));
afterEach(() => {
for (const key of envKeys) {
const value = savedEnv.get(key);
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
}); });
function context(): ExtensionContext { function context(): ExtensionContext {
@@ -45,6 +35,7 @@ describe("config", () => {
], ],
runtime, runtime,
5_000, 5_000,
defaultSettings(),
); );
assert.deepEqual(config, { assert.deepEqual(config, {
@@ -62,19 +53,20 @@ describe("config", () => {
}); });
it("supports privacy, threshold, focus, and status template settings", () => { it("supports privacy, threshold, focus, and status template settings", () => {
process.env.PI_NOTIFY_ENABLED = "off";
process.env.PI_NOTIFY_TITLE = "{status_icon} {status} {tool_error_count}";
process.env.PI_NOTIFY_BODY = "{message}";
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none";
process.env.PI_NOTIFY_MIN_SECONDS = "10.5";
process.env.PI_NOTIFY_VISIBILITY = "invisible";
process.env.PI_NOTIFY_ACTION = "none";
process.env.PI_NOTIFY_SOUND_CMD = " say done ";
const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], { const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], {
notificationId: "pi-session", notificationId: "pi-session",
toolErrorCount: 2, toolErrorCount: 2,
}, 2_000); }, 2_000, {
...defaultSettings(),
enabled: false,
title: "{status_icon} {status} {tool_error_count}",
body: "{message}",
messageSource: "none",
minDurationMs: 10_500,
visibility: "invisible",
action: "none",
soundCommand: "say done",
});
assert.equal(config.title, "⚠ ready · 2 tool errors 2"); assert.equal(config.title, "⚠ ready · 2 tool errors 2");
assert.equal(config.body, "Ready for input"); assert.equal(config.body, "Ready for input");
@@ -88,15 +80,13 @@ describe("config", () => {
}); });
it("applies selected-message truncation and a session override", () => { it("applies selected-message truncation and a session override", () => {
process.env.PI_NOTIFY_MESSAGE_SOURCE = "user";
process.env.PI_NOTIFY_MESSAGE_MAX = "5";
process.env.PI_NOTIFY_MIN_SECONDS = "0";
const config = getConfig( const config = getConfig(
context(), context(),
1_000, 1_000,
[{ role: "user", content: "hello world" }], [{ role: "user", content: "hello world" }],
runtime, runtime,
2_000, 2_000,
{ ...defaultSettings(), messageSource: "user", messageMax: 5, minDurationMs: 0 },
); );
assert.equal(config.body, "hell…"); assert.equal(config.body, "hell…");
+17 -18
View File
@@ -1,23 +1,24 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { afterEach, describe, it } from "node:test"; import { describe, it } from "node:test";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent"; import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { createPiNotifyExtension } from "../src/index.ts"; import { createPiNotifyExtension } from "../src/index.ts";
import type { NotifySettings } from "../src/settings.ts";
import type { NotifyConfig } from "../src/types.ts"; import type { NotifyConfig } from "../src/types.ts";
type Handler = (...args: any[]) => any; type Handler = (...args: any[]) => any;
const savedEnv = new Map( const DEFAULT_SETTINGS: NotifySettings = {
["PI_NOTIFY_MIN_SECONDS", "PI_NOTIFY_MESSAGE_SOURCE"].map((key) => [key, process.env[key]]), enabled: true,
); title: "π - {project}",
body: "{message}",
messageSource: "assistant",
messageMax: 80,
minDurationMs: 3_000,
visibility: "always",
action: "focus",
};
afterEach(() => { function harness(mode: ExtensionContext["mode"] = "tui", settings: Partial<NotifySettings> = {}) {
for (const [key, value] of savedEnv) {
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
});
function harness(mode: ExtensionContext["mode"] = "tui") {
const handlers = new Map<string, Handler>(); const handlers = new Map<string, Handler>();
const eventHandlers = new Map<string, Handler>(); const eventHandlers = new Map<string, Handler>();
const commands = new Map<string, { handler: Handler }>(); const commands = new Map<string, { handler: Handler }>();
@@ -45,6 +46,7 @@ function harness(mode: ExtensionContext["mode"] = "tui") {
createNotificationId: () => "pi-session-test", createNotificationId: () => "pi-session-test",
notify: (config) => notifications.push(config), notify: (config) => notifications.push(config),
getBackend: () => "kitty", getBackend: () => "kitty",
settings: { ...DEFAULT_SETTINGS, ...settings },
})(pi); })(pi);
const ctx = { const ctx = {
@@ -93,9 +95,8 @@ describe("pi-notify extension", () => {
}); });
it("does not write terminal notifications in RPC or non-interactive modes", async () => { it("does not write terminal notifications in RPC or non-interactive modes", async () => {
process.env.PI_NOTIFY_MIN_SECONDS = "0";
for (const mode of ["rpc", "json", "print"] as const) { for (const mode of ["rpc", "json", "print"] as const) {
const h = harness(mode); const h = harness(mode, { minDurationMs: 0 });
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
await h.handlers.get("agent_start")?.({}, h.ctx); await h.handlers.get("agent_start")?.({}, h.ctx);
await h.handlers.get("agent_settled")?.({}, h.ctx); await h.handlers.get("agent_settled")?.({}, h.ctx);
@@ -123,8 +124,7 @@ describe("pi-notify extension", () => {
}); });
it("notifies immediately when permission confirmation needs human input", async () => { it("notifies immediately when permission confirmation needs human input", async () => {
process.env.PI_NOTIFY_MIN_SECONDS = "3600"; const h = harness("tui", { minDurationMs: 3_600_000 });
const h = harness();
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
h.eventHandlers.get("permissions:ui_prompt")?.({ h.eventHandlers.get("permissions:ui_prompt")?.({
@@ -144,8 +144,7 @@ describe("pi-notify extension", () => {
}); });
it("keeps permission details private and honors the session off override", async () => { it("keeps permission details private and honors the session off override", async () => {
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none"; const h = harness("tui", { messageSource: "none" });
const h = harness();
await h.handlers.get("session_start")?.({}, h.ctx); await h.handlers.get("session_start")?.({}, h.ctx);
const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" }; const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" };
+107
View File
@@ -0,0 +1,107 @@
import assert from "node:assert/strict";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, it } from "node:test";
import { readNotifySettings } from "../src/settings.ts";
let tempDir: string | undefined;
afterEach(() => {
if (tempDir) {
rmSync(tempDir, { recursive: true, force: true });
tempDir = undefined;
}
});
function configPath(contents: string): string {
tempDir = mkdtempSync(join(tmpdir(), "pi-notify-settings-"));
const path = join(tempDir, "pi-notify.json");
writeFileSync(path, contents);
return path;
}
const expectedDefaults = {
enabled: true,
title: "π - {project}",
body: "{message}",
messageSource: "assistant",
messageMax: 80,
minDurationMs: 3_000,
visibility: "always",
action: "focus",
soundCommand: undefined,
};
describe("readNotifySettings", () => {
it("returns all defaults when the config file is missing", () => {
assert.deepEqual(readNotifySettings("/does/not/exist/pi-notify.json"), expectedDefaults);
});
it("returns all defaults for malformed or non-object JSON", () => {
for (const contents of ["not json", "[1,2]", "\"hi\"", "42"]) {
assert.deepEqual(readNotifySettings(configPath(contents)), expectedDefaults, contents);
}
});
it("adopts valid values from the config file", () => {
const settings = readNotifySettings(configPath(JSON.stringify({
enabled: false,
title: "π {status_icon} · {project}",
body: "{status}",
messageSource: "none",
messageMax: 120,
minSeconds: 30,
visibility: "unfocused",
action: "none",
soundCommand: "afplay /done.aiff",
})));
assert.deepEqual(settings, {
enabled: false,
title: "π {status_icon} · {project}",
body: "{status}",
messageSource: "none",
messageMax: 120,
minDurationMs: 30_000,
visibility: "unfocused",
action: "none",
soundCommand: "afplay /done.aiff",
});
});
it("falls back per-field when values have the wrong type or value", () => {
const settings = readNotifySettings(configPath(JSON.stringify({
enabled: {},
title: "",
body: null,
messageSource: ["assistant"],
messageMax: "bogus",
minSeconds: "bogus",
visibility: "bogus",
action: {},
})));
assert.deepEqual(settings, expectedDefaults);
});
it("clamps messageMax and minSeconds to their bounded ranges", () => {
const settings = readNotifySettings(configPath(JSON.stringify({ messageMax: 999, minSeconds: 4000 })));
assert.equal(settings.messageMax, 500);
assert.equal(settings.minDurationMs, 3_600_000);
const floor = readNotifySettings(configPath(JSON.stringify({ messageMax: -5, minSeconds: -1 })));
assert.equal(floor.messageMax, 0);
assert.equal(floor.minDurationMs, 0);
});
it("treats an empty or whitespace soundCommand as absent and trims values", () => {
for (const value of ["", " "]) {
assert.equal(readNotifySettings(configPath(JSON.stringify({ soundCommand: value }))).soundCommand, undefined);
}
assert.equal(
readNotifySettings(configPath(JSON.stringify({ soundCommand: " afplay /done.aiff " }))).soundCommand,
"afplay /done.aiff",
);
});
});
+18
View File
@@ -0,0 +1,18 @@
# my-pi 维护约定(pi-overflow)
- 完全自维护(无上游)的新结果展示截断扩展,负责"每个新 tool_result 给模型看多少":`capture.ts` 最先加载只暂存原文,`index.ts` 最后加载(在 `system-prompt.ts` 之前)生成有界视图并注册 `overflow_query`。
- 根 `pi.extensions` 顺序是硬约束:`./pi-overflow/capture.ts` 必须第一,`./pi-overflow/index.ts` 必须在所有其他扩展之后、`./extensions/system-prompt.ts` 之前;跨扩展暂存走 `globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]`,容量 64 条、淘汰最旧。
- `tool_result` handler 链中 capture 不得修改结果;index 改写 content 时必须原样回传 `structuredContent`、保留 image 部分、合并而非丢弃已有 `details`(如 RTK 压缩元数据、bash `fullOutputPath`)。
- `read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用(`parentToolCallId`)永远豁免;read 靠自身 offset/limit 分页,不得破坏 hashline 锚点。
- 视图与行号一律基于刚落盘的存储原文(不是 current 文本):文件来源用流式逐行扫描收集 head/环形 tail/signals,文本来源切同构片段;视图必须确定性、不超 `thresholdChars`、结尾永远可见;落盘原文优先取 `details.fullOutputPath` 文件(超 `maxEntryBytes` 只留头尾各半),其次暂存原文,再次当前文本。
- 超预算收缩优先级为 tail > signals > head:head 先缩到 min(10, headLines),signals 逐步减半至 10 再到 0,再按约 35/65 缩 head/tail,按字符截断兜底。
- signal 扫描与 `overflow_query` 的 search 正则只匹配每行前 10,000 字符;query 输出每行按 `maxLineChars` 截断。
- 存储位于 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),id 为 `sha1(sessionId + "\0" + toolCallId)` 前 12 位 hex;`overflow_query` 的 id 必须匹配 `^[0-9a-f]{12}$` 防路径穿越;`session_start` 按 `retentionDays`/`maxStoreBytes` 清理并删除超过 1 小时的 `*.tmp-*` 残留,失败不得抛出。
- 持久配置只读组合包 `config/pi-overflow.json`(`settings.ts` 逐字段校验、非法回退默认、signalPattern 编译失败回退默认且不敏感大小写);不得读写 `settings.json` 或以环境变量作持久默认。
- `overflow_query` 只读本扩展自己的存储目录;非法 id/正则/JSON/路径一律返回清楚错误文本,不得抛出。
- 与 pi-compaction 的唯一联动是被遮蔽 toolResult 的 `details.overflow.id` 公开字段——pi-compaction 只读它、不得 import 本包任何实现。
- 测试必须注入临时存储目录,不得写真实 agent 目录。
## 验证
- 在本目录内至少运行 `npm test` 与 `npm run check`;涉及加载入口、`pi.extensions` 顺序或 `files` 白名单时还要运行根 `node --test tests/extension-load.test.ts` 与 `tests/package-files.test.ts`。
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 sia
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+66
View File
@@ -0,0 +1,66 @@
# pi-overflow
Bounded display for oversized tool results. When a tool result exceeds
`thresholdChars`, pi-overflow persists the full original to disk and shows the
model a deterministic view — head lines, signal lines (error/fail/warn/…), and
tail lines — plus an `overflow_query` tool to search or read the stored
original by id.
## Two-stage load order
- `capture.ts` — **first** in `pi.extensions`. A `tool_result` handler that
never modifies anything; it stashes the untouched original text in
`globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]` (capacity 64) so the
bound stage can still reach the pre-RTK original.
- `index.ts` — **last** content extension, right before `system-prompt.ts`.
Its `tool_result` handler sees the final text after every other handler
(RTK semantic compaction included) has run, then:
- resolves the *original*: `details.fullOutputPath` file (streamed, capped
at `maxEntryBytes` head+tail) → stashed raw text → current text;
- if the current text still exceeds `thresholdChars`, replaces it with the
bounded view **built from the stored file** — line numbers and totals in
the view are exactly what `overflow_query` will read, even when RTK or
bash already cut the in-context text (file sources are scanned
line-by-line in a stream, never held in memory);
- if the current text fits but the original did not (an earlier extension
already cut it), keeps the text and appends a one-line
`[pi-overflow id=…]` recovery marker;
- preserves image parts and `structuredContent`, and merges
`details.overflow = { id, totalLines, totalBytes, truncated }` into the
existing details.
- Exempt tools (`read`, `edit`, `write`, `overflow_query`,
`ask_user_question`) and nested calls (`parentToolCallId`) are never
touched.
## Stored originals
`<agentDir>/my-pi/pi-overflow/<id>.txt` + `<id>.json` metadata, where
`id = sha1(sessionId + "\0" + toolCallId)[0:12]`. Directories are `0700`,
files `0600`, writes are atomic. `session_start` cleans entries older than
`retentionDays` and evicts oldest-first past `maxStoreBytes`.
## `overflow_query`
- `id` (required, `^[0-9a-f]{12}$` — path traversal safe)
- `search` — case-insensitive regex; hits come back with `context` lines and
adjacent hits merge; capped by `maxResults`
- `lines: { start, end }` — 1-based inclusive line range
- `jsonPath` — dot/bracket path into stored JSON (`a.b[3].c`)
- No selector → metadata + first-screen view. Output is always bounded by
`thresholdChars` and reports whether it was truncated.
## Configuration
`config/pi-overflow.json` inside the installed bundle. Every field is
validated independently and falls back to its default; an uncompilable
`signalPattern` falls back to the default pattern (case-insensitive).
## Development
```sh
npm test # node --test test/*.test.ts
npm run check # strip-types syntax check + tests
```
Tests inject a temporary store root — they never write to a real agent
directory.
+8
View File
@@ -0,0 +1,8 @@
# 上游来源
- 本项目完全自维护,无上游;未从任何外部仓库或 npm 包导入源码。
- 实现仅依赖 Pi 扩展宿主 API(`@earendil-works/pi-coding-agent`)与 `typebox`,两者均为组合包 peer 依赖。
## 本地改动记录
- 初始实现:`capture.ts`(首个加载,暂存原始 tool_result 文本)、`index.ts`(最后加载,生成有界视图 + 注册 `overflow_query`)、`src/`(视图、存储、查询、跨扩展暂存区)。
+30
View File
@@ -0,0 +1,30 @@
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { readOverflowSettings } from "./settings.ts";
import { getStash, stashResult } from "./src/stash.ts";
/**
* Pre-bound capture: loaded FIRST in pi.extensions, before RTK or any other
* result-rewriting extension. It only stashes the untouched original text —
* it never modifies the result. The bound stage (`index.ts`, loaded last)
* consumes the stash after all other handlers have run.
*/
export default function overflowCapture(pi: ExtensionAPI): void {
const settings = readOverflowSettings();
pi.on("tool_result", (event) => {
if (!settings.enabled) return undefined;
if (settings.exemptTools.includes(event.toolName)) return undefined;
if (event.parentToolCallId) return undefined;
const rawText = event.content
.filter((part): part is { type: "text"; text: string } => part.type === "text")
.map((part) => part.text)
.join("\n");
const fullOutputPath =
typeof event.details?.fullOutputPath === "string" ? event.details.fullOutputPath : undefined;
if (rawText.length <= settings.thresholdChars && !fullOutputPath) return undefined;
stashResult(getStash(), event.toolCallId, {
rawText,
...(fullOutputPath ? { fullOutputPath } : {}),
});
return undefined;
});
}
+167
View File
@@ -0,0 +1,167 @@
import { getAgentDir, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";
import { stat } from "node:fs/promises";
import { join } from "node:path";
import { readOverflowSettings, type OverflowSettings } from "./settings.ts";
import { getStash } from "./src/stash.ts";
import { buildView, collectViewFromFile, collectViewFromText } from "./src/view.ts";
import { cleanStore, entryId, storeEntry } from "./src/store.ts";
import { runOverflowQuery } from "./src/query.ts";
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
return `${(bytes / 1_048_576).toFixed(1)} MB`;
}
export interface OverflowExtensionOptions {
settings?: OverflowSettings;
storeRoot?: string;
}
/**
* Bound stage: loaded LAST among content extensions (before system-prompt).
* By the time this handler runs, every other tool_result handler — RTK
* included — has already transformed the result, so the bounded view is
* built from the final visible text while the stored original comes from
* capture.ts's stash or bash's fullOutputPath file.
*/
export default function overflowExtension(pi: ExtensionAPI, options: OverflowExtensionOptions = {}): void {
const settings = options.settings ?? readOverflowSettings();
const storeRoot = () => options.storeRoot ?? join(getAgentDir(), "my-pi", "pi-overflow");
pi.on("session_start", () => {
// Retention/size cleanup never throws (cleanStore swallows failures)
// and the returned promise lets tests await it.
return cleanStore(storeRoot(), settings.retentionDays, settings.maxStoreBytes);
});
pi.on("tool_result", async (event, ctx) => {
const stash = getStash();
const staged = stash.get(event.toolCallId);
try {
if (
!settings.enabled ||
settings.exemptTools.includes(event.toolName) ||
event.parentToolCallId
) {
return undefined;
}
const current = event.content
.filter((part): part is { type: "text"; text: string } => part.type === "text")
.map((part) => part.text)
.join("\n");
const fullOutputPath =
typeof event.details?.fullOutputPath === "string"
? event.details.fullOutputPath
: staged?.fullOutputPath;
// Resolve the original: bash's spilled file wins (it is complete even
// when the in-context text was already cut), then the capture stash,
// then the current text itself.
let source: { kind: "text"; text: string } | { kind: "file"; path: string };
let originalSize: number;
if (fullOutputPath) {
try {
const info = await stat(fullOutputPath);
if (!info.isFile()) throw new Error("not a file");
source = { kind: "file", path: fullOutputPath };
originalSize = info.size;
} catch {
const fallback = staged?.rawText ?? current;
source = { kind: "text", text: fallback };
originalSize = fallback.length;
}
} else {
const fallback = staged?.rawText ?? current;
source = { kind: "text", text: fallback };
originalSize = fallback.length;
}
if (current.length <= settings.thresholdChars && originalSize <= settings.thresholdChars) {
return undefined;
}
const sessionId = ctx.sessionManager?.getSessionId?.() ?? "session";
const id = entryId(sessionId, event.toolCallId);
const stored = await storeEntry(
storeRoot(),
id,
source,
{ tool: event.toolName, args: event.input },
settings.maxEntryBytes,
);
const overflow = {
id,
totalLines: stored.totalLines,
totalBytes: stored.totalBytes,
truncated: current.length > settings.thresholdChars,
};
const images = event.content.filter((part) => part.type !== "text");
let text: string;
if (current.length > settings.thresholdChars) {
// The view always describes the stored file — its line numbers and
// totals must match what overflow_query reads, which is not true
// of `current` when RTK rewrote or bash pre-truncated the result.
const viewInput =
stored.storedText !== undefined
? collectViewFromText(stored.storedText, settings)
: await collectViewFromFile(join(storeRoot(), `${id}.txt`), settings);
text = buildView(viewInput, settings, { id, tool: event.toolName });
} else {
text = `${current}\n[pi-overflow id=${id}: full original output stored (${stored.totalLines} lines, ${formatBytes(stored.totalBytes)}); use overflow_query to inspect it]`;
}
return {
content: [{ type: "text", text }, ...images],
structuredContent: event.structuredContent,
details: { ...event.details, overflow },
};
} finally {
stash.delete(event.toolCallId);
}
});
pi.registerTool({
name: "overflow_query",
label: "Overflow Query",
description:
"Inspect the full original of a tool output that pi-overflow stored because it was too long to show " +
"(search with regex, read line ranges, or extract a JSON path).",
promptSnippet: "Inspect a stored oversized tool output by id: regex search, line ranges, or JSON path.",
promptGuidelines: [
"When a tool result carries a [pi-overflow id=…] marker, the model is seeing a bounded view; the complete original is stored on disk.",
"Call overflow_query with the marker's id and a regex, line range, or jsonPath instead of re-running the tool or asking for the full dump.",
],
parameters: Type.Object({
id: Type.String({ description: "12-hex storage id from a [pi-overflow id=…] marker." }),
search: Type.Optional(
Type.String({ description: "Case-insensitive regular expression matched per line." }),
),
context: Type.Optional(
Type.Number({ minimum: 0, maximum: 10, description: "Context lines around each search hit (default 2)." }),
),
lines: Type.Optional(
Type.Object({
start: Type.Number({ minimum: 1, description: "First line to return (1-based, inclusive)." }),
end: Type.Number({ minimum: 1, description: "Last line to return (inclusive)." }),
}),
),
jsonPath: Type.Optional(
Type.String({ description: "Dot/bracket path into the stored JSON, e.g. snapshot.nodes[3].name." }),
),
maxResults: Type.Optional(
Type.Number({ minimum: 1, maximum: 200, description: "Maximum search hits to report (default 50)." }),
),
}),
async execute(_toolCallId, params) {
const result = await runOverflowQuery(storeRoot(), params, settings);
return {
content: [{ type: "text", text: result.text }],
details: { error: result.error === true, truncated: result.truncated },
isError: result.error === true,
};
},
});
}
+49
View File
@@ -0,0 +1,49 @@
{
"name": "pi-overflow",
"version": "0.1.0",
"description": "Bounded views for oversized tool results: store the full original, show head/signals/tail, recover with overflow_query.",
"private": true,
"type": "module",
"license": "MIT",
"main": "./index.ts",
"exports": {
".": "./index.ts"
},
"files": [
"index.ts",
"capture.ts",
"settings.ts",
"src",
"README.md",
"LICENSE"
],
"pi": {
"extensions": [
"./capture.ts",
"./index.ts"
]
},
"scripts": {
"test": "node --test test/*.test.ts",
"check": "node --experimental-strip-types --check index.ts && node --test test/*.test.ts"
},
"peerDependencies": {
"@earendil-works/pi-coding-agent": "*",
"typebox": "*"
},
"peerDependenciesMeta": {
"@earendil-works/pi-coding-agent": {
"optional": true
},
"typebox": {
"optional": true
}
},
"devDependencies": {
"@earendil-works/pi-coding-agent": "latest",
"typebox": "latest"
},
"engines": {
"node": ">=22.19.0"
}
}
+100
View File
@@ -0,0 +1,100 @@
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
export interface OverflowSettings {
enabled: boolean;
thresholdChars: number;
headLines: number;
tailLines: number;
maxSignalLines: number;
signalPattern: string;
signalRegex: RegExp;
maxLineChars: number;
exemptTools: string[];
retentionDays: number;
maxStoreBytes: number;
maxEntryBytes: number;
}
export const DEFAULT_SIGNAL_PATTERN =
"\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b";
export const OVERFLOW_DEFAULTS = {
enabled: true,
thresholdChars: 24_000,
headLines: 40,
tailLines: 120,
maxSignalLines: 40,
signalPattern: DEFAULT_SIGNAL_PATTERN,
maxLineChars: 500,
exemptTools: ["read", "edit", "write", "overflow_query", "ask_user_question"],
retentionDays: 7,
maxStoreBytes: 1_073_741_824,
maxEntryBytes: 67_108_864,
} as const;
// Bundle-level `config/` is the single source of truth: the extension reads
// `config/pi-overflow.json` from inside the installed bundle (resolved
// relative to this package-root source file), never `settings.json`.
export const OVERFLOW_CONFIG_PATH = fileURLToPath(
new URL("../config/pi-overflow.json", import.meta.url),
);
function isObject(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value);
}
function bool(value: unknown, fallback: boolean): boolean {
return typeof value === "boolean" ? value : fallback;
}
function intInRange(value: unknown, fallback: number, min: number, max: number): number {
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max
? value
: fallback;
}
function compileSignalPattern(value: unknown): { pattern: string; regex: RegExp } {
const pattern = typeof value === "string" && value.length > 0 ? value : DEFAULT_SIGNAL_PATTERN;
try {
return { pattern, regex: new RegExp(pattern, "i") };
} catch {
return { pattern: DEFAULT_SIGNAL_PATTERN, regex: new RegExp(DEFAULT_SIGNAL_PATTERN, "i") };
}
}
function stringList(value: unknown, fallback: readonly string[]): string[] {
if (!Array.isArray(value)) return [...fallback];
const list = value.filter((item): item is string => typeof item === "string" && item.length > 0);
return [...new Set(list)];
}
/**
* Read `config/pi-overflow.json`. Every field is validated independently and
* falls back to its bundle default when missing or invalid; an unreadable or
* non-object file yields all defaults. Tests inject an explicit path.
*/
export function readOverflowSettings(configPath: string = OVERFLOW_CONFIG_PATH): OverflowSettings {
let parsed: unknown = {};
try {
parsed = JSON.parse(readFileSync(configPath, "utf8"));
} catch {
parsed = {};
}
const config = isObject(parsed) ? parsed : {};
const signal = compileSignalPattern(config.signalPattern);
return {
enabled: bool(config.enabled, OVERFLOW_DEFAULTS.enabled),
thresholdChars: intInRange(config.thresholdChars, OVERFLOW_DEFAULTS.thresholdChars, 1, 10_000_000),
headLines: intInRange(config.headLines, OVERFLOW_DEFAULTS.headLines, 0, 10_000),
tailLines: intInRange(config.tailLines, OVERFLOW_DEFAULTS.tailLines, 0, 10_000),
maxSignalLines: intInRange(config.maxSignalLines, OVERFLOW_DEFAULTS.maxSignalLines, 0, 1_000),
signalPattern: signal.pattern,
signalRegex: signal.regex,
maxLineChars: intInRange(config.maxLineChars, OVERFLOW_DEFAULTS.maxLineChars, 10, 100_000),
exemptTools: stringList(config.exemptTools, OVERFLOW_DEFAULTS.exemptTools),
retentionDays: intInRange(config.retentionDays, OVERFLOW_DEFAULTS.retentionDays, 0, 3_650),
maxStoreBytes: intInRange(config.maxStoreBytes, OVERFLOW_DEFAULTS.maxStoreBytes, 1_024, 1_099_511_627_776),
maxEntryBytes: intInRange(config.maxEntryBytes, OVERFLOW_DEFAULTS.maxEntryBytes, 1_024, 1_099_511_627_776),
};
}
+188
View File
@@ -0,0 +1,188 @@
import { ID_PATTERN, readStoredEntry, type EntryMeta } from "./store.ts";
import type { OverflowSettings } from "../settings.ts";
export interface QueryParams {
id: string;
search?: string;
context?: number;
lines?: { start: number; end: number };
jsonPath?: string;
maxResults?: number;
}
export interface QueryResult {
text: string;
truncated: boolean;
error?: boolean;
}
function fail(text: string): QueryResult {
return { text, truncated: false, error: true };
}
function boundOutput(text: string, limit: number): QueryResult {
if (text.length <= limit) return { text, truncated: false };
const marker = `\n[output truncated at ${limit} chars; narrow the query (smaller line range, tighter regex, or jsonPath)]`;
const keep = Math.max(0, limit - marker.length);
return { text: text.slice(0, keep) + marker, truncated: true };
}
function metaLine(id: string, meta: EntryMeta | null): string {
if (!meta) return `[pi-overflow id=${id}]`;
return `[pi-overflow id=${id} tool=${meta.tool} ${meta.totalLines} lines / ${meta.totalBytes} bytes]`;
}
/** Regexes only inspect this prefix of each stored line — a model-supplied
* pattern must not backtrack catastrophically across a megabyte-long line. */
const MATCH_SCAN_CHARS = 10_000;
function clipLine(line: string, maxLineChars: number): string {
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
}
function evalJsonPath(root: unknown, path: string): { ok: true; value: unknown } | { ok: false; error: string } {
const tokens: Array<string | number> = [];
const pattern = /(?:^|\.)([A-Za-z_$][A-Za-z0-9_$]*)|\[\s*(\d+)\s*\]|\[\s*["']([^"']+)["']\s*\]/g;
let index = 0;
let match: RegExpExecArray | null;
while ((match = pattern.exec(path)) !== null) {
if (match.index !== index) return { ok: false, error: `cannot parse jsonPath near "${path.slice(index)}"` };
index = pattern.lastIndex;
if (match[1] !== undefined) tokens.push(match[1]);
else if (match[2] !== undefined) tokens.push(Number(match[2]));
else tokens.push(match[3] as string);
}
if (index !== path.length || tokens.length === 0) {
return { ok: false, error: `cannot parse jsonPath "${path}"` };
}
let value: unknown = root;
for (const token of tokens) {
if (value === null || value === undefined) {
return { ok: false, error: `jsonPath stops at null/undefined before "${String(token)}"` };
}
if (typeof token === "number") {
if (!Array.isArray(value) || token >= value.length) {
return { ok: false, error: `index [${token}] out of range` };
}
value = value[token];
} else {
if (typeof value !== "object" || Array.isArray(value)) {
return { ok: false, error: `"${token}" is not a property of this value` };
}
const record = value as Record<string, unknown>;
if (!(token in record)) return { ok: false, error: `property "${token}" not found` };
value = record[token];
}
}
return { ok: true, value };
}
function mergeRanges(hits: number[], context: number): Array<{ start: number; end: number }> {
const ranges: Array<{ start: number; end: number }> = [];
for (const hit of hits) {
const start = Math.max(1, hit - context);
const end = hit + context;
const last = ranges[ranges.length - 1];
if (last && start <= last.end + 1) {
last.end = Math.max(last.end, end);
} else {
ranges.push({ start, end });
}
}
return ranges;
}
/**
* Execute an overflow_query against the store. Never throws on bad input —
* invalid ids, regexes, JSON, and paths all come back as clear error text.
*/
export async function runOverflowQuery(
root: string,
params: QueryParams,
settings: OverflowSettings,
): Promise<QueryResult> {
if (typeof params.id !== "string" || !ID_PATTERN.test(params.id)) {
return fail(`invalid overflow id "${String(params.id)}" — expected 12 lowercase hex chars from a [pi-overflow id=…] marker`);
}
let stored: { text: string; meta: EntryMeta | null };
try {
stored = await readStoredEntry(root, params.id);
} catch {
return fail(`no stored output for id ${params.id} (it may have been cleaned up or never stored)`);
}
const lines = stored.text.split("\n");
const head = metaLine(params.id, stored.meta);
const context = Math.min(Math.max(Math.floor(params.context ?? 2), 0), 10);
const maxResults = Math.min(Math.max(Math.floor(params.maxResults ?? 50), 1), 200);
if (typeof params.search === "string") {
let regex: RegExp;
try {
regex = new RegExp(params.search, "i");
} catch (error) {
return fail(`invalid search regex: ${error instanceof Error ? error.message : String(error)}`);
}
const hits: number[] = [];
for (let index = 0; index < lines.length && hits.length < maxResults; index++) {
regex.lastIndex = 0;
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) hits.push(index + 1);
}
if (hits.length === 0) {
return boundOutput(`${head}\nno lines match /${params.search}/i`, settings.thresholdChars);
}
const parts = [head, `${hits.length}${hits.length >= maxResults ? "+" : ""} matching lines:`];
for (const range of mergeRanges(hits, context)) {
const end = Math.min(range.end, lines.length);
parts.push(`--- lines ${range.start}-${end} ---`);
for (let line = range.start; line <= end; line++) {
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
}
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
if (params.lines && typeof params.lines === "object") {
const start = Math.floor(params.lines.start);
const end = Math.floor(params.lines.end);
if (!Number.isFinite(start) || !Number.isFinite(end) || start < 1 || end < start) {
return fail(`invalid line range {start: ${params.lines.start}, end: ${params.lines.end}} — need 1 <= start <= end`);
}
const clippedEnd = Math.min(end, lines.length);
if (start > lines.length) {
return fail(`start ${start} is beyond the stored ${lines.length} lines`);
}
const parts = [head, `--- lines ${start}-${clippedEnd} ---`];
for (let line = start; line <= clippedEnd; line++) {
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
if (typeof params.jsonPath === "string") {
let parsed: unknown;
try {
parsed = JSON.parse(stored.text);
} catch (error) {
return fail(`stored output is not valid JSON: ${error instanceof Error ? error.message : String(error)}`);
}
const result = evalJsonPath(parsed, params.jsonPath);
if (!result.ok) return fail(`jsonPath error: ${result.error}`);
let rendered: string;
try {
rendered = JSON.stringify(result.value, null, 2) ?? "undefined";
} catch {
rendered = String(result.value);
}
return boundOutput(`${head}\njsonPath ${params.jsonPath}:\n${rendered}`, settings.thresholdChars);
}
// No selector: metadata plus a bounded head/tail preview.
const headCount = Math.min(settings.headLines, lines.length);
const tailCount = Math.min(settings.tailLines, Math.max(0, lines.length - headCount));
const parts = [head, `--- lines 1-${headCount} ---`, ...lines.slice(0, headCount)];
if (tailCount > 0) {
parts.push(`… ${lines.length - headCount - tailCount} lines omitted …`);
parts.push(`--- lines ${lines.length - tailCount + 1}-${lines.length} ---`, ...lines.slice(lines.length - tailCount));
}
return boundOutput(parts.join("\n"), settings.thresholdChars);
}
+35
View File
@@ -0,0 +1,35 @@
/**
* Cross-extension staging for pre-bound originals. `capture.ts` (loaded
* first) stashes the raw tool_result text; `index.ts` (loaded last) consumes
* it after every other handler — RTK in particular — has run. Sharing goes
* through `globalThis[Symbol.for(...)]`, the same mechanism pi-compaction
* uses for its metrics service.
*/
export interface StashedResult {
rawText: string;
fullOutputPath?: string;
}
const STASH_KEY = Symbol.for("my-pi.pi-overflow.stash.v1");
const STASH_LIMIT = 64;
export function getStash(): Map<string, StashedResult> {
const scope = globalThis as Record<symbol, unknown>;
let stash = scope[STASH_KEY] as Map<string, StashedResult> | undefined;
if (!(stash instanceof Map)) {
stash = new Map<string, StashedResult>();
scope[STASH_KEY] = stash;
}
return stash;
}
export function stashResult(stash: Map<string, StashedResult>, toolCallId: string, entry: StashedResult): void {
stash.delete(toolCallId);
stash.set(toolCallId, entry);
while (stash.size > STASH_LIMIT) {
const oldest = stash.keys().next().value;
if (oldest === undefined) break;
stash.delete(oldest);
}
}
+267
View File
@@ -0,0 +1,267 @@
import { createHash } from "node:crypto";
import { createReadStream, createWriteStream, type WriteStream } from "node:fs";
import { chmod, mkdir, open, readFile, readdir, rename, stat, unlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pipeline } from "node:stream/promises";
export interface EntryMeta {
tool: string;
argsSummary: string;
totalLines: number;
totalBytes: number;
createdAt: string;
omittedMiddle?: boolean;
}
export interface StoredEntry {
id: string;
totalLines: number;
totalBytes: number;
/** Present when the source was text: the exact content written to disk. */
storedText?: string;
}
export const ID_PATTERN = /^[0-9a-f]{12}$/;
const TMP_MAX_AGE_MS = 3_600_000;
export function entryId(sessionId: string, toolCallId: string): string {
return createHash("sha1").update(`${sessionId}\0${toolCallId}`).digest("hex").slice(0, 12);
}
async function ensureDir(dir: string): Promise<void> {
await mkdir(dir, { recursive: true, mode: 0o700 });
try {
await chmod(dir, 0o700);
} catch {
// Best-effort permission tightening only.
}
}
function tmpPath(path: string): string {
return `${path}.tmp-${process.pid}-${Math.random().toString(36).slice(2)}`;
}
async function writeFileAtomic(path: string, content: string | Buffer): Promise<void> {
const tmp = tmpPath(path);
await writeFile(tmp, content, { mode: 0o600 });
await rename(tmp, path);
}
function countNewlines(text: string): number {
let count = 0;
for (let index = 0; index < text.length; index++) {
if (text.charCodeAt(index) === 10) count++;
}
return count;
}
export function lineCount(text: string): number {
if (text.length === 0) return 0;
return countNewlines(text) + (text.endsWith("\n") ? 0 : 1);
}
function truncateSummary(value: string, maxChars = 300): string {
return value.length > maxChars ? `${value.slice(0, maxChars)}…` : value;
}
export function argsSummary(input: unknown): string {
try {
return truncateSummary(JSON.stringify(input) ?? "undefined");
} catch {
return "[unserializable]";
}
}
/**
* Copy `sourcePath` into `<root>/<id>.txt` without holding it all in memory.
* Files larger than `maxEntryBytes` keep the first and last halves with an
* omission marker between them.
*/
async function copyFileBounded(
sourcePath: string,
destPath: string,
maxEntryBytes: number,
): Promise<{ bytes: number; omittedMiddle: boolean }> {
const info = await stat(sourcePath);
if (!info.isFile()) throw new Error("full output path is not a file");
const size = info.size;
if (size <= maxEntryBytes) {
const tmp = tmpPath(destPath);
const out: WriteStream = createWriteStream(tmp, { mode: 0o600 });
await pipeline(createReadStream(sourcePath), out);
await rename(tmp, destPath);
return { bytes: size, omittedMiddle: false };
}
const half = Math.floor(maxEntryBytes / 2);
const omitted = size - half * 2;
const source = await open(sourcePath, "r");
try {
const head = Buffer.alloc(half);
const tail = Buffer.alloc(half);
await source.read(head, 0, half, 0);
await source.read(tail, 0, half, size - half);
const marker = Buffer.from(`\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n`);
await writeFileAtomic(destPath, Buffer.concat([head, marker, tail]));
} finally {
await source.close();
}
return { bytes: size, omittedMiddle: true };
}
async function countFileLines(path: string): Promise<number> {
let count = 0;
let lastByte = -1;
let seen = false;
for await (const chunk of createReadStream(path)) {
seen = true;
const buffer = chunk as Buffer;
for (let index = 0; index < buffer.length; index++) {
if (buffer[index] === 10) count++;
}
if (buffer.length > 0) lastByte = buffer[buffer.length - 1];
}
if (!seen) return 0;
return count + (lastByte === 10 ? 0 : 1);
}
/**
* Persist an original tool output under `<root>/<id>.txt` plus `<id>.json`
* metadata. Directories are 0700, files 0600, writes are atomic. The caller
* chooses `root` (defaults come from getAgentDir() in index.ts; tests inject
* a temp directory).
*/
export async function storeEntry(
root: string,
id: string,
source: { kind: "text"; text: string } | { kind: "file"; path: string },
meta: { tool: string; args: unknown },
maxEntryBytes: number,
): Promise<StoredEntry> {
await ensureDir(root);
const textPath = join(root, `${id}.txt`);
const metaPath = join(root, `${id}.json`);
let totalLines = 0;
let totalBytes = 0;
let omittedMiddle = false;
let storedText: string | undefined;
if (source.kind === "text") {
let text = source.text;
const bytes = Buffer.byteLength(text, "utf8");
totalBytes = bytes;
if (bytes > maxEntryBytes) {
const halfChars = Math.floor(maxEntryBytes / 2);
const omitted = bytes - halfChars * 2;
text = `${text.slice(0, halfChars)}\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n${text.slice(text.length - halfChars)}`;
omittedMiddle = true;
}
totalLines = lineCount(text);
storedText = text;
await writeFileAtomic(textPath, text);
} else {
const copied = await copyFileBounded(source.path, textPath, maxEntryBytes);
totalBytes = copied.bytes;
omittedMiddle = copied.omittedMiddle;
totalLines = await countFileLines(textPath).catch(() => 0);
}
const entry: EntryMeta = {
tool: meta.tool,
argsSummary: argsSummary(meta.args),
totalLines,
totalBytes,
createdAt: new Date().toISOString(),
...(omittedMiddle ? { omittedMiddle: true } : {}),
};
await writeFileAtomic(metaPath, `${JSON.stringify(entry, null, 2)}\n`);
return { id, totalLines, totalBytes, ...(storedText !== undefined ? { storedText } : {}) };
}
export async function readStoredEntry(root: string, id: string): Promise<{ text: string; meta: EntryMeta | null }> {
if (!ID_PATTERN.test(id)) throw new Error(`invalid overflow id: ${id}`);
const textPath = join(root, `${id}.txt`);
const metaPath = join(root, `${id}.json`);
const text = await readFile(textPath, "utf8");
let meta: EntryMeta | null = null;
try {
const parsed = JSON.parse(await readFile(metaPath, "utf8"));
if (parsed && typeof parsed === "object") meta = parsed as EntryMeta;
} catch {
meta = null;
}
return { text, meta };
}
/**
* Delete tmp files left by interrupted writes (older than one hour), entries
* older than `retentionDays`, then evict oldest-first until the store is
* under `maxStoreBytes`. All failures are swallowed — cleanup must never
* break a session.
*/
export async function cleanStore(root: string, retentionDays: number, maxStoreBytes: number): Promise<void> {
try {
const now = Date.now();
const cutoff = now - retentionDays * 86_400_000;
const names = await readdir(root);
// Stale tmp files from interrupted writes never become entries; remove
// them once they are clearly abandoned.
for (const name of names) {
if (!name.includes(".tmp-")) continue;
try {
const info = await stat(join(root, name));
if (info.isFile() && info.mtimeMs < now - TMP_MAX_AGE_MS) {
await unlink(join(root, name));
}
} catch {
// Already gone or unreadable — nothing to do.
}
}
const ids = new Set(
names
.filter((name) => !name.includes(".tmp-"))
.map((name) => name.replace(/\.(txt|json)$/, "")),
);
const entries: Array<{ id: string; bytes: number; mtime: number }> = [];
for (const id of ids) {
let bytes = 0;
let mtime = 0;
let seen = false;
for (const suffix of ["txt", "json"] as const) {
const file = join(root, `${id}.${suffix}`);
try {
const info = await stat(file);
bytes += info.size;
mtime = Math.max(mtime, info.mtimeMs);
seen = true;
} catch {
// Missing half of a pair still counts toward cleanup.
}
}
if (seen) entries.push({ id, bytes, mtime });
}
const remove = async (id: string) => {
for (const suffix of ["txt", "json"] as const) {
await unlink(join(root, `${id}.${suffix}`)).catch(() => {});
}
};
let total = 0;
const survivors: typeof entries = [];
for (const entry of entries) {
if (entry.mtime < cutoff) {
await remove(entry.id);
} else {
total += entry.bytes;
survivors.push(entry);
}
}
if (total > maxStoreBytes) {
survivors.sort((a, b) => a.mtime - b.mtime);
for (const entry of survivors) {
if (total <= maxStoreBytes) break;
await remove(entry.id);
total -= entry.bytes;
}
}
} catch {
// Cleanup failures never propagate.
}
}
+246
View File
@@ -0,0 +1,246 @@
import { createReadStream } from "node:fs";
import { open, stat } from "node:fs/promises";
import type { OverflowSettings } from "../settings.ts";
export interface ViewMeta {
id: string;
tool: string;
}
export interface SignalHit {
line: number;
text: string;
}
/**
* Pre-collected view segments of the STORED original (never the current —
* possibly RTK- or bash-truncated — text). Line numbers here are the stored
* file's line numbers, which is exactly what overflow_query reads.
*/
export interface ViewInput {
head: string[];
signals: SignalHit[];
tail: string[];
totalLines: number;
totalBytes: number;
/** First ~thresholdChars chars of the stored content, for char fallback. */
rawHead: string;
/** Last ~thresholdChars chars of the stored content, for char fallback. */
rawTail: string;
}
/** Regexes only inspect this prefix of each line — catastrophic backtracking
* on a giant single-line blob is not allowed to stall the handler. */
export const MATCH_SCAN_CHARS = 10_000;
function clip(line: string, maxLineChars: number): string {
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
}
/** Slice a text original into the same view segments a file scan produces. */
export function collectViewFromText(text: string, settings: OverflowSettings): ViewInput {
const lines = text.split("\n");
const total = lines.length;
const headCount = Math.min(settings.headLines, total);
const tailCount = Math.min(settings.tailLines, Math.max(0, total - headCount));
const head = lines.slice(0, headCount).map((line) => clip(line, settings.maxLineChars));
const tail = tailCount > 0 ? lines.slice(total - tailCount).map((line) => clip(line, settings.maxLineChars)) : [];
const signals: SignalHit[] = [];
const regex = new RegExp(settings.signalRegex.source, "i");
for (
let index = headCount;
index < total - tailCount && signals.length < settings.maxSignalLines;
index++
) {
regex.lastIndex = 0;
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) {
signals.push({ line: index + 1, text: clip(lines[index], settings.maxLineChars) });
}
}
return {
head,
signals,
tail,
totalLines: total,
totalBytes: Buffer.byteLength(text, "utf8"),
rawHead: text.slice(0, settings.thresholdChars),
rawTail: text.slice(Math.max(0, text.length - settings.thresholdChars)),
};
}
/**
* Stream the stored file line by line — never materialize it. Collects the
* first headLines lines, keeps the last tailLines lines in a ring buffer,
* and records at most maxSignalLines signal hits (line number + clipped
* text); hits landing inside the final tail range are dropped afterwards.
*/
export async function collectViewFromFile(path: string, settings: OverflowSettings): Promise<ViewInput> {
const head: string[] = [];
const tail: string[] = [];
const signals: SignalHit[] = [];
let totalLines = 0;
let remainder = "";
const regex = new RegExp(settings.signalRegex.source, "i");
const handleLine = (line: string) => {
totalLines++;
if (head.length < settings.headLines) {
head.push(clip(line, settings.maxLineChars));
} else if (signals.length < settings.maxSignalLines) {
regex.lastIndex = 0;
if (regex.test(line.slice(0, MATCH_SCAN_CHARS))) {
signals.push({ line: totalLines, text: clip(line, settings.maxLineChars) });
}
}
tail.push(clip(line, settings.maxLineChars));
if (tail.length > settings.tailLines) tail.shift();
};
for await (const chunk of createReadStream(path, { encoding: "utf8" })) {
remainder += chunk;
let index = remainder.indexOf("\n");
while (index >= 0) {
handleLine(remainder.slice(0, index));
remainder = remainder.slice(index + 1);
index = remainder.indexOf("\n");
}
}
if (remainder.length > 0) handleLine(remainder);
const tailStart = totalLines - tail.length + 1;
const middleSignals = signals.filter((hit) => hit.line < tailStart);
const info = await stat(path);
const take = Math.min(settings.thresholdChars, info.size);
let rawHead = "";
let rawTail = "";
if (take > 0) {
const handle = await open(path, "r");
try {
const headBuffer = Buffer.alloc(take);
const tailBuffer = Buffer.alloc(take);
await handle.read(headBuffer, 0, take, 0);
await handle.read(tailBuffer, 0, take, info.size - take);
rawHead = headBuffer.toString("utf8");
rawTail = tailBuffer.toString("utf8");
} finally {
await handle.close();
}
}
return {
head,
signals: middleSignals,
tail,
totalLines,
totalBytes: info.size,
rawHead,
rawTail,
};
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
if (bytes < 1_073_741_824) return `${(bytes / 1_048_576).toFixed(1)} MB`;
return `${(bytes / 1_073_741_824).toFixed(2)} GB`;
}
function signalDescription(pattern: string): string {
const compact = pattern.replace(/\\b/g, "").replace(/[()]/g, "");
return compact.length > 60 ? `${compact.slice(0, 60)}…` : compact;
}
/**
* Deterministic bounded view of the stored original: head lines, signal
* lines from the middle section, and tail lines — all with the stored
* file's line numbers. Output never exceeds settings.thresholdChars and the
* tail of the stored content is always visible.
*
* Shrink order under budget pressure is tail > signals > head: head first
* collapses to min(10, headLines), signals halve stepwise to 10 then 0, then
* head/tail shrink ~35/65 with tail preferred, and char truncation of the
* stored head/tail is the final fallback.
*/
export function buildView(input: ViewInput, settings: OverflowSettings, meta: ViewMeta): string {
const budget = settings.thresholdChars;
const header =
`[pi-overflow id=${meta.id} tool=${meta.tool} ${input.totalLines} lines / ${formatBytes(input.totalBytes)}` +
` — full output stored; use overflow_query to search or read ranges]`;
const render = (headLines: string[], signals: SignalHit[], tailLines: string[]): string => {
const parts: string[] = [header];
if (headLines.length > 0) {
parts.push(`--- lines 1-${headLines.length} ---`, ...headLines);
}
if (signals.length > 0) {
parts.push(
`--- signals (lines matching ${signalDescription(settings.signalPattern)}, up to ${settings.maxSignalLines}) ---`,
...signals.map((hit) => `L${hit.line}: ${hit.text}`),
);
}
if (tailLines.length > 0) {
const first = input.totalLines - tailLines.length + 1;
parts.push(`--- lines ${first}-${input.totalLines} ---`, ...tailLines);
}
return parts.join("\n");
};
const charFallback = (): string => {
const markerReserve = 64;
const bodyBudget = Math.max(64, budget - header.length - markerReserve);
const headChars = Math.floor(bodyBudget * 0.35);
const tailChars = bodyBudget - headChars;
const headText = input.rawHead.slice(0, headChars);
const tailText = input.rawTail.slice(Math.max(0, input.rawTail.length - tailChars));
const omitted = Math.max(0, input.totalBytes - headText.length - tailText.length);
return `${header}\n${headText}\n… ${omitted} chars omitted …\n${tailText}`;
};
let headList = input.head;
let signalList = input.signals;
let tailList = input.tail;
let result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
// 1. Collapse head to its minimum useful window.
headList = headList.slice(0, Math.min(10, settings.headLines));
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
// 2. Halve signals stepwise until 10 remain, then drop them entirely.
while (signalList.length > 10) {
signalList = signalList.slice(0, Math.ceil(signalList.length / 2));
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
}
if (signalList.length > 0) {
signalList = [];
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
}
// When head+tail already span every stored line there is no line-level
// slack left: go straight to char truncation so the ending survives.
if (input.head.length + input.tail.length >= input.totalLines) return charFallback();
// 3. Shrink head/tail ~35/65; tail keeps the larger share and always keeps
// its last line (the stored ending).
while (headList.length + tailList.length > 0) {
result = render(headList, signalList, tailList);
if (result.length <= budget) return result;
const headChars = headList.reduce((sum, line) => sum + line.length + 1, 0);
const tailChars = tailList.reduce((sum, line) => sum + line.length + 1, 0);
if (tailList.length > 1 && (tailChars >= headChars * (65 / 35) || headList.length === 0)) {
tailList.shift();
} else if (headList.length > 0) {
headList.pop();
} else {
tailList.shift();
}
}
// 4. Last resort: keep the stored text's own head 35% / tail 65%.
return charFallback();
}
+314
View File
@@ -0,0 +1,314 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, readFile, readdir, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import overflowCapture from "../capture.ts";
import overflowExtension from "../index.ts";
import { getStash } from "../src/stash.ts";
import { readOverflowSettings, OVERFLOW_DEFAULTS, type OverflowSettings } from "../settings.ts";
function harness() {
const handlers = new Map<string, Function[]>();
const tools = new Map<string, any>();
const pi: any = {
on(name: string, handler: Function) {
const list = handlers.get(name) ?? [];
list.push(handler);
handlers.set(name, list);
},
registerTool(tool: any) { tools.set(tool.name, tool); },
};
return { pi, handlers, tools };
}
function ctx() {
return { sessionManager: { getSessionId: () => "session-test" } } as any;
}
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
}
function textEvent(toolCallId: string, text: string, extra: Record<string, unknown> = {}) {
return {
toolCallId,
toolName: "bash",
input: { command: "run" },
content: [{ type: "text", text }],
details: {},
...extra,
};
}
async function runHandlers(handlers: Map<string, Function[]>, event: any, context: any) {
let current = event;
for (const handler of handlers.get("tool_result") ?? []) {
const result = await handler(current, context);
if (result) current = { ...current, ...result };
}
return current;
}
async function makeStore() {
return mkdtemp(join(tmpdir(), "pi-overflow-test-"));
}
test("capture stashes the original before another handler shortens it; bound stage stores original but views current", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = Array.from({ length: 3000 }, (_, i) => `orig ${i}`).join("\n");
const event = textEvent("call-1", big);
const afterCapture = await runHandlers(capture.handlers, event, ctx());
assert.equal(afterCapture, event, "capture must not modify the result");
assert.ok(getStash().has("call-1"), "capture must stash the original");
// Simulate RTK: replace content with a shortened version that is still
// over threshold.
const rtkShortened = Array.from({ length: 900 }, (_, i) => `rtk ${i}`).join("\n") + "\n" + "z".repeat(30_000);
const mid = { ...afterCapture, content: [{ type: "text", text: rtkShortened }] };
const final = await runHandlers(bound.handlers, mid, ctx());
assert.ok(!getStash().has("call-1"), "stash entry must be consumed");
const text = final.content[0].text;
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash/);
assert.ok(text.length <= settings().thresholdChars);
const id = final.details.overflow.id;
const stored = await readFile(join(root, `${id}.txt`), "utf8");
assert.ok(stored.includes("orig 0"), "store must keep the ORIGINAL, not the RTK-shortened text");
assert.ok(stored.includes("orig 2999"));
assert.equal(final.details.overflow.truncated, true);
});
test("details.fullOutputPath file is used as the original", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const spill = join(root, "spill.txt");
await writeFile(spill, Array.from({ length: 5000 }, (_, i) => `spill ${i}`).join("\n"));
const event = textEvent("call-2", "x".repeat(30_000), { details: { fullOutputPath: spill } });
const final = await runHandlers(bound.handlers, event, ctx());
const stored = await readFile(join(root, `${final.details.overflow.id}.txt`), "utf8");
assert.ok(stored.includes("spill 0"));
assert.ok(stored.includes("spill 4999"));
});
test("view line numbers track the STORED file, not the current text", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
// bash spill: 5000 lines on disk, but the in-context result only carries
// the last ~100 lines (still over threshold via padding).
const spill = join(root, "spill-5000.txt");
await writeFile(
spill,
Array.from({ length: 5000 }, (_, i) => `spill line ${i + 1}`).join("\n"),
);
const current = Array.from({ length: 100 }, (_, i) => `spill line ${4901 + i}`).join("\n") + "\n" + "pad".repeat(10_000);
const event = textEvent("call-lines", current, { details: { fullOutputPath: spill } });
const final = await runHandlers(bound.handlers, event, ctx());
const text = final.content[0].text;
const id = final.details.overflow.id;
assert.match(text, /--- lines 4881-5000 ---/, "tail range must come from the stored 5000-line file");
assert.match(text, /spill line 5000/);
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash 5000 lines/);
// overflow_query with the view's tail range returns the same lines.
const tool = bound.tools.get("overflow_query");
const result = await tool.execute("q1", { id, lines: { start: 4881, end: 4883 } });
assert.match(result.content[0].text, /L4881: spill line 4881/);
assert.match(result.content[0].text, /L4883: spill line 4883/);
});
test("RTK scenario: view line numbers come from the stored original text", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const original = Array.from({ length: 3000 }, (_, i) => `orig ${i + 1}`).join("\n");
await runHandlers(capture.handlers, textEvent("call-rtk", original), ctx());
// RTK rewrites the visible text but it stays over threshold.
const mid = {
...textEvent("call-rtk", ""),
content: [{ type: "text", text: "z".repeat(30_000) }],
};
const final = await runHandlers(bound.handlers, mid, ctx());
const text = final.content[0].text;
assert.match(text, /--- lines 2881-3000 ---/, "view tail must be the stored original's tail");
assert.ok(text.includes("orig 3000"));
});
test("file sources over maxEntryBytes keep real head and tail halves", async () => {
const root = await makeStore();
const { storeEntry } = await import("../src/store.ts");
const src = join(root, "big.bin");
const content = "HEAD-".repeat(200) + "MID-".repeat(2000) + "TAIL-".repeat(200);
await writeFile(src, content);
const maxEntryBytes = 2000;
await storeEntry(root, "0123456789ab", { kind: "file", path: src }, { tool: "bash", args: {} }, maxEntryBytes);
const stored = await readFile(join(root, "0123456789ab.txt"), "utf8");
assert.ok(stored.startsWith("HEAD-"));
assert.ok(stored.endsWith("TAIL-"));
assert.match(stored, /bytes omitted/);
// Head and tail halves must equal the source's real bytes.
const half = Math.floor(maxEntryBytes / 2);
assert.equal(stored.slice(0, half), content.slice(0, half));
assert.equal(stored.slice(-half), content.slice(-half));
assert.ok(!stored.includes("\0".repeat(100)), "must not store zeroed content");
});
test("interrupted-write tmp files older than an hour are cleaned", async () => {
const root = await makeStore();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const staleTmp = join(root, "deadbeef.txt.tmp-999-zzz");
const freshTmp = join(root, "livebeef.txt.tmp-111-aaa");
await writeFile(staleTmp, "partial");
await writeFile(freshTmp, "partial");
const old = new Date(Date.now() - 2 * 3_600_000);
const { utimes } = await import("node:fs/promises");
await utimes(staleTmp, old, old);
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const { existsSync } = await import("node:fs");
assert.ok(!existsSync(staleTmp), "stale tmp must be removed");
assert.ok(existsSync(freshTmp), "fresh tmp must survive");
});
test("current under threshold but original over → one-line marker appended", async () => {
const root = await makeStore();
getStash().clear();
const capture = harness();
overflowCapture(capture.pi);
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = "a".repeat(30_000);
const event = textEvent("call-3", big);
await runHandlers(capture.handlers, event, ctx());
const mid = { ...event, content: [{ type: "text", text: "short summary" }] };
const final = await runHandlers(bound.handlers, mid, ctx());
const text = final.content[0].text;
assert.ok(text.startsWith("short summary"));
assert.match(text, /\[pi-overflow id=[0-9a-f]{12}: full original output stored \(\d+ lines, .*; use overflow_query/);
assert.equal(final.details.overflow.truncated, false);
assert.ok(final.details.overflow.id);
});
test("exempt tools and nested calls are ignored", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const big = "x".repeat(30_000);
const exempt = await runHandlers(bound.handlers, textEvent("c-ex", big, { toolName: "read" }), ctx());
assert.equal(exempt.content[0].text, big, "read results must pass through untouched");
const nested = await runHandlers(bound.handlers, textEvent("c-nest", big, { parentToolCallId: "parent" }), ctx());
assert.equal(nested.content[0].text, big, "nested calls must pass through untouched");
});
test("image parts and structuredContent survive", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const event = textEvent("call-4", "x".repeat(30_000));
event.content.push({ type: "image", data: "b64" } as any);
(event as any).structuredContent = { rows: 3 };
const final = await runHandlers(bound.handlers, event, ctx());
assert.deepEqual(final.structuredContent, { rows: 3 });
assert.equal(final.content.at(-1).type, "image");
assert.equal(final.content.at(-1).data, "b64");
});
test("existing details fields survive the merge", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const event = textEvent("call-5", "x".repeat(30_000), { details: { rtk: { saved: 100 }, fullOutputPath: undefined } });
const final = await runHandlers(bound.handlers, event, ctx());
assert.deepEqual(final.details.rtk, { saved: 100 });
assert.ok(final.details.overflow.id);
});
test("files are 0600 and directories 0700", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
const final = await runHandlers(bound.handlers, textEvent("call-6", "x".repeat(30_000)), ctx());
const id = final.details.overflow.id;
const dirMode = (await stat(root)).mode & 0o777;
const fileMode = (await stat(join(root, `${id}.txt`))).mode & 0o777;
const metaMode = (await stat(join(root, `${id}.json`))).mode & 0o777;
assert.equal(dirMode, 0o700);
assert.equal(fileMode, 0o600);
assert.equal(metaMode, 0o600);
});
test("retention and total-size cleanup remove old entries", async () => {
const root = await makeStore();
const bound = harness();
overflowExtension(bound.pi, {
storeRoot: root,
settings: settings({ retentionDays: 1, maxStoreBytes: 100 }),
});
// An old entry (mtime in the past) must be deleted by age.
const oldId = "aaaaaaaaaaaa";
await writeFile(join(root, `${oldId}.txt`), "old");
await writeFile(join(root, `${oldId}.json`), "{}");
const past = new Date(Date.now() - 10 * 86_400_000);
const { utimes } = await import("node:fs/promises");
await utimes(join(root, `${oldId}.txt`), past, past);
await utimes(join(root, `${oldId}.json`), past, past);
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const names = await readdir(root);
assert.ok(!names.some((n) => n.startsWith(oldId)), "old entry must be deleted");
// Fresh entries exceeding maxStoreBytes get evicted oldest-first.
for (const [i, id] of ["bbbbbbbbbbbb", "cccccccccccc", "dddddddddddd"].entries()) {
await writeFile(join(root, `${id}.txt`), "y".repeat(80));
const t = new Date(Date.now() - (3 - i) * 60_000);
await utimes(join(root, `${id}.txt`), t, t);
}
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
const after = (await readdir(root)).filter((n) => n.endsWith(".txt"));
assert.ok(after.length <= 2, `expected eviction under maxStoreBytes, got ${after.join(",")}`);
assert.ok(!after.includes("bbbbbbbbbbbb.txt"), "oldest entry must be evicted first");
});
test("disabled setting clears the stash and returns undefined", async () => {
const root = await makeStore();
getStash().clear();
const bound = harness();
overflowExtension(bound.pi, { storeRoot: root, settings: settings({ enabled: false }) });
getStash().set("c-off", { rawText: "x".repeat(30_000) });
const event = textEvent("c-off", "x".repeat(30_000));
const result = await runHandlers(bound.handlers, event, ctx());
assert.equal(result, event);
assert.ok(!getStash().has("c-off"));
});
+112
View File
@@ -0,0 +1,112 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { runOverflowQuery } from "../src/query.ts";
import { storeEntry } from "../src/store.ts";
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
}
async function store(text: string, id = "0123456789ab") {
const root = await mkdtemp(join(tmpdir(), "pi-overflow-query-"));
await storeEntry(root, id, { kind: "text", text }, { tool: "bash", args: { command: "run" } }, 1_000_000);
return root;
}
test("search returns hits with merged context", async () => {
const lines = Array.from({ length: 100 }, (_, i) => (i === 9 || i === 11 || i === 50 ? `line ${i + 1} FAIL` : `line ${i + 1}`));
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 1 }, settings());
assert.ok(!result.error);
assert.match(result.text, /L10: line 10 FAIL/);
assert.match(result.text, /L12: line 12 FAIL/);
assert.match(result.text, /L51: line 51 FAIL/);
// hits at lines 10 and 12 with context 1 merge into one range 9-13
assert.match(result.text, /--- lines 9-13 ---/);
assert.match(result.text, /--- lines 50-52 ---/);
});
test("line-range retrieval returns numbered lines", async () => {
const lines = Array.from({ length: 50 }, (_, i) => `row ${i + 1}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 10, end: 15 } }, settings());
assert.match(result.text, /--- lines 10-15 ---/);
assert.match(result.text, /L10: row 10/);
assert.match(result.text, /L15: row 15/);
assert.ok(!result.text.includes("row 16"));
});
test("jsonPath extracts nested values", async () => {
const doc = JSON.stringify({ snapshot: { nodes: [{ name: "a" }, { name: "b" }, { name: "c" }] } });
const root = await store(doc);
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "snapshot.nodes[2].name" }, settings());
assert.match(result.text, /"c"/);
});
test("invalid id, regex, JSON and path return clear errors without throwing", async () => {
const root = await store('{"a": {"b": 1}}');
assert.ok((await runOverflowQuery(root, { id: "../etc/passwd" }, settings())).error);
assert.ok((await runOverflowQuery(root, { id: "zzzzzzzzzzzz" }, settings())).error);
assert.ok((await runOverflowQuery(root, { id: "0123456789ab", search: "([" }, settings())).error);
const goodPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b" }, settings());
assert.ok(!goodPath.error, "valid path on valid JSON must succeed");
const badPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b.c!!" }, settings());
assert.ok(badPath.error);
const missing = await runOverflowQuery(root, { id: "ffffffffffff" }, settings());
assert.ok(missing.error);
});
test("non-JSON content + jsonPath reports the parse failure", async () => {
const root = await store("not json at all");
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a" }, settings());
assert.ok(result.error);
assert.match(result.text, /not valid JSON/);
});
test("output is bounded by thresholdChars and marked truncated", async () => {
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i + 1} pad ${"p".repeat(40)}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(
root,
{ id: "0123456789ab", lines: { start: 1, end: 5000 } },
settings({ thresholdChars: 4000 }),
);
assert.ok(result.text.length <= 4000);
assert.equal(result.truncated, true);
assert.match(result.text, /truncated/);
});
test("lines and search output clip each line to maxLineChars", async () => {
const long = `row ${"q".repeat(2_000)}`;
const root = await store(`first\n${long}\n${long} ERROR\nlast`);
const s = settings({ maxLineChars: 60 });
const lines = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 2, end: 3 } }, s);
assert.ok(lines.text.includes("q".repeat(56) + "…"));
assert.ok(!lines.text.includes("q".repeat(61)));
const search = await runOverflowQuery(root, { id: "0123456789ab", search: "ERROR", context: 0 }, s);
assert.ok(search.text.includes("q".repeat(56) + "…"));
assert.ok(!search.text.includes("q".repeat(61)));
});
test("search only scans the first 10000 chars of each line", async () => {
const earlyHit = `hit ${"a".repeat(100)} FAIL`;
const lateHit = `${"a".repeat(10_500)} FAIL`;
const root = await store(`${earlyHit}\n${lateHit}\nplain`);
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 0 }, settings());
assert.match(result.text, /L1: hit /);
assert.ok(!result.text.includes("L2:"), "match beyond char 10000 must not be reported");
});
test("default query returns metadata and a first-screen view", async () => {
const lines = Array.from({ length: 300 }, (_, i) => `row ${i + 1}`);
const root = await store(lines.join("\n"));
const result = await runOverflowQuery(root, { id: "0123456789ab" }, settings());
assert.ok(!result.error);
assert.match(result.text, /\[pi-overflow id=0123456789ab tool=bash \d+ lines \/ \d+ bytes\]/);
assert.match(result.text, /--- lines 1-40 ---/);
assert.ok(result.text.includes("row 300"), "tail of the stored file must appear");
});
+86
View File
@@ -0,0 +1,86 @@
import test from "node:test";
import assert from "node:assert/strict";
import { buildView, collectViewFromText } from "../src/view.ts";
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
return {
...readOverflowSettings("/nonexistent/pi-overflow.json"),
...overrides,
};
}
const meta = { id: "abc123def456", tool: "bash" };
function view(text: string, s = settings()) {
return buildView(collectViewFromText(text, s), s, meta);
}
test("below threshold remains unchanged", () => {
const out = view("line one\nline two\nline three");
assert.ok(out.includes("line one"));
assert.ok(out.includes("line three"));
});
test("above threshold includes header, line ranges, signal lines and tail", () => {
const lines: string[] = [];
for (let i = 1; i <= 1000; i++) lines.push(`row ${i} ${i === 500 ? "ERROR boom" : "ok"}`);
const out = view(lines.join("\n"));
assert.match(out, /\[pi-overflow id=abc123def456 tool=bash 1000 lines/);
assert.match(out, /--- lines 1-40 ---/);
assert.match(out, /L500: row 500 ERROR boom/);
assert.match(out, /--- lines 881-1000 ---/);
assert.ok(out.includes("row 1000"));
});
test("output never exceeds thresholdChars", () => {
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i} ${"x".repeat(60)}`);
const s = settings({ thresholdChars: 2000 });
const out = view(lines.join("\n"), s);
assert.ok(out.length <= 2000, `view ${out.length} exceeds budget`);
assert.ok(out.includes("line 4999"), "tail must survive");
});
test("few long lines still expose the ending", () => {
const lines = ["start", ...Array.from({ length: 5 }, (_, i) => `middle${i} ${"y".repeat(4000)}`), "THE_END_MARKER"];
const s = settings({ thresholdChars: 3000, headLines: 40, tailLines: 120 });
const out = view(lines.join("\n"), s);
assert.ok(out.includes("THE_END_MARKER"));
assert.ok(out.length <= 3000 + 100, `view ${out.length} far exceeds budget`);
});
test("same input produces identical output", () => {
const lines = Array.from({ length: 800 }, (_, i) => `entry ${i} ${i % 77 === 0 ? "warning" : "fine"}`);
const text = lines.join("\n");
assert.equal(view(text), view(text));
});
test("a single line over maxLineChars is truncated", () => {
const long = "z".repeat(2000);
const out = view(`a\n${long}\nb`, settings({ maxLineChars: 50 }));
assert.ok(out.includes("z".repeat(50) + "…"));
assert.ok(!out.includes("z".repeat(51)));
});
test("signals outrank head and survive when head+tail exceed the budget", () => {
// ~150-char lines: head 40 + tail 120 = 160 lines * 150 = ~24K plus
// overhead, over budget. Shrink order must keep some signals while head
// collapses to 10 lines before the 35/65 phase kicks in.
const lines = Array.from({ length: 400 }, (_, i) => {
const marker = i === 200 ? "ERROR detected here" : "ok";
return `line ${i + 1} ${marker} ${"x".repeat(140)}`;
});
const out = view(lines.join("\n"));
assert.match(out, /--- signals/);
assert.match(out, /L201: line 201 ERROR detected here/);
// head collapsed below the configured 40 lines
assert.match(out, /--- lines 1-(?:[0-9]|10) ---/);
assert.ok(out.includes("line 400"));
});
test("signal hits inside the tail region are not duplicated", () => {
const lines = Array.from({ length: 100 }, (_, i) => `line ${i + 1} ${"y".repeat(40)}`);
lines[99] = "line 100 tail-side ERROR";
const out = view(lines.join("\n"));
assert.ok(!out.includes("L100:"), "tail-region hit must not appear in signals");
});
+1 -1
View File
@@ -8,7 +8,7 @@
- reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。 - reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。
- `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。 - `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。
- `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。 - `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。
- 策略默认允许常规只读与低风险工具;`web_fetch`、`ctx_fetch_and_index`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。 - 策略默认允许常规只读与低风险工具;`web_fetch`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。
- `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。 - `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。
- 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。 - 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。
- reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。 - reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。
+1
View File
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed ### Changed
- **Descend into bash control-flow bodies when enumerating command units.** `for`/`while`/`until` loops, `if`/`elif`/`else` branches, `case` items, `{ … }` brace groups, function definitions, and `!`-negated commands were previously matched only as whole-statement text, so a denied command hidden in a loop body or behind `!` (e.g. `for d in */; do find "$d" …; done`, `! rm -rf x`) bypassed every bash pattern — as did a substitution inside a value (`x=$(grep foo f)`, `export X=$(find .)`). The enumerator still emits each enclosing statement as a unit and now also emits every condition, body, and negated command — plus substitutions hosted in value lists, case subjects, assignments, declarations, and other non-command positions — so the most restrictive decision wins; pure body containers (`do …; done`, `elif`/`else` clauses, `case` items) no longer produce their own units. Hard guards share the same enumeration and now reach those nested commands too.
- **Replace the AutoReview text-JSON chat reviewer with a conversational tool-call reviewer.** The reviewer makes one independent model call per ask — a fixed review prompt, the rendered transcript, and a bounded JSON description of the planned action — and reaches its verdict only by calling one of two tools, `approve` or `reject`, each carrying a `reason`. A reply with no tool call counts as no verdict; multiple calls, unknown tools, and missing reasons count as invalid responses; neither is retried. An experimental TypeSafe Jev classifier rewrite explored in the working tree was dropped before release and never shipped. - **Replace the AutoReview text-JSON chat reviewer with a conversational tool-call reviewer.** The reviewer makes one independent model call per ask — a fixed review prompt, the rendered transcript, and a bounded JSON description of the planned action — and reaches its verdict only by calling one of two tools, `approve` or `reject`, each carrying a `reason`. A reply with no tool call counts as no verdict; multiple calls, unknown tools, and missing reasons count as invalid responses; neither is retried. An experimental TypeSafe Jev classifier rewrite explored in the working tree was dropped before release and never shipped.
- **The review model follows the session by default.** With `autoReview.provider` and `autoReview.model` both unset, the reviewer calls the session's current model (`ctx.model`), resolved live per ask so a mid-session switch applies to the next request. Setting both pins a fixed reviewer; setting exactly one is a misconfiguration that warns once at `session_start` and defers every ask. - **The review model follows the session by default.** With `autoReview.provider` and `autoReview.model` both unset, the reviewer calls the session's current model (`ctx.model`), resolved live per ask so a mid-session switch applies to the next request. Setting both pins a fixed reviewer; setting exactly one is a misconfiguration that warns once at `session_start` and defers every ask.
- **A rejection defers instead of denying.** `reject` maps to `defer` and the model's `reason` is shown in the human permission prompt, labelled as model-generated. Every review failure — timeout, provider error, malformed or truncated reply, unresolved model — also defers with a fixed system-authored note. Pressing Esc during a review aborts it and denies outright, so no human dialog appears after the user already refused. - **A rejection defers instead of denying.** `reject` maps to `defer` and the model's `reason` is shown in the human permission prompt, labelled as model-generated. Every review failure — timeout, provider error, malformed or truncated reply, unresolved model — also defers with a fixed system-authored note. Pressing Esc during a review aborts it and denies outright, so no human dialog appears after the user already refused.
+4
View File
@@ -21,3 +21,7 @@ AutoReview 源码已整合进 `src/auto-review/`,不再保留或加载第二
## 本地维护 ## 本地维护
本目录不是 submodule,不保留嵌套 `.git`、上游 `.pi` 本地状态、`dist/` 或构建产物。同步任一上游时必须从其源码仓库的明确 tag/commit 移植并与本地改动对照审查,不得整体覆盖,也不得使用 npm 预编译产物作为导入源。 本目录不是 submodule,不保留嵌套 `.git`、上游 `.pi` 本地状态、`dist/` 或构建产物。同步任一上游时必须从其源码仓库的明确 tag/commit 移植并与本地改动对照审查,不得整体覆盖,也不得使用 npm 预编译产物作为导入源。
## 本地改动
- Bash 命令枚举在保留整条控制流语句为独立 command unit 的基础上递归下降:`for`/`while`/`until`、`if`/`elif`/`else`、`case` 分支、`{ … }` 组合语句、函数定义与 `!` 取反命令体内的命令也按 bash pattern 逐条判定;赋值与声明值(`x=$(…)`、`export X=$(…)`)、值列表、case 词与 `[[ … ]]` 测试中的 `$(…)` 替换同样枚举,而 `do …; done`、`elif`/`else`、`case` 分支等纯容器不再单独产生 unit。藏在循环体、取反前缀或赋值载荷里的 `find`/`grep`/`rm` 等不再绕过 deny 或 hard guard。
+4 -1
View File
@@ -370,7 +370,10 @@ This holds wherever the substitution appears, not only in argument position.
A substitution in a **redirect target** (`echo hi > $(rm *.txt)`, `cat < <(rm c)`, ``echo hi 2> `rm d` ``) and one in an **interpolating heredoc body** (`cat <<EOF` with `$(rm e)` in the body) are evaluated the same way. A substitution in a **redirect target** (`echo hi > $(rm *.txt)`, `cat < <(rm c)`, ``echo hi 2> `rm d` ``) and one in an **interpolating heredoc body** (`cat <<EOF` with `$(rm e)` in the body) are evaluated the same way.
A quoted heredoc delimiter (`<<'EOF'` or `<<"EOF"`) does not interpolate, so its body is literal text and nothing in it is evaluated as a command. A quoted heredoc delimiter (`<<'EOF'` or `<<"EOF"`) does not interpolate, so its body is literal text and nothing in it is evaluated as a command.
The enclosing command is still matched without its redirect, so a rule like `npm install` keeps matching `npm install > out.txt`. The enclosing command is still matched without its redirect, so a rule like `npm install` keeps matching `npm install > out.txt`.
Control-flow bodies (`if`/`while`/`for`/`case`) and `{ … }` brace groups are not descended into; their contents are matched as part of the enclosing statement's text. Control-flow statements (`for`/`while`/`until` loops, `if`/`elif`/`else` branches, `case` items, `{ … }` brace groups, and function definitions) are evaluated as a whole unit *and* descended into: every command in their conditions and bodies is matched against the bash patterns in addition to the enclosing statement's text.
So `for d in src docs; do find "$d" -name '*.ts'; done` evaluates both the whole loop and `find "$d" -name '*.ts'`; if `find *` is denied, the whole invocation is denied.
Non-command parts of a control-flow statement — the `for` value list, the `case` subject and patterns, `[[ … ]]` tests — are not matched as commands themselves, but substitutions nested inside them still are, so `for f in $(find …)` and `if [[ "$(rm e)" = x ]]` gate the inner command.
Negation and shell bookkeeping get the same treatment: `! grep -q x f` evaluates both `! grep -q x f` and the inner `grep -q x f`, and a substitution inside a value — `x=$(grep foo f)`, `export X=$(find .)`, `local y=$(rg z)` — is gated as a command substitution even though the assignment or declaration itself runs no command.
A leading environment-variable assignment prefix is stripped before matching, so the rule gates the underlying command rather than the prefix. A leading environment-variable assignment prefix is stripped before matching, so the rule gates the underlying command rather than the prefix.
So `AWS_PROFILE=prod aws ec2 …` is matched as `aws ec2 …` — a `aws *` rule applies even though the invocation begins with `AWS_PROFILE=`. So `AWS_PROFILE=prod aws ec2 …` is matched as `aws ec2 …` — a `aws *` rule applies even though the invocation begins with `AWS_PROFILE=`.
@@ -1,6 +1,7 @@
import { import {
EXECUTION_HOST_TYPES, EXECUTION_HOST_TYPES,
forEachNestedExecution, forEachNestedExecution,
NESTED_EXECUTION_CONTEXTS,
} from "#src/access-intent/bash/nested-execution"; } from "#src/access-intent/bash/nested-execution";
import type { TSNode } from "#src/access-intent/bash/parser"; import type { TSNode } from "#src/access-intent/bash/parser";
import { import {
@@ -72,6 +73,62 @@ const COMMAND_ENUM_DESCEND = new Set([
*/ */
const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]); const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]);
/**
* Compound statement node types whose bodies really run commands:
* `for`/`while`/`until` loops (tree-sitter-bash folds `until` into
* `while_statement`), `if`/`case` statements, `{ … }` brace groups
* (`compound_statement`), function definitions, and `!`-negated commands.
*
* These were once emitted whole without descending (#306 deferred that), which
* let a gated command hide inside a loop body — a `for …; do find …; done`
* matched only patterns starting with `for`. Each is now emitted whole AND
* enumerated child-by-child; a `negated_command` emits `! grep …` whole and
* then `grep …` so the `!` prefix cannot defeat a command pattern either.
*/
const CONTROL_FLOW_TYPES = new Set([
"for_statement",
"c_style_for_statement",
"while_statement",
"if_statement",
"case_statement",
"compound_statement",
"function_definition",
"negated_command",
]);
/**
* Control-flow nodes that are pure body containers rather than statements in
* their own right: `do …; done`, `elif`/`else` clauses, and `case` items.
* They are descended exactly like {@link CONTROL_FLOW_TYPES} members but are
* not emitted as whole units — nobody writes a command pattern against
* `do …; done` or `a) find . ;;`, so emitting them only adds noise to prompts
* and approval suggestions.
*/
const CONTROL_FLOW_CONTAINERS = new Set([
"do_group",
"elif_clause",
"else_clause",
"case_item",
]);
/**
* Named children of a control-flow node that are statements in their own
* right and get full command enumeration (whole-emit rules, wrapper
* classification, substitution descent). Any other named child — words,
* `variable_name`, a for-loop's value list, a case subject or pattern, a
* `test_command` `[[ … ]]`, a comment — contributes only the nested
* `$(…)`/`<(…)` executions it hosts.
*/
const CONTROL_FLOW_STATEMENT_TYPES = new Set([
"command",
"list",
"pipeline",
"redirected_statement",
"subshell",
...CONTROL_FLOW_TYPES,
...CONTROL_FLOW_CONTAINERS,
]);
/** /**
* Enumerate the command units of a bash program, in source order. * Enumerate the command units of a bash program, in source order.
* *
@@ -82,8 +139,19 @@ const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]);
* subshells (`( … )`) — emitting each inner command as its own unit *in * subshells (`( … )`) — emitting each inner command as its own unit *in
* addition to* the enclosing command, since those inner commands really execute * addition to* the enclosing command, since those inner commands really execute
* (#306). * (#306).
* Control-flow bodies and `{ … }` brace groups are emitted whole without * Control-flow statements (loops, `if`/`elif`/`else`, `case`, `{ … }` groups,
* descending (deferred). * function definitions, `!` negation) are likewise emitted whole and then
* descended child-by-child: statement children (including nested control flow
* and conditions like `if grep -q …; then`) are enumerated as commands, while
* non-statement children (the for-loop value list, case subject/patterns,
* `[[ … ]]` tests, names) contribute only the nested executions they host.
* Pure body containers (`do …; done`, `elif`/`else` clauses, `case` items)
* are descended without a whole-unit emit of their own.
* Body commands keep the enclosing execution context — a loop inside `$(…)`
* runs there, a top-level loop's body runs in the current shell.
* Remaining named statements (`variable_assignment`, `declaration_command`,
* `test_command`, …) emit whole and still report the nested executions they
* host, so `x=$(grep foo f)` and `export X=$(find .)` gate the inner command.
* *
* The enclosing command/subshell is always still emitted whole, so adding the * The enclosing command/subshell is always still emitted whole, so adding the
* nested units can only ever produce a more-restrictive decision, never weaker. * nested units can only ever produce a more-restrictive decision, never weaker.
@@ -135,9 +203,57 @@ function collectCommandsInto(
return; return;
} }
// Any other named statement (compound_statement `{ … }`, if/while/for/case, if (CONTROL_FLOW_TYPES.has(node.type)) {
// function_definition): emit whole, do not descend — deferred (#306). out.push(makeUnit(node.text, context)); // never-weaker whole emit
descendControlFlowChildren(node, context, out);
return;
}
if (CONTROL_FLOW_CONTAINERS.has(node.type)) {
// Pure body container — enumerate its contents, emit no unit of its own.
descendControlFlowChildren(node, context, out);
return;
}
// Any other named statement (variable_assignment, declaration_command,
// test_command, …) keeps the whole-emit fallback, but a substitution in its
// payload still really runs — `x=$(grep foo f)` or `export X=$(find .)` —
// so hosted executions are enumerated in addition to the whole text.
out.push(makeUnit(node.text, context)); out.push(makeUnit(node.text, context));
collectHostedCommands(node, out);
}
/**
* Enumerate inside a control-flow statement, keeping the enclosing context.
*
* Statement children (`command`, `list`, `pipeline`, `redirected_statement`,
* `subshell`, or nested control flow such as a `do_group`)
* get full enumeration. Other named children are scanned only for nested
* executions: a `command_substitution` child directly (the `$(…)` in
* `for f in $(…)` or `case $(…) in`) is a context node in its own right — the
* strict-subtree {@link forEachNestedExecution} walk would not see it — so it
* is descended with its own context, while words, `variable_name`, test and
* expression subtrees go through the hosted-execution search.
*/
function descendControlFlowChildren(
node: TSNode,
context: BashCommandContext | undefined,
out: BashCommand[],
): void {
for (let i = 0; i < node.childCount; i++) {
const child = node.child(i);
if (!child?.isNamed) continue;
if (CONTROL_FLOW_STATEMENT_TYPES.has(child.type)) {
collectCommandsInto(child, context, out);
continue;
}
const nestedContext = NESTED_EXECUTION_CONTEXTS.get(child.type);
if (nestedContext) {
descendCommandChildren(child, nestedContext, out);
} else {
collectHostedCommands(child, out);
}
}
} }
function makeUnit( function makeUnit(
@@ -0,0 +1,179 @@
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
resetWarmBashParser,
warmBashParser,
} from "#src/access-intent/bash/parser";
import { parseBashCommandsSync } from "#src/access-intent/bash/sync-commands";
describe("collectCommands — control-flow descent", () => {
beforeEach(async () => {
resetWarmBashParser();
await warmBashParser();
});
afterEach(() => {
resetWarmBashParser();
});
it("emits a for-loop's body commands in addition to the whole statement", () => {
expect(
parseBashCommandsSync('for d in a b; do find "$d" -name x; done'),
).toEqual([
{ text: 'for d in a b; do find "$d" -name x; done' },
{ text: 'find "$d" -name x' },
]);
});
it("emits a command substitution in the for value list as a nested context", () => {
expect(
parseBashCommandsSync('for f in $(find . -name x); do echo "$f"; done'),
).toEqual([
{ text: 'for f in $(find . -name x); do echo "$f"; done' },
{ text: "find . -name x", context: "command_substitution" },
{ text: 'echo "$f"' },
]);
});
it("emits the condition and body commands of a redirected while loop", () => {
expect(
parseBashCommandsSync('while read l; do grep x "$l"; done < list'),
).toEqual([
{ text: 'while read l; do grep x "$l"; done' },
{ text: "read l" },
{ text: 'grep x "$l"' },
]);
});
it("emits an if's condition, branch, and else commands", () => {
expect(
parseBashCommandsSync("if grep -q x f; then echo y; else rm z; fi"),
).toEqual([
{ text: "if grep -q x f; then echo y; else rm z; fi" },
{ text: "grep -q x f" },
{ text: "echo y" },
{ text: "rm z" },
]);
});
it("emits a case item's commands", () => {
expect(parseBashCommandsSync("case $x in a) find . ;; esac")).toEqual([
{ text: "case $x in a) find . ;; esac" },
{ text: "find ." },
]);
});
it("emits a command substitution in the case subject as a nested context", () => {
expect(
parseBashCommandsSync("case $(rm x) in a) find . ;; esac"),
).toEqual([
{ text: "case $(rm x) in a) find . ;; esac" },
{ text: "rm x", context: "command_substitution" },
{ text: "find ." },
]);
});
it("descends a brace group", () => {
expect(parseBashCommandsSync("{ grep x f; }")).toEqual([
{ text: "{ grep x f; }" },
{ text: "grep x f" },
]);
});
it("descends a function definition's body", () => {
expect(parseBashCommandsSync("f() { grep x f; }")).toEqual([
{ text: "f() { grep x f; }" },
{ text: "{ grep x f; }" },
{ text: "grep x f" },
]);
});
it("descends nested loops", () => {
expect(
parseBashCommandsSync("for a in 1; do for b in 2; do find .; done; done"),
).toEqual([
{ text: "for a in 1; do for b in 2; do find .; done; done" },
{ text: "for b in 2; do find .; done" },
{ text: "find ." },
]);
});
it("descends a c-style for loop", () => {
expect(
parseBashCommandsSync("for ((i=0;i<3;i++)); do echo i; done"),
).toEqual([
{ text: "for ((i=0;i<3;i++)); do echo i; done" },
{ text: "echo i" },
]);
});
it("emits no container units for a clean loop body", () => {
expect(parseBashCommandsSync('for d in a b; do echo "$d"; done')).toEqual([
{ text: 'for d in a b; do echo "$d"; done' },
{ text: 'echo "$d"' },
]);
});
it("does not emit a [[ ]] test as a separate command unit", () => {
expect(parseBashCommandsSync("if [[ -f x ]]; then echo y; fi")).toEqual([
{ text: "if [[ -f x ]]; then echo y; fi" },
{ text: "echo y" },
]);
});
it("catches a substitution hosted inside a [[ ]] test", () => {
expect(
parseBashCommandsSync('if [[ "$(rm x)" = y ]]; then echo y; fi'),
).toEqual([
{ text: 'if [[ "$(rm x)" = y ]]; then echo y; fi' },
{ text: "rm x", context: "command_substitution" },
{ text: "echo y" },
]);
});
});
describe("collectCommands — assignment, declaration, and negation", () => {
beforeEach(async () => {
resetWarmBashParser();
await warmBashParser();
});
afterEach(() => {
resetWarmBashParser();
});
it("emits a substitution in a plain assignment value as a nested context", () => {
expect(parseBashCommandsSync("x=$(grep foo f)")).toEqual([
{ text: "x=$(grep foo f)" },
{ text: "grep foo f", context: "command_substitution" },
]);
});
it("emits a substitution in an export declaration value", () => {
expect(parseBashCommandsSync("export X=$(find .)")).toEqual([
{ text: "export X=$(find .)" },
{ text: "find .", context: "command_substitution" },
]);
});
it("emits a substitution in a local declaration inside a function body", () => {
expect(parseBashCommandsSync("f() { local y=$(rg z); }")).toEqual([
{ text: "f() { local y=$(rg z); }" },
{ text: "{ local y=$(rg z); }" },
{ text: "rg z", context: "command_substitution" },
]);
});
it("emits a negated command's inner command as a normal unit", () => {
expect(parseBashCommandsSync("! grep -q x f")).toEqual([
{ text: "! grep -q x f" },
{ text: "grep -q x f" },
]);
});
it("emits a negated condition's inner command inside an if", () => {
expect(parseBashCommandsSync("if ! grep -q x f; then :; fi")).toEqual([
{ text: "if ! grep -q x f; then :; fi" },
{ text: "! grep -q x f" },
{ text: "grep -q x f" },
{ text: ":" },
]);
});
});
@@ -14,10 +14,13 @@ import { BashProgram } from "#src/access-intent/bash/program";
import { resolveBashCommandCheck } from "#src/handlers/gates/bash-command"; import { resolveBashCommandCheck } from "#src/handlers/gates/bash-command";
import { pathFlavorForPlatform } from "#src/path/path-flavor"; import { pathFlavorForPlatform } from "#src/path/path-flavor";
import { PathNormalizer } from "#src/path-normalizer"; import { PathNormalizer } from "#src/path-normalizer";
import { PermissionResolver } from "#src/permission-resolver";
import type { ScopedPermissionResolver } from "#src/permission-resolver"; import type { ScopedPermissionResolver } from "#src/permission-resolver";
import type { PermissionState } from "#src/types"; import { SessionRules } from "#src/session-rules";
import type { DenyWithReason, PermissionState } from "#src/types";
import { makeCheckResult } from "#test/helpers/handler-fixtures"; import { makeCheckResult } from "#test/helpers/handler-fixtures";
import { createInMemoryManager } from "#test/helpers/manager-harness";
/** Decision strength ordering: deny (2) > ask (1) > allow (0). */ /** Decision strength ordering: deny (2) > ask (1) > allow (0). */
const STRENGTH: Record<PermissionState, number> = { const STRENGTH: Record<PermissionState, number> = {
@@ -146,3 +149,103 @@ describe("bash command gate — nested execution hosts do not weaken", () => {
expect(await decide("cat <<'EOF'\n$(rm x)\nEOF", resolver)).toBe("allow"); expect(await decide("cat <<'EOF'\n$(rm x)\nEOF", resolver)).toBe("allow");
}); });
}); });
/**
* The same totality property for control-flow bodies.
*
* A `for`/`while`/`if`/`case`/function was once emitted whole without
* descending, so `for d in …; do find …; done` only ever matched patterns
* starting with `for` — a `find*: deny` rule was bypassed. The enumerator now
* emits the whole statement AND its body commands, so hosting a gated command
* inside control flow cannot weaken its decision.
*/
describe("bash command gate — control-flow bodies do not weaken", () => {
/** Real pattern matching over a bash map: `*` allow, the rest deny-with-reason. */
function makePolicyResolver(
bash: Record<string, PermissionState | DenyWithReason>,
): ScopedPermissionResolver {
const manager = createInMemoryManager({
global: { permission: { "*": "allow", bash } },
});
return new PermissionResolver(manager, new SessionRules());
}
async function checkWith(
command: string,
resolver: ScopedPermissionResolver,
) {
const program = await BashProgram.parse(
command,
new PathNormalizer(pathFlavorForPlatform(process.platform), "/cwd"),
);
return resolveBashCommandCheck(command, program.commands(), undefined, resolver);
}
it("denies the reported repro: find inside a for-loop command substitution", async () => {
const resolver = makePolicyResolver({
"*": "allow",
"find*": { action: "deny", reason: "use the find tool" },
"grep*": { action: "deny", reason: "use the grep tool" },
});
const result = await checkWith(
'cd /Users/sia/code/my-pi && ls -1 extensions/ && echo "=== LOC by ext ===" && for d in pi-*/; do n=$(find "$d" -name \'*.ts\' -not -path \'*/node_modules/*\' -not -path \'*/dist/*\' -not -path \'*/test*\' | xargs wc -l 2>/dev/null | tail -1 | awk \'{print $1}\'); echo "$d $n"; done && echo "=== extensions ts ===" && wc -l extensions/*.ts | tail -3',
resolver,
);
expect(result.state).toBe("deny");
expect(result.command).toBe(
'find "$d" -name \'*.ts\' -not -path \'*/node_modules/*\' -not -path \'*/dist/*\' -not -path \'*/test*\'',
);
expect(result.matchedPattern).toBe("find*");
expect(result.reason).toBe("use the find tool");
expect(result.commandContext).toBe("command_substitution");
});
it.each([
'for d in a b; do find "$d" -name x; done',
'while read l; do grep x "$l"; done < list',
"if grep -q x f; then echo y; else echo n; fi",
"case $x in a) find . ;; esac",
"{ grep x f; }",
"f() { grep x f; }",
"for a in 1; do for b in 2; do find .; done; done",
"x=$(grep foo f)",
"export X=$(find .)",
"f() { local y=$(rg z); }",
"! grep -q x f",
"if ! grep -q x f; then :; fi",
])("denies a gated command inside `%s`", async (command) => {
const resolver = makePolicyResolver({
"*": "allow",
"find*": "deny",
"grep*": "deny",
"rg*": "deny",
});
expect((await checkWith(command, resolver)).state).toBe("deny");
});
it("denies a hard-guard payload hidden in a loop body", async () => {
const resolver = makePolicyResolver({
"*": "allow",
"rm -rf*": "deny",
});
const result = await checkWith("for f in a; do rm -rf /; done", resolver);
expect(result.state).toBe("deny");
expect(result.matchedPattern).toBe("rm -rf*");
});
it("allows a control-flow statement whose body commands are allowed", async () => {
const resolver = makePolicyResolver({
"*": "allow",
"find*": "deny",
"grep*": "deny",
});
expect(
(await checkWith('for d in a b; do echo "$d"; done', resolver)).state,
).toBe("allow");
});
});
@@ -7,6 +7,7 @@ import {
autoReviewConfigFileSchema, autoReviewConfigFileSchema,
normalizeAutoReviewConfig, normalizeAutoReviewConfig,
} from "#src/auto-review/config"; } from "#src/auto-review/config";
import { stripJsonComments } from "#src/config-loader";
import { unifiedConfigSchema } from "#src/config-schema"; import { unifiedConfigSchema } from "#src/config-schema";
/** The bundle's authoritative policy seed, as shipped. */ /** The bundle's authoritative policy seed, as shipped. */
@@ -16,10 +17,10 @@ const SEED_PATH = fileURLToPath(
function readSeed(): Record<string, unknown> { function readSeed(): Record<string, unknown> {
const raw = readFileSync(SEED_PATH, "utf8"); const raw = readFileSync(SEED_PATH, "utf8");
const stripped = raw // Parse with the same string-aware stripper the loader uses: a naive regex
.replace(/^\s*\/\/.*$/gm, "") // strips `/*`-looking runs inside string values (e.g. a `'src/**/*.ts'`
.replace(/\/\*[\s\S]*?\*\//g, ""); // glob in a deny reason) and corrupts the JSON.
return JSON.parse(stripped) as Record<string, unknown>; return JSON.parse(stripJsonComments(raw)) as Record<string, unknown>;
} }
describe("bundled policy seed", () => { describe("bundled policy seed", () => {
+2 -1
View File
@@ -2,9 +2,10 @@
- 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。 - 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。
- 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。 - 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。
- RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合和兜底截断;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。 - RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合;兜底截断默认关闭,新结果的展示截断由 pi-overflow 统一负责;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。
- RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。 - RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。
- FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。 - FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。
- RTK 不处理带 `parentToolCallId` 的嵌套调用(codemode 脚本内调用),否则会丢失 bash 的 structuredContent。
- `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。 - `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。
- read 压缩属于有损能力。任何默认值调整都必须同时说明哪些内容可能被省略、精确读取如何恢复原文,以及对 `edit` 文本匹配和排障证据的影响。 - read 压缩属于有损能力。任何默认值调整都必须同时说明哪些内容可能被省略、精确读取如何恢复原文,以及对 `edit` 文本匹配和排障证据的影响。
- 工具输出压缩可能损失证据。排障和审计相关改动应优先保证原始输出可恢复,并覆盖锚点完整性和截断边界。 - 工具输出压缩可能损失证据。排障和审计相关改动应优先保证原始输出可恢复,并覆盖锚点完整性和截断边界。
+2 -2
View File
@@ -154,7 +154,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
|--------|------|---------|-------|-------------| |--------|------|---------|-------|-------------|
| `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled | | `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled |
| `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation | | `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation |
| `outputCompaction.truncate.enabled` | boolean | `true` | — | Enable hard character truncation | | `outputCompaction.truncate.enabled` | boolean | `false` | — | Enable hard character truncation (off in this bundle — pi-overflow owns display truncation) |
| `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output | | `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output |
### Source Code Filtering Levels ### Source Code Filtering Levels
@@ -195,7 +195,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
"maxLines": 220 "maxLines": 220
}, },
"truncate": { "truncate": {
"enabled": true, "enabled": false,
"maxChars": 12000 "maxChars": 12000
} }
} }
+1
View File
@@ -13,3 +13,4 @@
- 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。 - 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。
- `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。 - `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。
- `outputCompaction.truncate` 兜底截断默认关闭(组合包 `config/rtk-optimizer.json`),新结果的展示截断由 pi-overflow 统一负责;RTK 只保留语义压缩。

Some files were not shown because too many files have changed in this diff Show More