mirror of
https://bitbucket.org/siakitem/my-pi.git
synced 2026-10-26 10:45:22 +00:00
Compare commits
10
Commits
0c45b1a438
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
87940da51a | ||
|
|
2b21163a18 | ||
|
|
dae1a01f23 | ||
|
|
833b9514d4 | ||
|
|
f9a7f743dd | ||
|
|
56cbc83296 | ||
|
|
2a6bf8c929 | ||
|
|
a6d8416c76 | ||
|
|
a254d4fd68 | ||
|
|
e996f59469 |
@@ -1,5 +0,0 @@
|
|||||||
# CodeGraph data files — local to each machine, not for committing.
|
|
||||||
# Ignore everything in .codegraph/ except this file itself, so transient
|
|
||||||
# files (the database, daemon.pid, sockets, logs) never show up in git.
|
|
||||||
*
|
|
||||||
!.gitignore
|
|
||||||
@@ -18,10 +18,11 @@
|
|||||||
| `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` |
|
| `pi-hermes-memory/` | Hermes 风格持久记忆:policy-only 注入、六记忆工具、`skill_manage` |
|
||||||
| `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 |
|
| `pi-minimal-footer/` | 紧凑 footer:上下文仪表、订阅用量条、扩展状态 |
|
||||||
| `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 |
|
| `pi-notify/` | Kitty OSC 99 完成与权限确认通知,精确聚焦来源窗口 |
|
||||||
| `pi-compaction/` | 增量 Session segment、确定性压缩、后台语义摘要、effective token、native compaction |
|
| `pi-compaction/` | 压力触发的观察遮蔽(Pi context_edit)、阈值整段摘要、effective token、native compaction |
|
||||||
| `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep`/`multi_grep` 与 `@` 补全 |
|
| `pi-overflow/` | 新工具结果统一截断:超阈值原文落盘 + 有界视图 + `overflow_query` |
|
||||||
|
| `extensions/fff-override.ts` | FFF `override` 模式接管 `find`/`grep` 与 `@` 补全,`multi_grep` 按上游保持关闭 |
|
||||||
|
| `extensions/bash-override.ts` | 用 Pi 的 `createBashToolDefinition` 重新注册 `bash`,仅替换模型可见描述(追加 `tools.jsonc` `descriptionAppend`);执行逻辑、输出结构、codemode 说明不变 |
|
||||||
| `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) |
|
| `extensions/codegraph.ts` | 原生 `codegraph_explore`,自有 stdio JSON-RPC 直连 `codegraph serve`(`codegraph-client.ts`) |
|
||||||
| `extensions/context-mode.ts` | 经 `prompt-guard.ts` 加载 context-mode adapter 与 skills(`context-mode-filter.ts` 过滤冲突锚点) |
|
|
||||||
| `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) |
|
| `extensions/hashline.ts` | 部署 `config/pi-hashline-edit.json` 后加载 pi-hashline-edit 覆盖 `read`/`edit`(`hashline-config.ts`) |
|
||||||
| `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` |
|
| `extensions/permission-system.ts` | 直接加载仓库内 `pi-permission-system/src/index.ts` |
|
||||||
| `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) |
|
| `extensions/lsp.ts` | 物化包内 typescript-language-server、部署 LSP 配置并加载 pi-lsp(`lsp-config.ts`) |
|
||||||
@@ -35,13 +36,13 @@
|
|||||||
## 全局硬规则
|
## 全局硬规则
|
||||||
|
|
||||||
- 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。
|
- 系统提示词单一所有权:`extensions/system-prompt.ts` 排在 `pi.extensions` 最后,是唯一允许在 `before_agent_start` 返回 `systemPrompt` 的扩展;其他扩展只能经 `systemPromptOptions.sections.<name>` 贡献分区,第三方扩展必须经 `extensions/` wrapper + `prompt-guard.ts` 加载,新增工具必须在 `config/prompt/tools.jsonc` 登记。宿主传入 `customPrompt`(子 Agent / SYSTEM.md)时不覆盖其开场白;子 Agent 有独立提示词画像 `config/prompt/subagent/`(preamble + 内置 Agent),判定规则与路由 scope/`subagentExclude` 细则见 `extensions/AGENTS.md`。
|
||||||
- 加载顺序:`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。
|
- 加载顺序:`pi-overflow/capture.ts` 必须排在最前(先于 RTK 等结果改写扩展暂存原文),`permission-system` → `subagents` → `pi-ssh` → `pi-chrome` → `pi-tool-search` → `pi-overflow/index.ts` → `system-prompt`(最后);`pi-hermes-memory` 必须排在 `pi-compaction` 之前(`session_before_compact` 取消语义),`pi-compaction` 在 `pi-minimal-footer` 与 `context-view` 之前。
|
||||||
- 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前仅 `better-sqlite3@12.11.1`;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不新增放行)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。
|
- 依赖:外部依赖在根 `package.json` 用精确版本,自维护扩展一律 `file:` 本地依赖并更新锁文件;Pi 核心包只作 optional host peer,根 runtime deps 不含 Pi runtime,任何目录不得再装一套 Pi runtime;各子包用于测试的 `@earendil-works/*` devDependencies 一律写 `"latest"`,经 `scripts/update-pi-devdeps.sh` 统一刷新;原生构建依赖只按锁定版本进根 `allowScripts`(当前无放行项;pi-hermes-memory 的 `better-sqlite3@13` 用包内 N-API 预编译,不经根 `allowScripts`)。不要用仓库级 `.pi/settings.json` 代替组合包依赖。根 `files` 对子包按运行时子路径白名单列出(源码、package.json、README/LICENSE/CHANGELOG/UPSTREAM),不得整目录列入,以免打入嵌套 node_modules、dist 和测试;由 `tests/package-files.test.ts` 守护。
|
||||||
- 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。
|
- 配置:`config/` 下全部文件是权威源、必须提交;不得读写 `settings.json` 自定义段、agent 目录独立配置文件或以环境变量作持久默认值;`config/` 只放机器无关值,路径类默认值在代码中运行时计算。`<agentDir>/my-pi/` 只放可丢弃运行时状态(日志、缓存),连同旧 `extensions/pi-permission-system/` 残留、构建产物与依赖目录都不提交;用户数据(记忆、会话、`auth.json` 等)不属于组合包配置,也不得放入该目录。
|
||||||
- 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。
|
- 组合包不加载任何 MCP adapter/SDK:Exa 与 CodeGraph 均用自有直连客户端。
|
||||||
- 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。
|
- 导入/同步上游:只从官方 Git 仓库的明确 tag/commit 取源码快照,禁止 npm tarball/缓存/`node_modules`/`dist/` 等预编译产物作导入源;保留上游 LICENSE、作者、测试与来源说明,移除嵌套 `.git`、`.pi` 状态、`node_modules` 与构建产物;同步前先核对本地改动再按明确范围移植,不得整体覆盖。各目录的维护约定见 `AGENTS.md`,每个自维护目录必须有中文 `UPSTREAM.md` 记录来源,上游原文文件不翻译不改写。
|
||||||
- 脚本:`install.sh`、`update.sh`、`uninstall.sh`、`configure.sh` 与 `scripts/lib.sh` 保持 POSIX `sh` 兼容并留在 `files` 中,行为变化同步 README;安装默认来源 `git:git@bitbucket.org:siakitem/my-pi.git`,可由 `PI_PACKAGE_SOURCE` 覆盖(update/uninstall 同规则);组合包安装失败立即停止,机器级依赖失败继续检查其余项并最终返回非零。
|
- 脚本:`install.sh`、`update.sh`、`uninstall.sh`、`configure.sh` 与 `scripts/lib.sh` 保持 POSIX `sh` 兼容并留在 `files` 中,行为变化同步 README;安装默认来源 `git:git@bitbucket.org:siakitem/my-pi.git`,可由 `PI_PACKAGE_SOURCE` 覆盖(update/uninstall 同规则);组合包安装失败立即停止,机器级依赖失败继续检查其余项并最终返回非零。
|
||||||
- 安装/升级纪律:机器级依赖与终端配置逐项询问、默认拒绝;`update.sh` 只升级已安装项、缺失项跳过,组件升级先比较本地与远端版本、一致不重装,成功升级后把 git 来源的受管 checkout `git reset --hard` 还原到已发布版本(本地路径来源不 reset、不 `git clean`);`pi update` 失败 fail-fast——Pi 1.3.0 会据 `.pi-update-incomplete` 标记在下次 update 自愈,不做预清理或人工恢复。`pi update` 只在有新版本时整体覆盖已安装副本(含 `config/`),不逐字段迁移本机改动;需长期保留的自定义必须改仓库 `config/` 源文件。`.zshrc` 只改 `# >>> my-pi:<id> >>>` 受管块并先建时间戳备份,不覆盖用户 `~/.p10k.zsh`;卸载不得删除用户记忆数据(`.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/`)或共享机器级工具。
|
- 安装/升级纪律:机器级依赖与终端配置逐项询问、默认拒绝;`update.sh` 只升级已安装项、缺失项跳过,组件升级先比较本地与远端版本、一致不重装,成功升级后把 git 来源的受管 checkout `git reset --hard` 还原到已发布版本(本地路径来源不 reset、不 `git clean`);`update.sh` 先 `pi update --self` 同步 Pi 最新 release(版本比较由 Pi 自身完成,失败记录后继续),再升级组合包;组合包 `pi update` 失败 fail-fast——Pi 1.3.0 会据 `.pi-update-incomplete` 标记在下次 update 自愈,不做预清理或人工恢复。`pi update` 只在有新版本时整体覆盖已安装副本(含 `config/`),不逐字段迁移本机改动;需长期保留的自定义必须改仓库 `config/` 源文件。`.zshrc` 只改 `# >>> my-pi:<id> >>>` 受管块并先建时间戳备份,不覆盖用户 `~/.p10k.zsh`;卸载不得删除用户记忆数据(`.pi/agent/pi-hermes-memory/`、`projects-memory/`、`.hippo/`)或共享机器级工具。
|
||||||
- 未经明确要求不执行发布、提交、推送、运行安装/卸载脚本、安装到用户 Pi 目录等外部写操作;凭据不进入命令参数、日志、会话或仓库(`auth.json` 权限 `600`、不提交)。
|
- 未经明确要求不执行发布、提交、推送、运行安装/卸载脚本、安装到用户 Pi 目录等外部写操作;凭据不进入命令参数、日志、会话或仓库(`auth.json` 权限 `600`、不提交)。
|
||||||
- 优先在目标扩展目录内完成改动;保留原项目 LICENSE 与版权;不让一个扩展依赖另一个扩展的未公开内部实现。
|
- 优先在目标扩展目录内完成改动;保留原项目 LICENSE 与版权;不让一个扩展依赖另一个扩展的未公开内部实现。
|
||||||
|
|
||||||
|
|||||||
@@ -2,9 +2,8 @@
|
|||||||
|
|
||||||
个人维护的 Pi 扩展组合包。安装一次即可加载并配置:
|
个人维护的 Pi 扩展组合包。安装一次即可加载并配置:
|
||||||
|
|
||||||
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find`、`grep` 和 `multi_grep`。
|
- `@ff-labs/pi-fff@0.10.5`:使用官方 `override` 模式接管 `find` 和 `grep`;上游默认关闭的 `multi_grep` 保持关闭。
|
||||||
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
|
- 本仓库维护的 `pi-rtk-optimizer`:只处理 Bash/read 等输出压缩,默认不改写命令,也不处理搜索工具结果。
|
||||||
- `context-mode@1.0.169`:通过 Pi adapter 提供 `ctx_*` 工具、上下文隔离和会话连续性。
|
|
||||||
- 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。
|
- 本仓库自维护的 `pi-web-search`:对模型只暴露 `web_search` / `web_fetch` 两个工具;`web_search` 由当前主模型在 Tavily / Exa / Keenable 间路由,`web_fetch` 固定走 Exa,三家均以纯 `fetch` 直连(Exa 为 `mcp.exa.ai` 的 JSON-RPC,不经 MCP adapter)。
|
||||||
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。
|
- `extensions/codegraph.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性启动常驻 `codegraph serve --mcp` 子进程,由组合包自有 stdio JSON-RPC 客户端直连(无 MCP adapter/SDK);serve 的文件监听自动同步索引,无结果缓存;未初始化项目经确认后才 `codegraph init`,拒绝记入会话不再重复询问。
|
||||||
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
|
- `extensions/context-view.ts`:通过 `/context` 只读显示 `pi-compaction` 的 effective context,不建立第二套上下文口径。
|
||||||
@@ -12,11 +11,12 @@
|
|||||||
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
|
- `pi-hermes-memory/`:从官方 `pi-hermes-memory v0.9.10` 源码导入的持久记忆扩展,提供全局/项目/用户/失败四类记忆、SQLite FTS5 会话与记忆检索、`SKILL.md` 过程记忆、correction 检测与自动 consolidation。
|
||||||
- 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。
|
- 本仓库维护的 `pi-minimal-footer`:基于 `@ogulcancelik/pi-minimal-footer@0.1.10`,用紧凑的上下文仪表和订阅用量条替换默认 footer。
|
||||||
- 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。
|
- 本仓库维护的 `pi-notify`:基于 `@smoose/pi-notify@0.1.1`,使用 Kitty OSC 99 在 Pi 完全 settled 或权限系统即将请求人工确认时通知,并支持点击精确聚焦来源 Kitty 窗口。
|
||||||
- 本仓库维护的 `pi-compaction`:以增量投影统一确定性工具结果压缩、后台语义摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。
|
- 本仓库维护的 `pi-compaction`:压力触发时以 Pi `context_edit` 批量遮蔽旧工具结果、空闲时按阈值触发整段摘要、effective token 计量、`context_tree_query`、portable checkpoint 与 Codex native compaction。
|
||||||
|
- 本仓库自维护的 `pi-overflow`:统一的新工具结果展示截断——超阈值原文落盘、向模型展示「头 + 信号行 + 尾」的有界视图,并提供 `overflow_query` 按 id 搜索、行区间或 JSON 路径取回原文。
|
||||||
- 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。
|
- 本仓库维护的 `pi-ask-user`:提供模型主动调用的 `ask_user_question`;TUI 中保留单题、多题、选择、文本、自定义回答与提交前复核,Paseo 等兼容 RPC 客户端通过标准 Pi extension UI 逐题交互。
|
||||||
- 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。
|
- 本仓库自维护的 `pi-worktree`:纯斜杠命令(`/wt` / `/worktree`)git worktree 管理,`/wt new` 一键拆分、`/wt merge` 一键合并回来源分支(默认 ff-only,可选 `--squash`),通过 `SessionManager.forkFrom` + `switchSession` 切换会话 cwd;登记状态在 `<agentDir>/pi-worktree/state.json`,不注册模型工具、不 push。
|
||||||
- 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。
|
- 本仓库维护的 `pi-tool-search`:从完整工具定义生成并缓存经过校验的工作流分组,默认以最多 5 个动态组、28 个动态工具的 LRU 策略加载原始完整 schema,并支持原子批量激活。
|
||||||
- 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;大体积页面快照写入 Session 临时文件并交给 Context Mode 隔离分析,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。
|
- 本仓库维护的 `pi-chrome`:从上游 `0.15.46` 源码导入,通过显式会话授权接入用户当前已登录的 Chrome Profile;页面快照 JSON 直接作为工具结果返回,超长时由 pi-overflow 截断并用 `overflow_query` 查询,21 个 `chrome_*` 工具仍由 Tool Search 按工作流加载。
|
||||||
- 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。
|
- 本仓库维护的 `pi-permission-system`:只读取一份全局普通策略与一份独立 Hard Guards,按 pattern 具体度裁决 allow/ask/deny,并整合 Codex Guardian 风格 AutoReview 与人工 `defer` 兜底;自动评审模型完全由配置文件指定,未配置时提示并转人工审批;工具显隐仍只由 Tool Search 管理。
|
||||||
- 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。
|
- 本仓库维护的 `pi-subagents`:从 `@tintinweb/pi-subagents@0.18.0` 源码导入并默认加载,提供隔离子 Agent、后台执行、steering、resume、嵌套代理与自定义 Agent;通过 mandatory permission wrapper 接入权限 child lifecycle、根 authority `ask` 转发和根 transcript auto-review 身份桥接,其 3 个编排工具由 Tool Search 的 `subagents` 组按需加载。子 Agent 使用独立提示词画像 `config/prompt/subagent/`(自有 preamble 与内置 Agent 定义),不嵌入父会话提示词。
|
||||||
- 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。
|
- 本仓库维护的 `pi-ssh`:通过纯 Node `ssh2` 持久连接、显式 `ssh_cd` 远端工作区切换、SFTP 与自适应有界搜索提供独立的远端工具;Agent 只在用户明确指定已导入主机及具体任务后发起受 AutoReview 复核的连接。
|
||||||
@@ -68,13 +68,14 @@ Powerlevel10k 主题/配置和各 Zsh 插件使用 `# >>> my-pi:<id> >>>` 到
|
|||||||
./update.sh
|
./update.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
升级脚本先执行:
|
升级脚本先把 Pi 本体同步到最新发布版本,再升级组合包:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
pi update --self
|
||||||
pi update git:git@bitbucket.org:siakitem/my-pi.git
|
pi update git:git@bitbucket.org:siakitem/my-pi.git
|
||||||
```
|
```
|
||||||
|
|
||||||
脚本不会在升级前破坏性预清理安装缓存:`pi update` 失败时立即停止后续升级——Pi 1.3.0 会在下一次 `pi update` 时依据自己的 `.pi-update-incomplete` 标记自动重新清理并恢复未完成的升级,重新运行 `update.sh` 即可,脚本不做人工恢复。
|
`pi update --self` 由 Pi 自己比较本地与最新 release,已是最新时直接跳过;它失败只记录为升级失败,不阻断组合包升级。脚本不会在升级前破坏性预清理安装缓存:组合包的 `pi update` 失败时立即停止后续升级——Pi 1.3.0 会在下一次 `pi update` 时依据自己的 `.pi-update-incomplete` 标记自动重新清理并恢复未完成的升级,重新运行 `update.sh` 即可,脚本不做人工恢复。
|
||||||
|
|
||||||
随后只处理已经安装的项目,未安装项直接跳过且不会询问安装:Kitty 及已启用的
|
随后只处理已经安装的项目,未安装项直接跳过且不会询问安装:Kitty 及已启用的
|
||||||
Solarized Dark、Oh My Zsh、Powerlevel10k、两个第三方 Zsh 插件、CodeGraph,以及由
|
Solarized Dark、Oh My Zsh、Powerlevel10k、两个第三方 Zsh 插件、CodeGraph,以及由
|
||||||
@@ -137,11 +138,11 @@ CodeGraph 的更新或遥测设置。
|
|||||||
|
|
||||||
### 工具与搜索路由
|
### 工具与搜索路由
|
||||||
|
|
||||||
`config/prompt/routing.jsonc` 定义跨工具路由规则,由 `extensions/system-prompt.ts` 在每轮开始前按当前激活工具渲染为 `<tool_routing>` 分区。代码结构、调用关系和待修改 symbol 优先使用 CodeGraph,定义、引用、类型和修改后诊断优先使用 LSP;仓库字面搜索先用 FFF `find` 缩小文件或目录范围,再在已收敛的路径中用 `grep`/`multi_grep` 获取行号。宽泛搜索命中大量结果、发生截断或达到上限时,应继续缩小路径、glob 或 pattern,而不是提高 limit 或输出全部结果。
|
`config/prompt/routing.jsonc` 定义跨工具路由规则,由 `extensions/system-prompt.ts` 在每轮开始前按当前激活工具渲染为 `<tool_routing>` 分区。代码结构、调用关系和待修改 symbol 优先使用 CodeGraph,定义、引用、类型和修改后诊断优先使用 LSP;本地字面搜索按已知信息直接选工具:确切标识符或文本用 `grep` 字面模式并限定 path/exclude,多个命名变体合并为一次 `grep` 正则 alternation,文件名、路径片段或功能概念用 FFF `find` 模糊全路径匹配,然后只 `read` 命中区域。对搜索结果做提取、去重、计数或聚合时,在 codemode 脚本中调用 `tools.grep` / `tools.find` 并用 JavaScript 后处理;bash 中不得运行 `grep`/`rg`/`find`/`fd`/`git grep`(策略 deny 的 reason 指明替代工具),程序需要文件清单时传 `$(git ls-files '<glob>')`。宽泛搜索命中大量结果、发生截断或达到上限时,应继续缩小路径、glob 或 pattern,而不是提高 limit 或输出全部结果。
|
||||||
|
|
||||||
在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。
|
在线搜索默认按任务只选一个服务:关键词不确定、新闻/近期话题或需要较多候选来源时优先 Tavily;官方/API 文档、版本信息、论文和高确定性事实确认优先 Exa;中文政策、政府公告、中文网页、站点限定和日期筛选优先 Keenable。只有来源质量或不确定性确实需要时才跨服务复核,避免默认把三套结果全部塞进上下文。
|
||||||
|
|
||||||
日志、测试/构建输出、工作区内大文件的探索/分析/总结以及不可预测的大输出优先交给 Context Mode。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。
|
日志、测试/构建输出以及不可预测的大输出由 pi-overflow 截断展示,原文可用 `overflow_query` 按 id 检索。只有确实需要精确源码或准备修改时,才用带 `offset/limit` 的 Hashline `read` 读取最小区域并取得 `LINE#HASH` 锚点,随后用锚定 `edit` 修改;成功编辑返回的新锚点可继续复用,只有锚点过期或下一目标区域尚未展示时才重新读取。已有工具结果足够时停止检索,避免对同一问题依次重复调用 CodeGraph、FFF 和 `read`。
|
||||||
|
|
||||||
使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。
|
使用 `/dump-system-prompt` 可将当前扩展所见的有效提示词写入当前项目的 `.pi-debug/effective-system-prompt.md`。`.pi-debug/` 属于本地诊断输出,默认不提交。
|
||||||
|
|
||||||
@@ -149,7 +150,7 @@ CodeGraph 的更新或遥测设置。
|
|||||||
|
|
||||||
`extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。
|
`extensions/system-prompt.ts` 是组合包唯一的系统提示词出口,排在 `pi.extensions` 最后加载。提示词内容全部由 `config/prompt/` 管理:`preamble.md` 替换 Pi 默认开场白(其余 `<tools>`、`<rules>`、`<skills>`、`<project_context>`、`<cwd>` 等分区及顺序原样保留),`tools.jsonc` 覆盖各工具的简介与规则,`routing.jsonc` 渲染 `<tool_routing>` 分区,`sections.jsonc` 决定各扩展贡献分区的顺序。
|
||||||
|
|
||||||
其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP、Context Mode)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。
|
其他扩展不直接改写提示词:pi-chrome、pi-ssh、pi-hermes-memory 等通过 `systemPromptOptions.sections.<name>` 贡献结构化分区,pi-rtk-optimizer 通过 `promptGuidelines` 补充规则;第三方扩展(FFF、Hashline、LSP)经 `extensions/` 下的包装文件与 `prompt-guard.ts` 代理加载,拦截直接的提示词改写并按 `tools.jsonc` 应用工具说明覆盖。
|
||||||
|
|
||||||
`config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。
|
`config/prompt/` 每轮重新读取(按 mtime 缓存),修改已安装组合包内的文件后下一轮即生效;配置解析失败时警告一次并保持 Pi 默认提示词,不阻止会话。该目录由组合包直接维护并随组合包发布;对已安装副本的本地修改不做保护,`pi update` 会直接覆盖;要长期自定义请直接编辑仓库中的 `config/prompt/`。
|
||||||
|
|
||||||
@@ -159,7 +160,7 @@ CodeGraph 的更新或遥测设置。
|
|||||||
|
|
||||||
授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。
|
授权后注册的 21 个 `chrome_*` 工具不会常驻提示词;Tool Search 将它们分为 `chrome-navigation`、`chrome-interaction` 和 `chrome-debugging`。普通网页资料发现仍使用 Tavily/Exa/Keenable,只有需要读取登录态页面或实际操作当前 Chrome 时才加载 Chrome 组。
|
||||||
|
|
||||||
`chrome_snapshot`、`chrome_find` 和带 `includeSnapshot=true` 的交互不会再把完整页面结构直接塞进模型上下文:完整 JSON 只写入当前项目 `.pi/chrome-context/` 下按 Session/目标隔离的临时文件,工具结果仅返回相对路径和小型摘要。模型应立即用 `ctx_execute_file` 从该文件提取下一步所需的 uid、文本或状态,不能用 `read` / `cat` 重新读入全文。每个目标只保留最新快照;页面操作后旧快照失效,Session 结束、`/chrome revoke` 或授权过期时清理。
|
`chrome_snapshot` 和带 `includeSnapshot=true` 的交互把页面快照 JSON 直接作为工具结果返回,`details` 只保留元素数、匹配数等小型元数据;`chrome_find` 只返回紧凑的排序匹配。快照超长时结果会带 `[pi-overflow id=…]` 标记,模型用 `overflow_query` 的 `jsonPath`(如 `elements[0].uid`)或 `search` 提取所需的 uid、文本或状态,而不是重复抓取快照。页面交互后旧快照即失效,需要重新抓取;早期版本写入 Session 临时文件的快照机制已移除。
|
||||||
|
|
||||||
### 按需加载工具 schema
|
### 按需加载工具 schema
|
||||||
|
|
||||||
@@ -167,7 +168,7 @@ CodeGraph 的更新或遥测设置。
|
|||||||
|
|
||||||
本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。
|
本地源码直接使用 `@earendil-works/pi-coding-agent`、当前 `typebox`、`ModelRegistry.complete()` 与 `setActiveTools()`,不安装、别名映射或加载旧 `@mariozechner` runtime,也不使用 provider payload 改写、代理执行或隐藏 `sendMessage` 循环。
|
||||||
|
|
||||||
新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、`ctx_execute`、`ctx_execute_file`、`ctx_batch_execute` 和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Context Mode 执行/知识库/观测/管理、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。
|
新 session 默认常驻 Pi 核心 `read`、`write`、`edit`、`bash`、`grep`、`find`,以及 `codegraph_explore`、`lsp_diagnostics`、六个记忆工具、`context_tree_query`、`overflow_query`、Pi 内置 `codemode`(exposure `model-only`)和 `tool_search`;Windows 上额外把内置 `powershell` 与 `bash` 作为同类本地 Shell 核心工具固定启用,非 Windows 平台则从目录中过滤它。exposure 为 `hidden` 或 `codemode`(script-only)的工具一律不进入目录。`codemode` 在权限策略中为 `allow`:脚本本身无副作用,脚本发起的每个嵌套工具调用仍携带 `parentToolCallId` 经过同一权限门裁决;RTK 跳过所有嵌套调用,保证 bash 的 `structuredContent` 原样传给脚本。组合包在 `pi-tool-search/extensions/bundle-groups.ts` 中为自身暴露的工具预置权威分组,包括本地文件导航、SSH 远端文件/命令、Chrome 导航/交互/调试、用户交互、CodeGraph/LSP、Tavily、Exa、Keenable、Memory 查询/维护与 Skill 管理。不可用、已固定或尚未授权而未注册的可选工具会自动从动态组中滤除。
|
||||||
|
|
||||||
标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。
|
标准组合包的全部隐藏工具都能命中预置目录,因此首次使用不调用模型、不生成用户缓存,也不把完整隐藏 schema 发送给 provider。只有用户另外安装了未识别工具,并且当前调用不是精确加载已有分组时,才可使用当前已认证模型补充目录;精确已有分组始终立即按当前确定性目录激活,不等待无关未知工具的模型增强。模型结果必须保留组合包预置分组,否则直接拒绝。有效增强缓存以 `0600` 写到 agent 目录的 `my-pi/tool-search/catalog-v1.json`,调用用量计入工具结果。用户 `groupOverrides` 的优先级高于预置目录;`/tool-search-rebuild` 会恢复预置目录,`/tool-search-status` 显示目录来源、容量、LRU 顺序和最近淘汰原因。
|
||||||
|
|
||||||
@@ -180,16 +181,15 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
|
|||||||
"alwaysEnabled": [
|
"alwaysEnabled": [
|
||||||
"codegraph_explore",
|
"codegraph_explore",
|
||||||
"lsp_diagnostics",
|
"lsp_diagnostics",
|
||||||
"ctx_execute",
|
|
||||||
"ctx_execute_file",
|
|
||||||
"ctx_batch_execute",
|
|
||||||
"memory_search",
|
"memory_search",
|
||||||
"session_search",
|
"session_search",
|
||||||
"memory_add",
|
"memory_add",
|
||||||
"memory_replace",
|
"memory_replace",
|
||||||
"memory_remove",
|
"memory_remove",
|
||||||
"skill_manage",
|
"skill_manage",
|
||||||
"context_tree_query"
|
"context_tree_query",
|
||||||
|
"overflow_query",
|
||||||
|
"codemode"
|
||||||
],
|
],
|
||||||
"showToolSearchFooterStatus": false,
|
"showToolSearchFooterStatus": false,
|
||||||
"maxActiveGroups": 5,
|
"maxActiveGroups": 5,
|
||||||
@@ -205,29 +205,32 @@ Tool Search 的配置文件是已安装组合包内的 `config/tool-search.json`
|
|||||||
|
|
||||||
组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。
|
组合包固定加载 `pi-hashline-edit@0.8.3`,以带 `LINE#HASH` 行锚点的实现覆盖 Pi 内置 `read` 和 `edit`。`extensions/hashline.ts` 在加载扩展前把 `config/pi-hashline-edit.json` 同步到 `~/.pi/agent/hashline.json`:`hashLength` 保持最小的 `2`,`grep` 强制关闭以避免与 FFF 冲突,`replaceText` 关闭以要求修改使用可验证锚点。
|
||||||
|
|
||||||
Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位、再用 Context Mode 隔离分析,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。
|
Hashline 不负责压缩大文件;每行锚点本身还会增加少量 token。上下文节省来自先用 CodeGraph/LSP 定位,最后只对真正准备修改的区域执行小范围 Hashline `read`。因此不应为了探索、计数、比较或总结而直接读取完整大文件。
|
||||||
|
|
||||||
### 搜索归 FFF
|
### 搜索归 FFF
|
||||||
|
|
||||||
`extensions/fff-override.ts` 强制设置 FFF 官方的 `PI_FFF_MODE=override`(显式传入
|
`extensions/fff-override.ts` 强制设置 FFF 官方的 `PI_FFF_MODE=override`(显式传入
|
||||||
`--fff-mode` 仍按 FFF 官方优先级覆盖它)。FFF 替换
|
`--fff-mode` 仍按 FFF 官方优先级覆盖它)。FFF 替换
|
||||||
Pi 的 `find` 和 `grep`,并提供 `multi_grep`;RTK 对所有 `grep` 工具结果直接跳过,
|
Pi 的 `find` 和 `grep`;`multi_grep` 上游标注为有害、默认关闭,wrapper 在加载前
|
||||||
|
删除 `PI_FFF_MULTIGREP` 环境变量,使其永远无法注册。RTK 对所有 `grep` 工具结果直接跳过,
|
||||||
也默认关闭 Bash 命令改写,因此不会把 `rg`、`grep`、`find` 或 `fd` 改写到 RTK。
|
也默认关闭 Bash 命令改写,因此不会把 `rg`、`grep`、`find` 或 `fd` 改写到 RTK。
|
||||||
|
|
||||||
|
`extensions/bash-override.ts` 用 Pi 的 `createBashToolDefinition` 重新注册内置 `bash`,
|
||||||
|
把「不要用 bash 搜索」的警告经 `tools.jsonc` 的 `descriptionAppend` 写进工具 schema,
|
||||||
|
让模型在调用时就看到替代工具;执行逻辑、输出结构和 codemode 的 `tools.bash` 说明完全沿用上游定义。
|
||||||
|
|
||||||
### 输出压缩归 RTK
|
### 输出压缩归 RTK
|
||||||
|
|
||||||
RTK 保留 Bash、read、build、test、lint 和 Git 输出压缩。默认 `read` 的有损压缩
|
RTK 保留 Bash、read、build、test、lint 和 Git 输出压缩。默认 `read` 的有损压缩
|
||||||
仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有
|
仍关闭;可用 `/rtk` 查看或调整。旧机器已有的 RTK 配置若没有
|
||||||
`commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。
|
`commandRewritingEnabled` 字段,也会按 `false` 归一化,不需要迁移配置。
|
||||||
|
|
||||||
### 大输出归 Context Mode
|
### 大输出归 pi-overflow
|
||||||
|
|
||||||
Context Mode 直接加载 npm 包内置的 Pi adapter 和 skills,不需要额外写入
|
新工具结果的展示截断统一由 `pi-overflow` 负责:超阈值的原文落盘、模型只看到
|
||||||
`~/.pi/agent/mcp.json`,也不需要全局安装 `context-mode` 命令。它提供 `ctx_*` 工具,
|
「头 + 信号行 + 尾」的有界视图,需要原文时用 `overflow_query` 检索(详见下文
|
||||||
将批量读取、命令研究和网页内容先在隔离进程中处理,再把收敛后的结果送入上下文。
|
「Pi Overflow」小节)。FFF 仍负责精确字面搜索;RTK 继续对 Bash、build、test、lint
|
||||||
|
和 Git 输出做语义压缩,其兜底硬截断已关闭。
|
||||||
FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash、build、test、lint
|
|
||||||
和 Git 输出。Context Mode 不替换这两个扩展的现有配置。
|
|
||||||
|
|
||||||
### SSH 服务器配置
|
### SSH 服务器配置
|
||||||
|
|
||||||
@@ -245,10 +248,10 @@ FFF 仍负责精确字面搜索;RTK 继续压缩未走 Context Mode 的 Bash
|
|||||||
|
|
||||||
### 在线搜索服务
|
### 在线搜索服务
|
||||||
|
|
||||||
运行 `./configure.sh search` 可用隐藏输入依次配置 Tavily、Exa 和 Keenable;也可一次无交互配置:
|
运行 `./configure.sh search` 可用隐藏输入依次配置 Tavily、Exa 和 Keenable;key 不得出现在命令行参数中,非交互方式从标准输入读取(每次调用最多一个 provider):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./configure.sh search --tavily '...' --exa '...' --keenable '...'
|
printf '%s' "$TAVILY_API_KEY" | ./configure.sh search --tavily -
|
||||||
```
|
```
|
||||||
|
|
||||||
只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的
|
只传部分参数时保留其他服务的已有值。凭据写入 Pi 自己的
|
||||||
@@ -366,23 +369,31 @@ compaction 和排队 follow-up 完成前不会提前通知;同时监听 `pi-pe
|
|||||||
`permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的
|
`permissions:ui_prompt`,仅在权限系统即将显示人工确认界面时立即通知,策略或 AutoReview 已直接完成的
|
||||||
决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`,
|
决定不会通知。Kitty 后端使用 Base64 OSC 99、每个 Pi Session 独立的稳定通知 ID、显式 `a=focus`,
|
||||||
并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty
|
并默认设置 `o=always`:即使来源 Kitty window/pane 正在接收键盘输入也会显示通知,点击通知由 Kitty
|
||||||
返回原始来源;仍可通过环境变量改用 `unfocused` 或 `invisible`。
|
返回原始来源;仍可通过 `visibility` 改用 `unfocused` 或 `invisible`。
|
||||||
|
|
||||||
默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制
|
默认忽略不足 3 秒的短任务,但权限确认通知不受该时长门槛影响。`/notify on|off|test|status` 可控制
|
||||||
当前 Session 的两类通知;持久默认通过 `PI_NOTIFY_ENABLED`、`PI_NOTIFY_MIN_SECONDS`、
|
当前 Session 的两类通知;持久默认通过组合包 `config/pi-notify.json` 的 `enabled`、`minSeconds`、
|
||||||
`PI_NOTIFY_VISIBILITY`、`PI_NOTIFY_ACTION`、`PI_NOTIFY_MESSAGE_SOURCE` 等环境变量配置。
|
`visibility`、`action`、`messageSource` 等字段配置。`messageSource: "none"`
|
||||||
`PI_NOTIFY_MESSAGE_SOURCE=none` 会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要
|
会同时隐藏回复摘要以及权限值和请求者细节。Kitty 位于 tmux 内时需要
|
||||||
`set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见
|
`set -g allow-passthrough all` 并重启 tmux server。完整配置和上游差异见
|
||||||
[`pi-notify/README.md`](pi-notify/README.md)。
|
[`pi-notify/README.md`](pi-notify/README.md)。
|
||||||
|
|
||||||
### Pi Compaction
|
### Pi Compaction
|
||||||
|
|
||||||
组合包只加载仓库内维护的 `pi-compaction/`。它以增量 Segment Index 管理已经进入 Session 的历史:已闭合工具结果先同步替换为有界的确定性证据 stub,大于阈值的结果再由单 Session 有界后台 worker 生成语义摘要;Provider 请求不会等待后台摘要。原始工具证据不重复写入 custom entry,可通过 `context_tree_query` 使用 source key 或 tool call ID 有界恢复。
|
组合包只加载仓库内维护的 `pi-compaction/`。它管理已经进入 Session 的历史投影:上下文压力达到阈值时,在 `turn_end` 批量提交 Pi 原生 `context_edit`,把最近 `keepRecentTurns` 轮之外的大型工具结果遮蔽为指向 `context_tree_query` 的单行占位符;遮蔽成批发生、受 trigger/target 回差与 `minClearTokens` 约束。原始工具结果保留在 Session 中(UI、导出不变),可通过 `context_tree_query` 按 tool call ID 有界恢复。
|
||||||
|
|
||||||
普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。Pi Core 的 raw threshold 只负责唤醒:只有 effective token 下界达到默认 90% 才执行 threshold compaction;manual 与 overflow 始终允许继续并在 native 失败时回退 Pi 默认 compaction。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。
|
普通 Provider、阈值判断、Codex native compaction、`/context`、`/compaction-status` 和 minimal footer 都读取同一个 effective projection snapshot。整段摘要只在空闲 `agent_settled` 且达到绝对 token 阈值(默认 400K,受窗口比例约束,272K 窗口时约为 90%)时经 `ctx.compact` 触发一次;Pi 自身的阈值压缩作为最终安全网始终有效,`session_before_compact` 不再按扩展自身阈值取消。Codex native API 直接消费 effective ResponseItems,同时保存有界 portable summary 供模型或 Provider 切换。
|
||||||
|
|
||||||
唯一配置位于已安装组合包的 `config/pi-compaction.json`,默认 soft/hard/target 比例为 72%/90%/55%,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和后台任务。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。
|
唯一配置位于已安装组合包的 `config/pi-compaction.json`:`masking` 段为遮蔽触发/目标阈值、`keepRecentTurns`、`minEntryTokens` 与 `excludeTools`,`summary` 段为整段摘要阈值与 thinking 档位,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和已遮蔽结果数。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。
|
||||||
|
|
||||||
原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。
|
原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。
|
||||||
|
|
||||||
本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 Context Mode 职责互补:Context Mode 防止大型原始内容进入会话,Pi Compaction 管理已经进入会话的长期历史。
|
本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 pi-overflow 职责互补:pi-overflow 在新工具结果进入会话时即截断展示,Pi Compaction 管理已经进入会话的长期历史。
|
||||||
|
|
||||||
|
### Pi Overflow
|
||||||
|
|
||||||
|
组合包自维护的 `pi-overflow/` 统一负责「每个新工具结果给模型看多少」。它有两个入口:`capture.ts` 排在 `pi.extensions` 最前,只把原始结果文本暂存到进程内 Map(容量 64),绝不修改结果;`index.ts` 排在 `system-prompt.ts` 之前、所有其他扩展之后,此时 RTK 等 handler 已经完成变换——它用最终可见文本判断是否超阈值,落盘的原文取 bash 的 `details.fullOutputPath` 文件、暂存原文或当前文本三者中的最完整版本,视图与其中的行号则一律基于落盘原文生成,保证与 `overflow_query` 读取的内容一致。
|
||||||
|
|
||||||
|
当最终文本仍超过 `thresholdChars`(默认 24000)时,模型看到的是确定性的「头部 40 行 + 中段匹配 error/warn 等信号行(至多 40 条)+ 尾部 120 行」视图,并写入 `details.overflow.id`;当上游已把结果压短但原文超阈值时,只追加一行 `[pi-overflow id=…]` 恢复标记。原文以 `sha1(sessionId+toolCallId)` 前 12 位 hex 为 id 存入 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),session 启动时按 `retentionDays`/`maxStoreBytes` 清理。模型可用 `overflow_query` 按 id 正则搜索(带上下文合并)、按行区间或 `jsonPath` 读取;`read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用永远豁免。配置为组合包内 `config/pi-overflow.json`,详见 [`pi-overflow/README.md`](pi-overflow/README.md)。
|
||||||
|
|
||||||
|
RTK 的兜底截断与 pi-chrome 的 30K 字符截断已收编进本扩展:`config/rtk-optimizer.json` 中 `outputCompaction.truncate.enabled` 默认关闭,chrome_* 工具返回完整文本;pi-web-search 的 `web_fetch` 默认 `max_characters` 提高到 20000(上限 100000),`web_search` 的 `max_results` 约束在 1–20。
|
||||||
|
|||||||
+13
-13
@@ -1,20 +1,20 @@
|
|||||||
{
|
{
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"softRatio": 0.72,
|
"masking": {
|
||||||
"hardRatio": 0.9,
|
|
||||||
"targetRatio": 0.55,
|
|
||||||
"deterministic": {
|
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"maxStubTokens": 1024
|
"triggerTokens": 200000,
|
||||||
|
"triggerRatio": 0.6,
|
||||||
|
"targetTokens": 100000,
|
||||||
|
"targetRatio": 0.35,
|
||||||
|
"minClearTokens": 50000,
|
||||||
|
"keepRecentTurns": 10,
|
||||||
|
"minEntryTokens": 256,
|
||||||
|
"excludeTools": ["ask_user_question"]
|
||||||
},
|
},
|
||||||
"semantic": {
|
"summary": {
|
||||||
"enabled": true,
|
"triggerTokens": 400000,
|
||||||
"model": "default",
|
"triggerRatio": 0.9,
|
||||||
"thinking": "low",
|
"thinking": "low"
|
||||||
"concurrency": 1,
|
|
||||||
"minInputChars": 4000,
|
|
||||||
"maxInputChars": 24000,
|
|
||||||
"maxRetries": 2
|
|
||||||
},
|
},
|
||||||
"native": {
|
"native": {
|
||||||
"enabled": true
|
"enabled": true
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"showCwd": true,
|
||||||
|
"showBranch": true
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"enabled": true,
|
||||||
|
"title": "π - {project}",
|
||||||
|
"body": "{message}",
|
||||||
|
"messageSource": "assistant",
|
||||||
|
"messageMax": 80,
|
||||||
|
"minSeconds": 3,
|
||||||
|
"visibility": "always",
|
||||||
|
"action": "focus",
|
||||||
|
"soundCommand": ""
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"enabled": true,
|
||||||
|
"thresholdChars": 24000,
|
||||||
|
"headLines": 40,
|
||||||
|
"tailLines": 120,
|
||||||
|
"maxSignalLines": 40,
|
||||||
|
"signalPattern": "\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b",
|
||||||
|
"maxLineChars": 500,
|
||||||
|
"exemptTools": ["read", "edit", "write", "overflow_query", "ask_user_question"],
|
||||||
|
"retentionDays": 7,
|
||||||
|
"maxStoreBytes": 1073741824,
|
||||||
|
"maxEntryBytes": 67108864
|
||||||
|
}
|
||||||
@@ -37,7 +37,6 @@
|
|||||||
"edit": "allow",
|
"edit": "allow",
|
||||||
"grep": "allow",
|
"grep": "allow",
|
||||||
"find": "allow",
|
"find": "allow",
|
||||||
"multi_grep": "allow",
|
|
||||||
"ls": "allow",
|
"ls": "allow",
|
||||||
"codegraph_explore": "allow",
|
"codegraph_explore": "allow",
|
||||||
"lsp_diagnostics": "allow",
|
"lsp_diagnostics": "allow",
|
||||||
@@ -45,13 +44,6 @@
|
|||||||
"lsp_references": "allow",
|
"lsp_references": "allow",
|
||||||
"lsp_hover": "allow",
|
"lsp_hover": "allow",
|
||||||
"lsp_symbols": "allow",
|
"lsp_symbols": "allow",
|
||||||
"ctx_execute": "allow",
|
|
||||||
"ctx_execute_file": "allow",
|
|
||||||
"ctx_batch_execute": "allow",
|
|
||||||
"ctx_search": "allow",
|
|
||||||
"ctx_index": "allow",
|
|
||||||
"ctx_stats": "allow",
|
|
||||||
"ctx_doctor": "allow",
|
|
||||||
"tool_search": "allow",
|
"tool_search": "allow",
|
||||||
"ask_user_question": "allow",
|
"ask_user_question": "allow",
|
||||||
// Launching, steering or reading a subagent has no side effect itself;
|
// Launching, steering or reading a subagent has no side effect itself;
|
||||||
@@ -86,10 +78,15 @@
|
|||||||
// index and never touches the workspace. It is pinned in Tool Search's
|
// index and never touches the workspace. It is pinned in Tool Search's
|
||||||
// alwaysEnabled set, so it must not pay an AutoReview round trip per turn.
|
// alwaysEnabled set, so it must not pay an AutoReview round trip per turn.
|
||||||
"context_tree_query": "allow",
|
"context_tree_query": "allow",
|
||||||
|
// pi-overflow's inspector reads only the extension's own stored output
|
||||||
|
// directory (id-validated); pinned in Tool Search's alwaysEnabled set.
|
||||||
|
"overflow_query": "allow",
|
||||||
|
// A codemode script does nothing by itself: every nested tool call it makes
|
||||||
|
// runs through this same gate (Pi emits tool_call with parentToolCallId).
|
||||||
|
"codemode": "allow",
|
||||||
|
|
||||||
// Model-chosen URLs can exfiltrate data — same reason curl/wget ask.
|
// Model-chosen URLs can exfiltrate data — same reason curl/wget ask.
|
||||||
"web_fetch": "ask",
|
"web_fetch": "ask",
|
||||||
"ctx_fetch_and_index": "ask",
|
|
||||||
|
|
||||||
"ssh_connect": "ask",
|
"ssh_connect": "ask",
|
||||||
"ssh_cd": "ask",
|
"ssh_cd": "ask",
|
||||||
@@ -105,10 +102,10 @@
|
|||||||
},
|
},
|
||||||
"bash": {
|
"bash": {
|
||||||
"*": "allow",
|
"*": "allow",
|
||||||
"rg*": "deny",
|
"rg*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
|
||||||
"grep*": "deny",
|
"grep*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
|
||||||
"find*": "deny",
|
"find*": { "action": "deny", "reason": "Bash file search is disabled. Locally use the find tool (fuzzy path, or a glob in path such as 'src/**/*.ts'); to pass a file list to a program use $(git ls-files '<glob>'). Over SSH use ssh_find." },
|
||||||
"fd*": "deny",
|
"fd*": { "action": "deny", "reason": "Bash file search is disabled. Locally use the find tool (fuzzy path, or a glob in path such as 'src/**/*.ts'); to pass a file list to a program use $(git ls-files '<glob>'). Over SSH use ssh_find." },
|
||||||
"git *": "ask",
|
"git *": "ask",
|
||||||
"git status*": "allow",
|
"git status*": "allow",
|
||||||
"git diff*": "allow",
|
"git diff*": "allow",
|
||||||
@@ -116,7 +113,7 @@
|
|||||||
"git show*": "allow",
|
"git show*": "allow",
|
||||||
"git rev-parse*": "allow",
|
"git rev-parse*": "allow",
|
||||||
"git ls-files*": "allow",
|
"git ls-files*": "allow",
|
||||||
"git grep*": "deny",
|
"git grep*": { "action": "deny", "reason": "Bash content search is disabled. Locally use the grep tool (pattern + path/exclude; regex alternation for several names); to extract/dedupe/count matches, call tools.grep in codemode and process in JS; to filter command output, use overflow_query on the stored result. Over SSH use ssh_grep." },
|
||||||
"git branch": "allow",
|
"git branch": "allow",
|
||||||
"git branch --show-current*": "allow",
|
"git branch --show-current*": "allow",
|
||||||
"git remote -v": "allow",
|
"git remote -v": "allow",
|
||||||
|
|||||||
+21
-17
@@ -16,16 +16,25 @@
|
|||||||
"text": "Use LSP for symbol definitions, references, hover/type information, and post-edit diagnostics when the language server supports the file."
|
"text": "Use LSP for symbol definitions, references, hover/type information, and post-edit diagnostics when the language server supports the file."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["find", "grep", "multi_grep"],
|
"whenAny": ["find", "grep"],
|
||||||
"text": "For literal search, narrow in stages instead of requesting a repository-wide dump:\n 1. Use find to identify likely files or directories. Constrain path/glob and keep the result limit small.\n 2. Use grep or multi_grep only inside the narrowed path or file to obtain matching line numbers. Start with a specific or literal pattern, a small context window, and a bounded limit.\n 3. Use read with offset/limit only for the exact matching region when full source is still needed."
|
"text": "Pick the local search tool by what you already know:\n - Exact identifier or text: grep with a literal pattern, constrained by path/exclude. Do not run find first just to locate it.\n - Several identifiers or naming variants: one grep with a regex alternation (e.g. user_id|userId|UserId) instead of several greps.\n - A filename, path fragment, or feature concept: find (fuzzy whole-path match; use a glob in path for exact names).\n - Then read only the matching region with offset/limit."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["find", "grep", "multi_grep"],
|
"whenAny": ["find", "grep"],
|
||||||
"text": "Do not run a broad search expected to return hundreds or thousands of files or matches. If results truncate or hit a limit, narrow the path, glob, or pattern instead of increasing the limit or dumping all results."
|
"text": "Do not run a broad search expected to return hundreds or thousands of files or matches. If results truncate or hit a limit, narrow the path, glob, or pattern instead of increasing the limit or dumping all results."
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"whenAll": ["codemode"],
|
||||||
|
"whenAny": ["find", "grep"],
|
||||||
|
"text": "To extract, deduplicate, count, or aggregate across search matches (what grep -o | sort -u | wc would do), call tools.grep / tools.find inside one codemode script and post-process the results in JavaScript."
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["bash"],
|
"whenAny": ["bash"],
|
||||||
"text": "Use bash for builds, tests, git, package scripts, and other programs. Never search files by running grep, find, rg, or fd through bash; use the find/grep tools."
|
"text": "Use bash for builds, tests, git, package scripts, and other programs. Never run grep, rg, find, fd, or git grep in bash — standalone, piped, inside a loop, or in $(…). Search files with the grep/find tools; when a program needs a file list (wc, node, tsc), pass $(git ls-files '<glob>') as arguments rather than piping into xargs."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"whenAll": ["bash", "overflow_query"],
|
||||||
|
"text": "To filter a command's output, run it without | grep; if the result carries a [pi-overflow id=…] marker, search it with overflow_query instead of re-running."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["ssh_connect"],
|
"whenAny": ["ssh_connect"],
|
||||||
@@ -50,29 +59,24 @@
|
|||||||
"text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches."
|
"text": "Use web_search for current, real-world, or citable web information; provider selection is handled internally. Keep queries focused and use site/published_* filters only when they materially narrow results. Read selected pages with web_fetch instead of repeating similar searches."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAll": ["web_fetch", "ctx_fetch_and_index"],
|
"whenAll": ["chrome_snapshot", "overflow_query"],
|
||||||
"text": "Read a web page with web_fetch. Use ctx_fetch_and_index only when the content should be indexed into the context-mode knowledge base for repeated ctx_search queries."
|
"text": "chrome_snapshot (and Chrome actions with includeSnapshot=true) return the page snapshot JSON directly. When it is large the result carries a [pi-overflow id=…] marker: extract only the uids, text, or state you need with overflow_query (jsonPath or search) instead of taking another snapshot, and take a fresh snapshot after any page interaction because the previous one is stale."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["ctx_execute", "ctx_execute_file", "ctx_batch_execute"],
|
"whenAny": ["overflow_query"],
|
||||||
"text": "Use Context Mode for logs, test/build output, generated data, large-file exploration/analysis/summarization, and any command whose output may be large or unpredictable. Prefer one ctx_batch_execute over several ctx_execute calls for multi-command research, and print only the derived answer needed for the task."
|
"text": "When a tool result starts with or ends with a [pi-overflow id=…] marker, the full original output is stored; inspect it with overflow_query (search, line ranges, or jsonPath) instead of re-running the command or dumping it again."
|
||||||
},
|
|
||||||
{
|
|
||||||
"whenAny": ["ctx_execute_file"],
|
|
||||||
"text": "ctx_execute_file analyzes a file and never edits it; make changes with read + edit. It is confined to the current project root: do not call it for absolute paths outside the workspace, ../ traversal, or symlinks that resolve outside the workspace; host permission approval does not bypass this boundary. Use another explicitly authorized tool for those files."
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"whenAll": ["chrome_snapshot", "ctx_execute_file"],
|
|
||||||
"text": "After chrome_snapshot or a Chrome action with includeSnapshot=true, immediately analyze the returned workspace-relative capture path with ctx_execute_file; never use read or cat for the raw JSON. Print only the uids, text, or state needed for the next action, and take a fresh snapshot after any interaction because the previous capture may be stale."
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAny": ["read"],
|
"whenAny": ["read"],
|
||||||
"text": "Use read only when exact source is needed or the file is small. Before editing, request the smallest useful offset/limit range; do not read a complete large file merely to copy, compare, hash, count, explore, or summarize it."
|
"text": "Use read only when exact source is needed or the file is small. Before editing, request the smallest useful offset/limit range; do not read a complete large source file merely to copy, compare, hash, count, or explore it. Reading project overview docs such as a README in full is fine."
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"whenAll": ["read", "edit"],
|
"whenAll": ["read", "edit"],
|
||||||
"text": "For existing-file changes, obtain fresh LINE#HASH anchors with a small read (find/grep output has no anchors), then use anchored edit operations. Reuse fresh anchors returned by successful edits for chained changes; re-read only when anchors are stale or the next target region was not shown."
|
"text": "For existing-file changes, obtain fresh LINE#HASH anchors with a small read (find/grep output has no anchors), then use anchored edit operations. Reuse fresh anchors returned by successful edits for chained changes; re-read only when anchors are stale or the next target region was not shown."
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"text": "When a tool call is denied by policy, use the alternative named in the denial and continue the subtask; do not drop it or present unverified facts as checked."
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"text": "Stop searching once the current evidence is sufficient. Avoid repeating the same lookup through CodeGraph, find, grep, and read without a concrete information gap."
|
"text": "Stop searching once the current evidence is sufficient. Avoid repeating the same lookup through CodeGraph, find, grep, and read without a concrete information gap."
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ You locate code and answer "where is X / what references Y" questions for the de
|
|||||||
## Hard limits
|
## Hard limits
|
||||||
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
|
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
|
||||||
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
|
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
|
||||||
- Never search through bash (grep, rg, find, fd); use the find and grep tools.
|
- Never search through bash (grep, rg, find, fd); use the find and grep tools. For several names use one grep with a regex alternation (a|b); when a program needs a file list, pass $(git ls-files '<glob>').
|
||||||
|
|
||||||
## Approach
|
## Approach
|
||||||
- Match the requested breadth: "quick" is one targeted lookup, "medium" covers a few related locations, "very thorough" searches multiple locations and naming conventions.
|
- Match the requested breadth: "quick" is one targeted lookup, "medium" covers a few related locations, "very thorough" searches multiple locations and naming conventions.
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ You explore the codebase and return an implementation plan to the delegating age
|
|||||||
## Hard limits
|
## Hard limits
|
||||||
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
|
- Do not create, modify, move, copy, or delete files, including temporary files anywhere (also /tmp).
|
||||||
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
|
- Use bash only for read-only commands such as git status, git log, git diff, git show, and ls. Never use output redirection, heredocs, or any command that changes system state.
|
||||||
- Never search through bash (grep, rg, find, fd); use the find and grep tools.
|
- Never search through bash (grep, rg, find, fd); use the find and grep tools. For several names use one grep with a regex alternation (a|b); when a program needs a file list, pass $(git ls-files '<glob>').
|
||||||
|
|
||||||
## Process
|
## Process
|
||||||
1. Restate the goal and the constraints you were given. Record anything ambiguous as an open question instead of guessing.
|
1. Restate the goal and the constraints you were given. Record anything ambiguous as an open question instead of guessing.
|
||||||
|
|||||||
+27
-23
@@ -2,23 +2,32 @@
|
|||||||
// Bundle-authoritative tool guidance, owned by extensions/system-prompt.ts.
|
// Bundle-authoritative tool guidance, owned by extensions/system-prompt.ts.
|
||||||
//
|
//
|
||||||
// Every tool the bundle can expose must have an entry. Per field:
|
// Every tool the bundle can expose must have an entry. Per field:
|
||||||
// "snippet" — replaces the tool's one-line <tools> entry (null hides it)
|
// "snippet" — replaces the tool's one-line <tools> entry (null hides it)
|
||||||
// "guidelines" — replaces the tool's <rules> lines
|
// "guidelines" — replaces the tool's <rules> lines
|
||||||
// "description" — replaces the tool description at registration; only
|
// "description" — replaces the tool description at registration
|
||||||
// applied to third-party tools loaded through a bundle
|
// "descriptionAppend" — appended to the description at registration (on top
|
||||||
// adapter (in-repo tools edit their own source instead)
|
// of "description" when both are set)
|
||||||
|
// "description"/"descriptionAppend" only apply to tools registered through a
|
||||||
|
// bundle adapter (prompt-guard), which includes the bash override and the
|
||||||
|
// third-party wrappers; other in-repo tools edit their own source instead.
|
||||||
// An absent field keeps the upstream text. "inherit": true means the
|
// An absent field keeps the upstream text. "inherit": true means the
|
||||||
// upstream text was reviewed against this bundle and is kept as-is.
|
// upstream text was reviewed against this bundle and is kept as-is.
|
||||||
"version": 1,
|
"version": 1,
|
||||||
"tools": {
|
"tools": {
|
||||||
"bash": {
|
"bash": {
|
||||||
"snippet": "Run shell commands for builds, tests, git, package scripts, and other programs; search files with the find/grep tools, not bash"
|
"snippet": "Run shell commands for builds, tests, git, package scripts, and other programs; search files with the find/grep tools, not bash",
|
||||||
|
"descriptionAppend": "Not for searching files: use the grep tool for file contents and the find tool for paths (e.g. `grep -n X src/*.ts` → grep {pattern: \"X\", path: \"src/*.ts\"}); grep, rg, find, fd, and git grep are denied here. When a program needs many files, pass $(git ls-files '<glob>') as arguments instead of piping into xargs, which triggers a permission review.",
|
||||||
|
"guidelines": [
|
||||||
|
"You can inspect PI_* environment variables for current model and session details.",
|
||||||
|
"Search commands are denied in bash; call the tools instead: `grep -n X src/*.ts` → grep {pattern: \"X\", path: \"src/*.ts\"}; `grep -rl X src` → grep {pattern: \"X\", path: \"src/\"} and take the file names; `find . -name '*.ts'` → find {pattern: \"\", path: \"**/*.ts\"}.",
|
||||||
|
"When a program needs many files (wc, node, tsc), pass $(git ls-files '<glob>') as arguments; piping into xargs triggers a permission review."
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"read": { "inherit": true },
|
"read": { "inherit": true },
|
||||||
"edit": {
|
"edit": {
|
||||||
"guidelines": [
|
"guidelines": [
|
||||||
"Use edit with LINE#HASH anchors from the latest read or edit result for all file changes; batch every change to one file into a single edit call.",
|
"Use edit with LINE#HASH anchors from the latest read or edit result for all file changes; batch every change to one file into a single edit call.",
|
||||||
"find/grep/multi_grep results carry plain line numbers, not LINE#HASH anchors; read the exact target range before editing.",
|
"find/grep results carry plain line numbers, not LINE#HASH anchors; read the exact target range before editing.",
|
||||||
"Never reuse anchors from an earlier read or edit response once a newer one exists for the same file; treat only the most recent response as carrying valid anchors.",
|
"Never reuse anchors from an earlier read or edit response once a newer one exists for the same file; treat only the most recent response as carrying valid anchors.",
|
||||||
"After a successful edit, the returned --- Anchors --- block replaces a re-read for nearby follow-up edits.",
|
"After a successful edit, the returned --- Anchors --- block replaces a re-read for nearby follow-up edits.",
|
||||||
"On [E_STALE_ANCHOR], the error lists the stale refs and any content-matched candidate anchors; re-read the file to get current anchors before retrying."
|
"On [E_STALE_ANCHOR], the error lists the stale refs and any content-matched candidate anchors; re-read the file to get current anchors before retrying."
|
||||||
@@ -28,12 +37,18 @@
|
|||||||
"write": { "inherit": true },
|
"write": { "inherit": true },
|
||||||
"ls": { "inherit": true },
|
"ls": { "inherit": true },
|
||||||
"powershell": { "inherit": true },
|
"powershell": { "inherit": true },
|
||||||
"grep": { "inherit": true },
|
"grep": { "snippet": "Search file contents via the FFF index (git-aware, smart-case, frecency-ranked, regex or literal); replaces bash grep/rg" },
|
||||||
"find": { "inherit": true },
|
"find": { "snippet": "Find files by fuzzy path or glob via the FFF index; replaces bash find/fd" },
|
||||||
"multi_grep": { "inherit": true },
|
|
||||||
"tool_search": { "inherit": true },
|
"tool_search": { "inherit": true },
|
||||||
|
"codemode": {
|
||||||
|
"guidelines": [
|
||||||
|
"Use codemode to batch two or more independent tool calls (Promise.allSettled), chain dependent lookups, or filter large output down to what you need; call a single tool directly instead of wrapping it in a script.",
|
||||||
|
"Call edit, write, ssh_connect, ssh_cd, ask_user_question, and Agent directly, never from a codemode script.",
|
||||||
|
"Inside scripts, search with tools.grep or tools.find and do extraction, dedup, and counting in JavaScript; never run grep, find, rg, or fd through tools.bash."
|
||||||
|
]
|
||||||
|
},
|
||||||
|
|
||||||
"codegraph_explore": { "inherit": true },
|
"codegraph_explore": { "snippet": "Explore code structure from the CodeGraph index: symbols, relationships, call flows, and their source" },
|
||||||
"lsp_diagnostics": { "inherit": true },
|
"lsp_diagnostics": { "inherit": true },
|
||||||
"lsp_definition": { "inherit": true },
|
"lsp_definition": { "inherit": true },
|
||||||
"lsp_references": { "inherit": true },
|
"lsp_references": { "inherit": true },
|
||||||
@@ -43,18 +58,6 @@
|
|||||||
"web_search": { "inherit": true },
|
"web_search": { "inherit": true },
|
||||||
"web_fetch": { "inherit": true },
|
"web_fetch": { "inherit": true },
|
||||||
|
|
||||||
"ctx_execute": { "inherit": true },
|
|
||||||
"ctx_execute_file": { "inherit": true },
|
|
||||||
"ctx_batch_execute": { "inherit": true },
|
|
||||||
"ctx_index": { "inherit": true },
|
|
||||||
"ctx_search": { "inherit": true },
|
|
||||||
"ctx_fetch_and_index": { "inherit": true },
|
|
||||||
"ctx_stats": { "inherit": true },
|
|
||||||
"ctx_doctor": { "inherit": true },
|
|
||||||
"ctx_insight": { "inherit": true },
|
|
||||||
"ctx_upgrade": { "inherit": true },
|
|
||||||
"ctx_purge": { "inherit": true },
|
|
||||||
|
|
||||||
"memory_search": { "inherit": true },
|
"memory_search": { "inherit": true },
|
||||||
"session_search": { "inherit": true },
|
"session_search": { "inherit": true },
|
||||||
"memory_add": { "inherit": true },
|
"memory_add": { "inherit": true },
|
||||||
@@ -62,6 +65,7 @@
|
|||||||
"memory_remove": { "inherit": true },
|
"memory_remove": { "inherit": true },
|
||||||
"skill_manage": { "inherit": true },
|
"skill_manage": { "inherit": true },
|
||||||
"context_tree_query": { "inherit": true },
|
"context_tree_query": { "inherit": true },
|
||||||
|
"overflow_query": { "inherit": true },
|
||||||
|
|
||||||
"ask_user_question": { "inherit": true },
|
"ask_user_question": { "inherit": true },
|
||||||
|
|
||||||
|
|||||||
@@ -11,7 +11,7 @@
|
|||||||
"enabled": false
|
"enabled": false
|
||||||
},
|
},
|
||||||
"truncate": {
|
"truncate": {
|
||||||
"enabled": true,
|
"enabled": false,
|
||||||
"maxChars": 12000
|
"maxChars": 12000
|
||||||
},
|
},
|
||||||
"sourceCodeFilteringEnabled": false,
|
"sourceCodeFilteringEnabled": false,
|
||||||
|
|||||||
@@ -2,16 +2,15 @@
|
|||||||
"alwaysEnabled": [
|
"alwaysEnabled": [
|
||||||
"codegraph_explore",
|
"codegraph_explore",
|
||||||
"lsp_diagnostics",
|
"lsp_diagnostics",
|
||||||
"ctx_execute",
|
|
||||||
"ctx_execute_file",
|
|
||||||
"ctx_batch_execute",
|
|
||||||
"memory_search",
|
"memory_search",
|
||||||
"session_search",
|
"session_search",
|
||||||
"memory_add",
|
"memory_add",
|
||||||
"memory_replace",
|
"memory_replace",
|
||||||
"memory_remove",
|
"memory_remove",
|
||||||
"skill_manage",
|
"skill_manage",
|
||||||
"context_tree_query"
|
"context_tree_query",
|
||||||
|
"overflow_query",
|
||||||
|
"codemode"
|
||||||
],
|
],
|
||||||
"showToolSearchFooterStatus": false,
|
"showToolSearchFooterStatus": false,
|
||||||
"maxActiveGroups": 5,
|
"maxActiveGroups": 5,
|
||||||
|
|||||||
+21
-13
@@ -24,24 +24,25 @@ search_usage() {
|
|||||||
cat <<'EOF'
|
cat <<'EOF'
|
||||||
用法:
|
用法:
|
||||||
./configure.sh search
|
./configure.sh search
|
||||||
./configure.sh search --tavily KEY --exa KEY --keenable KEY
|
./configure.sh search --tavily - 从标准输入读取 key
|
||||||
./configure.sh search --migrate [search.env 路径]
|
./configure.sh search --migrate [search.env 路径]
|
||||||
./configure.sh search --list
|
./configure.sh search --list
|
||||||
|
|
||||||
选项:
|
选项:
|
||||||
--tavily KEY 配置 tavily api_key 凭据
|
--tavily - 从标准输入读取 tavily api_key 凭据
|
||||||
--exa KEY 配置 exa api_key 凭据
|
--exa - 从标准输入读取 exa api_key 凭据
|
||||||
--keenable KEY 配置 keenable api_key 凭据
|
--keenable - 从标准输入读取 keenable api_key 凭据
|
||||||
--migrate 从旧 search.env 导入缺失的 key
|
--migrate 从旧 search.env 导入缺失的 key
|
||||||
--list 显示各 provider 配置状态(不输出 key)
|
--list 显示各 provider 配置状态(不输出 key)
|
||||||
--auth FILE 覆盖 auth.json 路径
|
--auth FILE 覆盖 auth.json 路径
|
||||||
-h, --help 显示帮助
|
-h, --help 显示帮助
|
||||||
|
|
||||||
不传 key 参数时进入隐藏输入的交互模式;传入任意 key 参数时不交互,
|
key 不得出现在命令行参数中:非交互方式用 `--<provider> -` 从标准输入
|
||||||
未指定的 provider 保留已有值。凭据写入 Pi 的 auth.json
|
读取,每次调用最多一个 provider 可经标准输入配置;不传参数时进入
|
||||||
($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,权限 600)。
|
隐藏输入的交互模式,未指定的 provider 保留已有值。凭据写入 Pi 的
|
||||||
key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR} 插值、
|
auth.json($PI_CODING_AGENT_DIR/auth.json 或 ~/.pi/agent/auth.json,
|
||||||
以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。
|
权限 600)。key 支持 auth.json 原生语义:字面量、$ENV_VAR / ${ENV_VAR}
|
||||||
|
插值、以及 !command(执行命令取 stdout,如 "!op read op://Vault/Tavily/key")。
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -52,6 +53,7 @@ configure_search() {
|
|||||||
SEARCH_AUTH_FILE=
|
SEARCH_AUTH_FILE=
|
||||||
search_mode=interactive
|
search_mode=interactive
|
||||||
search_did_work=0
|
search_did_work=0
|
||||||
|
search_stdin_provider=
|
||||||
|
|
||||||
run_search_auth() {
|
run_search_auth() {
|
||||||
if [ -n "$SEARCH_AUTH_FILE" ]; then
|
if [ -n "$SEARCH_AUTH_FILE" ]; then
|
||||||
@@ -69,7 +71,7 @@ configure_search() {
|
|||||||
prompt_label=$1
|
prompt_label=$1
|
||||||
prompt_configured=$2
|
prompt_configured=$2
|
||||||
if [ ! -t 0 ]; then
|
if [ ! -t 0 ]; then
|
||||||
warn "交互模式需要终端;请改用 --tavily/--exa/--keenable 参数。"
|
warn "交互模式需要终端;请改用 '--<provider> -' 从标准输入提供 key。"
|
||||||
return 2
|
return 2
|
||||||
fi
|
fi
|
||||||
if [ "$prompt_configured" -eq 1 ]; then
|
if [ "$prompt_configured" -eq 1 ]; then
|
||||||
@@ -88,9 +90,15 @@ configure_search() {
|
|||||||
while [ "$#" -gt 0 ]; do
|
while [ "$#" -gt 0 ]; do
|
||||||
case $1 in
|
case $1 in
|
||||||
--tavily|--exa|--keenable)
|
--tavily|--exa|--keenable)
|
||||||
[ "$#" -ge 2 ] || { warn "$1 缺少 KEY"; exit 2; }
|
[ "$#" -ge 2 ] || { warn "$1 缺少 KEY 或 -"; exit 2; }
|
||||||
|
if [ "$2" != "-" ]; then
|
||||||
|
warn "$1 不接受明文 key:key 不得出现在命令行参数中;请用交互模式或 '$1 -' 从标准输入提供。"
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
[ -z "$search_stdin_provider" ] || { warn "每次调用最多一个 provider 可经标准输入读取 key。"; exit 2; }
|
||||||
|
search_stdin_provider=${1#--}
|
||||||
search_mode=noninteractive; search_did_work=1
|
search_mode=noninteractive; search_did_work=1
|
||||||
run_search_auth --set "${1#--}=$2" || exit $?
|
run_search_auth --set-stdin "$search_stdin_provider" || exit $?
|
||||||
shift 2 ;;
|
shift 2 ;;
|
||||||
--migrate)
|
--migrate)
|
||||||
search_mode=noninteractive; search_did_work=1
|
search_mode=noninteractive; search_did_work=1
|
||||||
@@ -121,7 +129,7 @@ configure_search() {
|
|||||||
if search_has_key "$provider"; then configured=1; else configured=0; fi
|
if search_has_key "$provider"; then configured=1; else configured=0; fi
|
||||||
value=$(search_prompt_key "$label" "$configured") || exit $?
|
value=$(search_prompt_key "$label" "$configured") || exit $?
|
||||||
if [ -n "$value" ]; then
|
if [ -n "$value" ]; then
|
||||||
run_search_auth --set "$provider=$value" || exit $?
|
printf '%s' "$value" | run_search_auth --set-stdin "$provider" || exit $?
|
||||||
search_did_work=1
|
search_did_work=1
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|||||||
@@ -24,10 +24,10 @@
|
|||||||
|
|
||||||
## 各 wrapper 约定
|
## 各 wrapper 约定
|
||||||
|
|
||||||
- `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep`、`multi_grep` 和 `@` 补全;FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。
|
- `fff-override.ts`:强制 FFF 官方 `override` 模式,统一接管 `find`、`grep` 和 `@` 补全;`multi_grep` 按上游保持关闭(加载前删除 `PI_FFF_MULTIGREP` 环境变量)。FFF 独占本地字面搜索,远端服务器搜索由 `ssh_find` / `ssh_grep` 负责,RTK 不得接管搜索工具(见 `pi-rtk-optimizer/AGENTS.md`)。显式 CLI flag 仍遵循 FFF 官方优先级。
|
||||||
|
- `bash-override.ts`:用 `createBashToolDefinition` 重新注册内置 `bash`,目的只是让 `tools.jsonc` 的 `descriptionAppend` 落到模型可见描述里(prompt-guard 在注册时拼接)。ExtensionContext 不暴露 session settings,因此 `shellPath`/`commandPrefix` 在首次 execute 时按 `ctx.cwd` 经 `SettingsManager.create(ctx.cwd, getAgentDir())` 惰性解析并按 cwd 缓存,metadata 来自 `process.cwd()` 的定义;执行、输出结构与 codemode 说明完全委托上游定义,不得改动上游执行逻辑,也不得用 prepareLoadout 或 provider-payload 改写实现同样效果。
|
||||||
- `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。
|
- `codegraph.ts` + `codegraph-client.ts`:注册原生 `codegraph_explore` 工具,按索引根惰性常驻 `codegraph serve --mcp -p <root>` 子进程,经组合包自有 stdio 逐行 JSON-RPC 客户端直连(不经 shell、不带 pi-mcp-adapter 或 MCP SDK);serve 进程内置文件监听自动同步索引,无结果缓存。子进程 10 分钟无调用自动关闭、会话结束统一关闭;调用中死亡换新客户端透明重试一次;其启动的上游 daemon(`.codegraph/daemon.sock`)退出后可多驻留 ≤5 分钟,属上游行为,组合包不管理。
|
||||||
- CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。
|
- CodeGraph 是机器级依赖,`install.sh`/`update.sh` 经官方安装器管理 `codegraph` CLI,组合包不内置其运行时。安装、升级和普通启动流程不得执行 `codegraph init`;只有 `codegraph_explore` 在未初始化项目上被调用、且用户在 `ctx.ui.confirm` 中明确确认后才可 `codegraph init` 当前目录,拒绝记入会话级 declined 不再重复询问;init 失败必须如实报告并让模型回退内置工具;不得改动 CodeGraph 更新或遥测设置。CLI 缺失、init 被拒或服务调用失败都不得抛出,必须以文本结果打回并引导模型改用 `find`/`grep`/`read`/LSP。已移除的 `plugin-init.ts`(原 `/plugin_init`)职责已并入本文件的惰性确认,不得恢复。
|
||||||
- `context-mode.ts` + `context-mode-filter.ts`:经 `prompt-guard.ts` 代理加载上游 `context-mode` Pi adapter 与 skills,提供 `ctx_*` 工具、隔离式大输出处理和会话连续性;Context Mode 负责避免批量读取、命令研究和网页原始内容直接撑大上下文。filter 只丢弃其 context 注入中与本组合包路由重复且冲突的锚点段。
|
|
||||||
- `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
|
- `hashline.ts` + `hashline-config.ts`:部署组合包权威 `config/pi-hashline-edit.json` 后加载 `pi-hashline-edit`;默认以 2 字符行哈希覆盖内置 `read` / `edit`,`grep` 与 `replaceText` 均关闭。它保证锚定编辑而不负责压缩,大文件上下文节省必须来自先定位/隔离分析、再做最小范围 `offset/limit` 读取。
|
||||||
- `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。
|
- `permission-system.ts`:直接加载仓库内 `pi-permission-system/src/index.ts`,自身不部署、不播种、不迁移任何配置;运行时直接读取已安装组合包内 `config/pi-permission-policy.jsonc` / `config/pi-permission-hard-guards.jsonc`;日志保留在 `<agentDir>/my-pi/permission-system/logs/`。旧 `extensions/pi-permission-system/` 位置的残留文件被忽略,不迁移也不清理;项目级 policy 不读取。
|
||||||
- `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。
|
- `lsp.ts` + `lsp-config.ts`:TypeScript/JavaScript 后端由根包固定依赖提供,用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件。
|
||||||
|
|||||||
@@ -0,0 +1,38 @@
|
|||||||
|
import {
|
||||||
|
createBashToolDefinition,
|
||||||
|
type ExtensionAPI,
|
||||||
|
getAgentDir,
|
||||||
|
SettingsManager,
|
||||||
|
} from "@earendil-works/pi-coding-agent";
|
||||||
|
import { guardExtensionApi } from "./prompt-guard.ts";
|
||||||
|
|
||||||
|
type BashDefinition = ReturnType<typeof createBashToolDefinition>;
|
||||||
|
|
||||||
|
/** Re-register Pi's bash so tools.jsonc can append to its model-facing description; execution stays upstream. */
|
||||||
|
export default function bashOverrideExtension(pi: ExtensionAPI): void {
|
||||||
|
const sessionDefinitions = new Map<string, BashDefinition>();
|
||||||
|
const sessionDefinition = (cwd: string): BashDefinition => {
|
||||||
|
let definition = sessionDefinitions.get(cwd);
|
||||||
|
if (definition === undefined) {
|
||||||
|
const settings = SettingsManager.create(cwd, getAgentDir());
|
||||||
|
definition = createBashToolDefinition(cwd, {
|
||||||
|
shellPath: settings.getShellPath(),
|
||||||
|
commandPrefix: settings.getShellCommandPrefix(),
|
||||||
|
});
|
||||||
|
sessionDefinitions.set(cwd, definition);
|
||||||
|
}
|
||||||
|
return definition;
|
||||||
|
};
|
||||||
|
|
||||||
|
guardExtensionApi(pi).registerTool({
|
||||||
|
...sessionDefinition(process.cwd()),
|
||||||
|
execute: (toolCallId, params, signal, onUpdate, ctx) =>
|
||||||
|
sessionDefinition(ctx?.cwd ?? process.cwd()).execute(
|
||||||
|
toolCallId,
|
||||||
|
params,
|
||||||
|
signal,
|
||||||
|
onUpdate,
|
||||||
|
ctx,
|
||||||
|
),
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -222,6 +222,8 @@ export class CodegraphRpcClient {
|
|||||||
// ── internals ────────────────────────────────────────────────────────────
|
// ── internals ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
private attach(child: ChildProcess): void {
|
private attach(child: ChildProcess): void {
|
||||||
|
// EPIPE on a write after the child died is reported via exit/close.
|
||||||
|
child.stdin?.on("error", () => {});
|
||||||
child.stdout?.setEncoding("utf8");
|
child.stdout?.setEncoding("utf8");
|
||||||
child.stdout?.on("data", (chunk: string) => {
|
child.stdout?.on("data", (chunk: string) => {
|
||||||
this.stdoutBuffer += chunk;
|
this.stdoutBuffer += chunk;
|
||||||
|
|||||||
@@ -1,36 +0,0 @@
|
|||||||
/**
|
|
||||||
* Pure filter applied after context-mode's `context` handler: its upstream
|
|
||||||
* build appends one `{ role: "user", content }` message whose parts are
|
|
||||||
* `"\n\n"`-joined and whose first part is a routing anchor that duplicates —
|
|
||||||
* and contradicts — the bundle-owned `config/prompt/routing.jsonc`. The anchor
|
|
||||||
* part is stripped; remaining parts (active memory, resume snapshot) stay.
|
|
||||||
* No other message is touched; when nothing changes the input array is
|
|
||||||
* returned unchanged (same reference).
|
|
||||||
*/
|
|
||||||
|
|
||||||
export const CONTEXT_MODE_ANCHOR_PREFIX = "context-mode active. Hierarchy:";
|
|
||||||
|
|
||||||
type MessageLike = { role?: unknown; content?: unknown } & Record<string, unknown>;
|
|
||||||
|
|
||||||
export function filterContextModeMessages<T extends MessageLike>(
|
|
||||||
before: readonly T[],
|
|
||||||
after: readonly T[],
|
|
||||||
): T[] {
|
|
||||||
if (after.length <= before.length) return after as T[];
|
|
||||||
let result: T[] | null = null;
|
|
||||||
for (let i = after.length - 1; i >= before.length; i--) {
|
|
||||||
const message = after[i];
|
|
||||||
if (message === null || typeof message !== "object") continue;
|
|
||||||
if (message.role !== "user" || typeof message.content !== "string") continue;
|
|
||||||
const parts = message.content.split("\n\n");
|
|
||||||
const kept = parts.filter((part) => !part.startsWith(CONTEXT_MODE_ANCHOR_PREFIX));
|
|
||||||
if (kept.length === parts.length) continue;
|
|
||||||
result ??= after.slice();
|
|
||||||
if (kept.length === 0) {
|
|
||||||
result.splice(i, 1);
|
|
||||||
} else {
|
|
||||||
result[i] = { ...message, content: kept.join("\n\n") };
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return result ?? (after as T[]);
|
|
||||||
}
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
||||||
// The adapter is a prebuilt .js bundle, not a TypeScript source.
|
|
||||||
import piExtension from "../node_modules/context-mode/build/adapters/pi/extension.js";
|
|
||||||
|
|
||||||
import { filterContextModeMessages } from "./context-mode-filter.ts";
|
|
||||||
import { guardExtensionApi } from "./prompt-guard.ts";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Load context-mode through the prompt guard: the bundle keeps sole ownership
|
|
||||||
* of the system prompt, so the upstream routing anchor appended to the
|
|
||||||
* `context` message list is filtered out (bundle routing lives in
|
|
||||||
* `config/prompt/routing.jsonc`), while upstream active-memory/resume parts
|
|
||||||
* pass through untouched. The upstream code this filter depends on is pinned
|
|
||||||
* by `tests/system-prompt.test.ts` against the exact `context-mode` version.
|
|
||||||
*/
|
|
||||||
export default function bundledContextModeExtension(pi: ExtensionAPI): void {
|
|
||||||
piExtension(guardExtensionApi(pi, { contextFilter: filterContextModeMessages }));
|
|
||||||
}
|
|
||||||
@@ -17,7 +17,7 @@ export default function contextView(pi: ExtensionAPI): void {
|
|||||||
`Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`,
|
`Raw: ${formatNumber(metrics.rawTokens)} / ${formatNumber(metrics.contextWindow)}`,
|
||||||
`Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`,
|
`Reclaimed: ${formatNumber(metrics.reclaimedTokens)}`,
|
||||||
`Segments: ${formatNumber(metrics.segmentCount)}`,
|
`Segments: ${formatNumber(metrics.segmentCount)}`,
|
||||||
`Semantic jobs: ${formatNumber(metrics.semanticJobs)}`,
|
`Masked tool results: ${formatNumber(metrics.maskedResults)}`,
|
||||||
`Generation: ${formatNumber(metrics.generation)}`,
|
`Generation: ${formatNumber(metrics.generation)}`,
|
||||||
].join("\n")
|
].join("\n")
|
||||||
: "No pi-compaction projection is available yet. Send one message to build the first snapshot.";
|
: "No pi-compaction projection is available yet. Send one message to build the first snapshot.";
|
||||||
|
|||||||
@@ -5,5 +5,6 @@ import { guardExtensionApi } from "./prompt-guard.ts";
|
|||||||
/** Load FFF as the bundle's canonical `find` and `grep` implementation. */
|
/** Load FFF as the bundle's canonical `find` and `grep` implementation. */
|
||||||
export default function fffOverrideExtension(pi: ExtensionAPI): void {
|
export default function fffOverrideExtension(pi: ExtensionAPI): void {
|
||||||
process.env.PI_FFF_MODE = "override";
|
process.env.PI_FFF_MODE = "override";
|
||||||
|
delete process.env.PI_FFF_MULTIGREP;
|
||||||
fffExtension(guardExtensionApi(pi));
|
fffExtension(guardExtensionApi(pi));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,11 +1,28 @@
|
|||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
|
||||||
import { loadToolCatalog, type ToolCatalog } from "./system-prompt-config.ts";
|
import {
|
||||||
|
loadToolCatalog,
|
||||||
|
type ToolCatalog,
|
||||||
|
type ToolCatalogEntry,
|
||||||
|
} from "./system-prompt-config.ts";
|
||||||
|
|
||||||
type AnyMessage = { role?: string; content?: unknown } & Record<string, unknown>;
|
type AnyMessage = { role?: string; content?: unknown } & Record<string, unknown>;
|
||||||
|
|
||||||
type AnyHandler = (event: never, ctx: never) => unknown;
|
type AnyHandler = (event: never, ctx: never) => unknown;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The description a tool registers with under its catalog entry: a `description`
|
||||||
|
* override replaces the upstream text, a `descriptionAppend` is joined onto
|
||||||
|
* whichever base wins (override first, upstream otherwise) with a blank line.
|
||||||
|
*/
|
||||||
|
export function catalogDescription(description: string, entry: ToolCatalogEntry): string {
|
||||||
|
if ("inherit" in entry) return description;
|
||||||
|
const base = typeof entry.description === "string" ? entry.description : description;
|
||||||
|
return typeof entry.descriptionAppend === "string"
|
||||||
|
? `${base.trimEnd()}\n\n${entry.descriptionAppend}`
|
||||||
|
: base;
|
||||||
|
}
|
||||||
|
|
||||||
export interface PromptGuardOptions {
|
export interface PromptGuardOptions {
|
||||||
/**
|
/**
|
||||||
* Post-process the messages a wrapped `context` handler leaves behind.
|
* Post-process the messages a wrapped `context` handler leaves behind.
|
||||||
@@ -19,9 +36,11 @@ export interface PromptGuardOptions {
|
|||||||
* Proxy an `ExtensionAPI` for third-party extensions so they cannot bypass the
|
* Proxy an `ExtensionAPI` for third-party extensions so they cannot bypass the
|
||||||
* bundle's single system-prompt owner (`extensions/system-prompt.ts`):
|
* bundle's single system-prompt owner (`extensions/system-prompt.ts`):
|
||||||
*
|
*
|
||||||
* - `registerTool` applies the tool's `description` override from
|
* - `registerTool` applies the tool's `description` override and
|
||||||
* `config/prompt/tools.jsonc` when the catalog carries one (lazily loaded;
|
* `descriptionAppend` suffix from `config/prompt/tools.jsonc` when the
|
||||||
* a catalog error keeps the upstream description).
|
* catalog carries them (lazily loaded; a catalog error keeps the upstream
|
||||||
|
* description). An append composes onto the override when both are set, or
|
||||||
|
* onto the upstream description when only the append is set.
|
||||||
* - `before_agent_start` handlers keep every returned field except
|
* - `before_agent_start` handlers keep every returned field except
|
||||||
* `systemPrompt`, which is dropped.
|
* `systemPrompt`, which is dropped.
|
||||||
* - `context` handlers are post-processed through `opts.contextFilter`.
|
* - `context` handlers are post-processed through `opts.contextFilter`.
|
||||||
@@ -42,10 +61,12 @@ export function guardExtensionApi(pi: ExtensionAPI, opts: PromptGuardOptions = {
|
|||||||
|
|
||||||
const registerTool: ExtensionAPI["registerTool"] = (tool) => {
|
const registerTool: ExtensionAPI["registerTool"] = (tool) => {
|
||||||
const entry = catalog()?.tools[tool.name];
|
const entry = catalog()?.tools[tool.name];
|
||||||
if (entry !== undefined && !("inherit" in entry) && typeof entry.description === "string") {
|
const description =
|
||||||
pi.registerTool({ ...tool, description: entry.description });
|
entry === undefined ? tool.description : catalogDescription(tool.description, entry);
|
||||||
} else {
|
if (description === tool.description) {
|
||||||
pi.registerTool(tool);
|
pi.registerTool(tool);
|
||||||
|
} else {
|
||||||
|
pi.registerTool({ ...tool, description });
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -10,7 +10,12 @@ import { fileURLToPath } from "node:url";
|
|||||||
/** Entry in `config/prompt/tools.jsonc`. */
|
/** Entry in `config/prompt/tools.jsonc`. */
|
||||||
export type ToolCatalogEntry =
|
export type ToolCatalogEntry =
|
||||||
| { inherit: true }
|
| { inherit: true }
|
||||||
| { snippet?: string | null; guidelines?: string[]; description?: string };
|
| {
|
||||||
|
snippet?: string | null;
|
||||||
|
guidelines?: string[];
|
||||||
|
description?: string;
|
||||||
|
descriptionAppend?: string;
|
||||||
|
};
|
||||||
|
|
||||||
export interface ToolCatalog {
|
export interface ToolCatalog {
|
||||||
version: 1;
|
version: 1;
|
||||||
@@ -152,7 +157,7 @@ function isStringArray(value: unknown): value is string[] {
|
|||||||
return Array.isArray(value) && value.every((entry) => typeof entry === "string");
|
return Array.isArray(value) && value.every((entry) => typeof entry === "string");
|
||||||
}
|
}
|
||||||
|
|
||||||
const TOOL_ENTRY_KEYS = new Set(["inherit", "snippet", "guidelines", "description"]);
|
const TOOL_ENTRY_KEYS = new Set(["inherit", "snippet", "guidelines", "description", "descriptionAppend"]);
|
||||||
|
|
||||||
function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
|
function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
|
||||||
if (!isPlainObject(raw)) {
|
if (!isPlainObject(raw)) {
|
||||||
@@ -170,9 +175,9 @@ function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
|
|||||||
return { inherit: true };
|
return { inherit: true };
|
||||||
}
|
}
|
||||||
const hasField =
|
const hasField =
|
||||||
"snippet" in raw || "guidelines" in raw || "description" in raw;
|
"snippet" in raw || "guidelines" in raw || "description" in raw || "descriptionAppend" in raw;
|
||||||
if (!hasField) {
|
if (!hasField) {
|
||||||
fail("tools.jsonc", `tool "${name}" needs "inherit": true or at least one of snippet/guidelines/description`);
|
fail("tools.jsonc", `tool "${name}" needs "inherit": true or at least one of snippet/guidelines/description/descriptionAppend`);
|
||||||
}
|
}
|
||||||
if ("snippet" in raw && raw.snippet !== null && typeof raw.snippet !== "string") {
|
if ("snippet" in raw && raw.snippet !== null && typeof raw.snippet !== "string") {
|
||||||
fail("tools.jsonc", `tool "${name}".snippet must be a string or null`);
|
fail("tools.jsonc", `tool "${name}".snippet must be a string or null`);
|
||||||
@@ -183,10 +188,13 @@ function validateToolEntry(name: string, raw: unknown): ToolCatalogEntry {
|
|||||||
if ("description" in raw && typeof raw.description !== "string") {
|
if ("description" in raw && typeof raw.description !== "string") {
|
||||||
fail("tools.jsonc", `tool "${name}".description must be a string`);
|
fail("tools.jsonc", `tool "${name}".description must be a string`);
|
||||||
}
|
}
|
||||||
|
if ("descriptionAppend" in raw && typeof raw.descriptionAppend !== "string") {
|
||||||
|
fail("tools.jsonc", `tool "${name}".descriptionAppend must be a string`);
|
||||||
|
}
|
||||||
return raw as ToolCatalogEntry;
|
return raw as ToolCatalogEntry;
|
||||||
}
|
}
|
||||||
|
|
||||||
function validateToolCatalog(raw: unknown): ToolCatalog {
|
export function validateToolCatalog(raw: unknown): ToolCatalog {
|
||||||
if (!isPlainObject(raw) || raw.version !== 1 || !isPlainObject(raw.tools)) {
|
if (!isPlainObject(raw) || raw.version !== 1 || !isPlainObject(raw.tools)) {
|
||||||
fail("tools.jsonc", 'expected { "version": 1, "tools": { ... } }');
|
fail("tools.jsonc", 'expected { "version": 1, "tools": { ... } }');
|
||||||
}
|
}
|
||||||
@@ -321,8 +329,8 @@ export function isSubagentCustomPrompt(customPrompt: unknown): boolean {
|
|||||||
* Apply the tool catalog to the mutable prompt options: a `snippet` string
|
* Apply the tool catalog to the mutable prompt options: a `snippet` string
|
||||||
* replaces the tool's `<tools>` line (`null` hides it), `guidelines` replaces
|
* replaces the tool's `<tools>` line (`null` hides it), `guidelines` replaces
|
||||||
* the tool's `<rules>` lines; `{inherit:true}` entries are left untouched.
|
* the tool's `<rules>` lines; `{inherit:true}` entries are left untouched.
|
||||||
* `description` is not applied here — it is a registration-time concern handled
|
* `description` and `descriptionAppend` are not applied here — they are
|
||||||
* by `extensions/prompt-guard.ts`.
|
* registration-time concerns handled by `extensions/prompt-guard.ts`.
|
||||||
*
|
*
|
||||||
* Returns the selected tool names that have no catalog entry at all.
|
* Returns the selected tool names that have no catalog entry at all.
|
||||||
*/
|
*/
|
||||||
|
|||||||
Generated
+66
-1553
File diff suppressed because it is too large
Load Diff
+89
-19
@@ -10,15 +10,76 @@
|
|||||||
],
|
],
|
||||||
"files": [
|
"files": [
|
||||||
"extensions",
|
"extensions",
|
||||||
"pi-ask-user",
|
"pi-ask-user/index.ts",
|
||||||
"pi-compaction",
|
"pi-ask-user/src",
|
||||||
"pi-chrome",
|
"pi-ask-user/package.json",
|
||||||
"pi-hermes-memory",
|
"pi-ask-user/README.md",
|
||||||
"pi-rtk-optimizer",
|
"pi-ask-user/LICENSE",
|
||||||
"pi-minimal-footer",
|
"pi-ask-user/UPSTREAM.md",
|
||||||
"pi-subagents",
|
"pi-compaction/index.ts",
|
||||||
"pi-notify",
|
"pi-compaction/src",
|
||||||
"pi-ssh",
|
"pi-compaction/package.json",
|
||||||
|
"pi-compaction/README.md",
|
||||||
|
"pi-compaction/CHANGELOG.md",
|
||||||
|
"pi-compaction/LICENSES",
|
||||||
|
"pi-compaction/UPSTREAM.md",
|
||||||
|
"pi-chrome/extensions",
|
||||||
|
"pi-chrome/package.json",
|
||||||
|
"pi-chrome/README.md",
|
||||||
|
"pi-chrome/CHANGELOG.md",
|
||||||
|
"pi-chrome/SECURITY.md",
|
||||||
|
"pi-chrome/LICENSE",
|
||||||
|
"pi-chrome/UPSTREAM.md",
|
||||||
|
"pi-hermes-memory/src",
|
||||||
|
"pi-hermes-memory/package.json",
|
||||||
|
"pi-hermes-memory/README.md",
|
||||||
|
"pi-hermes-memory/CHANGELOG.md",
|
||||||
|
"pi-hermes-memory/LICENSE",
|
||||||
|
"pi-hermes-memory/UPSTREAM.md",
|
||||||
|
"pi-rtk-optimizer/index.ts",
|
||||||
|
"pi-rtk-optimizer/src",
|
||||||
|
"pi-rtk-optimizer/package.json",
|
||||||
|
"pi-rtk-optimizer/README.md",
|
||||||
|
"pi-rtk-optimizer/CHANGELOG.md",
|
||||||
|
"pi-rtk-optimizer/LICENSE",
|
||||||
|
"pi-rtk-optimizer/UPSTREAM.md",
|
||||||
|
"pi-minimal-footer/index.ts",
|
||||||
|
"pi-minimal-footer/status.ts",
|
||||||
|
"pi-minimal-footer/settings.ts",
|
||||||
|
"pi-minimal-footer/assets",
|
||||||
|
"pi-minimal-footer/package.json",
|
||||||
|
"pi-minimal-footer/README.md",
|
||||||
|
"pi-minimal-footer/LICENSE",
|
||||||
|
"pi-minimal-footer/UPSTREAM.md",
|
||||||
|
"pi-subagents/src",
|
||||||
|
"pi-subagents/package.json",
|
||||||
|
"pi-subagents/README.md",
|
||||||
|
"pi-subagents/CHANGELOG.md",
|
||||||
|
"pi-subagents/SECURITY.md",
|
||||||
|
"pi-subagents/LICENSE",
|
||||||
|
"pi-subagents/UPSTREAM.md",
|
||||||
|
"pi-overflow/index.ts",
|
||||||
|
"pi-overflow/capture.ts",
|
||||||
|
"pi-overflow/settings.ts",
|
||||||
|
"pi-overflow/src",
|
||||||
|
"pi-overflow/package.json",
|
||||||
|
"pi-overflow/README.md",
|
||||||
|
"pi-overflow/LICENSE",
|
||||||
|
"pi-overflow/UPSTREAM.md",
|
||||||
|
"pi-notify/src",
|
||||||
|
"pi-notify/package.json",
|
||||||
|
"pi-notify/README.md",
|
||||||
|
"pi-notify/LICENSE",
|
||||||
|
"pi-notify/UPSTREAM.md",
|
||||||
|
"pi-ssh/index.ts",
|
||||||
|
"pi-ssh/permission-integration.ts",
|
||||||
|
"pi-ssh/src",
|
||||||
|
"pi-ssh/scripts",
|
||||||
|
"pi-ssh/package.json",
|
||||||
|
"pi-ssh/README.md",
|
||||||
|
"pi-ssh/CHANGELOG.md",
|
||||||
|
"pi-ssh/LICENSE",
|
||||||
|
"pi-ssh/UPSTREAM.md",
|
||||||
"pi-tool-search/extensions",
|
"pi-tool-search/extensions",
|
||||||
"pi-tool-search/docs",
|
"pi-tool-search/docs",
|
||||||
"pi-tool-search/package.json",
|
"pi-tool-search/package.json",
|
||||||
@@ -26,8 +87,20 @@
|
|||||||
"pi-tool-search/CHANGELOG.md",
|
"pi-tool-search/CHANGELOG.md",
|
||||||
"pi-tool-search/LICENSE",
|
"pi-tool-search/LICENSE",
|
||||||
"pi-tool-search/UPSTREAM.md",
|
"pi-tool-search/UPSTREAM.md",
|
||||||
"pi-web-search",
|
"pi-web-search/index.ts",
|
||||||
"pi-worktree",
|
"pi-web-search/auth.ts",
|
||||||
|
"pi-web-search/router.ts",
|
||||||
|
"pi-web-search/clients",
|
||||||
|
"pi-web-search/package.json",
|
||||||
|
"pi-web-search/README.md",
|
||||||
|
"pi-web-search/LICENSE",
|
||||||
|
"pi-web-search/UPSTREAM.md",
|
||||||
|
"pi-worktree/index.ts",
|
||||||
|
"pi-worktree/src",
|
||||||
|
"pi-worktree/package.json",
|
||||||
|
"pi-worktree/README.md",
|
||||||
|
"pi-worktree/LICENSE",
|
||||||
|
"pi-worktree/UPSTREAM.md",
|
||||||
"pi-permission-system/LICENSES",
|
"pi-permission-system/LICENSES",
|
||||||
"pi-permission-system/docs/auto-review",
|
"pi-permission-system/docs/auto-review",
|
||||||
"pi-permission-system/src",
|
"pi-permission-system/src",
|
||||||
@@ -36,6 +109,7 @@
|
|||||||
"pi-permission-system/README.md",
|
"pi-permission-system/README.md",
|
||||||
"pi-permission-system/CHANGELOG.md",
|
"pi-permission-system/CHANGELOG.md",
|
||||||
"pi-permission-system/LICENSE",
|
"pi-permission-system/LICENSE",
|
||||||
|
"pi-permission-system/UPSTREAM.md",
|
||||||
"config",
|
"config",
|
||||||
"scripts",
|
"scripts",
|
||||||
"install.sh",
|
"install.sh",
|
||||||
@@ -47,12 +121,13 @@
|
|||||||
],
|
],
|
||||||
"pi": {
|
"pi": {
|
||||||
"extensions": [
|
"extensions": [
|
||||||
|
"./pi-overflow/capture.ts",
|
||||||
"./extensions/fff-override.ts",
|
"./extensions/fff-override.ts",
|
||||||
|
"./extensions/bash-override.ts",
|
||||||
"./pi-web-search/index.ts",
|
"./pi-web-search/index.ts",
|
||||||
"./extensions/codegraph.ts",
|
"./extensions/codegraph.ts",
|
||||||
"./pi-rtk-optimizer/index.ts",
|
"./pi-rtk-optimizer/index.ts",
|
||||||
"./extensions/hashline.ts",
|
"./extensions/hashline.ts",
|
||||||
"./extensions/context-mode.ts",
|
|
||||||
"./extensions/lsp.ts",
|
"./extensions/lsp.ts",
|
||||||
"./pi-hermes-memory/src/index.ts",
|
"./pi-hermes-memory/src/index.ts",
|
||||||
"./pi-compaction/index.ts",
|
"./pi-compaction/index.ts",
|
||||||
@@ -66,10 +141,8 @@
|
|||||||
"./pi-worktree/index.ts",
|
"./pi-worktree/index.ts",
|
||||||
"./pi-chrome/extensions/chrome-profile-bridge/index.ts",
|
"./pi-chrome/extensions/chrome-profile-bridge/index.ts",
|
||||||
"./pi-tool-search/extensions/index.ts",
|
"./pi-tool-search/extensions/index.ts",
|
||||||
|
"./pi-overflow/index.ts",
|
||||||
"./extensions/system-prompt.ts"
|
"./extensions/system-prompt.ts"
|
||||||
],
|
|
||||||
"skills": [
|
|
||||||
"./node_modules/context-mode/skills"
|
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
@@ -78,13 +151,13 @@
|
|||||||
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
|
"@ogulcancelik/pi-minimal-footer": "file:./pi-minimal-footer",
|
||||||
"@smoose/pi-notify": "file:./pi-notify",
|
"@smoose/pi-notify": "file:./pi-notify",
|
||||||
"@tintinweb/pi-subagents": "file:./pi-subagents",
|
"@tintinweb/pi-subagents": "file:./pi-subagents",
|
||||||
"context-mode": "1.0.169",
|
|
||||||
"jiti": "2.7.0",
|
"jiti": "2.7.0",
|
||||||
"pi-ask-user": "file:./pi-ask-user",
|
"pi-ask-user": "file:./pi-ask-user",
|
||||||
"pi-chrome": "file:./pi-chrome",
|
"pi-chrome": "file:./pi-chrome",
|
||||||
"pi-compaction": "file:./pi-compaction",
|
"pi-compaction": "file:./pi-compaction",
|
||||||
"pi-hashline-edit": "0.8.3",
|
"pi-hashline-edit": "0.8.3",
|
||||||
"pi-hermes-memory": "file:./pi-hermes-memory",
|
"pi-hermes-memory": "file:./pi-hermes-memory",
|
||||||
|
"pi-overflow": "file:./pi-overflow",
|
||||||
"pi-lsp": "0.1.7",
|
"pi-lsp": "0.1.7",
|
||||||
"pi-ssh": "file:./pi-ssh",
|
"pi-ssh": "file:./pi-ssh",
|
||||||
"pi-tool-search": "file:./pi-tool-search",
|
"pi-tool-search": "file:./pi-tool-search",
|
||||||
@@ -117,8 +190,5 @@
|
|||||||
},
|
},
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=22.19.0"
|
"node": ">=22.19.0"
|
||||||
},
|
|
||||||
"allowScripts": {
|
|
||||||
"better-sqlite3@12.11.1": true
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+2
-3
@@ -1,9 +1,8 @@
|
|||||||
# my-pi 维护约定(pi-chrome)
|
# my-pi 维护约定(pi-chrome)
|
||||||
|
|
||||||
- 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,Session 快照存储位于 `src/session-snapshot-store.ts`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。
|
- 扩展入口是 `pi-chrome/extensions/chrome-profile-bridge/index.ts`,Companion MV3 扩展位于 `browser-extension/`,测试保留在 `test/` 与 `test-suite/`,来源记录是 `UPSTREAM.md`。
|
||||||
- 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。
|
- 同步上游必须从官方 `tianrendong/pi-chrome` 的明确 tag/commit 移植并保留 README、CHANGELOG、SECURITY、文档、测试、作者和 LICENSE;不得导入嵌套 `.git`、`node_modules` 或构建产物。
|
||||||
- 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。
|
- 根包必须通过 `file:./pi-chrome` 提供依赖并直接加载仓库源码,不得同时安装或加载 npm 预编译入口。
|
||||||
- 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
|
- 控制默认锁定,每个 Pi session 必须显式 `/chrome authorize`;未授权时不注册 Chrome 工具。显式授权后出现的 21 个 `chrome_*` 工具必须由 Tool Search 分入 `chrome-navigation`、`chrome-interaction`、`chrome-debugging`,不得落入 unknown-tools 模型增强路径。授权撤销或过期时必须停用全部 21 个工具,包括 `chrome_find` / `chrome_inspect`;Tool Search 重新激活 schema 不能替代每次执行时的授权校验。
|
||||||
- 完整 Chrome snapshot 不得进入普通工具文本或 `details`;只能原子写入当前项目 `.pi/chrome-context/<session>/<target>/snapshot.json`,使用哈希目录、POSIX `700` 目录 / `600` 文件,并只返回工作区相对路径和有界元数据。快照不得跨 Session 持久化,Session 启动/关闭、撤销和授权过期必须清理。
|
- `chrome_snapshot` 与带 `includeSnapshot=true` 的动作把快照 JSON 直接放进工具结果文本返回;超长结果的展示截断由 pi-overflow 统一负责(原文落盘,模型用 `overflow_query` 的 jsonPath/search 提取)。`details` 只放元素数、匹配数等小型元数据,不得把完整快照写进 `details`(会进入会话文件),也不再写 Session 临时快照文件。`chrome_find` 只返回有界匹配摘要,不保存文件。
|
||||||
- `chrome_snapshot` 和带 `includeSnapshot=true` 的动作必须引导模型立即使用 `ctx_execute_file`,不得使用 `read` / `cat` 把原始 JSON 重新塞入上下文;任何可能改变页面或目标的成功操作必须使旧快照失效,`chrome_find` 只可额外返回有界匹配摘要。
|
|
||||||
- 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。
|
- 修改 bridge、Companion 扩展、授权 lifecycle、快照存储或加载入口后至少运行包内 `npm test`、根 Tool Routing/Tool Search/扩展联合加载测试、`npm pack --dry-run --json` 和实际 packed tarball 隔离安装验证;未经明确要求不得连接或控制真实 Chrome。
|
||||||
|
|||||||
+4
-10
@@ -6,7 +6,7 @@
|
|||||||
|
|
||||||
```text
|
```text
|
||||||
You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
|
You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
|
||||||
Agent: chrome_tab(list) → chrome_snapshot(...) → ctx_execute_file(capture) → chrome_screenshot(...)
|
Agent: chrome_tab(list) → chrome_snapshot(...) → overflow_query(jsonPath) → chrome_screenshot(...)
|
||||||
✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
|
✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
|
||||||
You: [keeps coding — agent never asked you to log in]
|
You: [keeps coding — agent never asked you to log in]
|
||||||
```
|
```
|
||||||
@@ -74,17 +74,11 @@ Second doctor run should show all checks passing.
|
|||||||
|
|
||||||
Tool parameters and gotchas are documented inline in Pi.
|
Tool parameters and gotchas are documented inline in Pi.
|
||||||
|
|
||||||
### Session-local snapshots and Context Mode
|
### Snapshots and pi-overflow
|
||||||
|
|
||||||
In this maintained build, `chrome_snapshot`, `chrome_find`, and actions with `includeSnapshot=true` keep the full structured page snapshot out of the normal tool result. They atomically save only the latest capture for each target under:
|
In this maintained build, `chrome_snapshot` and actions with `includeSnapshot=true` return the full structured page snapshot as JSON directly in the tool result. `chrome_find` returns only up to 20 compact ranked matches.
|
||||||
|
|
||||||
```text
|
Large snapshots are bounded by the bundle's `pi-overflow` extension: the result keeps a bounded view and the full original is stored, marked with `[pi-overflow id=…]`. Extract the uids, text, or state you need with `overflow_query` — a `jsonPath` such as `elements[0].uid` or `summary.focused`, or a regex `search` — instead of taking another snapshot. Any page interaction makes earlier snapshots stale, so take a fresh one (or pass `includeSnapshot=true` on the action) when the page has changed.
|
||||||
.pi/chrome-context/<session-hash>/<target-hash>/snapshot.json
|
|
||||||
```
|
|
||||||
|
|
||||||
The tool returns a workspace-relative path plus bounded metadata. Analyze that path immediately with `ctx_execute_file` and print only the uids, text, or state needed for the next browser action. Do **not** use `read` or `cat` on the capture, because that would put the raw page back into model context.
|
|
||||||
|
|
||||||
Captures are deliberately ephemeral: a successful page interaction invalidates the old target capture, and session shutdown, `/chrome revoke`, or authorization expiry removes the session files. `chrome_find` still returns up to 20 compact ranked matches for direct use while storing the full capture privately.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -15,10 +15,9 @@
|
|||||||
|
|
||||||
导入后的本地改动:
|
导入后的本地改动:
|
||||||
|
|
||||||
- Chrome 快照以原子方式写入当前项目 `.pi/chrome-context/` 目录下的 owner-only、Session 临时文件,而不是把原始页面结构返回给模型上下文。
|
- Chrome 快照与 `includeSnapshot=true` 的动作结果把快照 JSON 直接放入工具文本返回,`details` 只保留元素数/匹配数等小型元数据;展示截断由组合包 pi-overflow 统一负责(原文落盘 + `overflow_query` 查询)。`chrome_find` 只返回紧凑排序匹配,不保存文件。早期版本曾把快照原子写入 `.pi/chrome-context/` 临时文件并引导 `ctx_execute_file` 分析;该 Session 临时文件机制已随 context-mode 一起移除,各项目中残留的 `.pi/chrome-context/` 目录不再创建,可手动删除。
|
||||||
- 每个 Session/目标只保留最新快照;浏览器交互会使陈旧捕获失效,Session 关闭时移除该 Session 目录。
|
|
||||||
- `chrome_find` 返回紧凑的排序匹配,同时把完整结构化快照保留在 Session 本地捕获中,供可选的 `ctx_execute_file` 分析使用。
|
|
||||||
- click/type/fill/key 上的 `includeSnapshot=true` 会保存动作后的快照并返回其捕获路径,而不是把快照嵌入工具结果。
|
|
||||||
- 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。
|
- 授权后的 `before_agent_start` 不再返回 `systemPrompt`:Chrome 使用提示写入 `event.systemPromptOptions.sections.chrome_profile_bridge` 结构化分区,由组合包单一提示词出口 `extensions/system-prompt.ts` 统一组装,顺序由 `config/prompt/sections.jsonc` 控制;未授权时不贡献任何提示词内容。
|
||||||
|
- 移除 `PI_CHROME_BRIDGE_HOST`/`PI_CHROME_BRIDGE_PORT` 环境变量,host/port 固定为 `127.0.0.1:17318`(Companion 扩展 `service_worker.js` 同样硬编码该地址,单侧可配只会断连且非回环 host 有安全风险)。
|
||||||
|
- 移除 `MAX_TEXT_CHARS`/`truncateText` 的 30K 字符文本截断:chrome_* 工具返回完整文本,超长结果的展示截断由 pi-overflow 统一负责;`MAX_ELEMENTS` 等条数上限保留不变。
|
||||||
|
|
||||||
同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期、隐私和 Context Mode 集成。
|
同步上游时,请以导入 commit 为基准比较并选择性移植改动。不得整体覆盖这些本地生命周期和组合包集成改动。
|
||||||
|
|||||||
@@ -4,11 +4,6 @@ import { existsSync, readFileSync, statSync } from "node:fs";
|
|||||||
import { mkdir, writeFile } from "node:fs/promises";
|
import { mkdir, writeFile } from "node:fs/promises";
|
||||||
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
|
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
|
||||||
import { dirname, join, resolve } from "node:path";
|
import { dirname, join, resolve } from "node:path";
|
||||||
import {
|
|
||||||
SessionSnapshotStore,
|
|
||||||
type SnapshotCaptureMetadata,
|
|
||||||
type SnapshotTarget,
|
|
||||||
} from "../../src/session-snapshot-store.ts";
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Existing-profile Chrome bridge for pi.
|
* Existing-profile Chrome bridge for pi.
|
||||||
@@ -64,17 +59,11 @@ const PI_CHROME_GLOBAL_KEY = "__piChromeProfileBridgeLoaded__";
|
|||||||
// reload) so a /reload — which tears down and re-evaluates the module — does not silently drop
|
// reload) so a /reload — which tears down and re-evaluates the module — does not silently drop
|
||||||
// an active /chrome authorize grant.
|
// an active /chrome authorize grant.
|
||||||
const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__";
|
const PI_CHROME_AUTH_KEY = "__piChromeProfileBridgeAuth__";
|
||||||
const DEFAULT_HOST = process.env.PI_CHROME_BRIDGE_HOST ?? "127.0.0.1";
|
const DEFAULT_HOST = "127.0.0.1";
|
||||||
const DEFAULT_PORT = Number(process.env.PI_CHROME_BRIDGE_PORT ?? "17318");
|
const DEFAULT_PORT = 17318;
|
||||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||||
const MAX_TEXT_CHARS = 30_000;
|
|
||||||
const MAX_ELEMENTS = 80;
|
const MAX_ELEMENTS = 80;
|
||||||
|
|
||||||
function truncateText(text: string, maxChars = MAX_TEXT_CHARS): string {
|
|
||||||
if (text.length <= maxChars) return text;
|
|
||||||
return `${text.slice(0, maxChars)}\n\n[truncated ${text.length - maxChars} characters]`;
|
|
||||||
}
|
|
||||||
|
|
||||||
function safeJson(value: unknown): string {
|
function safeJson(value: unknown): string {
|
||||||
return JSON.stringify(value, null, 2);
|
return JSON.stringify(value, null, 2);
|
||||||
}
|
}
|
||||||
@@ -95,13 +84,28 @@ function snapshotFromIncludedResult(raw: unknown): unknown {
|
|||||||
return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined;
|
return raw && typeof raw === "object" ? (raw as { snapshot?: unknown }).snapshot : undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function formatSnapshotCapture(metadata: SnapshotCaptureMetadata): string {
|
/**
|
||||||
const kib = Math.max(1, Math.ceil(metadata.bytes / 1024));
|
* Small, session-file-safe metadata describing a returned snapshot. The full
|
||||||
return [
|
* snapshot goes to the model as text — details must stay small because Pi
|
||||||
`Chrome snapshot saved to ${metadata.path} (${kib} KiB, ${metadata.elementCount} elements${metadata.matchCount ? `, ${metadata.matchCount} matches` : ""}).`,
|
* persists them in the session file.
|
||||||
"Analyze it immediately with ctx_execute_file; FILE_CONTENT is JSON and the page data is under .snapshot.",
|
*/
|
||||||
"This capture is replaced by the next snapshot for the same target and becomes stale after page interaction.",
|
export function snapshotStats(snapshot: unknown): { elementCount: number; matchCount: number } {
|
||||||
].join("\n");
|
const object = snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
|
||||||
|
? snapshot as { elements?: unknown; matches?: unknown }
|
||||||
|
: {};
|
||||||
|
return {
|
||||||
|
elementCount: Array.isArray(object.elements) ? object.elements.length : 0,
|
||||||
|
matchCount: Array.isArray(object.matches) ? object.matches.length : 0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Action summary followed by the inline snapshot JSON. Oversized results are
|
||||||
|
* bounded by pi-overflow and recoverable through overflow_query, so no local
|
||||||
|
* capture file is written.
|
||||||
|
*/
|
||||||
|
export function formatActionResultText(text: string, snapshot: unknown | undefined): string {
|
||||||
|
return snapshot === undefined ? text : `${text}\n\n${safeJson(snapshot)}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function formatChromeFindMatches(snapshot: any): string {
|
export function formatChromeFindMatches(snapshot: any): string {
|
||||||
@@ -142,7 +146,7 @@ function formatChromeInspect(inspect: any): string {
|
|||||||
lines.push("\n## Ancestors");
|
lines.push("\n## Ancestors");
|
||||||
for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`);
|
for (const a of inspect.ancestors.slice(0, 6)) lines.push(`- ${a.uid} ${a.role || a.tag} ${compactLine(a.label || a.selector, 120)}`);
|
||||||
}
|
}
|
||||||
return truncateText(lines.join("\n"));
|
return lines.join("\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
function extensionRoot(): string {
|
function extensionRoot(): string {
|
||||||
@@ -648,7 +652,6 @@ export default function (pi: ExtensionAPI): void {
|
|||||||
let countdownInterval: NodeJS.Timeout | undefined;
|
let countdownInterval: NodeJS.Timeout | undefined;
|
||||||
// Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy.
|
// Remembered so bridge sends can tag tabs with this session's group even when ctx isn't handy.
|
||||||
let sessionCtx: ExtensionContext | undefined;
|
let sessionCtx: ExtensionContext | undefined;
|
||||||
let snapshotStore: SessionSnapshotStore | undefined;
|
|
||||||
|
|
||||||
const clearAuthExpiryTimer = (): void => {
|
const clearAuthExpiryTimer = (): void => {
|
||||||
if (!authExpiryTimer) return;
|
if (!authExpiryTimer) return;
|
||||||
@@ -722,7 +725,6 @@ export default function (pi: ExtensionAPI): void {
|
|||||||
persistAuth();
|
persistAuth();
|
||||||
// Revoking control ends pi-chrome's automation for this session; tidy up the target we own.
|
// Revoking control ends pi-chrome's automation for this session; tidy up the target we own.
|
||||||
cleanupAutomationTargetBestEffort();
|
cleanupAutomationTargetBestEffort();
|
||||||
void snapshotStore?.clear().catch(() => undefined);
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const authSummary = (): string => {
|
const authSummary = (): string => {
|
||||||
@@ -853,45 +855,22 @@ export default function (pi: ExtensionAPI): void {
|
|||||||
return { ...params, foreground: !background } as T;
|
return { ...params, foreground: !background } as T;
|
||||||
};
|
};
|
||||||
|
|
||||||
const snapshotTargetFrom = (params: Record<string, unknown>): SnapshotTarget => ({
|
const formatActionResult = (raw: unknown, params: Record<string, unknown>, text: string): ToolTextResult => {
|
||||||
targetId: typeof params.targetId === "string" ? params.targetId : undefined,
|
|
||||||
urlIncludes: typeof params.urlIncludes === "string" ? params.urlIncludes : undefined,
|
|
||||||
titleIncludes: typeof params.titleIncludes === "string" ? params.titleIncludes : undefined,
|
|
||||||
});
|
|
||||||
|
|
||||||
const saveSnapshotCapture = async (snapshot: unknown, params: Record<string, unknown>): Promise<SnapshotCaptureMetadata> => {
|
|
||||||
if (!snapshotStore) throw new Error("Chrome snapshot storage is unavailable before session_start.");
|
|
||||||
const stored = await snapshotStore.save(snapshot, snapshotTargetFrom(params));
|
|
||||||
return stored.metadata;
|
|
||||||
};
|
|
||||||
|
|
||||||
const invalidateSnapshotCapture = async (params?: Record<string, unknown>): Promise<void> => {
|
|
||||||
if (!snapshotStore) return;
|
|
||||||
if (!params) await snapshotStore.clear();
|
|
||||||
else await snapshotStore.invalidate(snapshotTargetFrom(params));
|
|
||||||
};
|
|
||||||
|
|
||||||
const formatActionWithCapture = async (raw: unknown, params: Record<string, unknown>, text: string): Promise<ToolTextResult> => {
|
|
||||||
const snapshot = snapshotFromIncludedResult(raw);
|
const snapshot = snapshotFromIncludedResult(raw);
|
||||||
const result = params.includeSnapshot && raw && typeof raw === "object"
|
const result = params.includeSnapshot && raw && typeof raw === "object"
|
||||||
? (raw as { result?: unknown }).result
|
? (raw as { result?: unknown }).result
|
||||||
: raw;
|
: raw;
|
||||||
if (!snapshot) {
|
if (!snapshot) {
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text }], details: { result: result as Json } };
|
return { content: [{ type: "text", text }], details: { result: result as Json } };
|
||||||
}
|
}
|
||||||
const capture = await saveSnapshotCapture(snapshot, params);
|
|
||||||
return {
|
return {
|
||||||
content: [{ type: "text", text: `${text}\n\n${formatSnapshotCapture(capture)}` }],
|
content: [{ type: "text", text: formatActionResultText(text, snapshot) }],
|
||||||
details: { result: result as Json, capture },
|
details: { result: result as Json, snapshot: snapshotStats(snapshot) },
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
pi.on("session_start", async (_event, ctx) => {
|
pi.on("session_start", async (_event, ctx) => {
|
||||||
sessionCtx = ctx;
|
sessionCtx = ctx;
|
||||||
const sessionId = ctx.sessionManager.getSessionId?.() || "unknown-session";
|
|
||||||
snapshotStore = new SessionSnapshotStore(workspaceCwd(ctx), sessionId);
|
|
||||||
await snapshotStore.clear();
|
|
||||||
await bridge.start();
|
await bridge.start();
|
||||||
// Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis.
|
// Reestablish in-memory state after a /reload restored chromeAuthorizedUntil from globalThis.
|
||||||
if (chromeControlAuthorized()) {
|
if (chromeControlAuthorized()) {
|
||||||
@@ -909,10 +888,6 @@ export default function (pi: ExtensionAPI): void {
|
|||||||
pi.on("session_shutdown", async (event) => {
|
pi.on("session_shutdown", async (event) => {
|
||||||
clearAuthExpiryTimer();
|
clearAuthExpiryTimer();
|
||||||
clearCountdownInterval();
|
clearCountdownInterval();
|
||||||
// Snapshot files are intentionally ephemeral. A reload must take a fresh observation too,
|
|
||||||
// because the browser page may have changed while the extension was reloading.
|
|
||||||
await snapshotStore?.clear().catch(() => undefined);
|
|
||||||
snapshotStore = undefined;
|
|
||||||
// Keep the owned automation target across /reload, but clean it up on real session end.
|
// Keep the owned automation target across /reload, but clean it up on real session end.
|
||||||
if (event?.reason !== "reload") cleanupAutomationTargetBestEffort();
|
if (event?.reason !== "reload") cleanupAutomationTargetBestEffort();
|
||||||
bridge.stop();
|
bridge.stop();
|
||||||
@@ -939,12 +914,12 @@ Capability model (important):
|
|||||||
- Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates.
|
- Interactive controls (click/type/fill/key/hover/drag/scroll/tap) use Chrome's real input layer via chrome.debugger / CDP. Events satisfy normal user-activation gates.
|
||||||
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
|
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
|
||||||
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP.
|
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP.
|
||||||
- \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` are written to a private session-local JSON capture; raw page data is never returned inline. The tool result gives the path for immediate \`ctx_execute_file\` analysis.
|
- \`chrome_snapshot\` and snapshots requested with \`includeSnapshot=true\` return the page snapshot as JSON directly in the tool result. When the result is large it carries a \`[pi-overflow id=…]\` marker — extract only the uids, text, or state you need with \`overflow_query\` (jsonPath such as \`elements[0].uid\` or \`summary.focused\`, or a \`search\`) instead of taking another snapshot.
|
||||||
|
|
||||||
Usage rules:
|
Usage rules:
|
||||||
1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying.
|
1. If a chrome_* tool says Chrome control is locked, ask the user to run \`/chrome authorize\` before retrying.
|
||||||
2. Call \`chrome_snapshot\`, then immediately call \`ctx_execute_file\` on the returned path and print only the uids/state needed for the next action. Never use read/cat on the full capture.
|
2. Call \`chrome_snapshot\` and read the returned JSON (or query a pi-overflow marker with \`overflow_query\`); keep only the uids/state needed for the next action.
|
||||||
3. Prefer \`uid\` over \`selector\`. After any page interaction, treat the old capture as stale. Use \`includeSnapshot=true\` when a fresh post-action capture is needed.
|
3. Prefer \`uid\` over \`selector\`. After any page interaction, earlier snapshots are stale — take a fresh one (or pass \`includeSnapshot=true\` on the action) instead of reusing old uids.
|
||||||
4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm.
|
4. If \`chrome_evaluate\` returns null when you expected a value, the expression evaluated to null/undefined in the page; surface the value via \`JSON.stringify\` to confirm.
|
||||||
5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now).
|
5. \`chrome_navigate\` supports an optional \`initScript\` that runs at document_start in MAIN world for the next navigation (good for seeding localStorage or stubbing Date.now).
|
||||||
6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work.
|
6. By default chrome_* tools run in the background without focusing Chrome; pass \`background=false\` or run /chrome background off when the user wants to watch Chrome work.
|
||||||
@@ -1337,7 +1312,6 @@ Usage rules:
|
|||||||
const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs.";
|
const text = tabs.map((tab) => `${tab.id}\t${tab.active ? "*" : " "}\t${tab.group?.title ? `[${tab.group.title}] ` : ""}${tab.title || "(untitled)"}\t${tab.url}`).join("\n") || "No tabs.";
|
||||||
return { content: [{ type: "text", text }], details: { tabs } };
|
return { content: [{ type: "text", text }], details: { tabs } };
|
||||||
}
|
}
|
||||||
if (params.action !== "version") await invalidateSnapshotCapture();
|
|
||||||
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1346,8 +1320,8 @@ Usage rules:
|
|||||||
name: "chrome_snapshot",
|
name: "chrome_snapshot",
|
||||||
label: "Chrome Snapshot",
|
label: "Chrome Snapshot",
|
||||||
description:
|
description:
|
||||||
"Capture the current Chrome page into a private, session-local JSON file instead of placing the raw DOM snapshot in model context. The result returns a path for immediate ctx_execute_file analysis. Each target keeps only its latest capture; page interaction makes it stale. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.",
|
"Capture the current Chrome page and return the snapshot as JSON directly in the result. When it is large the result carries a [pi-overflow id=…] marker — extract only the uids, text, or state you need with overflow_query (jsonPath or search). Page interaction makes a snapshot stale; take a fresh one after navigating or acting. Use mode/query/nearUid to control what Chrome captures. Runs in the background by default; pass background=false so the user can watch.",
|
||||||
promptSnippet: "Save the current Chrome page to a session-local JSON capture, then analyze that path with ctx_execute_file for only the required uids and state.",
|
promptSnippet: "Capture the current Chrome page and return its JSON snapshot; query large results via overflow_query.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
targetId: Type.Optional(Type.String()),
|
targetId: Type.Optional(Type.String()),
|
||||||
urlIncludes: Type.Optional(Type.String()),
|
urlIncludes: Type.Optional(Type.String()),
|
||||||
@@ -1372,8 +1346,7 @@ Usage rules:
|
|||||||
DEFAULT_TIMEOUT_MS,
|
DEFAULT_TIMEOUT_MS,
|
||||||
signal,
|
signal,
|
||||||
);
|
);
|
||||||
const capture = await saveSnapshotCapture(snapshot, params);
|
return { content: [{ type: "text", text: safeJson(snapshot) }], details: { snapshot: snapshotStats(snapshot) } };
|
||||||
return { content: [{ type: "text", text: formatSnapshotCapture(capture) }], details: { capture } };
|
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1381,8 +1354,8 @@ Usage rules:
|
|||||||
name: "chrome_find",
|
name: "chrome_find",
|
||||||
label: "Chrome Find",
|
label: "Chrome Find",
|
||||||
description:
|
description:
|
||||||
"Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates; the full structured snapshot is saved to the same private session-local file used by chrome_snapshot for optional ctx_execute_file analysis.",
|
"Find elements, regions, or text on the live Chrome page. Returns only compact ranked matches with stable uids and coordinates.",
|
||||||
promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches while keeping the full snapshot out of model context.",
|
promptSnippet: "Find live Chrome controls/text/regions and return compact stable-uid matches.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }),
|
query: Type.String({ description: "What to find, e.g. 'merge button', 'email error', 'approve PR', 'search box'." }),
|
||||||
mode: Type.Optional(StringEnum(snapshotModeValues)),
|
mode: Type.Optional(StringEnum(snapshotModeValues)),
|
||||||
@@ -1403,10 +1376,9 @@ Usage rules:
|
|||||||
DEFAULT_TIMEOUT_MS,
|
DEFAULT_TIMEOUT_MS,
|
||||||
signal,
|
signal,
|
||||||
);
|
);
|
||||||
const capture = await saveSnapshotCapture(snapshot, params);
|
|
||||||
return {
|
return {
|
||||||
content: [{ type: "text", text: `${formatChromeFindMatches(snapshot)}\n\n${formatSnapshotCapture(capture)}` }],
|
content: [{ type: "text", text: formatChromeFindMatches(snapshot) }],
|
||||||
details: { capture },
|
details: { snapshot: snapshotStats(snapshot) },
|
||||||
};
|
};
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1452,9 +1424,8 @@ Usage rules:
|
|||||||
DEFAULT_TIMEOUT_MS,
|
DEFAULT_TIMEOUT_MS,
|
||||||
signal,
|
signal,
|
||||||
);
|
);
|
||||||
const capture = await saveSnapshotCapture(snapshot, params);
|
const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${safeJson(snapshot)}`;
|
||||||
const text = `chrome_inspect fallback: loaded Chrome extension does not yet support page.inspect; reload it at chrome://extensions for deep inspect.\n\n${formatSnapshotCapture(capture)}`;
|
return { content: [{ type: "text", text }], details: { snapshot: snapshotStats(snapshot), fallback: "page.snapshot" } };
|
||||||
return { content: [{ type: "text", text }], details: { capture, fallback: "page.snapshot" } };
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1481,7 +1452,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal);
|
const result = await authorizedBridgeSend("page.navigate", withBackground(params), (params.timeoutMs ?? 15_000) + 2_000, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Navigated to ${params.url}${params.initScript ? " (with initScript)" : ""}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1506,13 +1476,12 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const value = await authorizedBridgeSend("page.evaluate", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
const text = value === undefined
|
const text = value === undefined
|
||||||
? "undefined"
|
? "undefined"
|
||||||
: typeof value === "string"
|
: typeof value === "string"
|
||||||
? value
|
? value
|
||||||
: safeJson(value) ?? "undefined";
|
: safeJson(value) ?? "undefined";
|
||||||
return { content: [{ type: "text", text: truncateText(text) }], details: { value: value as Json } };
|
return { content: [{ type: "text", text }], details: { value: value as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1520,7 +1489,7 @@ Usage rules:
|
|||||||
name: "chrome_click",
|
name: "chrome_click",
|
||||||
label: "Chrome Click",
|
label: "Chrome Click",
|
||||||
description:
|
description:
|
||||||
"Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-click snapshot to the session-local file and return its ctx_execute_file path, never the raw snapshot.",
|
"Click a snapshot uid, CSS selector, or viewport coordinate using Chrome's real input layer. Earlier snapshots become stale after the click. Pass includeSnapshot=true to get a fresh post-click snapshot JSON inline.",
|
||||||
promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.",
|
promptSnippet: "Click page elements in Chrome by snapshot uid, selector, or viewport coordinate.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })),
|
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot. Prefer uid over selector after taking a snapshot." })),
|
||||||
@@ -1528,7 +1497,7 @@ Usage rules:
|
|||||||
x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })),
|
x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })),
|
||||||
y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })),
|
y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })),
|
||||||
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
|
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
|
||||||
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-click snapshot to the session-local capture file and return its path." })),
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-click snapshot JSON inline after the action summary." })),
|
||||||
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
||||||
targetId: Type.Optional(Type.String()),
|
targetId: Type.Optional(Type.String()),
|
||||||
urlIncludes: Type.Optional(Type.String()),
|
urlIncludes: Type.Optional(Type.String()),
|
||||||
@@ -1545,7 +1514,7 @@ Usage rules:
|
|||||||
const summary = summarizeActionResult(result);
|
const summary = summarizeActionResult(result);
|
||||||
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
|
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
|
||||||
const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`;
|
const text = summary ? `Clicked ${target} — ${summary}` : `Clicked ${target}`;
|
||||||
return formatActionWithCapture(raw, params, text);
|
return formatActionResult(raw, params, text);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1553,13 +1522,13 @@ Usage rules:
|
|||||||
name: "chrome_type",
|
name: "chrome_type",
|
||||||
label: "Chrome Type",
|
label: "Chrome Type",
|
||||||
description:
|
description:
|
||||||
"Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-type snapshot and return its ctx_execute_file path.",
|
"Focus an optional snapshot uid or CSS selector, then type text using Chrome's real keyboard input. Earlier snapshots become stale after typing. Pass includeSnapshot=true to get a fresh post-type snapshot JSON inline.",
|
||||||
promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.",
|
promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
text: Type.String(),
|
text: Type.String(),
|
||||||
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
|
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
|
||||||
selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })),
|
selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })),
|
||||||
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-type snapshot to the session-local capture file and return its path." })),
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-type snapshot JSON inline after the action summary." })),
|
||||||
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
||||||
pressEnter: Type.Optional(Type.Boolean()),
|
pressEnter: Type.Optional(Type.Boolean()),
|
||||||
targetId: Type.Optional(Type.String()),
|
targetId: Type.Optional(Type.String()),
|
||||||
@@ -1578,7 +1547,7 @@ Usage rules:
|
|||||||
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
|
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
|
||||||
const base = `Typed ${params.text.length} character(s)${into}.`;
|
const base = `Typed ${params.text.length} character(s)${into}.`;
|
||||||
const text = summary ? `${base} (${summary})` : base;
|
const text = summary ? `${base} (${summary})` : base;
|
||||||
return formatActionWithCapture(raw, params, text);
|
return formatActionResult(raw, params, text);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1586,7 +1555,7 @@ Usage rules:
|
|||||||
name: "chrome_fill",
|
name: "chrome_fill",
|
||||||
label: "Chrome Fill",
|
label: "Chrome Fill",
|
||||||
description:
|
description:
|
||||||
"Set the full value of a text input, textarea, or contenteditable element using Chrome input. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-fill snapshot and return its ctx_execute_file path.",
|
"Set the full value of a text input, textarea, or contenteditable element using Chrome input. Earlier snapshots become stale after filling. Pass includeSnapshot=true to get a fresh post-fill snapshot JSON inline.",
|
||||||
promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.",
|
promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
text: Type.String(),
|
text: Type.String(),
|
||||||
@@ -1594,7 +1563,7 @@ Usage rules:
|
|||||||
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
|
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
|
||||||
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
|
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
|
||||||
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
|
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
|
||||||
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-fill snapshot to the session-local capture file and return its path." })),
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-fill snapshot JSON inline after the action summary." })),
|
||||||
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
||||||
targetId: Type.Optional(Type.String()),
|
targetId: Type.Optional(Type.String()),
|
||||||
urlIncludes: Type.Optional(Type.String()),
|
urlIncludes: Type.Optional(Type.String()),
|
||||||
@@ -1612,7 +1581,7 @@ Usage rules:
|
|||||||
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
|
const into = params.uid || params.selector ? ` into ${params.uid ?? params.selector}` : "";
|
||||||
const base = `Filled ${params.text.length} character(s)${into}.`;
|
const base = `Filled ${params.text.length} character(s)${into}.`;
|
||||||
const text = summary ? `${base} (${summary})` : base;
|
const text = summary ? `${base} (${summary})` : base;
|
||||||
return formatActionWithCapture(raw, params, text);
|
return formatActionResult(raw, params, text);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1620,7 +1589,7 @@ Usage rules:
|
|||||||
name: "chrome_key",
|
name: "chrome_key",
|
||||||
label: "Chrome Key",
|
label: "Chrome Key",
|
||||||
description:
|
description:
|
||||||
"Send a keyboard key to an existing Chrome tab. The previous capture becomes stale. Pass includeSnapshot=true to save a fresh post-key snapshot and return its ctx_execute_file path. Runs in the background by default; pass background=false so the user can watch.",
|
"Send a keyboard key to an existing Chrome tab. Earlier snapshots become stale after the key press. Pass includeSnapshot=true to get a fresh post-key snapshot JSON inline. Runs in the background by default; pass background=false so the user can watch.",
|
||||||
promptSnippet: "Press keys in Chrome through the companion extension.",
|
promptSnippet: "Press keys in Chrome through the companion extension.",
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
key: Type.String(),
|
key: Type.String(),
|
||||||
@@ -1630,7 +1599,7 @@ Usage rules:
|
|||||||
altKey: Type.Optional(Type.Boolean()),
|
altKey: Type.Optional(Type.Boolean()),
|
||||||
metaKey: Type.Optional(Type.Boolean()),
|
metaKey: Type.Optional(Type.Boolean()),
|
||||||
}, { description: "Modifier keys to hold while pressing the key (chord)." })),
|
}, { description: "Modifier keys to hold while pressing the key (chord)." })),
|
||||||
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, save a fresh post-key snapshot to the session-local capture file and return its path." })),
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, return a fresh post-key snapshot JSON inline after the action summary." })),
|
||||||
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
||||||
targetId: Type.Optional(Type.String()),
|
targetId: Type.Optional(Type.String()),
|
||||||
urlIncludes: Type.Optional(Type.String()),
|
urlIncludes: Type.Optional(Type.String()),
|
||||||
@@ -1647,7 +1616,7 @@ Usage rules:
|
|||||||
const summary = summarizeActionResult(result);
|
const summary = summarizeActionResult(result);
|
||||||
const base = `Pressed ${params.key}.`;
|
const base = `Pressed ${params.key}.`;
|
||||||
const text = summary ? `${base} (${summary})` : base;
|
const text = summary ? `${base} (${summary})` : base;
|
||||||
return formatActionWithCapture(raw, params, text);
|
return formatActionResult(raw, params, text);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1669,7 +1638,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal);
|
const result = await authorizedBridgeSend("page.waitFor", params, (params.timeoutMs ?? 10_000) + 2_000, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Observed ${params.kind}: ${params.value}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1691,7 +1659,7 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.console.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1713,7 +1681,7 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.network.list", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1733,7 +1701,7 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.network.get", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
return { content: [{ type: "text", text: truncateText(safeJson(result)) }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: safeJson(result) }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1813,7 +1781,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.hover", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Hovered ${params.uid ?? params.selector ?? `${params.x},${params.y}`}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1840,7 +1807,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.drag", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Dragged from ${params.fromUid ?? params.fromSelector} to ${params.toUid ?? params.toSelector}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1863,7 +1829,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.tap", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
|
const target = params.uid ?? params.selector ?? `${params.x},${params.y}`;
|
||||||
return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Tapped ${target} (touch)` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
@@ -1887,7 +1852,6 @@ Usage rules:
|
|||||||
}),
|
}),
|
||||||
async execute(_id, params, signal): Promise<ToolTextResult> {
|
async execute(_id, params, signal): Promise<ToolTextResult> {
|
||||||
const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.scroll", withBackground(params), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Scrolled dy=${params.deltaY ?? 0} dx=${params.deltaX ?? 0}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -1910,7 +1874,6 @@ Usage rules:
|
|||||||
const cwd = workspaceCwd(ctx);
|
const cwd = workspaceCwd(ctx);
|
||||||
const paths = params.paths.map((p) => resolve(cwd, p));
|
const paths = params.paths.map((p) => resolve(cwd, p));
|
||||||
const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal);
|
const result = await authorizedBridgeSend("page.upload", withBackground({ ...params, paths }), DEFAULT_TIMEOUT_MS, signal);
|
||||||
await invalidateSnapshotCapture(params);
|
|
||||||
return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } };
|
return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}` }], details: { result: result as Json } };
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
"name": "pi-chrome",
|
"name": "pi-chrome",
|
||||||
"version": "0.15.46",
|
"version": "0.15.46",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/session-snapshot-store.test.ts",
|
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && jiti test/snapshot-result.test.ts",
|
||||||
"version": "node scripts/sync-manifest-version.js",
|
"version": "node scripts/sync-manifest-version.js",
|
||||||
"prepublishOnly": "node scripts/sync-manifest-version.js"
|
"prepublishOnly": "node scripts/sync-manifest-version.js"
|
||||||
},
|
},
|
||||||
@@ -52,7 +52,6 @@
|
|||||||
"type": "commonjs",
|
"type": "commonjs",
|
||||||
"files": [
|
"files": [
|
||||||
"extensions",
|
"extensions",
|
||||||
"src",
|
|
||||||
"test",
|
"test",
|
||||||
"docs",
|
"docs",
|
||||||
"test-suite",
|
"test-suite",
|
||||||
|
|||||||
@@ -1,140 +0,0 @@
|
|||||||
import { createHash, randomUUID } from "node:crypto";
|
|
||||||
import { chmod, mkdir, rename, rm, writeFile } from "node:fs/promises";
|
|
||||||
import { join, relative, resolve, sep } from "node:path";
|
|
||||||
|
|
||||||
type SnapshotObject = Record<string, unknown>;
|
|
||||||
|
|
||||||
export type SnapshotTarget = {
|
|
||||||
targetId?: string;
|
|
||||||
urlIncludes?: string;
|
|
||||||
titleIncludes?: string;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type SnapshotCaptureMetadata = {
|
|
||||||
version: 1;
|
|
||||||
captureId: string;
|
|
||||||
capturedAt: string;
|
|
||||||
path: string;
|
|
||||||
bytes: number;
|
|
||||||
targetKey: string;
|
|
||||||
targetId?: string | number;
|
|
||||||
mode?: string;
|
|
||||||
elementCount: number;
|
|
||||||
matchCount: number;
|
|
||||||
staleAfterInteraction: true;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type StoredSnapshotCapture = {
|
|
||||||
metadata: SnapshotCaptureMetadata;
|
|
||||||
absolutePath: string;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type SnapshotCaptureFile = {
|
|
||||||
version: 1;
|
|
||||||
captureId: string;
|
|
||||||
capturedAt: string;
|
|
||||||
targetKey: string;
|
|
||||||
snapshot: unknown;
|
|
||||||
};
|
|
||||||
|
|
||||||
function hashSegment(value: string): string {
|
|
||||||
return createHash("sha256").update(value).digest("hex").slice(0, 20);
|
|
||||||
}
|
|
||||||
|
|
||||||
function normalizeRelativePath(path: string): string {
|
|
||||||
return path.split(sep).join("/");
|
|
||||||
}
|
|
||||||
|
|
||||||
function snapshotObject(snapshot: unknown): SnapshotObject | undefined {
|
|
||||||
return snapshot && typeof snapshot === "object" && !Array.isArray(snapshot)
|
|
||||||
? snapshot as SnapshotObject
|
|
||||||
: undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function snapshotTargetKey(target: SnapshotTarget = {}): string {
|
|
||||||
if (target.targetId) return `target:${target.targetId}`;
|
|
||||||
if (target.urlIncludes) return `url:${target.urlIncludes}`;
|
|
||||||
if (target.titleIncludes) return `title:${target.titleIncludes}`;
|
|
||||||
return "default";
|
|
||||||
}
|
|
||||||
|
|
||||||
export class SessionSnapshotStore {
|
|
||||||
readonly workspaceRoot: string;
|
|
||||||
readonly sessionRoot: string;
|
|
||||||
readonly sessionHash: string;
|
|
||||||
|
|
||||||
constructor(workspaceRoot: string, sessionId: string) {
|
|
||||||
this.workspaceRoot = resolve(workspaceRoot);
|
|
||||||
this.sessionHash = hashSegment(sessionId || "unknown-session");
|
|
||||||
this.sessionRoot = join(this.workspaceRoot, ".pi", "chrome-context", this.sessionHash);
|
|
||||||
}
|
|
||||||
|
|
||||||
async save(snapshot: unknown, target: SnapshotTarget = {}): Promise<StoredSnapshotCapture> {
|
|
||||||
const targetKey = snapshotTargetKey(target);
|
|
||||||
const targetDirectory = join(this.sessionRoot, hashSegment(targetKey));
|
|
||||||
const absolutePath = join(targetDirectory, "snapshot.json");
|
|
||||||
const temporaryPath = join(targetDirectory, `.snapshot.${process.pid}.${randomUUID()}.tmp`);
|
|
||||||
const captureId = randomUUID();
|
|
||||||
const capturedAt = new Date().toISOString();
|
|
||||||
const capture: SnapshotCaptureFile = {
|
|
||||||
version: 1,
|
|
||||||
captureId,
|
|
||||||
capturedAt,
|
|
||||||
targetKey,
|
|
||||||
snapshot,
|
|
||||||
};
|
|
||||||
const serialized = `${JSON.stringify(capture, null, 2)}\n`;
|
|
||||||
|
|
||||||
await mkdir(targetDirectory, { recursive: true, mode: 0o700 });
|
|
||||||
await chmod(this.sessionRoot, 0o700).catch(() => undefined);
|
|
||||||
await chmod(targetDirectory, 0o700).catch(() => undefined);
|
|
||||||
try {
|
|
||||||
await writeFile(temporaryPath, serialized, { encoding: "utf8", mode: 0o600, flag: "wx" });
|
|
||||||
await chmod(temporaryPath, 0o600).catch(() => undefined);
|
|
||||||
await rename(temporaryPath, absolutePath);
|
|
||||||
await chmod(absolutePath, 0o600).catch(() => undefined);
|
|
||||||
} catch (error) {
|
|
||||||
await rm(temporaryPath, { force: true }).catch(() => undefined);
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
|
|
||||||
const object = snapshotObject(snapshot);
|
|
||||||
const elements = Array.isArray(object?.elements) ? object.elements : [];
|
|
||||||
const matches = Array.isArray(object?.matches) ? object.matches : [];
|
|
||||||
const targetId = typeof object?.targetId === "string" || typeof object?.targetId === "number"
|
|
||||||
? object.targetId
|
|
||||||
: undefined;
|
|
||||||
const mode = typeof object?.mode === "string" ? object.mode : undefined;
|
|
||||||
const path = normalizeRelativePath(relative(this.workspaceRoot, absolutePath));
|
|
||||||
|
|
||||||
return {
|
|
||||||
absolutePath,
|
|
||||||
metadata: {
|
|
||||||
version: 1,
|
|
||||||
captureId,
|
|
||||||
capturedAt,
|
|
||||||
path,
|
|
||||||
bytes: Buffer.byteLength(serialized),
|
|
||||||
targetKey,
|
|
||||||
targetId,
|
|
||||||
mode,
|
|
||||||
elementCount: elements.length,
|
|
||||||
matchCount: matches.length,
|
|
||||||
staleAfterInteraction: true,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
async invalidate(target?: SnapshotTarget): Promise<void> {
|
|
||||||
if (!target) {
|
|
||||||
await this.clear();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const targetDirectory = join(this.sessionRoot, hashSegment(snapshotTargetKey(target)));
|
|
||||||
await rm(targetDirectory, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
|
|
||||||
async clear(): Promise<void> {
|
|
||||||
await rm(this.sessionRoot, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
import assert from "node:assert/strict";
|
|
||||||
import { mkdtemp, readFile, stat } from "node:fs/promises";
|
|
||||||
import { tmpdir } from "node:os";
|
|
||||||
import { join } from "node:path";
|
|
||||||
import test from "node:test";
|
|
||||||
import { SessionSnapshotStore, snapshotTargetKey } from "../src/session-snapshot-store.ts";
|
|
||||||
import { CHROME_TOOL_NAMES, formatChromeFindMatches, formatSnapshotCapture } from "../extensions/chrome-profile-bridge/index.ts";
|
|
||||||
|
|
||||||
test("snapshotTargetKey prefers explicit target selectors", () => {
|
|
||||||
assert.equal(snapshotTargetKey(), "default");
|
|
||||||
assert.equal(snapshotTargetKey({ titleIncludes: "Inbox" }), "title:Inbox");
|
|
||||||
assert.equal(snapshotTargetKey({ urlIncludes: "example.com" }), "url:example.com");
|
|
||||||
assert.equal(snapshotTargetKey({ targetId: "42", urlIncludes: "ignored" }), "target:42");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("SessionSnapshotStore atomically keeps only the latest capture per target", async () => {
|
|
||||||
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-snapshot-"));
|
|
||||||
const store = new SessionSnapshotStore(workspace, "session/private/id");
|
|
||||||
const first = await store.save({ mode: "interactive", targetId: 42, elements: [{ uid: "el-1" }], matches: [] });
|
|
||||||
const second = await store.save({ mode: "changes", targetId: 42, elements: [{ uid: "el-2" }], matches: [{ uid: "el-2" }] });
|
|
||||||
|
|
||||||
assert.equal(first.metadata.path, second.metadata.path);
|
|
||||||
assert.notEqual(first.metadata.captureId, second.metadata.captureId);
|
|
||||||
assert.equal(second.metadata.elementCount, 1);
|
|
||||||
assert.equal(second.metadata.matchCount, 1);
|
|
||||||
assert.equal(second.metadata.targetId, 42);
|
|
||||||
assert.ok(!second.metadata.path.includes("private"));
|
|
||||||
|
|
||||||
const stored = JSON.parse(await readFile(second.absolutePath, "utf8"));
|
|
||||||
assert.equal(stored.version, 1);
|
|
||||||
assert.equal(stored.captureId, second.metadata.captureId);
|
|
||||||
assert.equal(stored.snapshot.elements[0].uid, "el-2");
|
|
||||||
|
|
||||||
if (process.platform !== "win32") {
|
|
||||||
assert.equal((await stat(store.sessionRoot)).mode & 0o777, 0o700);
|
|
||||||
assert.equal((await stat(second.absolutePath)).mode & 0o777, 0o600);
|
|
||||||
}
|
|
||||||
|
|
||||||
await store.clear();
|
|
||||||
await assert.rejects(stat(second.absolutePath), { code: "ENOENT" });
|
|
||||||
});
|
|
||||||
|
|
||||||
test("target invalidation and session cleanup do not affect another session", async () => {
|
|
||||||
const workspace = await mkdtemp(join(tmpdir(), "pi-chrome-isolation-"));
|
|
||||||
const firstSession = new SessionSnapshotStore(workspace, "session-a");
|
|
||||||
const secondSession = new SessionSnapshotStore(workspace, "session-b");
|
|
||||||
const firstTarget = { targetId: "101" };
|
|
||||||
const secondTarget = { targetId: "202" };
|
|
||||||
const first = await firstSession.save({ elements: [] }, firstTarget);
|
|
||||||
const otherTarget = await firstSession.save({ elements: [] }, secondTarget);
|
|
||||||
const otherSession = await secondSession.save({ elements: [] }, firstTarget);
|
|
||||||
|
|
||||||
await firstSession.invalidate(firstTarget);
|
|
||||||
await assert.rejects(stat(first.absolutePath), { code: "ENOENT" });
|
|
||||||
await stat(otherTarget.absolutePath);
|
|
||||||
await stat(otherSession.absolutePath);
|
|
||||||
|
|
||||||
await firstSession.clear();
|
|
||||||
await assert.rejects(stat(otherTarget.absolutePath), { code: "ENOENT" });
|
|
||||||
await stat(otherSession.absolutePath);
|
|
||||||
await secondSession.clear();
|
|
||||||
});
|
|
||||||
|
|
||||||
test("snapshot result formatting stays bounded and points to Context Mode", () => {
|
|
||||||
const text = formatSnapshotCapture({
|
|
||||||
version: 1,
|
|
||||||
captureId: "capture-1",
|
|
||||||
capturedAt: "2026-08-28T00:00:00.000Z",
|
|
||||||
path: ".pi/chrome-context/session/target/snapshot.json",
|
|
||||||
bytes: 80_000,
|
|
||||||
targetKey: "target:42",
|
|
||||||
targetId: 42,
|
|
||||||
mode: "interactive",
|
|
||||||
elementCount: 80,
|
|
||||||
matchCount: 3,
|
|
||||||
staleAfterInteraction: true,
|
|
||||||
});
|
|
||||||
|
|
||||||
assert.match(text, /ctx_execute_file/);
|
|
||||||
assert.match(text, /\.pi\/chrome-context\/session\/target\/snapshot\.json/);
|
|
||||||
assert.doesNotMatch(text, /raw-secret-page-content/);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
|
|
||||||
const matches = Array.from({ length: 25 }, (_, index) => ({
|
|
||||||
kind: "element",
|
|
||||||
uid: `el-${index}`,
|
|
||||||
role: "button",
|
|
||||||
label: `Button ${index}`,
|
|
||||||
rect: { x: index, y: index, width: 10, height: 10 },
|
|
||||||
}));
|
|
||||||
const text = formatChromeFindMatches({ query: "button", matches });
|
|
||||||
|
|
||||||
assert.match(text, /el-0/);
|
|
||||||
assert.match(text, /el-19/);
|
|
||||||
assert.doesNotMatch(text, /el-20/);
|
|
||||||
assert.equal(CHROME_TOOL_NAMES.length, 21);
|
|
||||||
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
|
|
||||||
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
|
|
||||||
});
|
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import test from "node:test";
|
||||||
|
import {
|
||||||
|
CHROME_TOOL_NAMES,
|
||||||
|
formatActionResultText,
|
||||||
|
formatChromeFindMatches,
|
||||||
|
snapshotStats,
|
||||||
|
} from "../extensions/chrome-profile-bridge/index.ts";
|
||||||
|
|
||||||
|
test("formatActionResultText appends the inline snapshot JSON", () => {
|
||||||
|
const snapshot = { elements: [{ uid: "el-1", role: "button", label: "Save" }], matches: [] };
|
||||||
|
const text = formatActionResultText("Clicked el-1", snapshot);
|
||||||
|
const [summary, json] = text.split("\n\n");
|
||||||
|
assert.equal(summary, "Clicked el-1");
|
||||||
|
assert.deepEqual(JSON.parse(json), snapshot);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("formatActionResultText returns the summary alone without a snapshot", () => {
|
||||||
|
assert.equal(formatActionResultText("Clicked el-1", undefined), "Clicked el-1");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("snapshotStats stays small: only element and match counts", () => {
|
||||||
|
assert.deepEqual(
|
||||||
|
snapshotStats({ elements: [{ uid: "a" }, { uid: "b" }], matches: [{ uid: "a" }] }),
|
||||||
|
{ elementCount: 2, matchCount: 1 },
|
||||||
|
);
|
||||||
|
assert.deepEqual(snapshotStats({ elements: "nope" }), { elementCount: 0, matchCount: 0 });
|
||||||
|
assert.deepEqual(snapshotStats(undefined), { elementCount: 0, matchCount: 0 });
|
||||||
|
assert.deepEqual(snapshotStats([1, 2, 3]), { elementCount: 0, matchCount: 0 });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("chrome_find formatting is capped and authorization tracks all 21 tools", () => {
|
||||||
|
const matches = Array.from({ length: 25 }, (_, index) => ({
|
||||||
|
kind: "element",
|
||||||
|
uid: `el-${index}`,
|
||||||
|
role: "button",
|
||||||
|
label: `Button ${index}`,
|
||||||
|
rect: { x: index, y: index, width: 10, height: 10 },
|
||||||
|
}));
|
||||||
|
const text = formatChromeFindMatches({ query: "button", matches });
|
||||||
|
|
||||||
|
assert.match(text, /el-0/);
|
||||||
|
assert.match(text, /el-19/);
|
||||||
|
assert.doesNotMatch(text, /el-20/);
|
||||||
|
assert.equal(CHROME_TOOL_NAMES.length, 21);
|
||||||
|
assert.ok(CHROME_TOOL_NAMES.includes("chrome_find"));
|
||||||
|
assert.ok(CHROME_TOOL_NAMES.includes("chrome_inspect"));
|
||||||
|
});
|
||||||
+13
-9
@@ -1,19 +1,23 @@
|
|||||||
# my-pi 维护约定(pi-compaction)
|
# my-pi 维护约定(pi-compaction)
|
||||||
|
|
||||||
- 扩展入口是 `pi-compaction/index.ts`,增量索引、投影、后台摘要、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。
|
- 扩展入口是 `pi-compaction/index.ts`,遮蔽规划、投影索引、遮蔽台账、服务与 Codex native 实现在 `src/`,测试位于 `test/`,双上游来源记录是 `UPSTREAM.md`。
|
||||||
- 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。
|
- 根包必须通过 `file:./pi-compaction` 提供依赖并直接加载源码;不得重新声明或加载 `pi-condense`、npm `@ogulcancelik/pi-codex-compaction` 或 npm `pi-context-view`。
|
||||||
- `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune` 或 `openai-codex-native-compaction` 状态;切换后应使用新 Session。配置不写回 `settings.json`,默认 soft/hard/target 比例为 72%/90%/55%。
|
- `pi-compaction` 只支持自己的 version 1 记录和 `config/pi-compaction.json` 配置,不读取或迁移旧 `context-prune-*`、`contextPrune`、`openai-codex-native-compaction` 或旧 `segment-summary` 状态;切换后应使用新 Session。配置不写回 `settings.json`。
|
||||||
- Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage;background worker 不得在 Provider 请求中被 await。
|
- Context Authority 是 compaction 唯一生产投影者。普通 Provider、Codex native、`/context`、`/compaction-status` 和 footer 必须消费同一个不可变 projection snapshot;`pi-minimal-footer` 优先显示同一 effective usage,snapshot 尚未建立时才回退宿主原始 usage。
|
||||||
- `pi-compaction` 负责已经进入 Session 的历史投影,与 Context Mode 的输入隔离职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
- `pi-compaction` 负责已经进入 Session 的历史投影,与 pi-overflow 的新结果入口截断职责互补。首次 Session hydration 允许一次完整扫描;后续普通 `context` hot path 必须复用不可变 snapshot 或只处理 delta,不得调用 summarizer、写 Session 或重新构建完整 branch。
|
||||||
- 已闭合工具结果先同步生成保留工具名、参数、状态、首尾证据和 source key 的有界确定性 stub;大于 `semantic.minInputChars` 的结果再进入单 Session 有界后台语义 worker。后台结果只在安全事件边界提交,陈旧 hash 必须丢弃,Provider 请求不得等待后台摘要。
|
- L2 遮蔽只用 Pi 原生 `context_edit`(`{ type: "context_edit", targetId, replacement }`)处理保护窗口外的旧 toolResult:成批提交、受 trigger/target 回差与 `minClearTokens` 约束,不逐条即时替换;不为遮蔽写 `segment-summary` 等 custom entry,原始条目、UI 与导出保持不变。
|
||||||
- Segment custom entry 只能保存摘要、hash 和 source reference,不得复制完整原始工具输出。`context_tree_query` 恢复结果必须有界,native checkpoint 的 portable summary 也必须有界。
|
- 不得改写最近 `keepRecentTurns` 轮;不得遮蔽 user/assistant 消息、`excludeTools` 中的工具、`SKILL.md` 的 `read` 结果、已遮蔽或已省略的条目、以及小于 `minEntryTokens` 的结果。
|
||||||
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。Pi Core raw threshold 只负责唤醒,只有 effective lower bound 达到 `hardRatio` 才执行 threshold compaction;manual 与 overflow 不得被阻断。
|
- 遮蔽占位符必须确定性、单行、以 `MASK_PREFIX`(`"[pi-compaction] Earlier "`)开头,并在文本中指向 `context_tree_query` 恢复;`context_tree_query` 与 native checkpoint 的 portable summary 都必须有界。被遮蔽的 toolResult 若带 `details.overflow.id`(pi-overflow 已存原文),占位符末尾追加 `overflow_query` 提示——只读该公开字段,不得 import pi-overflow 内部实现。
|
||||||
|
- L3 整段摘要只能从空闲的 `agent_settled` 通过 `ctx.compact` 触发(`ctx.compact` 会中止当前运行,非空闲时调用会破坏运行);同一 leaf 不重复触发。阈值以绝对 token 计并受窗口比例约束(`effectiveThreshold = min(tokens, floor(ratio × contextWindow))`)。
|
||||||
|
- `context` 钩子不得改写消息——非 checkpoint 路径只更新 snapshot 并返回 undefined;唯一例外是 native checkpoint 路径:checkpoint 之后的条目在投影时必须应用各 target 最新的 `context_edit`(null 省略、字符串规范成 text block),因为 `sessionEntryToContextMessages` 不处理 edits。
|
||||||
|
- `session_before_compact` 只负责生成 portable/native 摘要,不得按自身阈值取消 Pi 的压缩(Pi 的 `contextTokens > window - reserveTokens` 是最终安全网)。
|
||||||
|
- effective token 默认使用 Provider usage-backed baseline 加 projected delta;无可靠 baseline 时使用带上下界的估算。遮蔽台账在 `session_start` / `session_tree` / `session_compact` 从当前 branch 的 `context_edit` 条目重建,指标暴露 `maskedResults` 与 `reclaimedTokens`。
|
||||||
- Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。
|
- Codex native compaction 必须直接消费当前 projection 生成的 ResponseItems,不得从 raw branch 重建输入;native checkpoint 同时保存有界 portable summary 供模型或 Provider 切换。
|
||||||
- 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认在 turn boundary 达到 effective 下界 90% 时触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。
|
- 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认由 L3 阈值或 Pi 自身阈值触发。`src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。
|
||||||
- SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。
|
- SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。
|
||||||
- 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。
|
- 同步 Codex 协议代码必须从官方 `ogulcancelik/pi-extensions` 明确 commit 移植并保留对应 MIT 许可证;不得从 npm tarball、缓存、`node_modules` 或构建产物导入。
|
||||||
|
|
||||||
## 验证
|
## 验证
|
||||||
|
|
||||||
- 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。
|
- 在本目录内至少运行 `npm test`、`npm run check` 和 `npm pack --dry-run --json`;涉及依赖、加载入口、context writer、footer 或 observer 时还要验证根 `npm install` 幂等、锁文件仅包含预期本地 link 变化、根扩展联合加载、性能回归,以及实际 packed 根 tarball 的隔离安装/加载。
|
||||||
- 修改 projection、token accounting、threshold、checkpoint、模型切换、后台并发或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。
|
- 修改 projection、token accounting、遮蔽规划/台账、threshold、checkpoint、模型切换或恢复索引后,至少运行 `npm test`、`npm run check`、`npm pack --dry-run --json`、根扩展联合加载、packed 根 tarball 隔离加载和性能回归。性能门禁为 cache-hit P95 <2ms、delta projection P95 <10ms,普通 hot path 不得扫描完整 branch。
|
||||||
|
|||||||
+36
-31
@@ -1,19 +1,22 @@
|
|||||||
# pi-compaction
|
# pi-compaction
|
||||||
|
|
||||||
High-performance unified context projection and compaction for Pi.
|
Pressure-aware context masking and threshold compaction for Pi.
|
||||||
|
|
||||||
## Responsibilities
|
## Responsibilities
|
||||||
|
|
||||||
- incrementally index completed tool calls;
|
Pi context management is layered; this extension owns the middle two layers:
|
||||||
- replace large tool results with deterministic bounded evidence stubs;
|
|
||||||
- pass small tool results through unchanged whenever a stub would not be shorter;
|
|
||||||
- refine large stubs with non-blocking semantic summaries;
|
|
||||||
- expose one effective context projection to normal provider requests and Codex/SubAPI native compaction;
|
|
||||||
- cancel Pi's raw threshold compaction until the effective lower-bound reaches the configured threshold;
|
|
||||||
- retain original tool evidence through `context_tree_query`;
|
|
||||||
- publish read-only effective metrics for the footer and context observer.
|
|
||||||
|
|
||||||
Context Mode remains responsible for preventing large external data from entering the session. RTK remains responsible for ordinary tool-output cleanup.
|
- **L1 — entry truncation** (future `pi-overflow`): truncate new tool results as they are written. Not implemented here.
|
||||||
|
- **L2 — observation masking** (this extension): under real context pressure, replace old tool results outside a protected recent window with deterministic one-line placeholders via Pi `context_edit` entries. Originals stay in the session untouched and are recoverable with `context_tree_query`.
|
||||||
|
- **L3 — whole-context summary** (this extension): when the session idles (`agent_settled`) above the summary threshold, trigger `ctx.compact` once per leaf.
|
||||||
|
- **Pi native threshold compaction** is the final safety net and is never blocked; this extension no longer cancels `session_before_compact` based on its own thresholds.
|
||||||
|
|
||||||
|
Additionally it:
|
||||||
|
|
||||||
|
- publishes one effective-context snapshot for normal provider requests and Codex/SubAPI native compaction;
|
||||||
|
- maintains a masked-results ledger so `/context`, `/compaction-status`, the footer, and AutoReview see the same placeholders the model sees.
|
||||||
|
|
||||||
|
pi-overflow bounds new tool results as they arrive. RTK remains responsible for ordinary tool-output cleanup.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -22,21 +25,21 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
|
|||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"softRatio": 0.72,
|
"masking": {
|
||||||
"hardRatio": 0.9,
|
|
||||||
"targetRatio": 0.55,
|
|
||||||
"deterministic": {
|
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"maxStubTokens": 1024
|
"triggerTokens": 200000,
|
||||||
|
"triggerRatio": 0.6,
|
||||||
|
"targetTokens": 100000,
|
||||||
|
"targetRatio": 0.35,
|
||||||
|
"minClearTokens": 50000,
|
||||||
|
"keepRecentTurns": 10,
|
||||||
|
"minEntryTokens": 256,
|
||||||
|
"excludeTools": ["ask_user_question"]
|
||||||
},
|
},
|
||||||
"semantic": {
|
"summary": {
|
||||||
"enabled": true,
|
"triggerTokens": 400000,
|
||||||
"model": "default",
|
"triggerRatio": 0.9,
|
||||||
"thinking": "low",
|
"thinking": "low"
|
||||||
"concurrency": 1,
|
|
||||||
"minInputChars": 4000,
|
|
||||||
"maxInputChars": 24000,
|
|
||||||
"maxRetries": 2
|
|
||||||
},
|
},
|
||||||
"native": {
|
"native": {
|
||||||
"enabled": true
|
"enabled": true
|
||||||
@@ -44,7 +47,9 @@ Configuration lives in the installed bundle's `config/pi-compaction.json` (the b
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Invalid or missing fields are normalized in memory.
|
All thresholds are absolute token counts constrained by a context-window ratio: `effectiveThreshold = min(tokens, floor(ratio × contextWindow))`. Masking fires when `currentTokens ≥ triggerEff` and clears the oldest eligible results until `currentTokens - cleared ≤ targetEff`; if the clearable volume stays under `min(minClearTokens, triggerEff - targetEff)` nothing is masked, so masking is rare, batched, and hysteretic. `masking.target` is normalized below `masking.trigger` and `summary.trigger` above `masking.trigger` when a config would violate that ordering. Invalid or missing fields fall back to defaults in memory.
|
||||||
|
|
||||||
|
Masking never touches the most recent `keepRecentTurns` assistant turns, user/assistant messages, `excludeTools`, `SKILL.md` reads, entries already masked, or results smaller than `minEntryTokens`.
|
||||||
|
|
||||||
## Native providers
|
## Native providers
|
||||||
|
|
||||||
@@ -67,17 +72,17 @@ The gateway must have a compatible deployed version, model and available upstrea
|
|||||||
|
|
||||||
## Session format
|
## Session format
|
||||||
|
|
||||||
The only custom entry type is `pi-compaction`, version 1. Segment summary entries contain hashes, summaries, and source references; they never duplicate complete raw tool output. Native checkpoints use Pi compaction-entry details with the same kind/version.
|
The extension writes no `segment-summary` or other custom entries for masking: L2 uses Pi's native `context_edit` entries (`{ type: "context_edit", targetId, replacement }`), which alter only future model context. The raw session entries — and therefore UI, exports, and the evidence ledger — are unchanged; every masked result remains recoverable through `context_tree_query`. Masking placeholders are deterministic single lines starting with `[pi-compaction] Earlier `.
|
||||||
|
|
||||||
Projection replaces a tool result only when the resulting summary plus its `Source:` recovery line is strictly shorter than the original text content. Small tool results are therefore passed through byte-for-byte and keep their original content parts, including any non-text parts. Because a stub is only larger than the text it would replace when the original is short, this comparison is the sole gate; no separate size option exists. Every indexed result remains recoverable through `context_tree_query` whether or not the message was replaced.
|
Compaction entries still carry `pi-compaction` version 1 details; native checkpoints use the same kind/version. This implementation intentionally does not read legacy `context-prune-*`, `openai-codex-native-compaction`, or older `segment-summary` entries. Start a new session after switching to this extension.
|
||||||
|
|
||||||
This implementation intentionally does not read legacy `context-prune-*` or `openai-codex-native-compaction` entries. Start a new session after switching to this extension.
|
|
||||||
|
|
||||||
## Interfaces
|
## Interfaces
|
||||||
|
|
||||||
- `context_tree_query`: recover original bounded tool evidence by source key or tool call ID.
|
- `context_tree_query`: recover the original output of a tool result that pi-compaction cleared from context, by tool call ID (1–20 ids per call, bounded output). Use it when a `[pi-compaction] Earlier` placeholder lacks evidence needed for the current task.
|
||||||
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, and pending semantic jobs.
|
- pi-overflow linkage: when the masked tool result carries `details.overflow.id` (pi-overflow stored the oversized original), the placeholder ends with `Full original stored: overflow_query {"id":"…"}`. pi-compaction only reads that public field and never imports pi-overflow internals.
|
||||||
|
- `/compaction-status`: show raw/effective tokens, reclaimed tokens, confidence, segments, masked results, and generation.
|
||||||
|
- Service `my-pi.pi-compaction.service.v1`: `getSnapshot`, `getMetrics`, and `query(sessionId, ids)` — `query` returns a placeholder record only for tool call IDs that are currently masked, so consumers such as the permission system's AutoReview see the same evidence the model does.
|
||||||
|
|
||||||
## Performance model
|
## Performance model
|
||||||
|
|
||||||
The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots. The `context` hook performs no model call and no session write. Semantic summaries run in a bounded background worker and become visible only at later safe event boundaries.
|
The first session hydration scans the branch once. Appended turns use delta projection and immutable cached snapshots; the `context` hook performs no model call, no session write, and returns the message list unchanged except on the native-checkpoint path, where post-checkpoint `context_edit` entries are applied during projection. Masking drafts are produced in `turn_end` in one batch and only when both the trigger and the `minClear` hysteresis pass, so the hot path stays allocation-free for unprompted turns.
|
||||||
|
|||||||
@@ -29,3 +29,8 @@
|
|||||||
- 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。
|
- 协议证据:`backend/internal/service/openai_compact_body_signal.go` 指出原生远端压缩 v2 为 Responses 加 `stream: true` 与 `compaction_trigger`,并处理 Codex beta 特性。
|
||||||
- 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。
|
- 本地 `src/native-provider.ts` 适配器是针对该协议独立实现的。没有导入任何 Sub2API 源码;复用现有 Codex 序列化器、流解析器和 checkpoint 格式。
|
||||||
- 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。
|
- 该参考描述的是上游能力,不构成对任何特定部署、模型或账号的保证。本适配器不使用单独的 Responses compact 端点。
|
||||||
|
|
||||||
|
## 本地改动记录
|
||||||
|
|
||||||
|
- 上下文管理重构为 L2/L3:L2 在上下文压力下经 `turn_end` 提交 Pi 原生 `context_edit`,把保护窗口外的旧 toolResult 批量遮蔽为指向 `context_tree_query` 的确定性占位符(原始条目保留、不写 custom entry);L3 只在空闲 `agent_settled` 且达到绝对 token 阈值(受窗口比例约束)时调用一次 `ctx.compact`。删除逐条确定性存根、后台语义 worker、`softRatio`/`hardRatio`/`targetRatio` 配置与 `session_before_compact` 的自行阈值取消;Pi 原生阈值压缩保留为最终安全网。
|
||||||
|
- L2 遮蔽占位符联动 pi-overflow:被遮蔽的 toolResult 若携带 `details.overflow.id`,占位符末尾追加 `overflow_query {"id":"…"}` 恢复提示。
|
||||||
|
|||||||
+174
-76
@@ -14,6 +14,12 @@ import {
|
|||||||
type SessionEntry,
|
type SessionEntry,
|
||||||
} from "@earendil-works/pi-coding-agent";
|
} from "@earendil-works/pi-coding-agent";
|
||||||
import { DEFAULT_CONFIG, loadConfig } from "./src/config.js";
|
import { DEFAULT_CONFIG, loadConfig } from "./src/config.js";
|
||||||
|
import {
|
||||||
|
applyContextEdits,
|
||||||
|
planMasking,
|
||||||
|
rebuildMaskedLedger,
|
||||||
|
type MaskedLedgerEntry,
|
||||||
|
} from "./src/masking.js";
|
||||||
import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js";
|
import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js";
|
||||||
import {
|
import {
|
||||||
buildCompactionRequestBody,
|
buildCompactionRequestBody,
|
||||||
@@ -32,27 +38,28 @@ import {
|
|||||||
type NativeCompactionDetails,
|
type NativeCompactionDetails,
|
||||||
type ResponseItem,
|
type ResponseItem,
|
||||||
} from "./src/native-compaction.js";
|
} from "./src/native-compaction.js";
|
||||||
import { SegmentIndex } from "./src/segment-index.js";
|
import { contentText, SegmentIndex } from "./src/segment-index.js";
|
||||||
import { SemanticWorker } from "./src/semantic-worker.js";
|
|
||||||
import { shouldCompact } from "./src/token-accounting.js";
|
|
||||||
import {
|
import {
|
||||||
CompactionServiceRegistry,
|
CompactionServiceRegistry,
|
||||||
publishCompactionService,
|
publishCompactionService,
|
||||||
unpublishCompactionService,
|
unpublishCompactionService,
|
||||||
} from "./src/service.js";
|
} from "./src/service.js";
|
||||||
|
import { effectiveThreshold } from "./src/token-accounting.js";
|
||||||
import {
|
import {
|
||||||
COMPACTION_CUSTOM_TYPE,
|
COMPACTION_CUSTOM_TYPE,
|
||||||
COMPACTION_VERSION,
|
COMPACTION_VERSION,
|
||||||
QUERY_TOOL_NAME,
|
QUERY_TOOL_NAME,
|
||||||
|
type MaskedResultRecord,
|
||||||
type PiCompactionConfig,
|
type PiCompactionConfig,
|
||||||
type ProjectionSnapshot,
|
type ProjectionSnapshot,
|
||||||
type ToolHistoryRecord,
|
|
||||||
} from "./src/types.js";
|
} from "./src/types.js";
|
||||||
|
|
||||||
interface SessionRuntime {
|
interface SessionRuntime {
|
||||||
index: SegmentIndex;
|
index: SegmentIndex;
|
||||||
config: PiCompactionConfig;
|
config: PiCompactionConfig;
|
||||||
codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] };
|
codexCache?: { fingerprint: string; modelKey: string; toolsHash: string; items: ResponseItem[] };
|
||||||
|
masked: Map<string, MaskedLedgerEntry>;
|
||||||
|
lastAutoCompactLeafId: string | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface CachedPayload {
|
interface CachedPayload {
|
||||||
@@ -64,10 +71,6 @@ function errorMessage(error: unknown): string {
|
|||||||
return error instanceof Error ? error.message : String(error);
|
return error instanceof Error ? error.message : String(error);
|
||||||
}
|
}
|
||||||
|
|
||||||
function hash(value: string): string {
|
|
||||||
return createHash("sha256").update(value).digest("hex").slice(0, 20);
|
|
||||||
}
|
|
||||||
|
|
||||||
interface EnvironmentValue {
|
interface EnvironmentValue {
|
||||||
hash: string;
|
hash: string;
|
||||||
tokens: number;
|
tokens: number;
|
||||||
@@ -81,6 +84,10 @@ interface EnvironmentCacheEntry extends EnvironmentValue {
|
|||||||
tools: unknown[];
|
tools: unknown[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function hash(value: string): string {
|
||||||
|
return createHash("sha256").update(value).digest("hex").slice(0, 20);
|
||||||
|
}
|
||||||
|
|
||||||
function environment(
|
function environment(
|
||||||
pi: ExtensionAPI,
|
pi: ExtensionAPI,
|
||||||
ctx: ExtensionContext,
|
ctx: ExtensionContext,
|
||||||
@@ -182,34 +189,46 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
const runtimes = new Map<string, SessionRuntime>();
|
const runtimes = new Map<string, SessionRuntime>();
|
||||||
const payloads = new Map<string, CachedPayload>();
|
const payloads = new Map<string, CachedPayload>();
|
||||||
const environments = new Map<string, EnvironmentCacheEntry>();
|
const environments = new Map<string, EnvironmentCacheEntry>();
|
||||||
const worker = new SemanticWorker();
|
|
||||||
const service = new CompactionServiceRegistry();
|
const service = new CompactionServiceRegistry();
|
||||||
publishCompactionService(service);
|
publishCompactionService(service);
|
||||||
|
|
||||||
|
const maskedRecords = (sessionId: string, ids: string[]): MaskedResultRecord[] => {
|
||||||
|
const masked = runtimes.get(sessionId)?.masked;
|
||||||
|
if (!masked) return [];
|
||||||
|
return ids.flatMap((id) => {
|
||||||
|
const entry = masked.get(id);
|
||||||
|
return entry
|
||||||
|
? [{
|
||||||
|
toolCallId: id,
|
||||||
|
resultTimestamp: entry.resultTimestamp,
|
||||||
|
deterministicSummary: entry.placeholder,
|
||||||
|
}]
|
||||||
|
: [];
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
const runtimeFor = (ctx: ExtensionContext): SessionRuntime => {
|
const runtimeFor = (ctx: ExtensionContext): SessionRuntime => {
|
||||||
const sessionId = ctx.sessionManager.getSessionId();
|
const sessionId = ctx.sessionManager.getSessionId();
|
||||||
let runtime = runtimes.get(sessionId);
|
let runtime = runtimes.get(sessionId);
|
||||||
if (!runtime) {
|
if (!runtime) {
|
||||||
const index = new SegmentIndex(sessionId);
|
runtime = {
|
||||||
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], DEFAULT_CONFIG);
|
index: new SegmentIndex(sessionId),
|
||||||
runtime = { index, config: structuredClone(DEFAULT_CONFIG) };
|
config: structuredClone(DEFAULT_CONFIG),
|
||||||
|
masked: rebuildLedger(ctx),
|
||||||
|
lastAutoCompactLeafId: null,
|
||||||
|
};
|
||||||
runtimes.set(sessionId, runtime);
|
runtimes.set(sessionId, runtime);
|
||||||
service.setSession(sessionId, (ids) => index.query(ids));
|
service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
|
||||||
}
|
}
|
||||||
return runtime;
|
return runtime;
|
||||||
};
|
};
|
||||||
|
|
||||||
const persist = (record: unknown): void => {
|
const ledgerStats = (runtime: SessionRuntime): { maskedResults: number; reclaimedTokens: number } => {
|
||||||
pi.appendEntry(COMPACTION_CUSTOM_TYPE, record);
|
let reclaimedTokens = 0;
|
||||||
};
|
for (const entry of runtime.masked.values()) {
|
||||||
|
reclaimedTokens += Math.max(0, entry.originalTokens - entry.placeholderTokens);
|
||||||
const drainSemantic = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
|
|
||||||
const sessionId = ctx.sessionManager.getSessionId();
|
|
||||||
for (const result of worker.drain(sessionId)) {
|
|
||||||
const record = runtime.index.applySemantic(result.sourceKey, result.sourceHash, result.summary);
|
|
||||||
if (record) persist(record);
|
|
||||||
}
|
}
|
||||||
runtime.index.setSemanticJobs(worker.count(sessionId));
|
return { maskedResults: runtime.masked.size, reclaimedTokens };
|
||||||
};
|
};
|
||||||
|
|
||||||
const projectMessages = (
|
const projectMessages = (
|
||||||
@@ -218,16 +237,32 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
messages: readonly any[],
|
messages: readonly any[],
|
||||||
): ProjectionSnapshot => {
|
): ProjectionSnapshot => {
|
||||||
const env = environment(pi, ctx, environments);
|
const env = environment(pi, ctx, environments);
|
||||||
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId()));
|
const stats = ledgerStats(runtime);
|
||||||
const snapshot = runtime.index.project(messages, {
|
const snapshot = runtime.index.project(messages, {
|
||||||
environmentHash: env.hash,
|
environmentHash: env.hash,
|
||||||
environmentTokens: env.tokens,
|
environmentTokens: env.tokens,
|
||||||
contextWindow: env.contextWindow,
|
contextWindow: env.contextWindow,
|
||||||
|
maskedResults: stats.maskedResults,
|
||||||
|
reclaimedTokens: stats.reclaimedTokens,
|
||||||
});
|
});
|
||||||
service.setSnapshot(snapshot);
|
service.setSnapshot(snapshot);
|
||||||
return snapshot;
|
return snapshot;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const rebuildLedger = (ctx: ExtensionContext): Map<string, MaskedLedgerEntry> =>
|
||||||
|
rebuildMaskedLedger(
|
||||||
|
ctx.sessionManager.getBranch(),
|
||||||
|
ctx.sessionManager.getLeafId(),
|
||||||
|
(message) => estimateTokens(message as any),
|
||||||
|
);
|
||||||
|
|
||||||
|
const hydrateRuntime = (ctx: ExtensionContext, runtime: SessionRuntime): void => {
|
||||||
|
runtime.index.hydrate();
|
||||||
|
runtime.masked = rebuildLedger(ctx);
|
||||||
|
runtime.lastAutoCompactLeafId = null;
|
||||||
|
runtime.codexCache = undefined;
|
||||||
|
};
|
||||||
|
|
||||||
const branchMessages = (
|
const branchMessages = (
|
||||||
ctx: ExtensionContext,
|
ctx: ExtensionContext,
|
||||||
runtime: SessionRuntime,
|
runtime: SessionRuntime,
|
||||||
@@ -237,9 +272,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
const checkpoint = findNativeCheckpoint(branch);
|
const checkpoint = findNativeCheckpoint(branch);
|
||||||
let messages: any[];
|
let messages: any[];
|
||||||
if (checkpoint.status === "valid") {
|
if (checkpoint.status === "valid") {
|
||||||
messages = branch
|
// sessionEntryToContextMessages sees only raw entries; applyContextEdits
|
||||||
.slice(checkpoint.checkpoint.entryIndex + 1)
|
// additionally replays the latest context_edit per target (omissions and
|
||||||
.flatMap((entry) => sessionEntryToContextMessages(entry));
|
// placeholder replacements included).
|
||||||
|
messages = applyContextEdits(branch.slice(checkpoint.checkpoint.entryIndex + 1));
|
||||||
if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) {
|
if (checkpoint.checkpoint.details.modelKey !== (ctx.model ? modelKey(ctx.model) : "")) {
|
||||||
messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages];
|
messages = [asPortableMessage(checkpoint.checkpoint.details.portableSummary), ...messages];
|
||||||
}
|
}
|
||||||
@@ -278,15 +314,22 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
publishCompactionService(service);
|
publishCompactionService(service);
|
||||||
const sessionId = ctx.sessionManager.getSessionId();
|
const sessionId = ctx.sessionManager.getSessionId();
|
||||||
const config = await loadConfig();
|
const config = await loadConfig();
|
||||||
const index = new SegmentIndex(sessionId);
|
const runtime = runtimes.get(sessionId) ?? {
|
||||||
index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], config);
|
index: new SegmentIndex(sessionId),
|
||||||
runtimes.set(sessionId, { index, config });
|
config,
|
||||||
service.setSession(sessionId, (ids) => index.query(ids));
|
masked: new Map<string, MaskedLedgerEntry>(),
|
||||||
|
lastAutoCompactLeafId: null,
|
||||||
|
};
|
||||||
|
runtime.config = config;
|
||||||
|
runtime.index.hydrate();
|
||||||
|
runtime.masked = rebuildLedger(ctx);
|
||||||
|
runtime.lastAutoCompactLeafId = null;
|
||||||
|
runtimes.set(sessionId, runtime);
|
||||||
|
service.setSession(sessionId, (ids) => maskedRecords(sessionId, ids));
|
||||||
});
|
});
|
||||||
|
|
||||||
pi.on("session_shutdown", (_event, ctx) => {
|
pi.on("session_shutdown", (_event, ctx) => {
|
||||||
const sessionId = ctx.sessionManager.getSessionId();
|
const sessionId = ctx.sessionManager.getSessionId();
|
||||||
worker.cancelSession(sessionId);
|
|
||||||
runtimes.delete(sessionId);
|
runtimes.delete(sessionId);
|
||||||
payloads.delete(sessionId);
|
payloads.delete(sessionId);
|
||||||
environments.delete(sessionId);
|
environments.delete(sessionId);
|
||||||
@@ -296,15 +339,13 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
|
|
||||||
pi.on("session_tree", (_event, ctx) => {
|
pi.on("session_tree", (_event, ctx) => {
|
||||||
const runtime = runtimeFor(ctx);
|
const runtime = runtimeFor(ctx);
|
||||||
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config);
|
hydrateRuntime(ctx, runtime);
|
||||||
runtime.codexCache = undefined;
|
|
||||||
service.clearSnapshot(ctx.sessionManager.getSessionId());
|
service.clearSnapshot(ctx.sessionManager.getSessionId());
|
||||||
});
|
});
|
||||||
|
|
||||||
pi.on("session_compact", (_event, ctx) => {
|
pi.on("session_compact", (_event, ctx) => {
|
||||||
const runtime = runtimeFor(ctx);
|
const runtime = runtimeFor(ctx);
|
||||||
runtime.index.hydrate(ctx.sessionManager.getBranch() as SessionEntry[], runtime.config);
|
hydrateRuntime(ctx, runtime);
|
||||||
runtime.codexCache = undefined;
|
|
||||||
service.clearSnapshot(ctx.sessionManager.getSessionId());
|
service.clearSnapshot(ctx.sessionManager.getSessionId());
|
||||||
payloads.delete(ctx.sessionManager.getSessionId());
|
payloads.delete(ctx.sessionManager.getSessionId());
|
||||||
});
|
});
|
||||||
@@ -319,21 +360,9 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
|
|
||||||
pi.on("turn_end", (event, ctx) => {
|
pi.on("turn_end", (event, ctx) => {
|
||||||
const runtime = runtimeFor(ctx);
|
const runtime = runtimeFor(ctx);
|
||||||
drainSemantic(ctx, runtime);
|
if (!runtime.config.enabled) return undefined;
|
||||||
const previous = runtime.index.getSnapshot();
|
|
||||||
const captured = runtime.index.captureTurn(event.message, event.toolResults, runtime.config);
|
|
||||||
for (const record of captured) persist(runtime.index.toPersisted(record));
|
|
||||||
const pressure = previous && previous.contextWindow > 0
|
|
||||||
? previous.upperBound / previous.contextWindow >= runtime.config.softRatio
|
|
||||||
: false;
|
|
||||||
const semanticCandidates = pressure
|
|
||||||
? runtime.index.allRecords()
|
|
||||||
: captured.filter((record) => record.rawText.length >= runtime.config.semantic.maxInputChars);
|
|
||||||
for (const record of semanticCandidates) {
|
|
||||||
worker.enqueue(ctx.sessionManager.getSessionId(), record, ctx, runtime.config);
|
|
||||||
}
|
|
||||||
runtime.index.setSemanticJobs(worker.count(ctx.sessionManager.getSessionId()));
|
|
||||||
|
|
||||||
|
const previous = runtime.index.getSnapshot();
|
||||||
const usage = ctx.getContextUsage();
|
const usage = ctx.getContextUsage();
|
||||||
if (usage?.tokens && previous) {
|
if (usage?.tokens && previous) {
|
||||||
const env = environment(pi, ctx, environments);
|
const env = environment(pi, ctx, environments);
|
||||||
@@ -343,6 +372,50 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
env.hash,
|
env.hash,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const currentTokens = usage?.tokens
|
||||||
|
?? (event.context?.contextMessages ?? []).reduce((sum, message) => sum + estimateTokens(message as any), 0)
|
||||||
|
+ environment(pi, ctx, environments).tokens;
|
||||||
|
const plan = planMasking({
|
||||||
|
contextEntries: event.context?.contextEntries ?? [],
|
||||||
|
currentTokens,
|
||||||
|
contextWindow: ctx.model?.contextWindow ?? 0,
|
||||||
|
masking: runtime.config.masking,
|
||||||
|
estimate: (message) => estimateTokens(message as any),
|
||||||
|
});
|
||||||
|
if (!plan.drafts.length) return undefined;
|
||||||
|
for (const result of plan.masked) {
|
||||||
|
runtime.masked.set(result.toolCallId, {
|
||||||
|
placeholder: result.placeholder,
|
||||||
|
originalTokens: result.originalTokens,
|
||||||
|
placeholderTokens: result.placeholderTokens,
|
||||||
|
resultTimestamp: result.resultTimestamp,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return { entries: [...(event.entries ?? []), ...plan.drafts] };
|
||||||
|
});
|
||||||
|
|
||||||
|
pi.on("agent_settled", (_event, ctx) => {
|
||||||
|
const runtime = runtimeFor(ctx);
|
||||||
|
if (!runtime.config.enabled || ctx.isIdle?.() === false) return;
|
||||||
|
const snapshot = runtime.index.getSnapshot();
|
||||||
|
const tokens = ctx.getContextUsage()?.tokens ?? snapshot?.effectiveTokens;
|
||||||
|
if (tokens == null) return;
|
||||||
|
const contextWindow = ctx.model?.contextWindow ?? snapshot?.contextWindow ?? 0;
|
||||||
|
const threshold = effectiveThreshold(
|
||||||
|
runtime.config.summary.triggerTokens,
|
||||||
|
runtime.config.summary.triggerRatio,
|
||||||
|
contextWindow,
|
||||||
|
);
|
||||||
|
if (tokens < threshold) return;
|
||||||
|
const leafId = ctx.sessionManager.getLeafId();
|
||||||
|
if (leafId !== null && runtime.lastAutoCompactLeafId === leafId) return;
|
||||||
|
runtime.lastAutoCompactLeafId = leafId;
|
||||||
|
ctx.compact({
|
||||||
|
onError: (error) => {
|
||||||
|
if (ctx.hasUI) ctx.ui.notify(`pi-compaction auto summary failed: ${errorMessage(error)}`, "warning");
|
||||||
|
},
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
pi.on("context", (event, ctx) => {
|
pi.on("context", (event, ctx) => {
|
||||||
@@ -358,7 +431,10 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
const projected = branchMessages(ctx, runtime, branch);
|
const projected = branchMessages(ctx, runtime, branch);
|
||||||
return { messages: projected.snapshot.messages as any };
|
return { messages: projected.snapshot.messages as any };
|
||||||
}
|
}
|
||||||
return { messages: projectMessages(ctx, runtime, event.messages).messages as any };
|
// Non-checkpoint path only refreshes the shared snapshot; the message list
|
||||||
|
// is returned to Pi unchanged — masking is append-only via context_edit.
|
||||||
|
projectMessages(ctx, runtime, event.messages);
|
||||||
|
return undefined;
|
||||||
});
|
});
|
||||||
|
|
||||||
pi.on("before_provider_headers", (event, ctx) => {
|
pi.on("before_provider_headers", (event, ctx) => {
|
||||||
@@ -387,7 +463,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
pi.on("session_before_compact", async (event, ctx) => {
|
pi.on("session_before_compact", async (event, ctx) => {
|
||||||
const runtime = runtimeFor(ctx);
|
const runtime = runtimeFor(ctx);
|
||||||
if (!runtime.config.enabled) return undefined;
|
if (!runtime.config.enabled) return undefined;
|
||||||
drainSemantic(ctx, runtime);
|
|
||||||
const { snapshot, checkpoint } = branchMessages(
|
const { snapshot, checkpoint } = branchMessages(
|
||||||
ctx,
|
ctx,
|
||||||
runtime,
|
runtime,
|
||||||
@@ -395,8 +470,6 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
event.reason === "overflow" && event.willRetry,
|
event.reason === "overflow" && event.willRetry,
|
||||||
);
|
);
|
||||||
|
|
||||||
if (!shouldCompact(event.reason, snapshot, runtime.config)) return { cancel: true };
|
|
||||||
|
|
||||||
const model = ctx.model;
|
const model = ctx.model;
|
||||||
if (!model) return undefined;
|
if (!model) return undefined;
|
||||||
if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) {
|
if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) {
|
||||||
@@ -412,7 +485,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
event.signal,
|
event.signal,
|
||||||
event.customInstructions,
|
event.customInstructions,
|
||||||
undefined,
|
undefined,
|
||||||
runtime.config.semantic.thinking,
|
runtime.config.summary.thinking,
|
||||||
);
|
);
|
||||||
return {
|
return {
|
||||||
compaction: {
|
compaction: {
|
||||||
@@ -463,7 +536,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
replacementHistory: buildReplacementHistory(
|
replacementHistory: buildReplacementHistory(
|
||||||
input,
|
input,
|
||||||
remote.compactionItem,
|
remote.compactionItem,
|
||||||
Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * Math.max(0.05, runtime.config.targetRatio - 0.3)))),
|
Math.min(64_000, Math.max(4_096, Math.floor(snapshot.contextWindow * 0.25))),
|
||||||
),
|
),
|
||||||
portableSummary: portableSummary(snapshot.messages),
|
portableSummary: portableSummary(snapshot.messages),
|
||||||
sourceFingerprint: snapshot.fingerprint,
|
sourceFingerprint: snapshot.fingerprint,
|
||||||
@@ -504,7 +577,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
event.signal,
|
event.signal,
|
||||||
event.preparation.customInstructions,
|
event.preparation.customInstructions,
|
||||||
undefined,
|
undefined,
|
||||||
runtime.config.semantic.thinking,
|
runtime.config.summary.thinking,
|
||||||
undefined,
|
undefined,
|
||||||
undefined,
|
undefined,
|
||||||
undefined,
|
undefined,
|
||||||
@@ -531,38 +604,64 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
pi.registerTool({
|
pi.registerTool({
|
||||||
name: QUERY_TOOL_NAME,
|
name: QUERY_TOOL_NAME,
|
||||||
label: "Query Compacted Tool History",
|
label: "Query Compacted Tool History",
|
||||||
description: "Recover original tool outputs by source key or tool call id after pi-compaction replaced them with bounded summaries.",
|
description: "Recover the original output of a tool result that pi-compaction cleared from context, by tool call id.",
|
||||||
promptSnippet: "Recover original compacted tool evidence",
|
promptSnippet: "Recover original tool output cleared from context",
|
||||||
promptGuidelines: [
|
promptGuidelines: [
|
||||||
"Use context_tree_query only when a pi-compaction summary lacks evidence needed for the current task.",
|
"Use context_tree_query when you see a \"[pi-compaction] Earlier\" placeholder and still need the original tool output.",
|
||||||
],
|
],
|
||||||
parameters: Type.Object({
|
parameters: Type.Object({
|
||||||
toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }),
|
toolCallIds: Type.Array(Type.String(), { minItems: 1, maxItems: 20 }),
|
||||||
}),
|
}),
|
||||||
async execute(_id, params, _signal, _update, ctx) {
|
async execute(_id, params, _signal, _update, ctx) {
|
||||||
const records = service.query(ctx.sessionManager.getSessionId(), params.toolCallIds);
|
const branch = ctx.sessionManager.getBranch() as SessionEntry[];
|
||||||
const blocks = records.map((record: ToolHistoryRecord) => {
|
const calls = new Map<string, { toolName: string; args: unknown }>();
|
||||||
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, records.length)));
|
const results = new Map<string, any>();
|
||||||
const args = truncateHead(JSON.stringify(record.args), { maxLines: 20, maxBytes: 2_048 }).content;
|
for (const entry of branch) {
|
||||||
const truncated = truncateHead(record.rawText, {
|
const message = (entry as any)?.message;
|
||||||
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, records.length))),
|
if ((entry as any)?.type !== "message" || !message) continue;
|
||||||
|
if (message.role === "assistant" && Array.isArray(message.content)) {
|
||||||
|
for (const block of message.content) {
|
||||||
|
if (block?.type === "toolCall" && typeof block.id === "string") {
|
||||||
|
calls.set(block.id, {
|
||||||
|
toolName: String(block.name ?? "unknown"),
|
||||||
|
args: block.arguments ?? block.input ?? block.args ?? {},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if (message.role === "toolResult" && typeof message.toolCallId === "string") {
|
||||||
|
results.set(message.toolCallId, message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const found: string[] = [];
|
||||||
|
const blocks: string[] = [];
|
||||||
|
for (const id of params.toolCallIds) {
|
||||||
|
const result = results.get(id);
|
||||||
|
if (!result) continue;
|
||||||
|
found.push(id);
|
||||||
|
const call = calls.get(id) ?? { toolName: String(result.toolName ?? "unknown"), args: {} };
|
||||||
|
const perResultBytes = Math.max(1_024, Math.floor(DEFAULT_MAX_BYTES / Math.max(1, params.toolCallIds.length)));
|
||||||
|
const args = truncateHead(JSON.stringify(call.args ?? {}), { maxLines: 20, maxBytes: 2_048 }).content;
|
||||||
|
const rawText = contentText(result.content);
|
||||||
|
const truncated = truncateHead(rawText, {
|
||||||
|
maxLines: Math.max(20, Math.floor(DEFAULT_MAX_LINES / Math.max(1, params.toolCallIds.length))),
|
||||||
maxBytes: perResultBytes,
|
maxBytes: perResultBytes,
|
||||||
});
|
});
|
||||||
return [
|
blocks.push([
|
||||||
`## ${record.sourceKey}`,
|
`## ${id}`,
|
||||||
`Tool: ${record.toolName}`,
|
`Tool: ${call.toolName}`,
|
||||||
`Args: ${args}`,
|
`Args: ${args}`,
|
||||||
`Status: ${record.isError ? "ERROR" : "OK"}`,
|
`Status: ${result.isError ? "ERROR" : "OK"}`,
|
||||||
"",
|
"",
|
||||||
truncated.content,
|
truncated.content,
|
||||||
...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []),
|
...(truncated.truncated ? [`[truncated: ${truncated.outputLines}/${truncated.totalLines} lines]`] : []),
|
||||||
].join("\n");
|
].join("\n"));
|
||||||
});
|
}
|
||||||
const missing = params.toolCallIds.filter((id: string) => !records.some((record) => record.sourceKey === id || record.toolCallId === id));
|
const missing = params.toolCallIds.filter((id: string) => !results.has(id));
|
||||||
for (const id of missing) blocks.push(`## ${id}\n(not found)`);
|
for (const id of missing) blocks.push(`## ${id}\n(not found)`);
|
||||||
return {
|
return {
|
||||||
content: [{ type: "text", text: blocks.join("\n\n---\n\n") }],
|
content: [{ type: "text", text: blocks.join("\n\n---\n\n") }],
|
||||||
details: { sourceKeys: records.map((record) => record.sourceKey), missing },
|
details: { toolCallIds: found, missing },
|
||||||
};
|
};
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -576,12 +675,11 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
|
|||||||
`Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`,
|
`Effective: ${metrics.effectiveTokens}/${metrics.contextWindow} (${metrics.percent.toFixed(1)}%, ${metrics.confidence})`,
|
||||||
`Raw: ${metrics.rawTokens}/${metrics.contextWindow}`,
|
`Raw: ${metrics.rawTokens}/${metrics.contextWindow}`,
|
||||||
`Reclaimed: ${metrics.reclaimedTokens}`,
|
`Reclaimed: ${metrics.reclaimedTokens}`,
|
||||||
`Segments: ${metrics.segmentCount}; semantic jobs: ${metrics.semanticJobs}; generation: ${metrics.generation}`,
|
`Segments: ${metrics.segmentCount}; masked tool results: ${metrics.maskedResults}; generation: ${metrics.generation}`,
|
||||||
].join("\n")
|
].join("\n")
|
||||||
: "pi-compaction has no projection snapshot yet.";
|
: "pi-compaction has no projection snapshot yet.";
|
||||||
if (ctx.hasUI) ctx.ui.notify(text, "info");
|
if (ctx.hasUI) ctx.ui.notify(text, "info");
|
||||||
else process.stderr.write(`${text}\n`);
|
else process.stderr.write(`${text}\n`);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|||||||
+58
-35
@@ -12,27 +12,29 @@ export const CONFIG_PATH = fileURLToPath(
|
|||||||
|
|
||||||
export const DEFAULT_CONFIG: PiCompactionConfig = {
|
export const DEFAULT_CONFIG: PiCompactionConfig = {
|
||||||
enabled: true,
|
enabled: true,
|
||||||
softRatio: 0.72,
|
masking: {
|
||||||
hardRatio: 0.9,
|
|
||||||
targetRatio: 0.55,
|
|
||||||
deterministic: {
|
|
||||||
enabled: true,
|
enabled: true,
|
||||||
maxStubTokens: 1_024,
|
triggerTokens: 200_000,
|
||||||
|
triggerRatio: 0.6,
|
||||||
|
targetTokens: 100_000,
|
||||||
|
targetRatio: 0.35,
|
||||||
|
minClearTokens: 50_000,
|
||||||
|
keepRecentTurns: 10,
|
||||||
|
minEntryTokens: 256,
|
||||||
|
excludeTools: ["ask_user_question"],
|
||||||
},
|
},
|
||||||
semantic: {
|
summary: {
|
||||||
enabled: true,
|
triggerTokens: 400_000,
|
||||||
model: "default",
|
triggerRatio: 0.9,
|
||||||
thinking: "low",
|
thinking: "low",
|
||||||
concurrency: 1,
|
|
||||||
minInputChars: 4_000,
|
|
||||||
maxInputChars: 24_000,
|
|
||||||
maxRetries: 2,
|
|
||||||
},
|
},
|
||||||
native: {
|
native: {
|
||||||
enabled: true,
|
enabled: true,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh"];
|
||||||
|
|
||||||
function ratio(value: unknown, fallback: number): number {
|
function ratio(value: unknown, fallback: number): number {
|
||||||
return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback;
|
return typeof value === "number" && Number.isFinite(value) && value > 0 && value < 1 ? value : fallback;
|
||||||
}
|
}
|
||||||
@@ -41,46 +43,67 @@ function integer(value: unknown, fallback: number, min = 0, max = Number.MAX_SAF
|
|||||||
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback;
|
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max ? value : fallback;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function positiveInteger(value: unknown, fallback: number): number {
|
||||||
|
return integer(value, fallback, 1);
|
||||||
|
}
|
||||||
|
|
||||||
function object(value: unknown): Record<string, unknown> {
|
function object(value: unknown): Record<string, unknown> {
|
||||||
return value !== null && typeof value === "object" && !Array.isArray(value)
|
return value !== null && typeof value === "object" && !Array.isArray(value)
|
||||||
? value as Record<string, unknown>
|
? value as Record<string, unknown>
|
||||||
: {};
|
: {};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function stringArray(value: unknown, fallback: string[]): string[] {
|
||||||
|
if (!Array.isArray(value) || !value.every((entry) => typeof entry === "string")) return fallback;
|
||||||
|
return value.map((entry) => entry.trim()).filter((entry) => entry.length > 0);
|
||||||
|
}
|
||||||
|
|
||||||
export function normalizeConfig(value: unknown): PiCompactionConfig {
|
export function normalizeConfig(value: unknown): PiCompactionConfig {
|
||||||
const input = object(value);
|
const input = object(value);
|
||||||
const deterministic = object(input.deterministic);
|
const masking = object(input.masking);
|
||||||
const semantic = object(input.semantic);
|
const summary = object(input.summary);
|
||||||
const native = object(input.native);
|
const native = object(input.native);
|
||||||
const softRatio = ratio(input.softRatio, DEFAULT_CONFIG.softRatio);
|
|
||||||
const hardRatio = Math.max(softRatio + 0.01, ratio(input.hardRatio, DEFAULT_CONFIG.hardRatio));
|
|
||||||
const targetRatio = Math.min(softRatio - 0.01, ratio(input.targetRatio, DEFAULT_CONFIG.targetRatio));
|
|
||||||
const thinking = ["off", "minimal", "low", "medium", "high", "xhigh"].includes(String(semantic.thinking))
|
|
||||||
? semantic.thinking as PiCompactionConfig["semantic"]["thinking"]
|
|
||||||
: DEFAULT_CONFIG.semantic.thinking;
|
|
||||||
|
|
||||||
return {
|
const normalized: PiCompactionConfig = {
|
||||||
enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled,
|
enabled: typeof input.enabled === "boolean" ? input.enabled : DEFAULT_CONFIG.enabled,
|
||||||
softRatio,
|
masking: {
|
||||||
hardRatio: Math.min(0.99, hardRatio),
|
enabled: typeof masking.enabled === "boolean" ? masking.enabled : DEFAULT_CONFIG.masking.enabled,
|
||||||
targetRatio: Math.max(0.1, targetRatio),
|
triggerTokens: positiveInteger(masking.triggerTokens, DEFAULT_CONFIG.masking.triggerTokens),
|
||||||
deterministic: {
|
triggerRatio: ratio(masking.triggerRatio, DEFAULT_CONFIG.masking.triggerRatio),
|
||||||
enabled: typeof deterministic.enabled === "boolean" ? deterministic.enabled : DEFAULT_CONFIG.deterministic.enabled,
|
targetTokens: positiveInteger(masking.targetTokens, DEFAULT_CONFIG.masking.targetTokens),
|
||||||
maxStubTokens: integer(deterministic.maxStubTokens, DEFAULT_CONFIG.deterministic.maxStubTokens, 32, 4_096),
|
targetRatio: ratio(masking.targetRatio, DEFAULT_CONFIG.masking.targetRatio),
|
||||||
|
minClearTokens: positiveInteger(masking.minClearTokens, DEFAULT_CONFIG.masking.minClearTokens),
|
||||||
|
keepRecentTurns: integer(masking.keepRecentTurns, DEFAULT_CONFIG.masking.keepRecentTurns, 1, 100),
|
||||||
|
minEntryTokens: integer(masking.minEntryTokens, DEFAULT_CONFIG.masking.minEntryTokens, 0, 100_000),
|
||||||
|
excludeTools: stringArray(masking.excludeTools, DEFAULT_CONFIG.masking.excludeTools),
|
||||||
},
|
},
|
||||||
semantic: {
|
summary: {
|
||||||
enabled: typeof semantic.enabled === "boolean" ? semantic.enabled : DEFAULT_CONFIG.semantic.enabled,
|
triggerTokens: positiveInteger(summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens),
|
||||||
model: typeof semantic.model === "string" && semantic.model.trim() ? semantic.model.trim() : DEFAULT_CONFIG.semantic.model,
|
triggerRatio: ratio(summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio),
|
||||||
thinking,
|
thinking: THINKING_LEVELS.includes(String(summary.thinking))
|
||||||
concurrency: integer(semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency, 1, 4),
|
? summary.thinking as PiCompactionConfig["summary"]["thinking"]
|
||||||
minInputChars: integer(semantic.minInputChars, DEFAULT_CONFIG.semantic.minInputChars, 0, 1_000_000),
|
: DEFAULT_CONFIG.summary.thinking,
|
||||||
maxInputChars: integer(semantic.maxInputChars, DEFAULT_CONFIG.semantic.maxInputChars, 1_000, 1_000_000),
|
|
||||||
maxRetries: integer(semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries, 0, 5),
|
|
||||||
},
|
},
|
||||||
native: {
|
native: {
|
||||||
enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled,
|
enabled: typeof native.enabled === "boolean" ? native.enabled : DEFAULT_CONFIG.native.enabled,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Trigger/target must keep their hysteresis ordering; fix up rather than
|
||||||
|
// letting a degenerate pair mask-or-compact every turn.
|
||||||
|
if (normalized.masking.targetTokens >= normalized.masking.triggerTokens) {
|
||||||
|
normalized.masking.targetTokens = Math.floor(normalized.masking.triggerTokens / 2);
|
||||||
|
}
|
||||||
|
if (normalized.masking.targetRatio >= normalized.masking.triggerRatio) {
|
||||||
|
normalized.masking.targetRatio = normalized.masking.triggerRatio / 2;
|
||||||
|
}
|
||||||
|
if (normalized.summary.triggerTokens <= normalized.masking.triggerTokens) {
|
||||||
|
normalized.summary.triggerTokens = normalized.masking.triggerTokens * 2;
|
||||||
|
}
|
||||||
|
if (normalized.summary.triggerRatio <= normalized.masking.triggerRatio) {
|
||||||
|
normalized.summary.triggerRatio = Math.min(0.99, normalized.masking.triggerRatio + 0.01);
|
||||||
|
}
|
||||||
|
return normalized;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function loadConfig(
|
export async function loadConfig(
|
||||||
|
|||||||
@@ -0,0 +1,257 @@
|
|||||||
|
import {
|
||||||
|
buildContextEntries,
|
||||||
|
sessionEntryToContextMessages,
|
||||||
|
type ContextEditEntryDraft,
|
||||||
|
} from "@earendil-works/pi-coding-agent";
|
||||||
|
import { contentText, timestampOf } from "./segment-index.js";
|
||||||
|
import { effectiveThreshold } from "./token-accounting.js";
|
||||||
|
import type { MaskingConfig } from "./types.js";
|
||||||
|
|
||||||
|
// Every placeholder written through a context_edit starts with this prefix so
|
||||||
|
// re-masking, ledger rebuilds, and the model-facing recovery hint all key off
|
||||||
|
// one marker.
|
||||||
|
export const MASK_PREFIX = "[pi-compaction] Earlier ";
|
||||||
|
|
||||||
|
const MAX_PLACEHOLDER_ARGS_CHARS = 300;
|
||||||
|
const EDITABLE_ROLES = new Set(["user", "assistant", "toolResult", "custom"]);
|
||||||
|
|
||||||
|
export interface MaskingProjectedEntry {
|
||||||
|
sourceEntry: { id?: string; type?: string; message?: any };
|
||||||
|
messages: any[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MaskedResult {
|
||||||
|
toolCallId: string;
|
||||||
|
entryId: string;
|
||||||
|
placeholder: string;
|
||||||
|
originalTokens: number;
|
||||||
|
placeholderTokens: number;
|
||||||
|
resultTimestamp: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MaskingPlan {
|
||||||
|
drafts: ContextEditEntryDraft[];
|
||||||
|
clearedTokens: number;
|
||||||
|
masked: MaskedResult[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MaskedLedgerEntry {
|
||||||
|
placeholder: string;
|
||||||
|
originalTokens: number;
|
||||||
|
placeholderTokens: number;
|
||||||
|
resultTimestamp: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EMPTY_PLAN: MaskingPlan = { drafts: [], clearedTokens: 0, masked: [] };
|
||||||
|
|
||||||
|
function json(value: unknown): string {
|
||||||
|
try {
|
||||||
|
return JSON.stringify(value) ?? "undefined";
|
||||||
|
} catch {
|
||||||
|
return "[unserializable]";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function maskPlaceholder(
|
||||||
|
toolName: string,
|
||||||
|
args: unknown,
|
||||||
|
toolCallId: string,
|
||||||
|
originalTokens: number,
|
||||||
|
overflowId?: string,
|
||||||
|
): string {
|
||||||
|
let argsText = json(args);
|
||||||
|
if (argsText.length > MAX_PLACEHOLDER_ARGS_CHARS) {
|
||||||
|
argsText = `${argsText.slice(0, MAX_PLACEHOLDER_ARGS_CHARS)}…`;
|
||||||
|
}
|
||||||
|
// `details.overflow.id` is pi-overflow's public contract — read the field
|
||||||
|
// only, never import its implementation.
|
||||||
|
const overflowHint = typeof overflowId === "string" && overflowId.length > 0
|
||||||
|
? ` Full original stored: overflow_query {"id":"${overflowId}"}.`
|
||||||
|
: "";
|
||||||
|
return `${MASK_PREFIX}${toolName} result cleared to save context (~${originalTokens} tokens). ` +
|
||||||
|
`Call args: ${argsText}. Re-run the call if still needed, or recover the original with ` +
|
||||||
|
`context_tree_query {"toolCallIds":["${toolCallId}"]}.${overflowHint}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isSkillRead(toolName: string, args: unknown): boolean {
|
||||||
|
if (toolName !== "read" || args === null || typeof args !== "object") return false;
|
||||||
|
const record = args as Record<string, unknown>;
|
||||||
|
const raw = record.path ?? record.filePath ?? record.file;
|
||||||
|
if (typeof raw !== "string") return false;
|
||||||
|
return (raw.split(/[\\/]/).pop() ?? raw) === "SKILL.md";
|
||||||
|
}
|
||||||
|
|
||||||
|
function toolResultCalls(entries: readonly MaskingProjectedEntry[]): Map<string, { toolName: string; args: unknown }> {
|
||||||
|
const calls = new Map<string, { toolName: string; args: unknown }>();
|
||||||
|
for (const entry of entries) {
|
||||||
|
for (const message of entry.messages ?? []) {
|
||||||
|
if (message?.role !== "assistant" || !Array.isArray(message.content)) continue;
|
||||||
|
for (const block of message.content) {
|
||||||
|
if (block?.type === "toolCall" && typeof block.id === "string") {
|
||||||
|
calls.set(block.id, {
|
||||||
|
toolName: String(block.name ?? "unknown"),
|
||||||
|
args: block.arguments ?? block.input ?? block.args ?? {},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return calls;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function planMasking(params: {
|
||||||
|
contextEntries: readonly MaskingProjectedEntry[];
|
||||||
|
currentTokens: number;
|
||||||
|
contextWindow: number;
|
||||||
|
masking: MaskingConfig;
|
||||||
|
estimate: (message: any) => number;
|
||||||
|
}): MaskingPlan {
|
||||||
|
const { masking } = params;
|
||||||
|
const triggerEff = effectiveThreshold(masking.triggerTokens, masking.triggerRatio, params.contextWindow);
|
||||||
|
if (!masking.enabled || params.currentTokens < triggerEff) return EMPTY_PLAN;
|
||||||
|
const targetEff = effectiveThreshold(masking.targetTokens, masking.targetRatio, params.contextWindow);
|
||||||
|
const minClear = Math.min(masking.minClearTokens, Math.max(0, triggerEff - targetEff));
|
||||||
|
|
||||||
|
const entries = params.contextEntries;
|
||||||
|
// The last keepRecentTurns assistant messages and everything after them stay
|
||||||
|
// untouched; when fewer turns exist, the whole context is protected.
|
||||||
|
let protectedIndex = 0;
|
||||||
|
let turns = 0;
|
||||||
|
for (let index = entries.length - 1; index >= 0; index--) {
|
||||||
|
const entry = entries[index];
|
||||||
|
if (entry?.sourceEntry?.type === "message" && (entry.messages ?? []).some((m) => m?.role === "assistant")) {
|
||||||
|
turns++;
|
||||||
|
if (turns >= masking.keepRecentTurns) {
|
||||||
|
protectedIndex = index;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const calls = toolResultCalls(entries);
|
||||||
|
const needed = params.currentTokens - targetEff;
|
||||||
|
const masked: MaskedResult[] = [];
|
||||||
|
let clearedTokens = 0;
|
||||||
|
for (let index = 0; index < protectedIndex && clearedTokens < needed; index++) {
|
||||||
|
const entry = entries[index];
|
||||||
|
const entryId = entry?.sourceEntry?.id;
|
||||||
|
if (entry?.sourceEntry?.type !== "message" || typeof entryId !== "string") continue;
|
||||||
|
for (const message of entry.messages ?? []) {
|
||||||
|
if (clearedTokens >= needed) break;
|
||||||
|
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
|
||||||
|
const call = calls.get(message.toolCallId);
|
||||||
|
const toolName = call?.toolName ?? String(message.toolName ?? "unknown");
|
||||||
|
if (masking.excludeTools.includes(toolName)) continue;
|
||||||
|
const args = call?.args ?? {};
|
||||||
|
if (isSkillRead(toolName, args)) continue;
|
||||||
|
if (contentText(message.content).startsWith(MASK_PREFIX)) continue;
|
||||||
|
const originalTokens = params.estimate(message);
|
||||||
|
if (originalTokens < masking.minEntryTokens) continue;
|
||||||
|
const overflowId = (message.details as { overflow?: { id?: unknown } } | undefined)?.overflow?.id;
|
||||||
|
const placeholder = maskPlaceholder(
|
||||||
|
toolName,
|
||||||
|
args,
|
||||||
|
message.toolCallId,
|
||||||
|
originalTokens,
|
||||||
|
typeof overflowId === "string" ? overflowId : undefined,
|
||||||
|
);
|
||||||
|
const placeholderTokens = params.estimate({ ...message, content: [{ type: "text", text: placeholder }] });
|
||||||
|
clearedTokens += originalTokens - placeholderTokens;
|
||||||
|
masked.push({
|
||||||
|
toolCallId: message.toolCallId,
|
||||||
|
entryId,
|
||||||
|
placeholder,
|
||||||
|
originalTokens,
|
||||||
|
placeholderTokens,
|
||||||
|
resultTimestamp: timestampOf(message.timestamp),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (clearedTokens < minClear) return EMPTY_PLAN;
|
||||||
|
return {
|
||||||
|
drafts: masked.map((result) => ({
|
||||||
|
type: "context_edit",
|
||||||
|
targetId: result.entryId,
|
||||||
|
replacement: { content: result.placeholder },
|
||||||
|
})),
|
||||||
|
clearedTokens,
|
||||||
|
masked,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rebuild the masked-results ledger from a session branch after hydration:
|
||||||
|
* the latest context_edit per target wins, only edits carrying this
|
||||||
|
* extension's placeholder prefix count as masked, and only targets that are
|
||||||
|
* still part of the active context (post-compaction kept set) are counted.
|
||||||
|
*/
|
||||||
|
export function rebuildMaskedLedger(
|
||||||
|
branch: readonly any[],
|
||||||
|
leafId: string | null | undefined,
|
||||||
|
estimate: (message: any) => number,
|
||||||
|
): Map<string, MaskedLedgerEntry> {
|
||||||
|
const latestEdits = new Map<string, any>();
|
||||||
|
const targets = new Map<string, any>();
|
||||||
|
for (const entry of branch) {
|
||||||
|
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
|
||||||
|
latestEdits.set(entry.targetId, entry);
|
||||||
|
} else if (entry?.type === "message" && typeof entry.id === "string") {
|
||||||
|
targets.set(entry.id, entry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const contextIds = new Set(
|
||||||
|
buildContextEntries(branch as any, leafId).map((entry) => entry.id),
|
||||||
|
);
|
||||||
|
const masked = new Map<string, MaskedLedgerEntry>();
|
||||||
|
for (const [targetId, edit] of latestEdits) {
|
||||||
|
if (!contextIds.has(targetId)) continue;
|
||||||
|
const text = contentText(edit.replacement?.content);
|
||||||
|
if (!text.startsWith(MASK_PREFIX)) continue;
|
||||||
|
const message = targets.get(targetId)?.message;
|
||||||
|
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") continue;
|
||||||
|
masked.set(message.toolCallId, {
|
||||||
|
placeholder: text,
|
||||||
|
originalTokens: estimate(message),
|
||||||
|
placeholderTokens: estimate({ ...message, content: [{ type: "text", text }] }),
|
||||||
|
resultTimestamp: timestampOf(message.timestamp),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return masked;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Apply the latest context_edit per target the way buildSessionProjection does
|
||||||
|
* for message entries `sessionEntryToContextMessages` alone would miss: null
|
||||||
|
* omits the entry, a string replacement becomes one text block for
|
||||||
|
* assistant/toolResult roles.
|
||||||
|
*/
|
||||||
|
export function applyContextEdits(entries: readonly any[]): any[] {
|
||||||
|
const edits = new Map<string, { content: unknown } | null>();
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (entry?.type === "context_edit" && typeof entry.targetId === "string") {
|
||||||
|
edits.set(entry.targetId, entry.replacement ?? null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const projected: any[] = [];
|
||||||
|
for (const entry of entries) {
|
||||||
|
const messages = sessionEntryToContextMessages(entry);
|
||||||
|
const edit = edits.get(entry?.id);
|
||||||
|
if (edit === undefined) {
|
||||||
|
projected.push(...messages);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (edit === null) continue;
|
||||||
|
for (const message of messages) {
|
||||||
|
if (!EDITABLE_ROLES.has(message?.role)) {
|
||||||
|
projected.push(message);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const content = (message.role === "assistant" || message.role === "toolResult")
|
||||||
|
&& typeof edit.content === "string"
|
||||||
|
? [{ type: "text", text: edit.content }]
|
||||||
|
: edit.content;
|
||||||
|
projected.push({ ...message, content });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return projected;
|
||||||
|
}
|
||||||
@@ -1,16 +1,8 @@
|
|||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import { estimateTokens } from "@earendil-works/pi-coding-agent";
|
import { estimateTokens } from "@earendil-works/pi-coding-agent";
|
||||||
import {
|
import type { ProjectionSnapshot, UsageBaseline } from "./types.js";
|
||||||
COMPACTION_CUSTOM_TYPE,
|
|
||||||
COMPACTION_VERSION,
|
|
||||||
type PiCompactionConfig,
|
|
||||||
type ProjectionSnapshot,
|
|
||||||
type SegmentSummaryRecord,
|
|
||||||
type ToolHistoryRecord,
|
|
||||||
type UsageBaseline,
|
|
||||||
} from "./types.js";
|
|
||||||
|
|
||||||
function timestampOf(value: unknown): number {
|
export function timestampOf(value: unknown): number {
|
||||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||||
if (typeof value === "string") {
|
if (typeof value === "string") {
|
||||||
const parsed = Date.parse(value);
|
const parsed = Date.parse(value);
|
||||||
@@ -19,10 +11,6 @@ function timestampOf(value: unknown): number {
|
|||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function sourceKey(toolCallId: string, timestamp: unknown): string {
|
|
||||||
return `${toolCallId}@${timestampOf(timestamp)}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function contentText(content: unknown): string {
|
export function contentText(content: unknown): string {
|
||||||
if (typeof content === "string") return content;
|
if (typeof content === "string") return content;
|
||||||
if (!Array.isArray(content)) return "";
|
if (!Array.isArray(content)) return "";
|
||||||
@@ -37,42 +25,6 @@ function hash(value: string): string {
|
|||||||
return createHash("sha256").update(value).digest("hex").slice(0, 20);
|
return createHash("sha256").update(value).digest("hex").slice(0, 20);
|
||||||
}
|
}
|
||||||
|
|
||||||
function json(value: unknown): string {
|
|
||||||
try {
|
|
||||||
return JSON.stringify(value) ?? "undefined";
|
|
||||||
} catch {
|
|
||||||
return "[unserializable]";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function boundedEvidence(text: string, maxChars: number): string {
|
|
||||||
const normalized = text.replace(/\u001b\[[0-9;]*m/g, "").trim();
|
|
||||||
if (normalized.length <= maxChars) return normalized;
|
|
||||||
const marker = `\n… ${normalized.length - maxChars} chars omitted …\n`;
|
|
||||||
const available = Math.max(0, maxChars - marker.length);
|
|
||||||
const head = Math.ceil(available * 0.7);
|
|
||||||
const tail = Math.max(0, available - head);
|
|
||||||
return `${normalized.slice(0, head)}${marker}${tail ? normalized.slice(-tail) : ""}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function deterministicSummary(params: {
|
|
||||||
toolName: string;
|
|
||||||
args: unknown;
|
|
||||||
rawText: string;
|
|
||||||
isError: boolean;
|
|
||||||
maxStubTokens: number;
|
|
||||||
}): string {
|
|
||||||
const maxChars = params.maxStubTokens * 4;
|
|
||||||
const argsText = boundedEvidence(json(params.args), Math.min(600, Math.floor(maxChars * 0.25)));
|
|
||||||
const evidenceBudget = Math.max(128, maxChars - argsText.length - 120);
|
|
||||||
const evidence = boundedEvidence(params.rawText || "(no output)", evidenceBudget);
|
|
||||||
return [
|
|
||||||
`[pi-compaction tool=${params.toolName} status=${params.isError ? "error" : "ok"}]`,
|
|
||||||
`Args: ${argsText}`,
|
|
||||||
`Evidence:\n${evidence}`,
|
|
||||||
].join("\n");
|
|
||||||
}
|
|
||||||
|
|
||||||
function messageKey(message: any): string {
|
function messageKey(message: any): string {
|
||||||
const role = String(message?.role ?? "unknown");
|
const role = String(message?.role ?? "unknown");
|
||||||
const timestamp = timestampOf(message?.timestamp);
|
const timestamp = timestampOf(message?.timestamp);
|
||||||
@@ -85,24 +37,14 @@ function messageKey(message: any): string {
|
|||||||
return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`;
|
return `${role}:${timestamp}:${typeof message?.content === "string" ? message.content.length : (message?.content?.length ?? 0)}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function isSummaryRecord(value: unknown): value is SegmentSummaryRecord {
|
/**
|
||||||
if (!value || typeof value !== "object") return false;
|
* Incremental context projection: tracks the branch's message list, estimates
|
||||||
const record = value as Partial<SegmentSummaryRecord>;
|
* tokens, and maintains the usage-backed effective-token baseline. Projection
|
||||||
return record.version === COMPACTION_VERSION
|
* is identity — pi-compaction never rewrites messages on this path; context
|
||||||
&& record.kind === "segment-summary"
|
* pressure reduction happens through append-only context_edit entries.
|
||||||
&& typeof record.sourceKey === "string"
|
*/
|
||||||
&& typeof record.toolCallId === "string"
|
|
||||||
&& typeof record.resultTimestamp === "number"
|
|
||||||
&& typeof record.toolName === "string"
|
|
||||||
&& typeof record.sourceHash === "string"
|
|
||||||
&& typeof record.deterministicSummary === "string";
|
|
||||||
}
|
|
||||||
|
|
||||||
export class SegmentIndex {
|
export class SegmentIndex {
|
||||||
private readonly records = new Map<string, ToolHistoryRecord>();
|
|
||||||
private readonly ids = new Map<string, string[]>();
|
|
||||||
private generation = 0;
|
private generation = 0;
|
||||||
private semanticJobs = 0;
|
|
||||||
private usageBaseline?: UsageBaseline;
|
private usageBaseline?: UsageBaseline;
|
||||||
private sourceMessages: any[] = [];
|
private sourceMessages: any[] = [];
|
||||||
private sourceKeys: string[] = [];
|
private sourceKeys: string[] = [];
|
||||||
@@ -117,67 +59,13 @@ export class SegmentIndex {
|
|||||||
this.sessionId = sessionId;
|
this.sessionId = sessionId;
|
||||||
}
|
}
|
||||||
|
|
||||||
hydrate(branch: readonly any[], config: PiCompactionConfig): void {
|
hydrate(): void {
|
||||||
this.reset();
|
this.reset();
|
||||||
if (!config.enabled) return;
|
|
||||||
const summaries = new Map<string, SegmentSummaryRecord>();
|
|
||||||
for (const entry of branch) {
|
|
||||||
if (entry?.type === "custom" && entry.customType === COMPACTION_CUSTOM_TYPE && isSummaryRecord(entry.data)) {
|
|
||||||
summaries.set(entry.data.sourceKey, entry.data);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const pendingCalls = new Map<string, { toolName: string; args: unknown }>();
|
|
||||||
for (const entry of branch) {
|
|
||||||
if (entry?.type !== "message") continue;
|
|
||||||
const message = entry.message;
|
|
||||||
if (message?.role === "assistant" && Array.isArray(message.content)) {
|
|
||||||
for (const block of message.content) {
|
|
||||||
if (block?.type === "toolCall" && typeof block.id === "string") {
|
|
||||||
pendingCalls.set(block.id, {
|
|
||||||
toolName: String(block.name ?? "unknown"),
|
|
||||||
args: block.input ?? block.arguments ?? block.args ?? {},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else if (message?.role === "toolResult" && typeof message.toolCallId === "string") {
|
|
||||||
const call = pendingCalls.get(message.toolCallId) ?? {
|
|
||||||
toolName: String(message.toolName ?? "unknown"),
|
|
||||||
args: {},
|
|
||||||
};
|
|
||||||
const key = sourceKey(message.toolCallId, message.timestamp);
|
|
||||||
const rawText = contentText(message.content);
|
|
||||||
const persisted = summaries.get(key);
|
|
||||||
const record: ToolHistoryRecord = {
|
|
||||||
sourceKey: key,
|
|
||||||
toolCallId: message.toolCallId,
|
|
||||||
resultTimestamp: timestampOf(message.timestamp),
|
|
||||||
toolName: call.toolName,
|
|
||||||
args: call.args,
|
|
||||||
isError: Boolean(message.isError),
|
|
||||||
rawText,
|
|
||||||
sourceHash: hash(`${call.toolName}\0${json(call.args)}\0${rawText}`),
|
|
||||||
deterministicSummary: persisted?.deterministicSummary ?? deterministicSummary({
|
|
||||||
toolName: call.toolName,
|
|
||||||
args: call.args,
|
|
||||||
rawText,
|
|
||||||
isError: Boolean(message.isError),
|
|
||||||
maxStubTokens: config.deterministic.maxStubTokens,
|
|
||||||
}),
|
|
||||||
semanticSummary: persisted?.semanticSummary,
|
|
||||||
};
|
|
||||||
this.put(record);
|
|
||||||
pendingCalls.delete(message.toolCallId);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.generation++;
|
this.generation++;
|
||||||
}
|
}
|
||||||
|
|
||||||
reset(): void {
|
reset(): void {
|
||||||
this.records.clear();
|
|
||||||
this.ids.clear();
|
|
||||||
this.generation = 0;
|
this.generation = 0;
|
||||||
this.semanticJobs = 0;
|
|
||||||
this.usageBaseline = undefined;
|
this.usageBaseline = undefined;
|
||||||
this.sourceMessages = [];
|
this.sourceMessages = [];
|
||||||
this.sourceKeys = [];
|
this.sourceKeys = [];
|
||||||
@@ -187,86 +75,6 @@ export class SegmentIndex {
|
|||||||
this.cached = undefined;
|
this.cached = undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
private put(record: ToolHistoryRecord): void {
|
|
||||||
this.records.set(record.sourceKey, record);
|
|
||||||
const keys = this.ids.get(record.toolCallId) ?? [];
|
|
||||||
if (!keys.includes(record.sourceKey)) keys.push(record.sourceKey);
|
|
||||||
this.ids.set(record.toolCallId, keys);
|
|
||||||
}
|
|
||||||
|
|
||||||
captureTurn(message: any, toolResults: any[], config: PiCompactionConfig): ToolHistoryRecord[] {
|
|
||||||
if (!config.enabled || !config.deterministic.enabled || !Array.isArray(message?.content)) return [];
|
|
||||||
const calls = new Map<string, any>();
|
|
||||||
for (const block of message.content) {
|
|
||||||
if (block?.type === "toolCall" && typeof block.id === "string") calls.set(block.id, block);
|
|
||||||
}
|
|
||||||
const captured: ToolHistoryRecord[] = [];
|
|
||||||
for (const result of toolResults ?? []) {
|
|
||||||
if (!result || typeof result.toolCallId !== "string") continue;
|
|
||||||
const call = calls.get(result.toolCallId);
|
|
||||||
if (!call) continue;
|
|
||||||
const rawText = contentText(result.content);
|
|
||||||
const toolName = String(call.name ?? result.toolName ?? "unknown");
|
|
||||||
const args = call.input ?? call.arguments ?? call.args ?? {};
|
|
||||||
const key = sourceKey(result.toolCallId, result.timestamp);
|
|
||||||
const record: ToolHistoryRecord = {
|
|
||||||
sourceKey: key,
|
|
||||||
toolCallId: result.toolCallId,
|
|
||||||
resultTimestamp: timestampOf(result.timestamp),
|
|
||||||
toolName,
|
|
||||||
args,
|
|
||||||
isError: Boolean(result.isError),
|
|
||||||
rawText,
|
|
||||||
sourceHash: hash(`${toolName}\0${json(args)}\0${rawText}`),
|
|
||||||
deterministicSummary: deterministicSummary({
|
|
||||||
toolName,
|
|
||||||
args,
|
|
||||||
rawText,
|
|
||||||
isError: Boolean(result.isError),
|
|
||||||
maxStubTokens: config.deterministic.maxStubTokens,
|
|
||||||
}),
|
|
||||||
};
|
|
||||||
this.put(record);
|
|
||||||
captured.push(record);
|
|
||||||
}
|
|
||||||
if (captured.length) {
|
|
||||||
this.generation++;
|
|
||||||
this.cached = undefined;
|
|
||||||
}
|
|
||||||
return captured;
|
|
||||||
}
|
|
||||||
|
|
||||||
applySemantic(sourceKeyValue: string, sourceHash: string, summary: string): SegmentSummaryRecord | undefined {
|
|
||||||
const record = this.records.get(sourceKeyValue);
|
|
||||||
if (!record || record.sourceHash !== sourceHash || !summary.trim()) return undefined;
|
|
||||||
record.semanticSummary = summary.trim();
|
|
||||||
this.generation++;
|
|
||||||
this.cached = undefined;
|
|
||||||
return this.toPersisted(record);
|
|
||||||
}
|
|
||||||
|
|
||||||
toPersisted(record: ToolHistoryRecord): SegmentSummaryRecord {
|
|
||||||
const effective = record.semanticSummary ?? record.deterministicSummary;
|
|
||||||
return {
|
|
||||||
version: COMPACTION_VERSION,
|
|
||||||
kind: "segment-summary",
|
|
||||||
sourceKey: record.sourceKey,
|
|
||||||
toolCallId: record.toolCallId,
|
|
||||||
resultTimestamp: record.resultTimestamp,
|
|
||||||
toolName: record.toolName,
|
|
||||||
sourceHash: record.sourceHash,
|
|
||||||
deterministicSummary: record.deterministicSummary,
|
|
||||||
...(record.semanticSummary ? { semanticSummary: record.semanticSummary } : {}),
|
|
||||||
rawChars: record.rawText.length,
|
|
||||||
effectiveChars: effective.length,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
setSemanticJobs(count: number): void {
|
|
||||||
this.semanticJobs = Math.max(0, count);
|
|
||||||
if (this.cached) this.cached = { ...this.cached, semanticJobs: this.semanticJobs };
|
|
||||||
}
|
|
||||||
|
|
||||||
recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void {
|
recordUsage(tokens: number, messageEstimate: number, environmentHash: string): void {
|
||||||
if (!Number.isFinite(tokens) || tokens <= 0) return;
|
if (!Number.isFinite(tokens) || tokens <= 0) return;
|
||||||
this.usageBaseline = { tokens, messageEstimate, environmentHash };
|
this.usageBaseline = { tokens, messageEstimate, environmentHash };
|
||||||
@@ -277,11 +85,27 @@ export class SegmentIndex {
|
|||||||
environmentHash: string;
|
environmentHash: string;
|
||||||
environmentTokens: number;
|
environmentTokens: number;
|
||||||
contextWindow: number;
|
contextWindow: number;
|
||||||
|
maskedResults: number;
|
||||||
|
reclaimedTokens: number;
|
||||||
}): ProjectionSnapshot {
|
}): ProjectionSnapshot {
|
||||||
const sameTail = messages.length === this.sourceMessages.length
|
const sameTail = messages.length === this.sourceMessages.length
|
||||||
&& messages.length > 0
|
&& messages.length > 0
|
||||||
&& messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1];
|
&& messageKey(messages[messages.length - 1]) === this.sourceKeys[this.sourceKeys.length - 1];
|
||||||
if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) return this.cached;
|
if (this.cached && sameTail && this.cached.environmentHash === params.environmentHash) {
|
||||||
|
if (this.cached.maskedResults === params.maskedResults
|
||||||
|
&& this.cached.reclaimedTokens === params.reclaimedTokens) {
|
||||||
|
return this.cached;
|
||||||
|
}
|
||||||
|
// Masking stats changed but the message list did not: publish a new
|
||||||
|
// immutable snapshot object instead of re-projecting or mutating.
|
||||||
|
this.cached = {
|
||||||
|
...this.cached,
|
||||||
|
maskedResults: params.maskedResults,
|
||||||
|
reclaimedTokens: params.reclaimedTokens,
|
||||||
|
rawTokens: this.cached.effectiveTokens + params.reclaimedTokens,
|
||||||
|
};
|
||||||
|
return this.cached;
|
||||||
|
}
|
||||||
|
|
||||||
const appendOnly = this.sourceMessages.length > 0
|
const appendOnly = this.sourceMessages.length > 0
|
||||||
&& messages.length >= this.sourceMessages.length
|
&& messages.length >= this.sourceMessages.length
|
||||||
@@ -301,17 +125,14 @@ export class SegmentIndex {
|
|||||||
|
|
||||||
for (let index = start; index < messages.length; index++) {
|
for (let index = start; index < messages.length; index++) {
|
||||||
const message = messages[index];
|
const message = messages[index];
|
||||||
const projected = this.projectMessage(message);
|
|
||||||
this.sourceMessages.push(message);
|
this.sourceMessages.push(message);
|
||||||
this.sourceKeys.push(messageKey(message));
|
this.sourceKeys.push(messageKey(message));
|
||||||
this.projectedMessages.push(projected);
|
this.projectedMessages.push(message);
|
||||||
this.rawEstimates.push(estimateTokens(message as any));
|
this.rawEstimates.push(estimateTokens(message as any));
|
||||||
this.effectiveEstimates.push(estimateTokens(projected as any));
|
this.effectiveEstimates.push(estimateTokens(message as any));
|
||||||
}
|
}
|
||||||
|
|
||||||
const rawMessageTokens = this.rawEstimates.reduce((sum, value) => sum + value, 0);
|
|
||||||
const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0);
|
const effectiveMessageTokens = this.effectiveEstimates.reduce((sum, value) => sum + value, 0);
|
||||||
const rawTokens = rawMessageTokens + params.environmentTokens;
|
|
||||||
const baseline = this.usageBaseline;
|
const baseline = this.usageBaseline;
|
||||||
const usageBacked = baseline?.environmentHash === params.environmentHash;
|
const usageBacked = baseline?.environmentHash === params.environmentHash;
|
||||||
const effectiveTokens = usageBacked
|
const effectiveTokens = usageBacked
|
||||||
@@ -329,55 +150,19 @@ export class SegmentIndex {
|
|||||||
environmentHash: params.environmentHash,
|
environmentHash: params.environmentHash,
|
||||||
messages: this.projectedMessages,
|
messages: this.projectedMessages,
|
||||||
messageEstimate: effectiveMessageTokens,
|
messageEstimate: effectiveMessageTokens,
|
||||||
rawTokens,
|
rawTokens: effectiveTokens + params.reclaimedTokens,
|
||||||
effectiveTokens,
|
effectiveTokens,
|
||||||
reclaimedTokens: Math.max(0, rawTokens - effectiveTokens),
|
reclaimedTokens: params.reclaimedTokens,
|
||||||
contextWindow: params.contextWindow,
|
contextWindow: params.contextWindow,
|
||||||
confidence,
|
confidence,
|
||||||
lowerBound,
|
lowerBound,
|
||||||
upperBound,
|
upperBound,
|
||||||
segmentCount: this.sourceMessages.length,
|
segmentCount: this.sourceMessages.length,
|
||||||
semanticJobs: this.semanticJobs,
|
maskedResults: params.maskedResults,
|
||||||
};
|
};
|
||||||
return this.cached;
|
return this.cached;
|
||||||
}
|
}
|
||||||
|
|
||||||
private projectMessage(message: any): any {
|
|
||||||
if (message?.role !== "toolResult" || typeof message.toolCallId !== "string") return message;
|
|
||||||
const record = this.records.get(sourceKey(message.toolCallId, message.timestamp));
|
|
||||||
if (!record) return message;
|
|
||||||
const original = contentText(message.content);
|
|
||||||
const summary = record.semanticSummary ?? record.deterministicSummary;
|
|
||||||
const projected = `${summary}\nSource: ${record.sourceKey}; recover with context_tree_query.`;
|
|
||||||
if (projected.length >= original.length) return message;
|
|
||||||
return {
|
|
||||||
...message,
|
|
||||||
content: [{ type: "text", text: projected }],
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
query(ids: string[]): ToolHistoryRecord[] {
|
|
||||||
const found: ToolHistoryRecord[] = [];
|
|
||||||
const seen = new Set<string>();
|
|
||||||
for (const id of ids) {
|
|
||||||
const direct = this.records.get(id);
|
|
||||||
const keys = direct ? [id] : this.ids.get(id) ?? [];
|
|
||||||
for (const key of keys) {
|
|
||||||
if (seen.has(key)) continue;
|
|
||||||
const record = this.records.get(key);
|
|
||||||
if (record) {
|
|
||||||
found.push(record);
|
|
||||||
seen.add(key);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return found;
|
|
||||||
}
|
|
||||||
|
|
||||||
allRecords(): ToolHistoryRecord[] {
|
|
||||||
return [...this.records.values()];
|
|
||||||
}
|
|
||||||
|
|
||||||
getSnapshot(): ProjectionSnapshot | undefined {
|
getSnapshot(): ProjectionSnapshot | undefined {
|
||||||
return this.cached;
|
return this.cached;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,189 +0,0 @@
|
|||||||
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
||||||
import type { PiCompactionConfig, ToolHistoryRecord } from "./types.js";
|
|
||||||
|
|
||||||
interface Job {
|
|
||||||
sessionId: string;
|
|
||||||
sourceKey: string;
|
|
||||||
sourceHash: string;
|
|
||||||
toolName: string;
|
|
||||||
args: unknown;
|
|
||||||
isError: boolean;
|
|
||||||
rawText: string;
|
|
||||||
ctx: ExtensionContext;
|
|
||||||
config: PiCompactionConfig;
|
|
||||||
controller: AbortController;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface SemanticResult {
|
|
||||||
sessionId: string;
|
|
||||||
sourceKey: string;
|
|
||||||
sourceHash: string;
|
|
||||||
summary: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
function modelFor(config: PiCompactionConfig, ctx: ExtensionContext): any {
|
|
||||||
if (config.semantic.model === "default") return ctx.model;
|
|
||||||
const slash = config.semantic.model.indexOf("/");
|
|
||||||
if (slash <= 0) return ctx.model;
|
|
||||||
return ctx.modelRegistry.find(
|
|
||||||
config.semantic.model.slice(0, slash),
|
|
||||||
config.semantic.model.slice(slash + 1),
|
|
||||||
) ?? ctx.model;
|
|
||||||
}
|
|
||||||
|
|
||||||
function serialize(job: Job): string {
|
|
||||||
const args = (() => {
|
|
||||||
try { return JSON.stringify(job.args); } catch { return "[unserializable]"; }
|
|
||||||
})();
|
|
||||||
const max = job.config.semantic.maxInputChars;
|
|
||||||
const raw = job.rawText.length <= max
|
|
||||||
? job.rawText
|
|
||||||
: `${job.rawText.slice(0, Math.floor(max * 0.7))}\n… omitted …\n${job.rawText.slice(-Math.floor(max * 0.3))}`;
|
|
||||||
return `Summarize this completed coding-agent tool call for future context.\nKeep concrete outcomes, paths, identifiers, values, and exact errors.\nUse at most 6 concise bullets. Do not give advice and do not call tools.\n\nTool: ${job.toolName}\nStatus: ${job.isError ? "ERROR" : "OK"}\nArgs: ${args}\nOutput:\n${raw}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
async function run(job: Job): Promise<string> {
|
|
||||||
const model = modelFor(job.config, job.ctx);
|
|
||||||
if (!model) throw new Error("No summarizer model selected");
|
|
||||||
const message = await job.ctx.modelRegistry.complete(model, {
|
|
||||||
messages: [{
|
|
||||||
role: "user",
|
|
||||||
content: [{ type: "text", text: serialize(job) }],
|
|
||||||
timestamp: Date.now(),
|
|
||||||
}],
|
|
||||||
} as any, {
|
|
||||||
signal: job.controller.signal,
|
|
||||||
reasoningEffort: job.config.semantic.thinking === "off" ? undefined : job.config.semantic.thinking,
|
|
||||||
} as any);
|
|
||||||
if (message.stopReason === "length" || message.stopReason === "error" || message.stopReason === "aborted") {
|
|
||||||
throw new Error(message.errorMessage ?? `Semantic compaction stopped: ${message.stopReason}`);
|
|
||||||
}
|
|
||||||
const text = message.content
|
|
||||||
.flatMap((part: any) => part.type === "text" && typeof part.text === "string" ? [part.text] : [])
|
|
||||||
.join("\n")
|
|
||||||
.trim();
|
|
||||||
if (!text) throw new Error("Semantic compaction returned an empty summary");
|
|
||||||
return text;
|
|
||||||
}
|
|
||||||
|
|
||||||
function abortableDelay(ms: number, signal: AbortSignal): Promise<void> {
|
|
||||||
if (signal.aborted) return Promise.reject(new Error("Semantic compaction aborted"));
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const timer = setTimeout(done, ms);
|
|
||||||
function done(): void {
|
|
||||||
signal.removeEventListener("abort", aborted);
|
|
||||||
resolve();
|
|
||||||
}
|
|
||||||
function aborted(): void {
|
|
||||||
clearTimeout(timer);
|
|
||||||
signal.removeEventListener("abort", aborted);
|
|
||||||
reject(new Error("Semantic compaction aborted"));
|
|
||||||
}
|
|
||||||
signal.addEventListener("abort", aborted, { once: true });
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export class SemanticWorker {
|
|
||||||
private readonly queue: Job[] = [];
|
|
||||||
private readonly active = new Map<string, Job>();
|
|
||||||
private readonly ready: SemanticResult[] = [];
|
|
||||||
private readonly queuedKeys = new Set<string>();
|
|
||||||
|
|
||||||
enqueue(sessionId: string, record: ToolHistoryRecord, ctx: ExtensionContext, config: PiCompactionConfig): void {
|
|
||||||
if (!config.semantic.enabled || record.semanticSummary || record.rawText.length < config.semantic.minInputChars) return;
|
|
||||||
const key = `${sessionId}:${record.sourceKey}:${record.sourceHash}`;
|
|
||||||
if (this.queuedKeys.has(key)) return;
|
|
||||||
this.queuedKeys.add(key);
|
|
||||||
this.queue.push({
|
|
||||||
sessionId,
|
|
||||||
sourceKey: record.sourceKey,
|
|
||||||
sourceHash: record.sourceHash,
|
|
||||||
toolName: record.toolName,
|
|
||||||
args: record.args,
|
|
||||||
isError: record.isError,
|
|
||||||
rawText: record.rawText,
|
|
||||||
ctx,
|
|
||||||
config,
|
|
||||||
controller: new AbortController(),
|
|
||||||
});
|
|
||||||
this.pump(config.semantic.concurrency);
|
|
||||||
}
|
|
||||||
|
|
||||||
private pump(limit: number): void {
|
|
||||||
while (this.active.size < limit && this.queue.length > 0) {
|
|
||||||
const job = this.queue.shift()!;
|
|
||||||
const key = `${job.sessionId}:${job.sourceKey}:${job.sourceHash}`;
|
|
||||||
this.active.set(key, job);
|
|
||||||
void this.execute(key, job, limit);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private async execute(key: string, job: Job, limit: number): Promise<void> {
|
|
||||||
let lastError: unknown;
|
|
||||||
try {
|
|
||||||
for (let attempt = 0; attempt <= job.config.semantic.maxRetries; attempt++) {
|
|
||||||
try {
|
|
||||||
const summary = await run(job);
|
|
||||||
if (!job.controller.signal.aborted) {
|
|
||||||
this.ready.push({
|
|
||||||
sessionId: job.sessionId,
|
|
||||||
sourceKey: job.sourceKey,
|
|
||||||
sourceHash: job.sourceHash,
|
|
||||||
summary,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
} catch (error) {
|
|
||||||
lastError = error;
|
|
||||||
if (job.controller.signal.aborted || attempt === job.config.semantic.maxRetries) break;
|
|
||||||
await abortableDelay(Math.min(8_000, 1_000 * 2 ** attempt), job.controller.signal);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!job.controller.signal.aborted) {
|
|
||||||
console.error(`pi-compaction: semantic summary failed for ${job.sourceKey}:`, lastError);
|
|
||||||
}
|
|
||||||
} finally {
|
|
||||||
this.active.delete(key);
|
|
||||||
this.queuedKeys.delete(key);
|
|
||||||
this.pump(limit);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
drain(sessionId: string): SemanticResult[] {
|
|
||||||
const drained: SemanticResult[] = [];
|
|
||||||
for (let index = this.ready.length - 1; index >= 0; index--) {
|
|
||||||
if (this.ready[index]!.sessionId !== sessionId) continue;
|
|
||||||
drained.push(this.ready[index]!);
|
|
||||||
this.ready.splice(index, 1);
|
|
||||||
}
|
|
||||||
return drained.reverse();
|
|
||||||
}
|
|
||||||
|
|
||||||
count(sessionId: string): number {
|
|
||||||
return this.queue.filter((job) => job.sessionId === sessionId).length
|
|
||||||
+ [...this.active.values()].filter((job) => job.sessionId === sessionId).length;
|
|
||||||
}
|
|
||||||
|
|
||||||
cancelSession(sessionId: string): void {
|
|
||||||
for (const job of this.active.values()) {
|
|
||||||
if (job.sessionId === sessionId) job.controller.abort();
|
|
||||||
}
|
|
||||||
for (let index = this.queue.length - 1; index >= 0; index--) {
|
|
||||||
if (this.queue[index]!.sessionId === sessionId) {
|
|
||||||
const job = this.queue[index]!;
|
|
||||||
this.queuedKeys.delete(`${job.sessionId}:${job.sourceKey}:${job.sourceHash}`);
|
|
||||||
this.queue.splice(index, 1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for (let index = this.ready.length - 1; index >= 0; index--) {
|
|
||||||
if (this.ready[index]!.sessionId === sessionId) this.ready.splice(index, 1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
cancelAll(): void {
|
|
||||||
for (const job of this.active.values()) job.controller.abort();
|
|
||||||
this.queue.length = 0;
|
|
||||||
this.ready.length = 0;
|
|
||||||
this.queuedKeys.clear();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -2,8 +2,8 @@ import {
|
|||||||
SERVICE_SYMBOL,
|
SERVICE_SYMBOL,
|
||||||
type CompactionMetrics,
|
type CompactionMetrics,
|
||||||
type CompactionService,
|
type CompactionService,
|
||||||
|
type MaskedResultRecord,
|
||||||
type ProjectionSnapshot,
|
type ProjectionSnapshot,
|
||||||
type ToolHistoryRecord,
|
|
||||||
} from "./types.js";
|
} from "./types.js";
|
||||||
|
|
||||||
interface GlobalWithCompactionService {
|
interface GlobalWithCompactionService {
|
||||||
@@ -14,11 +14,11 @@ const root = globalThis as GlobalWithCompactionService;
|
|||||||
|
|
||||||
export class CompactionServiceRegistry implements CompactionService {
|
export class CompactionServiceRegistry implements CompactionService {
|
||||||
private readonly snapshots = new Map<string, ProjectionSnapshot>();
|
private readonly snapshots = new Map<string, ProjectionSnapshot>();
|
||||||
private readonly queryHandlers = new Map<string, (ids: string[]) => ToolHistoryRecord[]>();
|
private readonly queryHandlers = new Map<string, (ids: string[]) => MaskedResultRecord[]>();
|
||||||
|
|
||||||
setSession(
|
setSession(
|
||||||
sessionId: string,
|
sessionId: string,
|
||||||
query: (ids: string[]) => ToolHistoryRecord[],
|
query: (ids: string[]) => MaskedResultRecord[],
|
||||||
): void {
|
): void {
|
||||||
this.queryHandlers.set(sessionId, query);
|
this.queryHandlers.set(sessionId, query);
|
||||||
}
|
}
|
||||||
@@ -58,11 +58,11 @@ export class CompactionServiceRegistry implements CompactionService {
|
|||||||
percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0,
|
percent: snapshot.contextWindow > 0 ? 100 * snapshot.effectiveTokens / snapshot.contextWindow : 0,
|
||||||
confidence: snapshot.confidence,
|
confidence: snapshot.confidence,
|
||||||
segmentCount: snapshot.segmentCount,
|
segmentCount: snapshot.segmentCount,
|
||||||
semanticJobs: snapshot.semanticJobs,
|
maskedResults: snapshot.maskedResults,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
query(sessionId: string, ids: string[]): ToolHistoryRecord[] {
|
query(sessionId: string, ids: string[]): MaskedResultRecord[] {
|
||||||
return this.queryHandlers.get(sessionId)?.(ids) ?? [];
|
return this.queryHandlers.get(sessionId)?.(ids) ?? [];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,13 +1,5 @@
|
|||||||
import type { PiCompactionConfig, ProjectionSnapshot } from "./types.js";
|
// Effective trigger for one configured (tokens, ratio) pair: the absolute
|
||||||
|
// token figure wins on large windows, the ratio caps it on small ones.
|
||||||
export type CompactionReason = "manual" | "threshold" | "overflow";
|
export function effectiveThreshold(tokens: number, ratio: number, contextWindow: number): number {
|
||||||
|
return contextWindow > 0 ? Math.min(tokens, Math.floor(ratio * contextWindow)) : tokens;
|
||||||
export function shouldCompact(
|
|
||||||
reason: CompactionReason,
|
|
||||||
snapshot: Pick<ProjectionSnapshot, "lowerBound" | "contextWindow">,
|
|
||||||
config: Pick<PiCompactionConfig, "hardRatio">,
|
|
||||||
): boolean {
|
|
||||||
if (reason === "manual" || reason === "overflow") return true;
|
|
||||||
if (snapshot.contextWindow <= 0) return true;
|
|
||||||
return snapshot.lowerBound / snapshot.contextWindow >= config.hardRatio;
|
|
||||||
}
|
}
|
||||||
|
|||||||
+30
-48
@@ -5,58 +5,33 @@ export const COMPACTION_VERSION = 1 as const;
|
|||||||
export const QUERY_TOOL_NAME = "context_tree_query";
|
export const QUERY_TOOL_NAME = "context_tree_query";
|
||||||
export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1");
|
export const SERVICE_SYMBOL = Symbol.for("my-pi.pi-compaction.service.v1");
|
||||||
|
|
||||||
|
export interface MaskingConfig {
|
||||||
|
enabled: boolean;
|
||||||
|
triggerTokens: number;
|
||||||
|
triggerRatio: number;
|
||||||
|
targetTokens: number;
|
||||||
|
targetRatio: number;
|
||||||
|
minClearTokens: number;
|
||||||
|
keepRecentTurns: number;
|
||||||
|
minEntryTokens: number;
|
||||||
|
excludeTools: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SummaryConfig {
|
||||||
|
triggerTokens: number;
|
||||||
|
triggerRatio: number;
|
||||||
|
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
|
||||||
|
}
|
||||||
|
|
||||||
export interface PiCompactionConfig {
|
export interface PiCompactionConfig {
|
||||||
enabled: boolean;
|
enabled: boolean;
|
||||||
softRatio: number;
|
masking: MaskingConfig;
|
||||||
hardRatio: number;
|
summary: SummaryConfig;
|
||||||
targetRatio: number;
|
|
||||||
deterministic: {
|
|
||||||
enabled: boolean;
|
|
||||||
maxStubTokens: number;
|
|
||||||
};
|
|
||||||
semantic: {
|
|
||||||
enabled: boolean;
|
|
||||||
model: string;
|
|
||||||
thinking: "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
|
|
||||||
concurrency: number;
|
|
||||||
minInputChars: number;
|
|
||||||
maxInputChars: number;
|
|
||||||
maxRetries: number;
|
|
||||||
};
|
|
||||||
native: {
|
native: {
|
||||||
enabled: boolean;
|
enabled: boolean;
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface SegmentSummaryRecord {
|
|
||||||
version: typeof COMPACTION_VERSION;
|
|
||||||
kind: "segment-summary";
|
|
||||||
sourceKey: string;
|
|
||||||
toolCallId: string;
|
|
||||||
resultTimestamp: number;
|
|
||||||
toolName: string;
|
|
||||||
sourceHash: string;
|
|
||||||
deterministicSummary: string;
|
|
||||||
semanticSummary?: string;
|
|
||||||
rawChars: number;
|
|
||||||
effectiveChars: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
export type PiCompactionRecord = SegmentSummaryRecord;
|
|
||||||
|
|
||||||
export interface ToolHistoryRecord {
|
|
||||||
sourceKey: string;
|
|
||||||
toolCallId: string;
|
|
||||||
resultTimestamp: number;
|
|
||||||
toolName: string;
|
|
||||||
args: unknown;
|
|
||||||
isError: boolean;
|
|
||||||
rawText: string;
|
|
||||||
sourceHash: string;
|
|
||||||
deterministicSummary: string;
|
|
||||||
semanticSummary?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface UsageBaseline {
|
export interface UsageBaseline {
|
||||||
tokens: number;
|
tokens: number;
|
||||||
messageEstimate: number;
|
messageEstimate: number;
|
||||||
@@ -78,7 +53,7 @@ export interface ProjectionSnapshot {
|
|||||||
lowerBound: number;
|
lowerBound: number;
|
||||||
upperBound: number;
|
upperBound: number;
|
||||||
segmentCount: number;
|
segmentCount: number;
|
||||||
semanticJobs: number;
|
maskedResults: number;
|
||||||
codexItems?: ResponseItem[];
|
codexItems?: ResponseItem[];
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -99,6 +74,13 @@ export interface PortableCheckpointDetails {
|
|||||||
sourceFingerprint: string;
|
sourceFingerprint: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** What the service returns for one masked tool result — the same placeholder the model sees. */
|
||||||
|
export interface MaskedResultRecord {
|
||||||
|
toolCallId: string;
|
||||||
|
resultTimestamp: number;
|
||||||
|
deterministicSummary: string;
|
||||||
|
}
|
||||||
|
|
||||||
export interface CompactionMetrics {
|
export interface CompactionMetrics {
|
||||||
sessionId: string;
|
sessionId: string;
|
||||||
generation: number;
|
generation: number;
|
||||||
@@ -109,11 +91,11 @@ export interface CompactionMetrics {
|
|||||||
percent: number;
|
percent: number;
|
||||||
confidence: ProjectionSnapshot["confidence"];
|
confidence: ProjectionSnapshot["confidence"];
|
||||||
segmentCount: number;
|
segmentCount: number;
|
||||||
semanticJobs: number;
|
maskedResults: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface CompactionService {
|
export interface CompactionService {
|
||||||
getSnapshot(sessionId: string): ProjectionSnapshot | undefined;
|
getSnapshot(sessionId: string): ProjectionSnapshot | undefined;
|
||||||
getMetrics(sessionId: string): CompactionMetrics | undefined;
|
getMetrics(sessionId: string): CompactionMetrics | undefined;
|
||||||
query(sessionId: string, ids: string[]): ToolHistoryRecord[];
|
query(sessionId: string, ids: string[]): MaskedResultRecord[];
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,34 +2,82 @@ import test from "node:test";
|
|||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts";
|
import { DEFAULT_CONFIG, normalizeConfig } from "../src/config.ts";
|
||||||
|
|
||||||
test("normalizes one greenfield piCompaction config without legacy aliases", () => {
|
test("normalizes a full pi-compaction config", () => {
|
||||||
const config = normalizeConfig({
|
const config = normalizeConfig({
|
||||||
enabled: false,
|
enabled: false,
|
||||||
softRatio: 0.7,
|
masking: {
|
||||||
hardRatio: 0.91,
|
enabled: false,
|
||||||
targetRatio: 0.5,
|
triggerTokens: 150_000,
|
||||||
deterministic: { enabled: false, maxStubTokens: 512 },
|
triggerRatio: 0.5,
|
||||||
semantic: { enabled: false, concurrency: 2, maxRetries: 0 },
|
targetTokens: 60_000,
|
||||||
|
targetRatio: 0.2,
|
||||||
|
minClearTokens: 30_000,
|
||||||
|
keepRecentTurns: 5,
|
||||||
|
minEntryTokens: 512,
|
||||||
|
excludeTools: ["bash", "read"],
|
||||||
|
},
|
||||||
|
summary: { triggerTokens: 300_000, triggerRatio: 0.8, thinking: "medium" },
|
||||||
native: { enabled: false },
|
native: { enabled: false },
|
||||||
});
|
});
|
||||||
assert.equal(config.enabled, false);
|
assert.equal(config.enabled, false);
|
||||||
assert.equal(config.deterministic.enabled, false);
|
assert.deepEqual(config.masking, {
|
||||||
assert.equal(config.deterministic.maxStubTokens, 512);
|
enabled: false,
|
||||||
assert.equal(config.semantic.enabled, false);
|
triggerTokens: 150_000,
|
||||||
assert.equal(config.semantic.maxRetries, 0);
|
triggerRatio: 0.5,
|
||||||
|
targetTokens: 60_000,
|
||||||
|
targetRatio: 0.2,
|
||||||
|
minClearTokens: 30_000,
|
||||||
|
keepRecentTurns: 5,
|
||||||
|
minEntryTokens: 512,
|
||||||
|
excludeTools: ["bash", "read"],
|
||||||
|
});
|
||||||
|
assert.equal(config.summary.triggerTokens, 300_000);
|
||||||
|
assert.equal(config.summary.triggerRatio, 0.8);
|
||||||
|
assert.equal(config.summary.thinking, "medium");
|
||||||
assert.equal(config.native.enabled, false);
|
assert.equal(config.native.enabled, false);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("invalid ratios and bounds fail safe to bounded defaults", () => {
|
test("invalid fields fail safe to bounded defaults", () => {
|
||||||
const config = normalizeConfig({
|
const config = normalizeConfig({
|
||||||
softRatio: 2,
|
enabled: "yes",
|
||||||
hardRatio: -1,
|
masking: {
|
||||||
targetRatio: 0.99,
|
triggerTokens: -1,
|
||||||
semantic: { concurrency: 50, maxRetries: -1 },
|
triggerRatio: 2,
|
||||||
|
targetTokens: "a lot",
|
||||||
|
targetRatio: 0,
|
||||||
|
minClearTokens: 0,
|
||||||
|
keepRecentTurns: 500,
|
||||||
|
minEntryTokens: -3,
|
||||||
|
excludeTools: "read",
|
||||||
|
},
|
||||||
|
summary: { triggerTokens: Number.NaN, triggerRatio: 1.5, thinking: "insane" },
|
||||||
|
native: { enabled: 1 },
|
||||||
});
|
});
|
||||||
assert.equal(config.softRatio, DEFAULT_CONFIG.softRatio);
|
assert.equal(config.enabled, DEFAULT_CONFIG.enabled);
|
||||||
assert.ok(config.targetRatio < config.softRatio);
|
assert.deepEqual(config.masking, DEFAULT_CONFIG.masking);
|
||||||
assert.ok(config.hardRatio > config.softRatio);
|
assert.equal(config.summary.triggerTokens, DEFAULT_CONFIG.summary.triggerTokens);
|
||||||
assert.equal(config.semantic.concurrency, DEFAULT_CONFIG.semantic.concurrency);
|
assert.equal(config.summary.triggerRatio, DEFAULT_CONFIG.summary.triggerRatio);
|
||||||
assert.equal(config.semantic.maxRetries, DEFAULT_CONFIG.semantic.maxRetries);
|
assert.equal(config.summary.thinking, DEFAULT_CONFIG.summary.thinking);
|
||||||
|
assert.equal(config.native.enabled, DEFAULT_CONFIG.native.enabled);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("masking target is corrected below trigger and summary above masking", () => {
|
||||||
|
const config = normalizeConfig({
|
||||||
|
masking: { triggerTokens: 80_000, triggerRatio: 0.5, targetTokens: 120_000, targetRatio: 0.9 },
|
||||||
|
summary: { triggerTokens: 50_000, triggerRatio: 0.4 },
|
||||||
|
});
|
||||||
|
assert.equal(config.masking.targetTokens, 40_000);
|
||||||
|
assert.equal(config.masking.targetRatio, 0.25);
|
||||||
|
assert.equal(config.summary.triggerTokens, 160_000);
|
||||||
|
assert.ok(config.summary.triggerRatio > config.masking.triggerRatio);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("minClearTokens and excludeTools accept valid overrides", () => {
|
||||||
|
const config = normalizeConfig({
|
||||||
|
masking: { minClearTokens: 1, keepRecentTurns: 1, minEntryTokens: 0, excludeTools: [] },
|
||||||
|
});
|
||||||
|
assert.equal(config.masking.minClearTokens, 1);
|
||||||
|
assert.equal(config.masking.keepRecentTurns, 1);
|
||||||
|
assert.equal(config.masking.minEntryTokens, 0);
|
||||||
|
assert.deepEqual(config.masking.excludeTools, []);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
import test from "node:test";
|
import test from "node:test";
|
||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { performance } from "node:perf_hooks";
|
|
||||||
import piCompaction from "../index.ts";
|
import piCompaction from "../index.ts";
|
||||||
|
import { getCompactionService } from "../src/service.ts";
|
||||||
|
import { MASK_PREFIX } from "../src/masking.ts";
|
||||||
|
|
||||||
function harness() {
|
function harness() {
|
||||||
const handlers = new Map<string, Function[]>();
|
const handlers = new Map<string, Function[]>();
|
||||||
@@ -24,90 +25,253 @@ function harness() {
|
|||||||
return { pi, handlers, tools, commands, entries };
|
return { pi, handlers, tools, commands, entries };
|
||||||
}
|
}
|
||||||
|
|
||||||
function context(branch: any[] = []) {
|
function context(branch: any[] = [], tokens = 1_000) {
|
||||||
return {
|
const compactCalls: any[] = [];
|
||||||
|
const sessionId = `session-${Math.random().toString(36).slice(2)}`;
|
||||||
|
const ctx = {
|
||||||
sessionManager: {
|
sessionManager: {
|
||||||
getSessionId: () => "session-test",
|
getSessionId: () => sessionId,
|
||||||
getBranch: () => branch,
|
getBranch: () => branch,
|
||||||
|
getLeafId: () => "leaf-1",
|
||||||
},
|
},
|
||||||
model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 },
|
model: { provider: "test", api: "test", id: "model", contextWindow: 100_000 },
|
||||||
modelRegistry: {},
|
modelRegistry: {},
|
||||||
getSystemPrompt: () => "system",
|
getSystemPrompt: () => "system",
|
||||||
getContextUsage: () => ({ tokens: 1_000, contextWindow: 100_000, percent: 1 }),
|
getContextUsage: () => ({ tokens, contextWindow: 100_000, percent: 1 }),
|
||||||
|
compact(options: unknown) { compactCalls.push(options); },
|
||||||
|
isIdle: () => true,
|
||||||
hasUI: false,
|
hasUI: false,
|
||||||
abort() {},
|
abort() {},
|
||||||
} as any;
|
} as any;
|
||||||
|
return { ctx, compactCalls };
|
||||||
}
|
}
|
||||||
|
|
||||||
test("one runtime projects tool evidence and recovers it through context_tree_query", async () => {
|
function projectedEntry(id: string, message: any) {
|
||||||
|
return { sourceEntry: { id, type: "message", message }, messages: [message] };
|
||||||
|
}
|
||||||
|
|
||||||
|
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
|
||||||
|
return {
|
||||||
|
role: "assistant",
|
||||||
|
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
|
||||||
|
timestamp: id,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
|
||||||
|
return {
|
||||||
|
role: "toolResult",
|
||||||
|
toolCallId,
|
||||||
|
toolName,
|
||||||
|
content: [{ type: "text", text }],
|
||||||
|
timestamp: id,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// Default config: masking trigger = min(200K, 60% of window); at the harness
|
||||||
|
// 100K window that is 60K tokens and the target is 35K, so 60K used needs
|
||||||
|
// ~25K tokens of maskable results outside the 10-turn protected window.
|
||||||
|
function pressurizedContextEntries() {
|
||||||
|
const entries = [
|
||||||
|
projectedEntry("a1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }])),
|
||||||
|
projectedEntry("r1", toolResultMessage("r1", "c1", `failure evidence\n${"x".repeat(120_000)}\nstack tail`)),
|
||||||
|
];
|
||||||
|
for (let index = 0; index < 10; index++) {
|
||||||
|
entries.push(projectedEntry(`keep-${index}`, assistantMessage(`keep-${index}`, [])));
|
||||||
|
}
|
||||||
|
return entries;
|
||||||
|
}
|
||||||
|
|
||||||
|
test("turn_end masks old tool results under pressure and preserves prior drafts", async () => {
|
||||||
const h = harness();
|
const h = harness();
|
||||||
piCompaction(h.pi);
|
piCompaction(h.pi);
|
||||||
const ctx = context();
|
const { ctx } = context([], 60_000);
|
||||||
await h.handlers.get("session_start")![0]!({}, ctx);
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
const assistant = {
|
const prior = { type: "custom", customType: "other-extension", data: {} };
|
||||||
role: "assistant",
|
const result = await h.handlers.get("turn_end")![0]!({
|
||||||
content: [{ type: "toolCall", id: "call-1", name: "bash", arguments: { command: "npm test" } }],
|
entries: [prior],
|
||||||
timestamp: 10,
|
message: assistantMessage("a1", []),
|
||||||
};
|
toolResults: [],
|
||||||
const result = {
|
context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
|
||||||
role: "toolResult",
|
}, ctx);
|
||||||
toolCallId: "call-1",
|
|
||||||
toolName: "bash",
|
|
||||||
content: [{ type: "text", text: `failure evidence\n${"x".repeat(12_000)}\nstack tail` }],
|
|
||||||
isError: true,
|
|
||||||
timestamp: 11,
|
|
||||||
};
|
|
||||||
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx);
|
|
||||||
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx);
|
|
||||||
|
|
||||||
assert.match(projected.messages[1].content[0].text, /pi-compaction tool=bash/);
|
assert.ok(result?.entries);
|
||||||
const cacheStart = performance.now();
|
assert.strictEqual(result.entries[0], prior);
|
||||||
for (let iteration = 0; iteration < 1_000; iteration++) {
|
const edits = result.entries.slice(1);
|
||||||
await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx);
|
assert.equal(edits.length, 1);
|
||||||
}
|
assert.equal(edits[0]!.type, "context_edit");
|
||||||
const cacheAverageMs = (performance.now() - cacheStart) / 1_000;
|
assert.equal(edits[0]!.targetId, "r1");
|
||||||
assert.ok(cacheAverageMs < 2, `full context handler cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`);
|
assert.ok(edits[0]!.replacement.content.startsWith(MASK_PREFIX));
|
||||||
assert.equal(h.entries.length, 1);
|
// Masking is append-only context_edit; nothing is written as custom entries.
|
||||||
assert.equal(h.entries[0]!.customType, "pi-compaction");
|
assert.equal(h.entries.length, 0);
|
||||||
|
|
||||||
const query = h.tools.get("context_tree_query");
|
|
||||||
const recovered = await query.execute("q", { toolCallIds: ["call-1"] }, new AbortController().signal, undefined, ctx);
|
|
||||||
assert.match(recovered.content[0].text, /failure evidence/);
|
|
||||||
|
|
||||||
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("small tool results are passed through without a stub wrapper", async () => {
|
test("turn_end below the masking trigger returns undefined", async () => {
|
||||||
const h = harness();
|
const h = harness();
|
||||||
piCompaction(h.pi);
|
piCompaction(h.pi);
|
||||||
const ctx = context();
|
const { ctx } = context([], 40_000);
|
||||||
await h.handlers.get("session_start")![0]!({}, ctx);
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
const assistant = {
|
const result = await h.handlers.get("turn_end")![0]!({
|
||||||
role: "assistant",
|
entries: [],
|
||||||
content: [{ type: "toolCall", id: "call-small", name: "bash", arguments: { command: "ls" } }],
|
message: assistantMessage("a1", []),
|
||||||
timestamp: 20,
|
toolResults: [],
|
||||||
};
|
context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
|
||||||
const result = {
|
}, ctx);
|
||||||
role: "toolResult",
|
assert.equal(result, undefined);
|
||||||
toolCallId: "call-small",
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
toolName: "bash",
|
});
|
||||||
content: [{ type: "text", text: "ok\nfile-a.ts\nfile-b.ts" }],
|
|
||||||
timestamp: 21,
|
|
||||||
};
|
|
||||||
|
|
||||||
await h.handlers.get("turn_end")![0]!({ message: assistant, toolResults: [result] }, ctx);
|
test("context hook returns messages untouched on the non-checkpoint path", async () => {
|
||||||
const projected = await h.handlers.get("context")![0]!({ messages: [assistant, result] }, ctx);
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const { ctx } = context();
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
assert.strictEqual(projected.messages[1].content[0].text, "ok\nfile-a.ts\nfile-b.ts");
|
const messages = [assistantMessage("a1", []), toolResultMessage("r1", "c1", "visible\nfile-a.ts")];
|
||||||
assert.doesNotMatch(projected.messages[1].content[0].text, /pi-compaction tool=/);
|
const projected = await h.handlers.get("context")![0]!({ messages }, ctx);
|
||||||
|
assert.equal(projected, undefined);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("agent_settled compacts once above the summary threshold and never repeats on the same leaf", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const { ctx, compactCalls } = context([], 95_000);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
|
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
|
||||||
|
assert.equal(compactCalls.length, 1);
|
||||||
|
assert.equal(typeof compactCalls[0].onError, "function");
|
||||||
|
|
||||||
|
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
|
||||||
|
assert.equal(compactCalls.length, 1);
|
||||||
|
|
||||||
|
ctx.sessionManager.getLeafId = () => "leaf-2";
|
||||||
|
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
|
||||||
|
assert.equal(compactCalls.length, 2);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("agent_settled stays idle below the summary threshold", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const { ctx, compactCalls } = context([], 50_000);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
|
h.handlers.get("agent_settled")![0]!({ type: "agent_settled", aborted: false }, ctx);
|
||||||
|
assert.equal(compactCalls.length, 0);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("context_tree_query recovers the original tool output from the branch", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const branch = [
|
||||||
|
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
|
||||||
|
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "failure evidence\nstack tail") },
|
||||||
|
];
|
||||||
|
const { ctx } = context(branch);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
// Original evidence stays recoverable even when the message is passed through.
|
|
||||||
const query = h.tools.get("context_tree_query")!;
|
const query = h.tools.get("context_tree_query")!;
|
||||||
const recovered = await query.execute("q", { toolCallIds: ["call-small"] }, new AbortController().signal, undefined, ctx);
|
const recovered = await query.execute("q", { toolCallIds: ["c1"] }, new AbortController().signal, undefined, ctx);
|
||||||
assert.match(recovered.content[0].text, /file-a\.ts/);
|
assert.match(recovered.content[0].text, /failure evidence/);
|
||||||
|
assert.match(recovered.content[0].text, /stack tail/);
|
||||||
|
assert.deepEqual(recovered.details.missing, []);
|
||||||
|
|
||||||
|
const missing = await query.execute("q", { toolCallIds: ["nope"] }, new AbortController().signal, undefined, ctx);
|
||||||
|
assert.match(missing.content[0].text, /not found/);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("service query returns placeholders only for masked tool call ids", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const { ctx } = context([], 60_000);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
const sessionId = ctx.sessionManager.getSessionId();
|
||||||
|
|
||||||
|
await h.handlers.get("turn_end")![0]!({
|
||||||
|
entries: [],
|
||||||
|
message: assistantMessage("a1", []),
|
||||||
|
toolResults: [],
|
||||||
|
context: { contextEntries: pressurizedContextEntries(), contextMessages: [] },
|
||||||
|
}, ctx);
|
||||||
|
|
||||||
|
const records = getCompactionService()!.query(sessionId, ["c1", "never-masked"]);
|
||||||
|
assert.equal(records.length, 1);
|
||||||
|
assert.equal(records[0]!.toolCallId, "c1");
|
||||||
|
assert.ok(records[0]!.deterministicSummary.startsWith(MASK_PREFIX));
|
||||||
|
assert.equal(typeof records[0]!.resultTimestamp, "number");
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("hydrating a branch with this extension's context_edit rebuilds masking metrics", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const placeholder = `${MASK_PREFIX}bash result cleared to save context (~1_000 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["c1"]}.`;
|
||||||
|
const branch = [
|
||||||
|
{ type: "message", id: "a1", parentId: null, timestamp: "t", message: assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "npm test" } }]) },
|
||||||
|
{ type: "message", id: "r1", parentId: "a1", timestamp: "t", message: toolResultMessage("r1", "c1", "x".repeat(8_000)) },
|
||||||
|
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: placeholder } },
|
||||||
|
];
|
||||||
|
const { ctx } = context(branch);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
const sessionId = ctx.sessionManager.getSessionId();
|
||||||
|
|
||||||
|
await h.handlers.get("context")![0]!({ messages: [toolResultMessage("r1", "c1", "x".repeat(8_000))] }, ctx);
|
||||||
|
const metrics = getCompactionService()!.getMetrics(sessionId);
|
||||||
|
assert.equal(metrics!.maskedResults, 1);
|
||||||
|
assert.ok(metrics!.reclaimedTokens > 0);
|
||||||
|
assert.ok(metrics!.rawTokens > metrics!.effectiveTokens);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("session_before_compact no longer cancels threshold compactions", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const { ctx } = context();
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: false, error: "no credentials" });
|
||||||
|
|
||||||
|
const result = await h.handlers.get("session_before_compact")![0]!({
|
||||||
|
reason: "threshold", branchEntries: [], preparation: { firstKeptEntryId: "u1" },
|
||||||
|
signal: new AbortController().signal,
|
||||||
|
}, ctx);
|
||||||
|
assert.equal(result, undefined);
|
||||||
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("native checkpoint projection applies context edits made after the checkpoint", async () => {
|
||||||
|
const h = harness();
|
||||||
|
piCompaction(h.pi);
|
||||||
|
const details = {
|
||||||
|
kind: "pi-compaction",
|
||||||
|
version: 1,
|
||||||
|
checkpoint: "native",
|
||||||
|
modelKey: "test:test:model",
|
||||||
|
replacementHistory: [{ type: "compaction", encrypted_content: "opaque" }],
|
||||||
|
portableSummary: "checkpoint summary",
|
||||||
|
sourceFingerprint: "fp",
|
||||||
|
};
|
||||||
|
const branch = [
|
||||||
|
{ type: "compaction", id: "c1", parentId: null, timestamp: "t", summary: "marker", firstKeptEntryId: "r1", tokensBefore: 1_000, details },
|
||||||
|
{ type: "message", id: "r1", parentId: "c1", timestamp: "t", message: toolResultMessage("r1", "c1", "ORIGINAL-CONTENT-HERE") },
|
||||||
|
{ type: "context_edit", id: "e1", parentId: "r1", timestamp: "t", targetId: "r1", replacement: { content: `${MASK_PREFIX}bash result cleared to save context (~5 tokens). Call args: {}.` } },
|
||||||
|
{ type: "message", id: "u1", parentId: "e1", timestamp: "t", message: { role: "user", content: "next question", timestamp: 30 } },
|
||||||
|
] as any[];
|
||||||
|
const { ctx } = context(branch);
|
||||||
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
|
|
||||||
|
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
|
||||||
|
const serialized = JSON.stringify(projected.messages);
|
||||||
|
assert.doesNotMatch(serialized, /ORIGINAL-CONTENT-HERE/);
|
||||||
|
assert.match(serialized, /result cleared to save context/);
|
||||||
|
assert.match(serialized, /next question/);
|
||||||
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
await h.handlers.get("session_shutdown")![0]!({}, ctx);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -127,7 +291,7 @@ test("model switching after a native checkpoint projects the portable checkpoint
|
|||||||
{ type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details },
|
{ type: "compaction", id: "c1", parentId: null, timestamp: new Date().toISOString(), summary: "marker", firstKeptEntryId: "m0", tokensBefore: 1_000, details },
|
||||||
{ type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } },
|
{ type: "message", id: "m1", parentId: "c1", timestamp: new Date().toISOString(), message: { role: "user", content: "continue", timestamp: 20 } },
|
||||||
] as any[];
|
] as any[];
|
||||||
const ctx = context(branch);
|
const { ctx } = context(branch);
|
||||||
await h.handlers.get("session_start")![0]!({}, ctx);
|
await h.handlers.get("session_start")![0]!({}, ctx);
|
||||||
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
|
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
|
||||||
assert.match(projected.messages[0].content[0].text, /decisions and files/);
|
assert.match(projected.messages[0].content[0].text, /decisions and files/);
|
||||||
@@ -143,7 +307,7 @@ for (const provider of ["sub-api", "openai-codex"]) {
|
|||||||
type: "message", id: "u1", parentId: null,
|
type: "message", id: "u1", parentId: null,
|
||||||
message: { role: "user", content: "Remember the test decision.", timestamp: 1 },
|
message: { role: "user", content: "Remember the test decision.", timestamp: 1 },
|
||||||
}];
|
}];
|
||||||
const ctx = context(branch);
|
const { ctx } = context(branch);
|
||||||
ctx.model = {
|
ctx.model = {
|
||||||
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
|
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
|
||||||
id: "selected-model", input: ["text"], contextWindow: 100_000,
|
id: "selected-model", input: ["text"], contextWindow: 100_000,
|
||||||
@@ -231,7 +395,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
|
|||||||
type: "message", id: "u1", parentId: null,
|
type: "message", id: "u1", parentId: null,
|
||||||
message: { role: "user", content: "Test only.", timestamp: 1 },
|
message: { role: "user", content: "Test only.", timestamp: 1 },
|
||||||
}];
|
}];
|
||||||
const ctx = context(branch);
|
const { ctx } = context(branch);
|
||||||
ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" };
|
ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" };
|
||||||
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" });
|
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" });
|
||||||
const responses = [
|
const responses = [
|
||||||
@@ -266,7 +430,7 @@ test("SubAPI native failures and cancellation return control to Pi without a che
|
|||||||
test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => {
|
test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => {
|
||||||
const h = harness();
|
const h = harness();
|
||||||
piCompaction(h.pi);
|
piCompaction(h.pi);
|
||||||
const ctx = context();
|
const { ctx } = context();
|
||||||
try {
|
try {
|
||||||
for (const model of [
|
for (const model of [
|
||||||
{ provider: "other", api: "openai-responses", id: "test" },
|
{ provider: "other", api: "openai-responses", id: "test" },
|
||||||
@@ -302,7 +466,7 @@ test("native runtime delegates authentication to each provider without blocking
|
|||||||
type: "message", id: "u1", parentId: null,
|
type: "message", id: "u1", parentId: null,
|
||||||
message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 },
|
message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 },
|
||||||
}];
|
}];
|
||||||
const ctx = context(branch);
|
const { ctx } = context(branch);
|
||||||
const provider = scenario.provider ?? "sub-api";
|
const provider = scenario.provider ?? "sub-api";
|
||||||
ctx.model = {
|
ctx.model = {
|
||||||
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
|
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
|
||||||
|
|||||||
@@ -0,0 +1,209 @@
|
|||||||
|
import test from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { MASK_PREFIX, planMasking, rebuildMaskedLedger } from "../src/masking.ts";
|
||||||
|
import { effectiveThreshold } from "../src/token-accounting.ts";
|
||||||
|
import type { MaskingConfig } from "../src/types.ts";
|
||||||
|
|
||||||
|
const MASKING: MaskingConfig = {
|
||||||
|
enabled: true,
|
||||||
|
triggerTokens: 1_000,
|
||||||
|
triggerRatio: 0.9,
|
||||||
|
targetTokens: 400,
|
||||||
|
targetRatio: 0.35,
|
||||||
|
minClearTokens: 200,
|
||||||
|
keepRecentTurns: 2,
|
||||||
|
minEntryTokens: 10,
|
||||||
|
excludeTools: ["ask_user_question"],
|
||||||
|
};
|
||||||
|
|
||||||
|
const estimate = (message: any): number => Math.ceil(JSON.stringify(message).length / 4);
|
||||||
|
|
||||||
|
function assistantMessage(id: string, calls: Array<{ id: string; name: string; args?: unknown }>) {
|
||||||
|
return {
|
||||||
|
role: "assistant",
|
||||||
|
content: calls.map((call) => ({ type: "toolCall", id: call.id, name: call.name, arguments: call.args ?? {} })),
|
||||||
|
timestamp: id,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function toolResultMessage(id: string, toolCallId: string, text: string, toolName = "bash") {
|
||||||
|
return {
|
||||||
|
role: "toolResult",
|
||||||
|
toolCallId,
|
||||||
|
toolName,
|
||||||
|
content: [{ type: "text", text }],
|
||||||
|
timestamp: id,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function entry(id: string, message: any, messages = [message]) {
|
||||||
|
return { sourceEntry: { id, type: "message", message }, messages };
|
||||||
|
}
|
||||||
|
|
||||||
|
function protect(turns = MASKING.keepRecentTurns) {
|
||||||
|
return Array.from({ length: turns }, (_unused, index) =>
|
||||||
|
entry(`recent-${index}`, { role: "assistant", content: [{ type: "text", text: "ok" }], timestamp: `recent-${index}` }));
|
||||||
|
}
|
||||||
|
|
||||||
|
test("below the effective trigger plans no drafts", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: { command: "ls" } }])),
|
||||||
|
entry("e2", toolResultMessage("r1", "c1", "x".repeat(8_000))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 999, contextWindow: 0, masking: MASKING, estimate });
|
||||||
|
assert.equal(plan.drafts.length, 0);
|
||||||
|
assert.equal(plan.masked.length, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("above the trigger masks oldest first until context fits under target", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", assistantMessage("a1", [
|
||||||
|
{ id: "c1", name: "bash", args: { command: "one" } },
|
||||||
|
{ id: "c2", name: "bash", args: { command: "two" } },
|
||||||
|
{ id: "c3", name: "bash", args: { command: "three" } },
|
||||||
|
])),
|
||||||
|
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
|
||||||
|
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000))),
|
||||||
|
entry("e4", toolResultMessage("r3", "c3", "c".repeat(4_000))),
|
||||||
|
entry("e5", toolResultMessage("r4", "c4", "recent result".repeat(200))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
// needed = 2_000 - 400 = 1_600; each result clears ~980 tokens.
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 2_000, contextWindow: 0, masking: MASKING, estimate });
|
||||||
|
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e2", "e3"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("results inside the recent keepRecentTurns window are never masked", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", toolResultMessage("r1", "c1", "a".repeat(4_000))),
|
||||||
|
entry("recent-a", assistantMessage("ra", [{ id: "rc", name: "bash", args: {} }])),
|
||||||
|
entry("recent-r", toolResultMessage("rr", "rc", "b".repeat(4_000))),
|
||||||
|
entry("recent-b", assistantMessage("rb", [])),
|
||||||
|
];
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
|
||||||
|
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e1"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("excluded tools, SKILL.md reads, already-masked and small results are skipped", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", assistantMessage("a1", [
|
||||||
|
{ id: "c1", name: "ask_user_question", args: { question: "x" } },
|
||||||
|
{ id: "c2", name: "read", args: { path: "/repo/skills/foo/SKILL.md" } },
|
||||||
|
{ id: "c3", name: "bash", args: { command: "old" } },
|
||||||
|
{ id: "c4", name: "bash", args: { command: "tiny" } },
|
||||||
|
])),
|
||||||
|
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000), "ask_user_question")),
|
||||||
|
entry("e3", toolResultMessage("r2", "c2", "b".repeat(4_000), "read")),
|
||||||
|
entry("e4", toolResultMessage("r3", "c3", `${MASK_PREFIX}bash result cleared to save context (~900 tokens).`), ),
|
||||||
|
entry("e5", toolResultMessage("r4", "c4", "tiny")),
|
||||||
|
entry("e6", toolResultMessage("r5", "c5", "d".repeat(4_000))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const masking = { ...MASKING, minClearTokens: 100, minEntryTokens: 100 };
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
|
||||||
|
assert.deepEqual(plan.drafts.map((draft) => draft.targetId), ["e6"]);
|
||||||
|
assert.equal(plan.masked[0]!.toolCallId, "c5");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("when clearable volume stays under minClear nothing is masked", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", toolResultMessage("r1", "c1", "a".repeat(600))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking: MASKING, estimate });
|
||||||
|
// ~130 tokens clearable, below minClearTokens (200) → not worth invalidating cache.
|
||||||
|
assert.equal(plan.drafts.length, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("placeholder is deterministic, names the tool and call id, and truncates args at 300 chars", () => {
|
||||||
|
const longArgs = { command: `run ${"v".repeat(600)}` };
|
||||||
|
const entries = [
|
||||||
|
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash", args: longArgs }])),
|
||||||
|
entry("e2", toolResultMessage("r1", "c1", "a".repeat(4_000))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const masking = { ...MASKING, minClearTokens: 1 };
|
||||||
|
const first = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
|
||||||
|
const second = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
|
||||||
|
assert.equal(first.drafts.length, 1);
|
||||||
|
const placeholder = first.masked[0]!.placeholder;
|
||||||
|
assert.equal(placeholder, second.masked[0]!.placeholder);
|
||||||
|
assert.ok(placeholder.startsWith(MASK_PREFIX));
|
||||||
|
assert.match(placeholder, /bash result cleared/);
|
||||||
|
assert.match(placeholder, /"toolCallIds":\["c1"\]/);
|
||||||
|
const argsText = placeholder.match(/Call args: (.*)\. Re-run/)?.[1] ?? "";
|
||||||
|
assert.ok(argsText.length <= 301, `args excerpt ${argsText.length} exceeded 300+ellipsis`);
|
||||||
|
assert.ok(placeholder.includes("…"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a masked result carrying details.overflow.id gets an overflow_query hint in the placeholder", () => {
|
||||||
|
const withOverflow = {
|
||||||
|
...toolResultMessage("r1", "c1", "a".repeat(4_000)),
|
||||||
|
details: { overflow: { id: "0123456789ab", totalLines: 5, totalBytes: 4000, truncated: true } },
|
||||||
|
};
|
||||||
|
const withoutOverflow = toolResultMessage("r2", "c2", "b".repeat(4_000));
|
||||||
|
const entries = [
|
||||||
|
entry("e1", assistantMessage("a1", [{ id: "c1", name: "bash" }, { id: "c2", name: "bash" }])),
|
||||||
|
entry("e2", withOverflow),
|
||||||
|
entry("e3", withoutOverflow),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const masking = { ...MASKING, minClearTokens: 1 };
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 3_500, contextWindow: 0, masking, estimate });
|
||||||
|
assert.equal(plan.masked.length, 2);
|
||||||
|
const first = plan.masked.find((m) => m.toolCallId === "c1")!.placeholder;
|
||||||
|
const second = plan.masked.find((m) => m.toolCallId === "c2")!.placeholder;
|
||||||
|
assert.ok(first.includes(` Full original stored: overflow_query {"id":"0123456789ab"}.`));
|
||||||
|
assert.ok(!second.includes("overflow_query"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the small-window ratio caps both trigger and target", () => {
|
||||||
|
const masking: MaskingConfig = {
|
||||||
|
...MASKING,
|
||||||
|
triggerTokens: 200_000,
|
||||||
|
triggerRatio: 0.6,
|
||||||
|
targetTokens: 100_000,
|
||||||
|
targetRatio: 0.35,
|
||||||
|
minClearTokens: 50_000,
|
||||||
|
};
|
||||||
|
assert.equal(effectiveThreshold(masking.triggerTokens, masking.triggerRatio, 272_000), 163_200);
|
||||||
|
assert.equal(effectiveThreshold(masking.targetTokens, masking.targetRatio, 272_000), 95_200);
|
||||||
|
const entries = [
|
||||||
|
entry("e1", toolResultMessage("r1", "c1", "a".repeat(300_000))),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const below = planMasking({ contextEntries: entries, currentTokens: 163_199, contextWindow: 272_000, masking, estimate });
|
||||||
|
assert.equal(below.drafts.length, 0);
|
||||||
|
const above = planMasking({ contextEntries: entries, currentTokens: 163_200, contextWindow: 272_000, masking, estimate });
|
||||||
|
assert.equal(above.drafts.length, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("ledger rebuild counts only masked targets still inside the active context", () => {
|
||||||
|
const placeholder = (id: string) =>
|
||||||
|
`${MASK_PREFIX}bash result cleared to save context (~100 tokens). Call args: {}. Re-run the call if still needed, or recover the original with context_tree_query {"toolCallIds":["${id}"]}.`;
|
||||||
|
const branch = [
|
||||||
|
{ type: "message", id: "u1", parentId: null, timestamp: "t", message: { role: "user", content: "start", timestamp: 1 } },
|
||||||
|
{ type: "message", id: "r-old", parentId: "u1", timestamp: "t", message: toolResultMessage("m-old", "c-old", "o".repeat(2_000)) },
|
||||||
|
{ type: "context_edit", id: "e-old", parentId: "r-old", timestamp: "t", targetId: "r-old", replacement: { content: placeholder("c-old") } },
|
||||||
|
{ type: "message", id: "m-keep", parentId: "e-old", timestamp: "t", message: { role: "user", content: "kept question", timestamp: 2 } },
|
||||||
|
{ type: "compaction", id: "c1", parentId: "m-keep", timestamp: "t", summary: "summary", firstKeptEntryId: "m-keep", tokensBefore: 9_999 },
|
||||||
|
{ type: "message", id: "a-keep", parentId: "c1", timestamp: "t", message: assistantMessage("a-keep", []) },
|
||||||
|
{ type: "message", id: "r-new", parentId: "a-keep", timestamp: "t", message: toolResultMessage("m-new", "c-new", "n".repeat(2_000)) },
|
||||||
|
{ type: "context_edit", id: "e-new", parentId: "r-new", timestamp: "t", targetId: "r-new", replacement: { content: placeholder("c-new") } },
|
||||||
|
];
|
||||||
|
const ledger = rebuildMaskedLedger(branch, "e-new", estimate);
|
||||||
|
assert.equal(ledger.size, 1);
|
||||||
|
assert.ok(ledger.has("c-new"));
|
||||||
|
assert.ok(!ledger.has("c-old"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("user and assistant messages are never masked", () => {
|
||||||
|
const entries = [
|
||||||
|
entry("e1", { role: "user", content: "u".repeat(8_000), timestamp: "u1" }),
|
||||||
|
entry("e2", { role: "assistant", content: [{ type: "text", text: "a".repeat(8_000) }], timestamp: "a1" }),
|
||||||
|
...protect(),
|
||||||
|
];
|
||||||
|
const plan = planMasking({ contextEntries: entries, currentTokens: 9_000, contextWindow: 0, masking: { ...MASKING, keepRecentTurns: 1, minClearTokens: 1 }, estimate });
|
||||||
|
assert.equal(plan.drafts.length, 0);
|
||||||
|
});
|
||||||
@@ -10,9 +10,9 @@ function message(index: number) {
|
|||||||
|
|
||||||
test("cached projection and append-only delta stay inside the hot-path budget", () => {
|
test("cached projection and append-only delta stay inside the hot-path budget", () => {
|
||||||
const index = new SegmentIndex("perf");
|
const index = new SegmentIndex("perf");
|
||||||
index.hydrate([], DEFAULT_CONFIG);
|
index.hydrate();
|
||||||
const messages = Array.from({ length: 10_000 }, (_, value) => message(value));
|
const messages = Array.from({ length: 10_000 }, (_, value) => message(value));
|
||||||
const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000 };
|
const params = { environmentHash: "env", environmentTokens: 2_000, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
|
||||||
index.project(messages, params);
|
index.project(messages, params);
|
||||||
|
|
||||||
const cacheStart = performance.now();
|
const cacheStart = performance.now();
|
||||||
@@ -24,6 +24,7 @@ test("cached projection and append-only delta stay inside the hot-path budget",
|
|||||||
index.project(appended, params);
|
index.project(appended, params);
|
||||||
const deltaMs = performance.now() - deltaStart;
|
const deltaMs = performance.now() - deltaStart;
|
||||||
|
|
||||||
|
console.log(`cache-hit average ${cacheAverageMs.toFixed(3)}ms; delta projection ${deltaMs.toFixed(3)}ms`);
|
||||||
assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`);
|
assert.ok(cacheAverageMs < 2, `cache-hit average ${cacheAverageMs.toFixed(3)}ms exceeded 2ms`);
|
||||||
assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`);
|
assert.ok(deltaMs < 10, `delta projection ${deltaMs.toFixed(3)}ms exceeded 10ms`);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
import test from "node:test";
|
import test from "node:test";
|
||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { DEFAULT_CONFIG } from "../src/config.ts";
|
import { SegmentIndex } from "../src/segment-index.ts";
|
||||||
import { SegmentIndex, deterministicSummary } from "../src/segment-index.ts";
|
|
||||||
|
|
||||||
const assistant = {
|
const assistant = {
|
||||||
role: "assistant",
|
role: "assistant",
|
||||||
@@ -17,66 +16,96 @@ const result = {
|
|||||||
timestamp: 11,
|
timestamp: 11,
|
||||||
};
|
};
|
||||||
|
|
||||||
test("deterministic summary is bounded and preserves head/tail evidence", () => {
|
// Regression: a ~14.5K-char read result projected under a 1M window must reach
|
||||||
const summary = deterministicSummary({
|
// the model byte-for-byte unchanged — masking only happens via context_edit
|
||||||
toolName: "bash",
|
// drafts under real pressure, never on the hot projection path.
|
||||||
args: { command: "npm test" },
|
test("projection is byte-for-byte identity for a 300-line read result under a 1M window", () => {
|
||||||
rawText: result.content[0]!.text,
|
const readBody = Array.from({ length: 300 }, (_unused, index) => `line ${index + 1}: ${"y".repeat(38)}`).join("\n");
|
||||||
isError: true,
|
const readResult = {
|
||||||
maxStubTokens: 256,
|
role: "toolResult",
|
||||||
|
toolCallId: "call-read",
|
||||||
|
toolName: "read",
|
||||||
|
content: [{ type: "text", text: readBody }],
|
||||||
|
timestamp: 12,
|
||||||
|
};
|
||||||
|
assert.ok(readBody.length > 14_000 && readBody.length < 15_000, `fixture ${readBody.length} chars`);
|
||||||
|
|
||||||
|
const index = new SegmentIndex("session-1");
|
||||||
|
index.hydrate();
|
||||||
|
const snapshot = index.project([assistant, readResult], {
|
||||||
|
environmentHash: "env",
|
||||||
|
environmentTokens: 100,
|
||||||
|
contextWindow: 1_000_000,
|
||||||
|
maskedResults: 0,
|
||||||
|
reclaimedTokens: 0,
|
||||||
});
|
});
|
||||||
assert.match(summary, /tool=bash status=error/);
|
|
||||||
assert.match(summary, /failure/);
|
assert.strictEqual(snapshot.messages[1], readResult);
|
||||||
assert.match(summary, /stack tail/);
|
assert.strictEqual(snapshot.messages[1]!.content[0].text, readBody);
|
||||||
assert.ok(summary.length <= 1_100);
|
assert.ok(snapshot.messages[1]!.content[0].text.includes("line 150:"));
|
||||||
|
assert.equal(snapshot.reclaimedTokens, 0);
|
||||||
|
assert.equal(snapshot.maskedResults, 0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("projection incrementally replaces tool results and caches identical input", () => {
|
test("projection caches identical input and handles append-only deltas", () => {
|
||||||
const index = new SegmentIndex("session-1");
|
const index = new SegmentIndex("session-2");
|
||||||
index.hydrate([], DEFAULT_CONFIG);
|
index.hydrate();
|
||||||
const records = index.captureTurn(assistant, [result], DEFAULT_CONFIG);
|
|
||||||
assert.equal(records.length, 1);
|
|
||||||
|
|
||||||
const messages = [assistant, result];
|
const messages = [assistant, result];
|
||||||
const first = index.project(messages, {
|
const params = { environmentHash: "env", environmentTokens: 100, contextWindow: 300_000, maskedResults: 0, reclaimedTokens: 0 };
|
||||||
environmentHash: "env",
|
const first = index.project(messages, params);
|
||||||
environmentTokens: 100,
|
const second = index.project(messages, params);
|
||||||
contextWindow: 300_000,
|
|
||||||
});
|
|
||||||
const second = index.project(messages, {
|
|
||||||
environmentHash: "env",
|
|
||||||
environmentTokens: 100,
|
|
||||||
contextWindow: 300_000,
|
|
||||||
});
|
|
||||||
|
|
||||||
assert.strictEqual(first, second);
|
assert.strictEqual(first, second);
|
||||||
assert.ok(first.effectiveTokens < first.rawTokens);
|
assert.ok(first.rawTokens > 0);
|
||||||
assert.match(first.messages[1]!.content[0].text, /context_tree_query/);
|
|
||||||
|
|
||||||
const record = records[0]!;
|
const appended = index.project([...messages, { role: "user", content: "next", timestamp: 13 }], params);
|
||||||
assert.ok(index.applySemantic(record.sourceKey, record.sourceHash, "- npm test failed\n- stack tail"));
|
assert.notStrictEqual(appended, first);
|
||||||
const semantic = index.project(messages, {
|
assert.equal(appended.messages.length, 3);
|
||||||
|
assert.strictEqual(appended.messages[1], result);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("changed masking stats publish a new snapshot without mutating the old one", () => {
|
||||||
|
const index = new SegmentIndex("session-stats");
|
||||||
|
index.hydrate();
|
||||||
|
const params = {
|
||||||
environmentHash: "env",
|
environmentHash: "env",
|
||||||
environmentTokens: 100,
|
environmentTokens: 100,
|
||||||
contextWindow: 300_000,
|
contextWindow: 300_000,
|
||||||
});
|
maskedResults: 0,
|
||||||
assert.notStrictEqual(semantic, first);
|
reclaimedTokens: 0,
|
||||||
assert.match(semantic.messages[1]!.content[0].text, /npm test failed/);
|
};
|
||||||
|
const first = index.project([assistant, result], params);
|
||||||
|
const same = index.project([assistant, result], params);
|
||||||
|
assert.strictEqual(same, first);
|
||||||
|
|
||||||
|
const updated = index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 });
|
||||||
|
assert.notStrictEqual(updated, first);
|
||||||
|
assert.equal(updated.maskedResults, 1);
|
||||||
|
assert.equal(updated.reclaimedTokens, 500);
|
||||||
|
assert.equal(updated.rawTokens, updated.effectiveTokens + 500);
|
||||||
|
// The previously published object stays immutable.
|
||||||
|
assert.equal(first.maskedResults, 0);
|
||||||
|
assert.equal(first.reclaimedTokens, 0);
|
||||||
|
// Unchanged stats keep returning the same cached object.
|
||||||
|
assert.strictEqual(index.project([assistant, result], { ...params, maskedResults: 1, reclaimedTokens: 500 }), updated);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("usage-backed baseline is adjusted only by projected delta", () => {
|
test("usage-backed baseline is adjusted only by projected delta", () => {
|
||||||
const index = new SegmentIndex("session-2");
|
const index = new SegmentIndex("session-3");
|
||||||
index.hydrate([], DEFAULT_CONFIG);
|
index.hydrate();
|
||||||
const initial = index.project([assistant], {
|
const initial = index.project([assistant], {
|
||||||
environmentHash: "env",
|
environmentHash: "env",
|
||||||
environmentTokens: 50,
|
environmentTokens: 50,
|
||||||
contextWindow: 100_000,
|
contextWindow: 100_000,
|
||||||
|
maskedResults: 0,
|
||||||
|
reclaimedTokens: 0,
|
||||||
});
|
});
|
||||||
index.recordUsage(1_000, initial.messageEstimate, "env");
|
index.recordUsage(1_000, initial.messageEstimate, "env");
|
||||||
const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], {
|
const next = index.project([assistant, { role: "user", content: "next", timestamp: 12 }], {
|
||||||
environmentHash: "env",
|
environmentHash: "env",
|
||||||
environmentTokens: 50,
|
environmentTokens: 50,
|
||||||
contextWindow: 100_000,
|
contextWindow: 100_000,
|
||||||
|
maskedResults: 0,
|
||||||
|
reclaimedTokens: 0,
|
||||||
});
|
});
|
||||||
assert.equal(next.confidence, "usage-backed");
|
assert.equal(next.confidence, "usage-backed");
|
||||||
assert.ok(next.effectiveTokens >= 1_000);
|
assert.ok(next.effectiveTokens >= 1_000);
|
||||||
|
|||||||
@@ -1,19 +1,17 @@
|
|||||||
import test from "node:test";
|
import test from "node:test";
|
||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { shouldCompact } from "../src/token-accounting.ts";
|
import { effectiveThreshold } from "../src/token-accounting.ts";
|
||||||
|
|
||||||
test("raw pressure cannot trigger compaction while effective lower-bound stays below 90%", () => {
|
test("absolute token threshold wins on large windows", () => {
|
||||||
const config = { hardRatio: 0.9 };
|
assert.equal(effectiveThreshold(200_000, 0.6, 1_000_000), 200_000);
|
||||||
const snapshot = { lowerBound: 231_000, contextWindow: 300_000 };
|
|
||||||
assert.equal(shouldCompact("threshold", snapshot, config), false);
|
|
||||||
assert.equal(shouldCompact("manual", snapshot, config), true);
|
|
||||||
assert.equal(shouldCompact("overflow", snapshot, config), true);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("effective lower-bound at the configured threshold compacts", () => {
|
test("window ratio caps the threshold on small windows", () => {
|
||||||
assert.equal(shouldCompact(
|
assert.equal(effectiveThreshold(200_000, 0.6, 272_000), 163_200);
|
||||||
"threshold",
|
assert.equal(effectiveThreshold(100_000, 0.35, 272_000), 95_200);
|
||||||
{ lowerBound: 270_000, contextWindow: 300_000 },
|
});
|
||||||
{ hardRatio: 0.9 },
|
|
||||||
), true);
|
test("a missing context window falls back to the absolute threshold", () => {
|
||||||
|
assert.equal(effectiveThreshold(200_000, 0.6, 0), 200_000);
|
||||||
|
assert.equal(effectiveThreshold(200_000, 0.6, -1), 200_000);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,31 +0,0 @@
|
|||||||
You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.
|
|
||||||
|
|
||||||
Available tools:
|
|
||||||
- read: Read file contents
|
|
||||||
- bash: Execute bash commands (ls, grep, find, etc.)
|
|
||||||
- edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call
|
|
||||||
- write: Create or overwrite files
|
|
||||||
|
|
||||||
In addition to the tools above, you may have access to other custom tools depending on the project.
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
- Use bash for file operations like ls, rg, find
|
|
||||||
- Use read to examine files instead of cat or sed.
|
|
||||||
- You can inspect PI_* environment variables for current model and session details.
|
|
||||||
- Use edit for precise changes (edits[].oldText must match exactly)
|
|
||||||
- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls
|
|
||||||
- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.
|
|
||||||
- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.
|
|
||||||
- Use write only for new files or complete rewrites.
|
|
||||||
- Be concise in your responses
|
|
||||||
- Show file paths clearly when working with files
|
|
||||||
|
|
||||||
Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI):
|
|
||||||
- Main documentation: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/README.md
|
|
||||||
- Additional docs: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/docs
|
|
||||||
- Examples: /Users/sia/.nvm/versions/node/v24.16.0/lib/node_modules/@earendil-works/pi-coding-agent/examples (extensions, custom tools, SDK)
|
|
||||||
- When reading pi docs or examples, resolve docs/... under Additional docs and examples/... under Examples, not the current working directory
|
|
||||||
- When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md), environment variables (docs/environment-variables.md)
|
|
||||||
- When working on pi topics, read the docs and examples, and follow .md cross-references before implementing
|
|
||||||
- Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details)
|
|
||||||
Current working directory: /Users/sia/code/my-pi
|
|
||||||
@@ -19,7 +19,7 @@
|
|||||||
|
|
||||||
### 依赖
|
### 依赖
|
||||||
|
|
||||||
- `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,与 Context Mode 使用的根 `better-sqlite3@12.11.1` 并存;因此根 `allowScripts` 无需新增条目,也不得删除既有的 `better-sqlite3@12.11.1` 放行。
|
- `better-sqlite3@^13.0.3`:13.x 运行时实际 `dlopen` 的是包内 N-API 预编译二进制 `prebuilds/<platform>.node`,**不依赖 node-gyp 编译产物**。npm 会因该包存在 `binding.gyp` 而推断出隐式 `node-gyp rebuild`,但该步骤既无必要也不得批准。npm 会把它嵌套安装到本目录的 `node_modules/`,根 `allowScripts` 无需任何 `better-sqlite3` 放行条目(context-mode 移除后根依赖已不再包含 better-sqlite3@12)。
|
||||||
- `strip-ansi@7.2.0`:解析到根 `node_modules`。
|
- `strip-ansi@7.2.0`:解析到根 `node_modules`。
|
||||||
|
|
||||||
### 本地改动
|
### 本地改动
|
||||||
|
|||||||
@@ -24,10 +24,10 @@ const emit = async (event, data = {}) => {
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
process.env.PI_CODING_AGENT_DIR = root;
|
process.env.PI_CODING_AGENT_DIR = root;
|
||||||
process.env.PI_HERMES_MEMORY_CONFIG_PATH = join(root, "hermes-memory-config.json");
|
const configPath = join(root, "hermes-memory-config.json");
|
||||||
await mkdir(globalDir);
|
await mkdir(globalDir);
|
||||||
await mkdir(ctx.cwd);
|
await mkdir(ctx.cwd);
|
||||||
await writeFile(process.env.PI_HERMES_MEMORY_CONFIG_PATH, JSON.stringify({
|
await writeFile(configPath, JSON.stringify({
|
||||||
lazyInitialization: lazy, memoryMode: "policy-only",
|
lazyInitialization: lazy, memoryMode: "policy-only",
|
||||||
reviewEnabled: false, correctionDetection: false,
|
reviewEnabled: false, correctionDetection: false,
|
||||||
flushOnShutdown: false, flushOnCompact: false,
|
flushOnShutdown: false, flushOnCompact: false,
|
||||||
@@ -40,7 +40,10 @@ try {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const started = performance.now();
|
const started = performance.now();
|
||||||
const { default: register } = await import(pathToFileURL(resolve(entry)).href);
|
const module = await import(pathToFileURL(resolve(entry)).href);
|
||||||
|
const register = module.createHermesMemoryExtension
|
||||||
|
? module.createHermesMemoryExtension({ configPath })
|
||||||
|
: module.default;
|
||||||
const imported = performance.now();
|
const imported = performance.now();
|
||||||
register({
|
register({
|
||||||
on(event, handler) {
|
on(event, handler) {
|
||||||
|
|||||||
@@ -89,16 +89,7 @@ export const DEFAULT_CONFIG_PATH = fileURLToPath(
|
|||||||
new URL("../../config/pi-hermes-memory.json", import.meta.url),
|
new URL("../../config/pi-hermes-memory.json", import.meta.url),
|
||||||
);
|
);
|
||||||
|
|
||||||
/**
|
export function loadConfig(configPath = DEFAULT_CONFIG_PATH): MemoryConfig {
|
||||||
* Resolve the effective config path. `PI_HERMES_MEMORY_CONFIG_PATH` relocates
|
|
||||||
* the file (tests point it at a temp file; it is never a persisted config
|
|
||||||
* value).
|
|
||||||
*/
|
|
||||||
export function resolveConfigPath(): string {
|
|
||||||
return process.env.PI_HERMES_MEMORY_CONFIG_PATH ?? DEFAULT_CONFIG_PATH;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function loadConfig(configPath = resolveConfigPath()): MemoryConfig {
|
|
||||||
try {
|
try {
|
||||||
if (fs.existsSync(configPath)) {
|
if (fs.existsSync(configPath)) {
|
||||||
const raw = fs.readFileSync(configPath, "utf-8");
|
const raw = fs.readFileSync(configPath, "utf-8");
|
||||||
|
|||||||
@@ -52,7 +52,7 @@ import { registerPreviewContextCommand } from "./handlers/preview-context.js";
|
|||||||
import { registerStandingPinCommand } from "./handlers/standing-pin.js";
|
import { registerStandingPinCommand } from "./handlers/standing-pin.js";
|
||||||
import { StandingInstructions } from "./store/standing-instructions.js";
|
import { StandingInstructions } from "./store/standing-instructions.js";
|
||||||
import { STANDING_FILE } from "./constants.js";
|
import { STANDING_FILE } from "./constants.js";
|
||||||
import { loadConfig } from "./config.js";
|
import { DEFAULT_CONFIG_PATH, loadConfig } from "./config.js";
|
||||||
import { shouldWarnAutoConsolidationFailure } from "./auto-consolidation-warning.js";
|
import { shouldWarnAutoConsolidationFailure } from "./auto-consolidation-warning.js";
|
||||||
import { detectProject, detectProjectSkills } from "./project.js";
|
import { detectProject, detectProjectSkills } from "./project.js";
|
||||||
import { buildPromptContext } from "./prompt-context.js";
|
import { buildPromptContext } from "./prompt-context.js";
|
||||||
@@ -90,8 +90,12 @@ export function registerProjectSkillDiscoveryHandler(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
export default function (pi: ExtensionAPI) {
|
export interface HermesMemoryExtensionOptions {
|
||||||
const config = loadConfig();
|
configPath?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hermesMemoryExtension(pi: ExtensionAPI, configPath: string) {
|
||||||
|
const config = loadConfig(configPath);
|
||||||
const lazy = config.lazyInitialization === true && config.memoryMode === "policy-only";
|
const lazy = config.lazyInitialization === true && config.memoryMode === "policy-only";
|
||||||
let sessionContext: ExtensionContext | undefined;
|
let sessionContext: ExtensionContext | undefined;
|
||||||
|
|
||||||
@@ -480,3 +484,12 @@ export default function (pi: ExtensionAPI) {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function createHermesMemoryExtension(
|
||||||
|
options?: HermesMemoryExtensionOptions,
|
||||||
|
): (pi: ExtensionAPI) => void {
|
||||||
|
const configPath = options?.configPath ?? DEFAULT_CONFIG_PATH;
|
||||||
|
return (pi) => hermesMemoryExtension(pi, configPath);
|
||||||
|
}
|
||||||
|
|
||||||
|
export default createHermesMemoryExtension();
|
||||||
|
|||||||
@@ -8,10 +8,10 @@ import { after, beforeEach, afterEach, describe, it } from "node:test";
|
|||||||
// Bind the entry point to a disposable agent root, never the user's memory.
|
// Bind the entry point to a disposable agent root, never the user's memory.
|
||||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-lazy-startup-"));
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-lazy-startup-"));
|
||||||
const previousRoot = process.env.PI_CODING_AGENT_DIR;
|
const previousRoot = process.env.PI_CODING_AGENT_DIR;
|
||||||
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
|
||||||
process.env.PI_CODING_AGENT_DIR = root;
|
process.env.PI_CODING_AGENT_DIR = root;
|
||||||
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(root, "hermes-memory-config.json");
|
const configPath = path.join(root, "hermes-memory-config.json");
|
||||||
const { default: registerExtension } = await import("../src/index.js");
|
const { createHermesMemoryExtension } = await import("../src/index.js");
|
||||||
|
const registerExtension = createHermesMemoryExtension({ configPath });
|
||||||
const { MemoryStore } = await import("../src/store/memory-store.js");
|
const { MemoryStore } = await import("../src/store/memory-store.js");
|
||||||
const { DatabaseManager } = await import("../src/store/db.js");
|
const { DatabaseManager } = await import("../src/store/db.js");
|
||||||
const globalDir = path.join(root, "pi-hermes-memory");
|
const globalDir = path.join(root, "pi-hermes-memory");
|
||||||
@@ -24,7 +24,7 @@ let notifications: string[];
|
|||||||
let ctx: any;
|
let ctx: any;
|
||||||
|
|
||||||
async function configure(overrides: Record<string, unknown> = {}) {
|
async function configure(overrides: Record<string, unknown> = {}) {
|
||||||
await fs.writeFile(path.join(root, "hermes-memory-config.json"), JSON.stringify({
|
await fs.writeFile(configPath, JSON.stringify({
|
||||||
lazyInitialization: true,
|
lazyInitialization: true,
|
||||||
reviewEnabled: false, correctionDetection: false,
|
reviewEnabled: false, correctionDetection: false,
|
||||||
flushOnCompact: false, flushOnShutdown: false,
|
flushOnCompact: false, flushOnShutdown: false,
|
||||||
@@ -87,8 +87,6 @@ afterEach(async () => {
|
|||||||
after(async () => {
|
after(async () => {
|
||||||
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
||||||
else process.env.PI_CODING_AGENT_DIR = previousRoot;
|
else process.env.PI_CODING_AGENT_DIR = previousRoot;
|
||||||
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
|
||||||
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
|
|
||||||
await fs.rm(root, { recursive: true, force: true });
|
await fs.rm(root, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -18,11 +18,11 @@ describe("session project memory rebinding", () => {
|
|||||||
const launchDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-launch-"));
|
const launchDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-launch-"));
|
||||||
const targetDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-target-"));
|
const targetDir = await fs.mkdtemp(path.join(os.tmpdir(), "pi-project-rebind-target-"));
|
||||||
const previousAgentRoot = process.env.PI_CODING_AGENT_DIR;
|
const previousAgentRoot = process.env.PI_CODING_AGENT_DIR;
|
||||||
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
const configPath = path.join(agentRoot, "hermes-memory-config.json");
|
||||||
const previousCwd = process.cwd();
|
const previousCwd = process.cwd();
|
||||||
try {
|
try {
|
||||||
await fs.writeFile(
|
await fs.writeFile(
|
||||||
path.join(agentRoot, "hermes-memory-config.json"),
|
configPath,
|
||||||
JSON.stringify({
|
JSON.stringify({
|
||||||
memoryMode: "legacy-inject",
|
memoryMode: "legacy-inject",
|
||||||
reviewEnabled: false,
|
reviewEnabled: false,
|
||||||
@@ -47,10 +47,10 @@ describe("session project memory rebinding", () => {
|
|||||||
);
|
);
|
||||||
|
|
||||||
process.env.PI_CODING_AGENT_DIR = agentRoot;
|
process.env.PI_CODING_AGENT_DIR = agentRoot;
|
||||||
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(agentRoot, "hermes-memory-config.json");
|
|
||||||
process.chdir(launchDir);
|
process.chdir(launchDir);
|
||||||
// Import after setting PI_CODING_AGENT_DIR so AGENT_ROOT is test-local.
|
// Import after setting PI_CODING_AGENT_DIR so AGENT_ROOT is test-local.
|
||||||
const { default: registerExtension } = await import("../src/index.js");
|
const { createHermesMemoryExtension } = await import("../src/index.js");
|
||||||
|
const registerExtension = createHermesMemoryExtension({ configPath });
|
||||||
const mockPi: MockPi = {
|
const mockPi: MockPi = {
|
||||||
handlers: {},
|
handlers: {},
|
||||||
tools: {},
|
tools: {},
|
||||||
@@ -103,8 +103,6 @@ describe("session project memory rebinding", () => {
|
|||||||
process.chdir(previousCwd);
|
process.chdir(previousCwd);
|
||||||
if (previousAgentRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
if (previousAgentRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
||||||
else process.env.PI_CODING_AGENT_DIR = previousAgentRoot;
|
else process.env.PI_CODING_AGENT_DIR = previousAgentRoot;
|
||||||
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
|
||||||
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
|
|
||||||
await fs.rm(agentRoot, { recursive: true, force: true });
|
await fs.rm(agentRoot, { recursive: true, force: true });
|
||||||
await fs.rm(launchDir, { recursive: true, force: true });
|
await fs.rm(launchDir, { recursive: true, force: true });
|
||||||
await fs.rm(targetDir, { recursive: true, force: true });
|
await fs.rm(targetDir, { recursive: true, force: true });
|
||||||
|
|||||||
@@ -24,10 +24,10 @@ import { after, beforeEach, describe, it } from "node:test";
|
|||||||
// Bind the entry point to a disposable agent root, never the user's memory.
|
// Bind the entry point to a disposable agent root, never the user's memory.
|
||||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-prompt-injection-"));
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "pi-prompt-injection-"));
|
||||||
const previousRoot = process.env.PI_CODING_AGENT_DIR;
|
const previousRoot = process.env.PI_CODING_AGENT_DIR;
|
||||||
const previousConfigPath = process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
|
||||||
process.env.PI_CODING_AGENT_DIR = root;
|
process.env.PI_CODING_AGENT_DIR = root;
|
||||||
process.env.PI_HERMES_MEMORY_CONFIG_PATH = path.join(root, "hermes-memory-config.json");
|
const configPath = path.join(root, "hermes-memory-config.json");
|
||||||
const { default: registerExtension } = await import("../src/index.js");
|
const { createHermesMemoryExtension } = await import("../src/index.js");
|
||||||
|
const registerExtension = createHermesMemoryExtension({ configPath });
|
||||||
const globalDir = path.join(root, "pi-hermes-memory");
|
const globalDir = path.join(root, "pi-hermes-memory");
|
||||||
const cwd = path.join(root, "workspace");
|
const cwd = path.join(root, "workspace");
|
||||||
|
|
||||||
@@ -68,7 +68,7 @@ let handlers: Record<string, Array<(event: any, ctx: any) => any>>;
|
|||||||
let ctx: any;
|
let ctx: any;
|
||||||
|
|
||||||
async function configure(overrides: Record<string, unknown> = {}) {
|
async function configure(overrides: Record<string, unknown> = {}) {
|
||||||
await fs.writeFile(path.join(root, "hermes-memory-config.json"), JSON.stringify({
|
await fs.writeFile(configPath, JSON.stringify({
|
||||||
memoryMode: "policy-only",
|
memoryMode: "policy-only",
|
||||||
memoryPolicyStyle: "full",
|
memoryPolicyStyle: "full",
|
||||||
lazyInitialization: true,
|
lazyInitialization: true,
|
||||||
@@ -115,8 +115,6 @@ beforeEach(async () => {
|
|||||||
after(async () => {
|
after(async () => {
|
||||||
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
if (previousRoot === undefined) delete process.env.PI_CODING_AGENT_DIR;
|
||||||
else process.env.PI_CODING_AGENT_DIR = previousRoot;
|
else process.env.PI_CODING_AGENT_DIR = previousRoot;
|
||||||
if (previousConfigPath === undefined) delete process.env.PI_HERMES_MEMORY_CONFIG_PATH;
|
|
||||||
else process.env.PI_HERMES_MEMORY_CONFIG_PATH = previousConfigPath;
|
|
||||||
try {
|
try {
|
||||||
await fs.rm(root, { recursive: true, force: true });
|
await fs.rm(root, { recursive: true, force: true });
|
||||||
} catch {
|
} catch {
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
# my-pi 维护约定(pi-minimal-footer)
|
||||||
|
|
||||||
|
- 扩展入口是 `pi-minimal-footer/index.ts`(`package.json` 的 `pi.extensions` 指向 `./index.ts`);`status.ts` 收集 `ctx.ui.setStatus()` 发布的非空扩展状态并渲染为 footer 片段,`settings.ts` 读取 footer 配置,`assets/` 为 provider 图标,测试在 `test/`。
|
||||||
|
- 持久配置只读组合包 `config/pi-minimal-footer.json`(`settings.ts` 的 `readFooterSettings` 解析 `showCwd` / `showBranch`,缺失或非法时回退默认 `true`);不读写 `settings.json`,不以环境变量作持久默认。
|
||||||
|
- 上下文仪表优先显示 pi-compaction 发布的 effective usage 快照(`globalThis[Symbol.for("my-pi.pi-compaction.service.v1")].getMetrics(sessionId)` 的 `percent` / `effectiveTokens`);无快照时回退宿主用量(`buildSessionContext` 取最后一条非 aborted assistant 消息的 `input + output + cacheRead + cacheWrite`)。
|
||||||
|
- 来源记录在 `UPSTREAM.md`:从 `ogulcancelik/pi-extensions` 的 `pi-minimal-footer@0.1.10`(commit `77bd1e1`)导入,保留上游 LICENSE 与作者;本地改动为扩展状态渲染与 bundle 配置文件读取。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
- 在本目录内运行 `npm test`(`node --experimental-strip-types --test test/*.test.ts`)与 `npm run check`(`node --experimental-strip-types --check index.ts`)。
|
||||||
@@ -34,14 +34,14 @@ pi install npm:@ogulcancelik/pi-minimal-footer
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
Environment variables (all optional):
|
Persistent defaults are read from `config/pi-minimal-footer.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
|
||||||
|
|
||||||
| Variable | Description | Default |
|
| Field | Description | Default |
|
||||||
| ------------------------------- | -------------------------------------------------------- | ------- |
|
| ------------ | -------------------------------------------------------- | ------- |
|
||||||
| `PI_MINIMAL_FOOTER_SHOW_CWD` | Show current working directory in footer status line | `1` |
|
| `showCwd` | Show current working directory in footer status line | `true` |
|
||||||
| `PI_MINIMAL_FOOTER_SHOW_BRANCH` | Show git branch/dirty/ahead/behind in footer status line | `1` |
|
| `showBranch` | Show git branch/dirty/ahead/behind in footer status line | `true` |
|
||||||
|
|
||||||
Accepted false values: `0`, `false`, `no`, `off` (case-insensitive).
|
Only `true`/`false` values are honoured; anything else falls back to the default for that field.
|
||||||
|
|
||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
|
|||||||
@@ -5,4 +5,4 @@
|
|||||||
- 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10`
|
- 导入包版本:`@ogulcancelik/pi-minimal-footer@0.1.10`
|
||||||
- 导入日期:2026-08-19
|
- 导入日期:2026-08-19
|
||||||
|
|
||||||
本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段)。
|
本目录由 `my-pi` 直接维护;它不是 submodule,也不保留上游 Git 元数据。本地改动新增了 Pi 扩展状态渲染(把 `ctx.ui.setStatus()` 发布的每个非空状态渲染为普通 footer 片段),并把 showCwd/showBranch 持久配置由 `PI_MINIMAL_FOOTER_*` 环境变量改为组合包 `config/pi-minimal-footer.json`。
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import { existsSync, readFileSync } from "node:fs";
|
|||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { homedir } from "node:os";
|
import { homedir } from "node:os";
|
||||||
import { collectFooterStatusItems } from "./status.ts";
|
import { collectFooterStatusItems } from "./status.ts";
|
||||||
|
import { readFooterSettings } from "./settings.ts";
|
||||||
|
|
||||||
// ============ Types ============
|
// ============ Types ============
|
||||||
|
|
||||||
@@ -43,19 +44,6 @@ interface GitCache {
|
|||||||
const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes
|
const USAGE_REFRESH_INTERVAL = 5 * 60_000; // 5 minutes
|
||||||
const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider
|
const usageCache = new Map<string, UsageSnapshot>(); // keyed by provider
|
||||||
|
|
||||||
// ============ Env Flags ============
|
|
||||||
|
|
||||||
function parseBooleanEnv(value: string | undefined, fallback: boolean): boolean {
|
|
||||||
if (value === undefined) return fallback;
|
|
||||||
|
|
||||||
const normalized = value.trim().toLowerCase();
|
|
||||||
if (!normalized) return fallback;
|
|
||||||
|
|
||||||
if (["1", "true", "yes", "on"].includes(normalized)) return true;
|
|
||||||
if (["0", "false", "no", "off"].includes(normalized)) return false;
|
|
||||||
return fallback;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ============ Identity Cache ============
|
// ============ Identity Cache ============
|
||||||
|
|
||||||
let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved
|
let codexEmailPrefix: string | null | undefined; // undefined = not yet resolved
|
||||||
@@ -754,9 +742,8 @@ export default function (pi: ExtensionAPI) {
|
|||||||
const BAR_FILLED = "━";
|
const BAR_FILLED = "━";
|
||||||
const BAR_EMPTY = "─";
|
const BAR_EMPTY = "─";
|
||||||
|
|
||||||
// Optional visibility toggles (default: enabled)
|
// Optional visibility toggles from bundle config (default: enabled)
|
||||||
const showCwd = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_CWD, true);
|
const { showCwd, showBranch } = readFooterSettings();
|
||||||
const showBranch = parseBooleanEnv(process.env.PI_MINIMAL_FOOTER_SHOW_BRANCH, true);
|
|
||||||
|
|
||||||
function formatTokenCount(tokens: number): string {
|
function formatTokenCount(tokens: number): string {
|
||||||
if (tokens >= 1_000_000) {
|
if (tokens >= 1_000_000) {
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
|
||||||
|
export interface FooterSettings {
|
||||||
|
showCwd: boolean;
|
||||||
|
showBranch: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bundle-level `config/` is the single source of truth: the extension reads
|
||||||
|
// `config/pi-minimal-footer.json` from inside the installed bundle (resolved
|
||||||
|
// relative to this source file), never environment variables.
|
||||||
|
export const FOOTER_CONFIG_PATH = fileURLToPath(
|
||||||
|
new URL("../config/pi-minimal-footer.json", import.meta.url),
|
||||||
|
);
|
||||||
|
|
||||||
|
export function readFooterSettings(configPath: string = FOOTER_CONFIG_PATH): FooterSettings {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
|
||||||
|
const raw =
|
||||||
|
parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
|
||||||
|
? (parsed as Record<string, unknown>)
|
||||||
|
: {};
|
||||||
|
return {
|
||||||
|
showCwd: typeof raw.showCwd === "boolean" ? raw.showCwd : true,
|
||||||
|
showBranch: typeof raw.showBranch === "boolean" ? raw.showBranch : true,
|
||||||
|
};
|
||||||
|
} catch {
|
||||||
|
return { showCwd: true, showBranch: true };
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { afterEach, describe, it } from "node:test";
|
||||||
|
|
||||||
|
import { readFooterSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
let tempDir: string | undefined;
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
if (tempDir) {
|
||||||
|
rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
tempDir = undefined;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
function configPath(contents: string): string {
|
||||||
|
tempDir = mkdtempSync(join(tmpdir(), "pi-minimal-footer-settings-"));
|
||||||
|
const path = join(tempDir, "pi-minimal-footer.json");
|
||||||
|
writeFileSync(path, contents);
|
||||||
|
return path;
|
||||||
|
}
|
||||||
|
|
||||||
|
const expectedDefaults = { showCwd: true, showBranch: true };
|
||||||
|
|
||||||
|
describe("readFooterSettings", () => {
|
||||||
|
it("returns all defaults when the config file is missing", () => {
|
||||||
|
assert.deepEqual(readFooterSettings("/does/not/exist/pi-minimal-footer.json"), expectedDefaults);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns all defaults for malformed or non-object JSON", () => {
|
||||||
|
for (const contents of ["not json", "[1,2]", "42"]) {
|
||||||
|
assert.deepEqual(readFooterSettings(configPath(contents)), expectedDefaults, contents);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("adopts explicit boolean values", () => {
|
||||||
|
assert.deepEqual(
|
||||||
|
readFooterSettings(configPath(JSON.stringify({ showCwd: false, showBranch: false }))),
|
||||||
|
{ showCwd: false, showBranch: false },
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back per-field for non-boolean values", () => {
|
||||||
|
assert.deepEqual(
|
||||||
|
readFooterSettings(configPath(JSON.stringify({ showCwd: "no", showBranch: 0 }))),
|
||||||
|
expectedDefaults,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
+1
-1
@@ -5,7 +5,7 @@
|
|||||||
- Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。
|
- Kitty OSC 99 必须保留 source-window focus、Session 间唯一 ID、Base64 payload 和 tmux passthrough;修改 visibility/action、事件生命周期、fallback、模板、隐私或命令时必须同步 README 和测试。
|
||||||
- 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。
|
- 完成通知默认只在 TUI `agent_settled` 后发送;自动重试、自动 compaction 和 follow-up 期间不得提前通知或重置总耗时。权限通知只在 `permissions:ui_prompt` 后立即发送,不得为策略或 AutoReview 已直接决定的请求通知,也不受完成通知时长门槛影响。非 TUI 输出完整性、策略/AutoReview 静默决定、隐私模式、自动 retry/compaction/follow-up 去重、总耗时、短任务阈值和多窗口点击聚焦都属于回归边界。
|
||||||
- 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。
|
- 默认 `o=always`、`a=focus`,每个 Pi Session 使用独立稳定 ID,标题和正文必须 Base64 编码。
|
||||||
- `/notify on|off` 只覆盖当前 Session;持久默认来自 `PI_NOTIFY_*` 环境变量。`PI_NOTIFY_MESSAGE_SOURCE=none` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。
|
- `/notify on|off` 只覆盖当前 Session;持久默认来自组合包 `config/pi-notify.json`(`src/settings.ts` 读取,文件缺失或字段非法逐字段回退默认)。`messageSource: "none"` 必须继续提供不泄露回复正文、权限值或请求者细节的隐私模式,非 TUI 模式不得写入 OSC 序列。
|
||||||
- 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。
|
- 根包必须通过 `file:./pi-notify` 提供依赖并直接加载源码,不得同时加载 npm `@smoose/pi-notify` 实现,否则会重复通知。
|
||||||
|
|
||||||
## 验证
|
## 验证
|
||||||
|
|||||||
+19
-15
@@ -28,19 +28,21 @@ Do not separately install upstream when using the bundle, or notifications will
|
|||||||
- `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus.
|
- `/notify test`: send a notification immediately; click it to verify exact Kitty-window focus.
|
||||||
- `/notify status`: show the active backend, visibility policy, and duration threshold.
|
- `/notify status`: show the active backend, visibility policy, and duration threshold.
|
||||||
|
|
||||||
Command overrides are session-local. Use environment variables for persistent defaults.
|
Command overrides are session-local. Persistent defaults come from the bundle's `config/pi-notify.json`.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
- `PI_NOTIFY_ENABLED`: `1`/`true` or `0`/`false`; default enabled.
|
Persistent defaults are read once per session from `config/pi-notify.json` inside the installed bundle; a missing or invalid file falls back to the defaults shown:
|
||||||
- `PI_NOTIFY_TITLE`: title template shared by completion and permission notifications; default `π - {project}`.
|
|
||||||
- `PI_NOTIFY_BODY`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary.
|
- `enabled`: `true`/`false`; default `true`.
|
||||||
- `PI_NOTIFY_MESSAGE_SOURCE`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details.
|
- `title`: title template shared by completion and permission notifications; default `π - {project}`.
|
||||||
- `PI_NOTIFY_MESSAGE_MAX`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`.
|
- `body`: completion-notification body template; default `{message}`. Permission prompts use a bounded `Permission required · <agent> · <surface>` summary.
|
||||||
- `PI_NOTIFY_MIN_SECONDS`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts.
|
- `messageSource`: `assistant`, `user`, or privacy mode `none`; default `assistant`. Privacy mode also hides permission values and requester details.
|
||||||
- `PI_NOTIFY_VISIBILITY`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`.
|
- `messageMax`: maximum completion-message or permission-value characters before ellipsis; default `80`, maximum `500`.
|
||||||
- `PI_NOTIFY_ACTION`: Kitty click action `focus` or `none`; default `focus`.
|
- `minSeconds`: minimum complete run duration; default `3`, range `0`–`3600`. This does not delay permission prompts.
|
||||||
- `PI_NOTIFY_SOUND_CMD`: optional shell command run after the notification. This is trusted local configuration and executes through a shell.
|
- `visibility`: Kitty policy `unfocused`, `invisible`, or `always`; default `always`.
|
||||||
|
- `action`: Kitty click action `focus` or `none`; default `focus`.
|
||||||
|
- `soundCommand`: optional shell command run after the notification. This is trusted local configuration and executes through a shell.
|
||||||
|
|
||||||
Visibility policies:
|
Visibility policies:
|
||||||
|
|
||||||
@@ -70,12 +72,14 @@ The active tmux client type is checked so a stale `KITTY_WINDOW_ID` cannot force
|
|||||||
- `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict.
|
- `{status_icon}`: `✓` or `⚠`. This reports observed tool errors, not a definitive task-success verdict.
|
||||||
- `{tool_error_count}`: observed tool error count.
|
- `{tool_error_count}`: observed tool error count.
|
||||||
|
|
||||||
Example for multiple projects without exposing assistant text on the lock screen:
|
Example for multiple projects without exposing assistant text on the lock screen (`config/pi-notify.json`):
|
||||||
|
|
||||||
```bash
|
```json
|
||||||
export PI_NOTIFY_TITLE='π {status_icon} · {project}'
|
{
|
||||||
export PI_NOTIFY_MESSAGE_SOURCE=none
|
"title": "π {status_icon} · {project}",
|
||||||
export PI_NOTIFY_VISIBILITY=unfocused
|
"messageSource": "none",
|
||||||
|
"visibility": "unfocused"
|
||||||
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications.
|
Kitty must be allowed under **System Settings → Notifications**. Self-built or unsigned Kitty binaries may not be able to publish macOS notifications.
|
||||||
|
|||||||
@@ -18,5 +18,6 @@
|
|||||||
- 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。
|
- 新增时长过滤、启用/隐私/动作设置、状态模板字段和 `/notify` 控制。
|
||||||
- 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。
|
- 保留 Base64 Kitty payload,加固 OSC 元数据/回退净化,并忽略分离进程的 spawn 错误。
|
||||||
- 新增生命周期、多窗口元数据、回退、解析器和配置测试。
|
- 新增生命周期、多窗口元数据、回退、解析器和配置测试。
|
||||||
|
- 持久配置由 `PI_NOTIFY_*` 环境变量改为组合包 `config/pi-notify.json`(`src/settings.ts` 读取并逐字段回退默认值)。
|
||||||
|
|
||||||
上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。
|
上游声明本项目使用 MIT 许可证,但在导入的 commit 和 npm tarball 中都没有包含 `LICENSE` 文件。本目录使用上游作者的 GitHub 身份和 commit 年份补充标准 MIT 文本,使声明的许可证随维护源码一起保留。
|
||||||
|
|||||||
+11
-17
@@ -1,13 +1,6 @@
|
|||||||
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
||||||
import { extractMessageText, truncateText } from "./messages.ts";
|
import { extractMessageText, truncateText } from "./messages.ts";
|
||||||
import {
|
import { readNotifySettings, type NotifySettings } from "./settings.ts";
|
||||||
parseEnabled,
|
|
||||||
parseKittyVisibility,
|
|
||||||
parseMessageMax,
|
|
||||||
parseMessageSource,
|
|
||||||
parseMinDurationMs,
|
|
||||||
parseNotificationAction,
|
|
||||||
} from "./parsers.ts";
|
|
||||||
import { renderTemplate } from "./template.ts";
|
import { renderTemplate } from "./template.ts";
|
||||||
import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts";
|
import type { NotificationRuntimeValues, NotifyConfig } from "./types.ts";
|
||||||
|
|
||||||
@@ -20,12 +13,13 @@ export function getConfig(
|
|||||||
messages: unknown[] | undefined,
|
messages: unknown[] | undefined,
|
||||||
runtime: NotificationRuntimeValues,
|
runtime: NotificationRuntimeValues,
|
||||||
now = Date.now(),
|
now = Date.now(),
|
||||||
|
settings: NotifySettings = readNotifySettings(),
|
||||||
): NotifyConfig {
|
): NotifyConfig {
|
||||||
const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt);
|
const durationMs = startedAt === undefined ? 0 : Math.max(0, now - startedAt);
|
||||||
const messageMax = parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX);
|
const messageMax = settings.messageMax;
|
||||||
const userMessage = truncateText(extractMessageText(messages, "user"), messageMax);
|
const userMessage = truncateText(extractMessageText(messages, "user"), messageMax);
|
||||||
const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax);
|
const assistantMessage = truncateText(extractMessageText(messages, "assistant"), messageMax);
|
||||||
const messageSource = parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE);
|
const messageSource = settings.messageSource;
|
||||||
const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : "";
|
const selectedMessage = messageSource === "user" ? userMessage : messageSource === "assistant" ? assistantMessage : "";
|
||||||
const message = selectedMessage || "Ready for input";
|
const message = selectedMessage || "Ready for input";
|
||||||
const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0);
|
const toolErrorCount = Math.max(0, runtime.toolErrorCount ?? 0);
|
||||||
@@ -39,15 +33,15 @@ export function getConfig(
|
|||||||
};
|
};
|
||||||
|
|
||||||
return {
|
return {
|
||||||
enabled: parseEnabled(process.env.PI_NOTIFY_ENABLED),
|
enabled: settings.enabled,
|
||||||
title: renderTemplate(process.env.PI_NOTIFY_TITLE || DEFAULT_TITLE, ctx, durationMs, values),
|
title: renderTemplate(settings.title || DEFAULT_TITLE, ctx, durationMs, values),
|
||||||
body: renderTemplate(process.env.PI_NOTIFY_BODY || DEFAULT_BODY, ctx, durationMs, values),
|
body: renderTemplate(settings.body || DEFAULT_BODY, ctx, durationMs, values),
|
||||||
durationMs,
|
durationMs,
|
||||||
minDurationMs: parseMinDurationMs(process.env.PI_NOTIFY_MIN_SECONDS),
|
minDurationMs: settings.minDurationMs,
|
||||||
notificationId: runtime.notificationId,
|
notificationId: runtime.notificationId,
|
||||||
visibility: parseKittyVisibility(process.env.PI_NOTIFY_VISIBILITY),
|
visibility: settings.visibility,
|
||||||
action: parseNotificationAction(process.env.PI_NOTIFY_ACTION),
|
action: settings.action,
|
||||||
soundCommand: process.env.PI_NOTIFY_SOUND_CMD?.trim() || undefined,
|
soundCommand: settings.soundCommand,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+10
-7
@@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto";
|
|||||||
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
||||||
import { getConfig, shouldNotify } from "./config.ts";
|
import { getConfig, shouldNotify } from "./config.ts";
|
||||||
import { truncateText } from "./messages.ts";
|
import { truncateText } from "./messages.ts";
|
||||||
import { parseMessageMax, parseMessageSource } from "./parsers.ts";
|
import { readNotifySettings, type NotifySettings } from "./settings.ts";
|
||||||
import { getNotificationBackend, sendNotification } from "./terminal.ts";
|
import { getNotificationBackend, sendNotification } from "./terminal.ts";
|
||||||
import type { NotifyConfig } from "./types.ts";
|
import type { NotifyConfig } from "./types.ts";
|
||||||
|
|
||||||
@@ -11,13 +11,14 @@ type ExtensionDependencies = {
|
|||||||
createNotificationId: () => string;
|
createNotificationId: () => string;
|
||||||
notify: (config: NotifyConfig) => void;
|
notify: (config: NotifyConfig) => void;
|
||||||
getBackend: typeof getNotificationBackend;
|
getBackend: typeof getNotificationBackend;
|
||||||
|
settings: NotifySettings;
|
||||||
};
|
};
|
||||||
|
|
||||||
const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt";
|
const PERMISSIONS_UI_PROMPT_CHANNEL = "permissions:ui_prompt";
|
||||||
|
|
||||||
function formatPermissionPromptBody(raw: unknown): string | undefined {
|
function formatPermissionPromptBody(raw: unknown, settings: NotifySettings): string | undefined {
|
||||||
if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined;
|
if (!isRecord(raw) || !readNonEmptyString(raw.requestId)) return undefined;
|
||||||
if (parseMessageSource(process.env.PI_NOTIFY_MESSAGE_SOURCE) === "none") {
|
if (settings.messageSource === "none") {
|
||||||
return "Permission confirmation required";
|
return "Permission confirmation required";
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -29,7 +30,7 @@ function formatPermissionPromptBody(raw: unknown): string | undefined {
|
|||||||
|
|
||||||
if (!value) return labels.join(" · ");
|
if (!value) return labels.join(" · ");
|
||||||
const normalizedValue = value.replace(/\s+/g, " ").trim();
|
const normalizedValue = value.replace(/\s+/g, " ").trim();
|
||||||
return `${labels.join(" · ")}\n${truncateText(normalizedValue, parseMessageMax(process.env.PI_NOTIFY_MESSAGE_MAX))}`;
|
return `${labels.join(" · ")}\n${truncateText(normalizedValue, settings.messageMax)}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function readNonEmptyString(value: unknown): string | undefined {
|
function readNonEmptyString(value: unknown): string | undefined {
|
||||||
@@ -47,6 +48,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
|
|||||||
const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`);
|
const createNotificationId = dependencies.createNotificationId ?? (() => `pi-${process.pid}-${randomUUID()}`);
|
||||||
const notify = dependencies.notify ?? sendNotification;
|
const notify = dependencies.notify ?? sendNotification;
|
||||||
const getBackend = dependencies.getBackend ?? getNotificationBackend;
|
const getBackend = dependencies.getBackend ?? getNotificationBackend;
|
||||||
|
const settings = dependencies.settings ?? readNotifySettings();
|
||||||
|
|
||||||
return function piNotify(pi: ExtensionAPI): void {
|
return function piNotify(pi: ExtensionAPI): void {
|
||||||
const notificationId = createNotificationId();
|
const notificationId = createNotificationId();
|
||||||
@@ -81,7 +83,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
|
|||||||
|
|
||||||
pi.on("agent_settled", (_event, ctx) => {
|
pi.on("agent_settled", (_event, ctx) => {
|
||||||
try {
|
try {
|
||||||
const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now());
|
const config = getConfig(ctx, startedAt, latestMessages, { notificationId, toolErrorCount }, now(), settings);
|
||||||
if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config);
|
if (ctx.mode === "tui" && shouldNotify(config, enabledOverride)) notify(config);
|
||||||
} finally {
|
} finally {
|
||||||
resetRun();
|
resetRun();
|
||||||
@@ -89,7 +91,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
|
|||||||
});
|
});
|
||||||
|
|
||||||
const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => {
|
const unsubscribePermissionPrompt = pi.events.on(PERMISSIONS_UI_PROMPT_CHANNEL, (raw) => {
|
||||||
const body = formatPermissionPromptBody(raw);
|
const body = formatPermissionPromptBody(raw, settings);
|
||||||
if (!body || currentCtx?.mode !== "tui") return;
|
if (!body || currentCtx?.mode !== "tui") return;
|
||||||
|
|
||||||
const config = getConfig(
|
const config = getConfig(
|
||||||
@@ -98,6 +100,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
|
|||||||
undefined,
|
undefined,
|
||||||
{ notificationId, toolErrorCount },
|
{ notificationId, toolErrorCount },
|
||||||
now(),
|
now(),
|
||||||
|
settings,
|
||||||
);
|
);
|
||||||
const permissionConfig = { ...config, body, minDurationMs: 0 };
|
const permissionConfig = { ...config, body, minDurationMs: 0 };
|
||||||
if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig);
|
if (shouldNotify(permissionConfig, enabledOverride)) notify(permissionConfig);
|
||||||
@@ -119,7 +122,7 @@ export function createPiNotifyExtension(dependencies: Partial<ExtensionDependenc
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now());
|
const config = getConfig(ctx, now() - 1000, undefined, { notificationId, toolErrorCount: 0 }, now(), settings);
|
||||||
if (action === "test") {
|
if (action === "test") {
|
||||||
if (ctx.mode !== "tui") {
|
if (ctx.mode !== "tui") {
|
||||||
ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning");
|
ctx.ui.notify("System notifications are only emitted in interactive TUI mode", "warning");
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
import {
|
||||||
|
parseEnabled,
|
||||||
|
parseKittyVisibility,
|
||||||
|
parseMessageMax,
|
||||||
|
parseMessageSource,
|
||||||
|
parseMinDurationMs,
|
||||||
|
parseNotificationAction,
|
||||||
|
} from "./parsers.ts";
|
||||||
|
import type { KittyVisibility, MessageSource, NotificationAction } from "./types.ts";
|
||||||
|
|
||||||
|
export interface NotifySettings {
|
||||||
|
enabled: boolean;
|
||||||
|
title: string;
|
||||||
|
body: string;
|
||||||
|
messageSource: MessageSource;
|
||||||
|
messageMax: number;
|
||||||
|
minDurationMs: number;
|
||||||
|
visibility: KittyVisibility;
|
||||||
|
action: NotificationAction;
|
||||||
|
soundCommand?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DEFAULT_TITLE = "π - {project}";
|
||||||
|
const DEFAULT_BODY = "{message}";
|
||||||
|
|
||||||
|
// Bundle-level `config/` is the single source of truth: the extension reads
|
||||||
|
// `config/pi-notify.json` from inside the installed bundle (resolved relative
|
||||||
|
// to this source file), never environment variables.
|
||||||
|
export const NOTIFY_CONFIG_PATH = fileURLToPath(new URL("../../config/pi-notify.json", import.meta.url));
|
||||||
|
|
||||||
|
function readConfigFile(configPath: string): Record<string, unknown> {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(readFileSync(configPath, "utf8")) as unknown;
|
||||||
|
return parsed !== null && typeof parsed === "object" && !Array.isArray(parsed)
|
||||||
|
? (parsed as Record<string, unknown>)
|
||||||
|
: {};
|
||||||
|
} catch {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function optionalString(value: unknown): string | undefined {
|
||||||
|
if (typeof value === "string") return value;
|
||||||
|
if (typeof value === "number" || typeof value === "boolean") return String(value);
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function readNotifySettings(configPath: string = NOTIFY_CONFIG_PATH): NotifySettings {
|
||||||
|
const raw = readConfigFile(configPath);
|
||||||
|
return {
|
||||||
|
enabled: parseEnabled(optionalString(raw.enabled)),
|
||||||
|
title: typeof raw.title === "string" && raw.title ? raw.title : DEFAULT_TITLE,
|
||||||
|
body: typeof raw.body === "string" && raw.body ? raw.body : DEFAULT_BODY,
|
||||||
|
messageSource: parseMessageSource(optionalString(raw.messageSource)),
|
||||||
|
messageMax: parseMessageMax(optionalString(raw.messageMax)),
|
||||||
|
minDurationMs: parseMinDurationMs(optionalString(raw.minSeconds)),
|
||||||
|
visibility: parseKittyVisibility(optionalString(raw.visibility)),
|
||||||
|
action: parseNotificationAction(optionalString(raw.action)),
|
||||||
|
soundCommand: typeof raw.soundCommand === "string" ? raw.soundCommand.trim() || undefined : undefined,
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -1,28 +1,18 @@
|
|||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { afterEach, describe, it } from "node:test";
|
import { describe, it } from "node:test";
|
||||||
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
||||||
import { getConfig, shouldNotify } from "../src/config.ts";
|
import { getConfig, shouldNotify } from "../src/config.ts";
|
||||||
|
import type { NotifySettings } from "../src/settings.ts";
|
||||||
|
|
||||||
const envKeys = [
|
const defaultSettings = (): NotifySettings => ({
|
||||||
"PI_NOTIFY_ENABLED",
|
enabled: true,
|
||||||
"PI_NOTIFY_TITLE",
|
title: "π - {project}",
|
||||||
"PI_NOTIFY_BODY",
|
body: "{message}",
|
||||||
"PI_NOTIFY_MESSAGE_SOURCE",
|
messageSource: "assistant",
|
||||||
"PI_NOTIFY_MESSAGE_MAX",
|
messageMax: 80,
|
||||||
"PI_NOTIFY_MIN_SECONDS",
|
minDurationMs: 3_000,
|
||||||
"PI_NOTIFY_VISIBILITY",
|
visibility: "always",
|
||||||
"PI_NOTIFY_ACTION",
|
action: "focus",
|
||||||
"PI_NOTIFY_SOUND_CMD",
|
|
||||||
] as const;
|
|
||||||
|
|
||||||
const savedEnv = new Map(envKeys.map((key) => [key, process.env[key]]));
|
|
||||||
|
|
||||||
afterEach(() => {
|
|
||||||
for (const key of envKeys) {
|
|
||||||
const value = savedEnv.get(key);
|
|
||||||
if (value === undefined) delete process.env[key];
|
|
||||||
else process.env[key] = value;
|
|
||||||
}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
function context(): ExtensionContext {
|
function context(): ExtensionContext {
|
||||||
@@ -45,6 +35,7 @@ describe("config", () => {
|
|||||||
],
|
],
|
||||||
runtime,
|
runtime,
|
||||||
5_000,
|
5_000,
|
||||||
|
defaultSettings(),
|
||||||
);
|
);
|
||||||
|
|
||||||
assert.deepEqual(config, {
|
assert.deepEqual(config, {
|
||||||
@@ -62,19 +53,20 @@ describe("config", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("supports privacy, threshold, focus, and status template settings", () => {
|
it("supports privacy, threshold, focus, and status template settings", () => {
|
||||||
process.env.PI_NOTIFY_ENABLED = "off";
|
|
||||||
process.env.PI_NOTIFY_TITLE = "{status_icon} {status} {tool_error_count}";
|
|
||||||
process.env.PI_NOTIFY_BODY = "{message}";
|
|
||||||
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none";
|
|
||||||
process.env.PI_NOTIFY_MIN_SECONDS = "10.5";
|
|
||||||
process.env.PI_NOTIFY_VISIBILITY = "invisible";
|
|
||||||
process.env.PI_NOTIFY_ACTION = "none";
|
|
||||||
process.env.PI_NOTIFY_SOUND_CMD = " say done ";
|
|
||||||
|
|
||||||
const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], {
|
const config = getConfig(context(), 1_000, [{ role: "assistant", content: "secret" }], {
|
||||||
notificationId: "pi-session",
|
notificationId: "pi-session",
|
||||||
toolErrorCount: 2,
|
toolErrorCount: 2,
|
||||||
}, 2_000);
|
}, 2_000, {
|
||||||
|
...defaultSettings(),
|
||||||
|
enabled: false,
|
||||||
|
title: "{status_icon} {status} {tool_error_count}",
|
||||||
|
body: "{message}",
|
||||||
|
messageSource: "none",
|
||||||
|
minDurationMs: 10_500,
|
||||||
|
visibility: "invisible",
|
||||||
|
action: "none",
|
||||||
|
soundCommand: "say done",
|
||||||
|
});
|
||||||
|
|
||||||
assert.equal(config.title, "⚠ ready · 2 tool errors 2");
|
assert.equal(config.title, "⚠ ready · 2 tool errors 2");
|
||||||
assert.equal(config.body, "Ready for input");
|
assert.equal(config.body, "Ready for input");
|
||||||
@@ -88,15 +80,13 @@ describe("config", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("applies selected-message truncation and a session override", () => {
|
it("applies selected-message truncation and a session override", () => {
|
||||||
process.env.PI_NOTIFY_MESSAGE_SOURCE = "user";
|
|
||||||
process.env.PI_NOTIFY_MESSAGE_MAX = "5";
|
|
||||||
process.env.PI_NOTIFY_MIN_SECONDS = "0";
|
|
||||||
const config = getConfig(
|
const config = getConfig(
|
||||||
context(),
|
context(),
|
||||||
1_000,
|
1_000,
|
||||||
[{ role: "user", content: "hello world" }],
|
[{ role: "user", content: "hello world" }],
|
||||||
runtime,
|
runtime,
|
||||||
2_000,
|
2_000,
|
||||||
|
{ ...defaultSettings(), messageSource: "user", messageMax: 5, minDurationMs: 0 },
|
||||||
);
|
);
|
||||||
|
|
||||||
assert.equal(config.body, "hell…");
|
assert.equal(config.body, "hell…");
|
||||||
|
|||||||
@@ -1,23 +1,24 @@
|
|||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { afterEach, describe, it } from "node:test";
|
import { describe, it } from "node:test";
|
||||||
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
||||||
import { createPiNotifyExtension } from "../src/index.ts";
|
import { createPiNotifyExtension } from "../src/index.ts";
|
||||||
|
import type { NotifySettings } from "../src/settings.ts";
|
||||||
import type { NotifyConfig } from "../src/types.ts";
|
import type { NotifyConfig } from "../src/types.ts";
|
||||||
|
|
||||||
type Handler = (...args: any[]) => any;
|
type Handler = (...args: any[]) => any;
|
||||||
|
|
||||||
const savedEnv = new Map(
|
const DEFAULT_SETTINGS: NotifySettings = {
|
||||||
["PI_NOTIFY_MIN_SECONDS", "PI_NOTIFY_MESSAGE_SOURCE"].map((key) => [key, process.env[key]]),
|
enabled: true,
|
||||||
);
|
title: "π - {project}",
|
||||||
|
body: "{message}",
|
||||||
|
messageSource: "assistant",
|
||||||
|
messageMax: 80,
|
||||||
|
minDurationMs: 3_000,
|
||||||
|
visibility: "always",
|
||||||
|
action: "focus",
|
||||||
|
};
|
||||||
|
|
||||||
afterEach(() => {
|
function harness(mode: ExtensionContext["mode"] = "tui", settings: Partial<NotifySettings> = {}) {
|
||||||
for (const [key, value] of savedEnv) {
|
|
||||||
if (value === undefined) delete process.env[key];
|
|
||||||
else process.env[key] = value;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
function harness(mode: ExtensionContext["mode"] = "tui") {
|
|
||||||
const handlers = new Map<string, Handler>();
|
const handlers = new Map<string, Handler>();
|
||||||
const eventHandlers = new Map<string, Handler>();
|
const eventHandlers = new Map<string, Handler>();
|
||||||
const commands = new Map<string, { handler: Handler }>();
|
const commands = new Map<string, { handler: Handler }>();
|
||||||
@@ -45,6 +46,7 @@ function harness(mode: ExtensionContext["mode"] = "tui") {
|
|||||||
createNotificationId: () => "pi-session-test",
|
createNotificationId: () => "pi-session-test",
|
||||||
notify: (config) => notifications.push(config),
|
notify: (config) => notifications.push(config),
|
||||||
getBackend: () => "kitty",
|
getBackend: () => "kitty",
|
||||||
|
settings: { ...DEFAULT_SETTINGS, ...settings },
|
||||||
})(pi);
|
})(pi);
|
||||||
|
|
||||||
const ctx = {
|
const ctx = {
|
||||||
@@ -93,9 +95,8 @@ describe("pi-notify extension", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("does not write terminal notifications in RPC or non-interactive modes", async () => {
|
it("does not write terminal notifications in RPC or non-interactive modes", async () => {
|
||||||
process.env.PI_NOTIFY_MIN_SECONDS = "0";
|
|
||||||
for (const mode of ["rpc", "json", "print"] as const) {
|
for (const mode of ["rpc", "json", "print"] as const) {
|
||||||
const h = harness(mode);
|
const h = harness(mode, { minDurationMs: 0 });
|
||||||
await h.handlers.get("session_start")?.({}, h.ctx);
|
await h.handlers.get("session_start")?.({}, h.ctx);
|
||||||
await h.handlers.get("agent_start")?.({}, h.ctx);
|
await h.handlers.get("agent_start")?.({}, h.ctx);
|
||||||
await h.handlers.get("agent_settled")?.({}, h.ctx);
|
await h.handlers.get("agent_settled")?.({}, h.ctx);
|
||||||
@@ -123,8 +124,7 @@ describe("pi-notify extension", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("notifies immediately when permission confirmation needs human input", async () => {
|
it("notifies immediately when permission confirmation needs human input", async () => {
|
||||||
process.env.PI_NOTIFY_MIN_SECONDS = "3600";
|
const h = harness("tui", { minDurationMs: 3_600_000 });
|
||||||
const h = harness();
|
|
||||||
await h.handlers.get("session_start")?.({}, h.ctx);
|
await h.handlers.get("session_start")?.({}, h.ctx);
|
||||||
|
|
||||||
h.eventHandlers.get("permissions:ui_prompt")?.({
|
h.eventHandlers.get("permissions:ui_prompt")?.({
|
||||||
@@ -144,8 +144,7 @@ describe("pi-notify extension", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("keeps permission details private and honors the session off override", async () => {
|
it("keeps permission details private and honors the session off override", async () => {
|
||||||
process.env.PI_NOTIFY_MESSAGE_SOURCE = "none";
|
const h = harness("tui", { messageSource: "none" });
|
||||||
const h = harness();
|
|
||||||
await h.handlers.get("session_start")?.({}, h.ctx);
|
await h.handlers.get("session_start")?.({}, h.ctx);
|
||||||
const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" };
|
const prompt = { requestId: "perm-secret", surface: "bash", value: "cat private-file" };
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { afterEach, describe, it } from "node:test";
|
||||||
|
import { readNotifySettings } from "../src/settings.ts";
|
||||||
|
|
||||||
|
let tempDir: string | undefined;
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
if (tempDir) {
|
||||||
|
rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
tempDir = undefined;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
function configPath(contents: string): string {
|
||||||
|
tempDir = mkdtempSync(join(tmpdir(), "pi-notify-settings-"));
|
||||||
|
const path = join(tempDir, "pi-notify.json");
|
||||||
|
writeFileSync(path, contents);
|
||||||
|
return path;
|
||||||
|
}
|
||||||
|
|
||||||
|
const expectedDefaults = {
|
||||||
|
enabled: true,
|
||||||
|
title: "π - {project}",
|
||||||
|
body: "{message}",
|
||||||
|
messageSource: "assistant",
|
||||||
|
messageMax: 80,
|
||||||
|
minDurationMs: 3_000,
|
||||||
|
visibility: "always",
|
||||||
|
action: "focus",
|
||||||
|
soundCommand: undefined,
|
||||||
|
};
|
||||||
|
|
||||||
|
describe("readNotifySettings", () => {
|
||||||
|
it("returns all defaults when the config file is missing", () => {
|
||||||
|
assert.deepEqual(readNotifySettings("/does/not/exist/pi-notify.json"), expectedDefaults);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns all defaults for malformed or non-object JSON", () => {
|
||||||
|
for (const contents of ["not json", "[1,2]", "\"hi\"", "42"]) {
|
||||||
|
assert.deepEqual(readNotifySettings(configPath(contents)), expectedDefaults, contents);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("adopts valid values from the config file", () => {
|
||||||
|
const settings = readNotifySettings(configPath(JSON.stringify({
|
||||||
|
enabled: false,
|
||||||
|
title: "π {status_icon} · {project}",
|
||||||
|
body: "{status}",
|
||||||
|
messageSource: "none",
|
||||||
|
messageMax: 120,
|
||||||
|
minSeconds: 30,
|
||||||
|
visibility: "unfocused",
|
||||||
|
action: "none",
|
||||||
|
soundCommand: "afplay /done.aiff",
|
||||||
|
})));
|
||||||
|
|
||||||
|
assert.deepEqual(settings, {
|
||||||
|
enabled: false,
|
||||||
|
title: "π {status_icon} · {project}",
|
||||||
|
body: "{status}",
|
||||||
|
messageSource: "none",
|
||||||
|
messageMax: 120,
|
||||||
|
minDurationMs: 30_000,
|
||||||
|
visibility: "unfocused",
|
||||||
|
action: "none",
|
||||||
|
soundCommand: "afplay /done.aiff",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back per-field when values have the wrong type or value", () => {
|
||||||
|
const settings = readNotifySettings(configPath(JSON.stringify({
|
||||||
|
enabled: {},
|
||||||
|
title: "",
|
||||||
|
body: null,
|
||||||
|
messageSource: ["assistant"],
|
||||||
|
messageMax: "bogus",
|
||||||
|
minSeconds: "bogus",
|
||||||
|
visibility: "bogus",
|
||||||
|
action: {},
|
||||||
|
})));
|
||||||
|
|
||||||
|
assert.deepEqual(settings, expectedDefaults);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("clamps messageMax and minSeconds to their bounded ranges", () => {
|
||||||
|
const settings = readNotifySettings(configPath(JSON.stringify({ messageMax: 999, minSeconds: 4000 })));
|
||||||
|
assert.equal(settings.messageMax, 500);
|
||||||
|
assert.equal(settings.minDurationMs, 3_600_000);
|
||||||
|
|
||||||
|
const floor = readNotifySettings(configPath(JSON.stringify({ messageMax: -5, minSeconds: -1 })));
|
||||||
|
assert.equal(floor.messageMax, 0);
|
||||||
|
assert.equal(floor.minDurationMs, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("treats an empty or whitespace soundCommand as absent and trims values", () => {
|
||||||
|
for (const value of ["", " "]) {
|
||||||
|
assert.equal(readNotifySettings(configPath(JSON.stringify({ soundCommand: value }))).soundCommand, undefined);
|
||||||
|
}
|
||||||
|
assert.equal(
|
||||||
|
readNotifySettings(configPath(JSON.stringify({ soundCommand: " afplay /done.aiff " }))).soundCommand,
|
||||||
|
"afplay /done.aiff",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# my-pi 维护约定(pi-overflow)
|
||||||
|
|
||||||
|
- 完全自维护(无上游)的新结果展示截断扩展,负责"每个新 tool_result 给模型看多少":`capture.ts` 最先加载只暂存原文,`index.ts` 最后加载(在 `system-prompt.ts` 之前)生成有界视图并注册 `overflow_query`。
|
||||||
|
- 根 `pi.extensions` 顺序是硬约束:`./pi-overflow/capture.ts` 必须第一,`./pi-overflow/index.ts` 必须在所有其他扩展之后、`./extensions/system-prompt.ts` 之前;跨扩展暂存走 `globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]`,容量 64 条、淘汰最旧。
|
||||||
|
- `tool_result` handler 链中 capture 不得修改结果;index 改写 content 时必须原样回传 `structuredContent`、保留 image 部分、合并而非丢弃已有 `details`(如 RTK 压缩元数据、bash `fullOutputPath`)。
|
||||||
|
- `read`/`edit`/`write`/`overflow_query`/`ask_user_question` 与嵌套调用(`parentToolCallId`)永远豁免;read 靠自身 offset/limit 分页,不得破坏 hashline 锚点。
|
||||||
|
- 视图与行号一律基于刚落盘的存储原文(不是 current 文本):文件来源用流式逐行扫描收集 head/环形 tail/signals,文本来源切同构片段;视图必须确定性、不超 `thresholdChars`、结尾永远可见;落盘原文优先取 `details.fullOutputPath` 文件(超 `maxEntryBytes` 只留头尾各半),其次暂存原文,再次当前文本。
|
||||||
|
- 超预算收缩优先级为 tail > signals > head:head 先缩到 min(10, headLines),signals 逐步减半至 10 再到 0,再按约 35/65 缩 head/tail,按字符截断兜底。
|
||||||
|
- signal 扫描与 `overflow_query` 的 search 正则只匹配每行前 10,000 字符;query 输出每行按 `maxLineChars` 截断。
|
||||||
|
- 存储位于 `<agentDir>/my-pi/pi-overflow/`(目录 `700`、文件 `600`、原子写入),id 为 `sha1(sessionId + "\0" + toolCallId)` 前 12 位 hex;`overflow_query` 的 id 必须匹配 `^[0-9a-f]{12}$` 防路径穿越;`session_start` 按 `retentionDays`/`maxStoreBytes` 清理并删除超过 1 小时的 `*.tmp-*` 残留,失败不得抛出。
|
||||||
|
- 持久配置只读组合包 `config/pi-overflow.json`(`settings.ts` 逐字段校验、非法回退默认、signalPattern 编译失败回退默认且不敏感大小写);不得读写 `settings.json` 或以环境变量作持久默认。
|
||||||
|
- `overflow_query` 只读本扩展自己的存储目录;非法 id/正则/JSON/路径一律返回清楚错误文本,不得抛出。
|
||||||
|
- 与 pi-compaction 的唯一联动是被遮蔽 toolResult 的 `details.overflow.id` 公开字段——pi-compaction 只读它、不得 import 本包任何实现。
|
||||||
|
- 测试必须注入临时存储目录,不得写真实 agent 目录。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
- 在本目录内至少运行 `npm test` 与 `npm run check`;涉及加载入口、`pi.extensions` 顺序或 `files` 白名单时还要运行根 `node --test tests/extension-load.test.ts` 与 `tests/package-files.test.ts`。
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 sia
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
# pi-overflow
|
||||||
|
|
||||||
|
Bounded display for oversized tool results. When a tool result exceeds
|
||||||
|
`thresholdChars`, pi-overflow persists the full original to disk and shows the
|
||||||
|
model a deterministic view — head lines, signal lines (error/fail/warn/…), and
|
||||||
|
tail lines — plus an `overflow_query` tool to search or read the stored
|
||||||
|
original by id.
|
||||||
|
|
||||||
|
## Two-stage load order
|
||||||
|
|
||||||
|
- `capture.ts` — **first** in `pi.extensions`. A `tool_result` handler that
|
||||||
|
never modifies anything; it stashes the untouched original text in
|
||||||
|
`globalThis[Symbol.for("my-pi.pi-overflow.stash.v1")]` (capacity 64) so the
|
||||||
|
bound stage can still reach the pre-RTK original.
|
||||||
|
- `index.ts` — **last** content extension, right before `system-prompt.ts`.
|
||||||
|
Its `tool_result` handler sees the final text after every other handler
|
||||||
|
(RTK semantic compaction included) has run, then:
|
||||||
|
- resolves the *original*: `details.fullOutputPath` file (streamed, capped
|
||||||
|
at `maxEntryBytes` head+tail) → stashed raw text → current text;
|
||||||
|
- if the current text still exceeds `thresholdChars`, replaces it with the
|
||||||
|
bounded view **built from the stored file** — line numbers and totals in
|
||||||
|
the view are exactly what `overflow_query` will read, even when RTK or
|
||||||
|
bash already cut the in-context text (file sources are scanned
|
||||||
|
line-by-line in a stream, never held in memory);
|
||||||
|
- if the current text fits but the original did not (an earlier extension
|
||||||
|
already cut it), keeps the text and appends a one-line
|
||||||
|
`[pi-overflow id=…]` recovery marker;
|
||||||
|
- preserves image parts and `structuredContent`, and merges
|
||||||
|
`details.overflow = { id, totalLines, totalBytes, truncated }` into the
|
||||||
|
existing details.
|
||||||
|
- Exempt tools (`read`, `edit`, `write`, `overflow_query`,
|
||||||
|
`ask_user_question`) and nested calls (`parentToolCallId`) are never
|
||||||
|
touched.
|
||||||
|
|
||||||
|
## Stored originals
|
||||||
|
|
||||||
|
`<agentDir>/my-pi/pi-overflow/<id>.txt` + `<id>.json` metadata, where
|
||||||
|
`id = sha1(sessionId + "\0" + toolCallId)[0:12]`. Directories are `0700`,
|
||||||
|
files `0600`, writes are atomic. `session_start` cleans entries older than
|
||||||
|
`retentionDays` and evicts oldest-first past `maxStoreBytes`.
|
||||||
|
|
||||||
|
## `overflow_query`
|
||||||
|
|
||||||
|
- `id` (required, `^[0-9a-f]{12}$` — path traversal safe)
|
||||||
|
- `search` — case-insensitive regex; hits come back with `context` lines and
|
||||||
|
adjacent hits merge; capped by `maxResults`
|
||||||
|
- `lines: { start, end }` — 1-based inclusive line range
|
||||||
|
- `jsonPath` — dot/bracket path into stored JSON (`a.b[3].c`)
|
||||||
|
- No selector → metadata + first-screen view. Output is always bounded by
|
||||||
|
`thresholdChars` and reports whether it was truncated.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
`config/pi-overflow.json` inside the installed bundle. Every field is
|
||||||
|
validated independently and falls back to its default; an uncompilable
|
||||||
|
`signalPattern` falls back to the default pattern (case-insensitive).
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm test # node --test test/*.test.ts
|
||||||
|
npm run check # strip-types syntax check + tests
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests inject a temporary store root — they never write to a real agent
|
||||||
|
directory.
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
# 上游来源
|
||||||
|
|
||||||
|
- 本项目完全自维护,无上游;未从任何外部仓库或 npm 包导入源码。
|
||||||
|
- 实现仅依赖 Pi 扩展宿主 API(`@earendil-works/pi-coding-agent`)与 `typebox`,两者均为组合包 peer 依赖。
|
||||||
|
|
||||||
|
## 本地改动记录
|
||||||
|
|
||||||
|
- 初始实现:`capture.ts`(首个加载,暂存原始 tool_result 文本)、`index.ts`(最后加载,生成有界视图 + 注册 `overflow_query`)、`src/`(视图、存储、查询、跨扩展暂存区)。
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { readOverflowSettings } from "./settings.ts";
|
||||||
|
import { getStash, stashResult } from "./src/stash.ts";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pre-bound capture: loaded FIRST in pi.extensions, before RTK or any other
|
||||||
|
* result-rewriting extension. It only stashes the untouched original text —
|
||||||
|
* it never modifies the result. The bound stage (`index.ts`, loaded last)
|
||||||
|
* consumes the stash after all other handlers have run.
|
||||||
|
*/
|
||||||
|
export default function overflowCapture(pi: ExtensionAPI): void {
|
||||||
|
const settings = readOverflowSettings();
|
||||||
|
pi.on("tool_result", (event) => {
|
||||||
|
if (!settings.enabled) return undefined;
|
||||||
|
if (settings.exemptTools.includes(event.toolName)) return undefined;
|
||||||
|
if (event.parentToolCallId) return undefined;
|
||||||
|
const rawText = event.content
|
||||||
|
.filter((part): part is { type: "text"; text: string } => part.type === "text")
|
||||||
|
.map((part) => part.text)
|
||||||
|
.join("\n");
|
||||||
|
const fullOutputPath =
|
||||||
|
typeof event.details?.fullOutputPath === "string" ? event.details.fullOutputPath : undefined;
|
||||||
|
if (rawText.length <= settings.thresholdChars && !fullOutputPath) return undefined;
|
||||||
|
stashResult(getStash(), event.toolCallId, {
|
||||||
|
rawText,
|
||||||
|
...(fullOutputPath ? { fullOutputPath } : {}),
|
||||||
|
});
|
||||||
|
return undefined;
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
import { getAgentDir, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { Type } from "typebox";
|
||||||
|
import { stat } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { readOverflowSettings, type OverflowSettings } from "./settings.ts";
|
||||||
|
import { getStash } from "./src/stash.ts";
|
||||||
|
import { buildView, collectViewFromFile, collectViewFromText } from "./src/view.ts";
|
||||||
|
import { cleanStore, entryId, storeEntry } from "./src/store.ts";
|
||||||
|
import { runOverflowQuery } from "./src/query.ts";
|
||||||
|
|
||||||
|
function formatBytes(bytes: number): string {
|
||||||
|
if (bytes < 1024) return `${bytes} B`;
|
||||||
|
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||||
|
return `${(bytes / 1_048_576).toFixed(1)} MB`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface OverflowExtensionOptions {
|
||||||
|
settings?: OverflowSettings;
|
||||||
|
storeRoot?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bound stage: loaded LAST among content extensions (before system-prompt).
|
||||||
|
* By the time this handler runs, every other tool_result handler — RTK
|
||||||
|
* included — has already transformed the result, so the bounded view is
|
||||||
|
* built from the final visible text while the stored original comes from
|
||||||
|
* capture.ts's stash or bash's fullOutputPath file.
|
||||||
|
*/
|
||||||
|
export default function overflowExtension(pi: ExtensionAPI, options: OverflowExtensionOptions = {}): void {
|
||||||
|
const settings = options.settings ?? readOverflowSettings();
|
||||||
|
const storeRoot = () => options.storeRoot ?? join(getAgentDir(), "my-pi", "pi-overflow");
|
||||||
|
|
||||||
|
pi.on("session_start", () => {
|
||||||
|
// Retention/size cleanup never throws (cleanStore swallows failures)
|
||||||
|
// and the returned promise lets tests await it.
|
||||||
|
return cleanStore(storeRoot(), settings.retentionDays, settings.maxStoreBytes);
|
||||||
|
});
|
||||||
|
|
||||||
|
pi.on("tool_result", async (event, ctx) => {
|
||||||
|
const stash = getStash();
|
||||||
|
const staged = stash.get(event.toolCallId);
|
||||||
|
try {
|
||||||
|
if (
|
||||||
|
!settings.enabled ||
|
||||||
|
settings.exemptTools.includes(event.toolName) ||
|
||||||
|
event.parentToolCallId
|
||||||
|
) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
const current = event.content
|
||||||
|
.filter((part): part is { type: "text"; text: string } => part.type === "text")
|
||||||
|
.map((part) => part.text)
|
||||||
|
.join("\n");
|
||||||
|
const fullOutputPath =
|
||||||
|
typeof event.details?.fullOutputPath === "string"
|
||||||
|
? event.details.fullOutputPath
|
||||||
|
: staged?.fullOutputPath;
|
||||||
|
|
||||||
|
// Resolve the original: bash's spilled file wins (it is complete even
|
||||||
|
// when the in-context text was already cut), then the capture stash,
|
||||||
|
// then the current text itself.
|
||||||
|
let source: { kind: "text"; text: string } | { kind: "file"; path: string };
|
||||||
|
let originalSize: number;
|
||||||
|
if (fullOutputPath) {
|
||||||
|
try {
|
||||||
|
const info = await stat(fullOutputPath);
|
||||||
|
if (!info.isFile()) throw new Error("not a file");
|
||||||
|
source = { kind: "file", path: fullOutputPath };
|
||||||
|
originalSize = info.size;
|
||||||
|
} catch {
|
||||||
|
const fallback = staged?.rawText ?? current;
|
||||||
|
source = { kind: "text", text: fallback };
|
||||||
|
originalSize = fallback.length;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
const fallback = staged?.rawText ?? current;
|
||||||
|
source = { kind: "text", text: fallback };
|
||||||
|
originalSize = fallback.length;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (current.length <= settings.thresholdChars && originalSize <= settings.thresholdChars) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
const sessionId = ctx.sessionManager?.getSessionId?.() ?? "session";
|
||||||
|
const id = entryId(sessionId, event.toolCallId);
|
||||||
|
const stored = await storeEntry(
|
||||||
|
storeRoot(),
|
||||||
|
id,
|
||||||
|
source,
|
||||||
|
{ tool: event.toolName, args: event.input },
|
||||||
|
settings.maxEntryBytes,
|
||||||
|
);
|
||||||
|
const overflow = {
|
||||||
|
id,
|
||||||
|
totalLines: stored.totalLines,
|
||||||
|
totalBytes: stored.totalBytes,
|
||||||
|
truncated: current.length > settings.thresholdChars,
|
||||||
|
};
|
||||||
|
|
||||||
|
const images = event.content.filter((part) => part.type !== "text");
|
||||||
|
let text: string;
|
||||||
|
if (current.length > settings.thresholdChars) {
|
||||||
|
// The view always describes the stored file — its line numbers and
|
||||||
|
// totals must match what overflow_query reads, which is not true
|
||||||
|
// of `current` when RTK rewrote or bash pre-truncated the result.
|
||||||
|
const viewInput =
|
||||||
|
stored.storedText !== undefined
|
||||||
|
? collectViewFromText(stored.storedText, settings)
|
||||||
|
: await collectViewFromFile(join(storeRoot(), `${id}.txt`), settings);
|
||||||
|
text = buildView(viewInput, settings, { id, tool: event.toolName });
|
||||||
|
} else {
|
||||||
|
text = `${current}\n[pi-overflow id=${id}: full original output stored (${stored.totalLines} lines, ${formatBytes(stored.totalBytes)}); use overflow_query to inspect it]`;
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
content: [{ type: "text", text }, ...images],
|
||||||
|
structuredContent: event.structuredContent,
|
||||||
|
details: { ...event.details, overflow },
|
||||||
|
};
|
||||||
|
} finally {
|
||||||
|
stash.delete(event.toolCallId);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
pi.registerTool({
|
||||||
|
name: "overflow_query",
|
||||||
|
label: "Overflow Query",
|
||||||
|
description:
|
||||||
|
"Inspect the full original of a tool output that pi-overflow stored because it was too long to show " +
|
||||||
|
"(search with regex, read line ranges, or extract a JSON path).",
|
||||||
|
promptSnippet: "Inspect a stored oversized tool output by id: regex search, line ranges, or JSON path.",
|
||||||
|
promptGuidelines: [
|
||||||
|
"When a tool result carries a [pi-overflow id=…] marker, the model is seeing a bounded view; the complete original is stored on disk.",
|
||||||
|
"Call overflow_query with the marker's id and a regex, line range, or jsonPath instead of re-running the tool or asking for the full dump.",
|
||||||
|
],
|
||||||
|
parameters: Type.Object({
|
||||||
|
id: Type.String({ description: "12-hex storage id from a [pi-overflow id=…] marker." }),
|
||||||
|
search: Type.Optional(
|
||||||
|
Type.String({ description: "Case-insensitive regular expression matched per line." }),
|
||||||
|
),
|
||||||
|
context: Type.Optional(
|
||||||
|
Type.Number({ minimum: 0, maximum: 10, description: "Context lines around each search hit (default 2)." }),
|
||||||
|
),
|
||||||
|
lines: Type.Optional(
|
||||||
|
Type.Object({
|
||||||
|
start: Type.Number({ minimum: 1, description: "First line to return (1-based, inclusive)." }),
|
||||||
|
end: Type.Number({ minimum: 1, description: "Last line to return (inclusive)." }),
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
jsonPath: Type.Optional(
|
||||||
|
Type.String({ description: "Dot/bracket path into the stored JSON, e.g. snapshot.nodes[3].name." }),
|
||||||
|
),
|
||||||
|
maxResults: Type.Optional(
|
||||||
|
Type.Number({ minimum: 1, maximum: 200, description: "Maximum search hits to report (default 50)." }),
|
||||||
|
),
|
||||||
|
}),
|
||||||
|
async execute(_toolCallId, params) {
|
||||||
|
const result = await runOverflowQuery(storeRoot(), params, settings);
|
||||||
|
return {
|
||||||
|
content: [{ type: "text", text: result.text }],
|
||||||
|
details: { error: result.error === true, truncated: result.truncated },
|
||||||
|
isError: result.error === true,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
{
|
||||||
|
"name": "pi-overflow",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"description": "Bounded views for oversized tool results: store the full original, show head/signals/tail, recover with overflow_query.",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"license": "MIT",
|
||||||
|
"main": "./index.ts",
|
||||||
|
"exports": {
|
||||||
|
".": "./index.ts"
|
||||||
|
},
|
||||||
|
"files": [
|
||||||
|
"index.ts",
|
||||||
|
"capture.ts",
|
||||||
|
"settings.ts",
|
||||||
|
"src",
|
||||||
|
"README.md",
|
||||||
|
"LICENSE"
|
||||||
|
],
|
||||||
|
"pi": {
|
||||||
|
"extensions": [
|
||||||
|
"./capture.ts",
|
||||||
|
"./index.ts"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"test": "node --test test/*.test.ts",
|
||||||
|
"check": "node --experimental-strip-types --check index.ts && node --test test/*.test.ts"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"@earendil-works/pi-coding-agent": "*",
|
||||||
|
"typebox": "*"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"@earendil-works/pi-coding-agent": {
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"typebox": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@earendil-works/pi-coding-agent": "latest",
|
||||||
|
"typebox": "latest"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=22.19.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
|
||||||
|
export interface OverflowSettings {
|
||||||
|
enabled: boolean;
|
||||||
|
thresholdChars: number;
|
||||||
|
headLines: number;
|
||||||
|
tailLines: number;
|
||||||
|
maxSignalLines: number;
|
||||||
|
signalPattern: string;
|
||||||
|
signalRegex: RegExp;
|
||||||
|
maxLineChars: number;
|
||||||
|
exemptTools: string[];
|
||||||
|
retentionDays: number;
|
||||||
|
maxStoreBytes: number;
|
||||||
|
maxEntryBytes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const DEFAULT_SIGNAL_PATTERN =
|
||||||
|
"\\b(error|errors|fail|failed|failure|warn|warning|exception|traceback|panic|fatal)\\b";
|
||||||
|
|
||||||
|
export const OVERFLOW_DEFAULTS = {
|
||||||
|
enabled: true,
|
||||||
|
thresholdChars: 24_000,
|
||||||
|
headLines: 40,
|
||||||
|
tailLines: 120,
|
||||||
|
maxSignalLines: 40,
|
||||||
|
signalPattern: DEFAULT_SIGNAL_PATTERN,
|
||||||
|
maxLineChars: 500,
|
||||||
|
exemptTools: ["read", "edit", "write", "overflow_query", "ask_user_question"],
|
||||||
|
retentionDays: 7,
|
||||||
|
maxStoreBytes: 1_073_741_824,
|
||||||
|
maxEntryBytes: 67_108_864,
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
// Bundle-level `config/` is the single source of truth: the extension reads
|
||||||
|
// `config/pi-overflow.json` from inside the installed bundle (resolved
|
||||||
|
// relative to this package-root source file), never `settings.json`.
|
||||||
|
export const OVERFLOW_CONFIG_PATH = fileURLToPath(
|
||||||
|
new URL("../config/pi-overflow.json", import.meta.url),
|
||||||
|
);
|
||||||
|
|
||||||
|
function isObject(value: unknown): value is Record<string, unknown> {
|
||||||
|
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
function bool(value: unknown, fallback: boolean): boolean {
|
||||||
|
return typeof value === "boolean" ? value : fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
function intInRange(value: unknown, fallback: number, min: number, max: number): number {
|
||||||
|
return typeof value === "number" && Number.isInteger(value) && value >= min && value <= max
|
||||||
|
? value
|
||||||
|
: fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
function compileSignalPattern(value: unknown): { pattern: string; regex: RegExp } {
|
||||||
|
const pattern = typeof value === "string" && value.length > 0 ? value : DEFAULT_SIGNAL_PATTERN;
|
||||||
|
try {
|
||||||
|
return { pattern, regex: new RegExp(pattern, "i") };
|
||||||
|
} catch {
|
||||||
|
return { pattern: DEFAULT_SIGNAL_PATTERN, regex: new RegExp(DEFAULT_SIGNAL_PATTERN, "i") };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function stringList(value: unknown, fallback: readonly string[]): string[] {
|
||||||
|
if (!Array.isArray(value)) return [...fallback];
|
||||||
|
const list = value.filter((item): item is string => typeof item === "string" && item.length > 0);
|
||||||
|
return [...new Set(list)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Read `config/pi-overflow.json`. Every field is validated independently and
|
||||||
|
* falls back to its bundle default when missing or invalid; an unreadable or
|
||||||
|
* non-object file yields all defaults. Tests inject an explicit path.
|
||||||
|
*/
|
||||||
|
export function readOverflowSettings(configPath: string = OVERFLOW_CONFIG_PATH): OverflowSettings {
|
||||||
|
let parsed: unknown = {};
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(readFileSync(configPath, "utf8"));
|
||||||
|
} catch {
|
||||||
|
parsed = {};
|
||||||
|
}
|
||||||
|
const config = isObject(parsed) ? parsed : {};
|
||||||
|
const signal = compileSignalPattern(config.signalPattern);
|
||||||
|
return {
|
||||||
|
enabled: bool(config.enabled, OVERFLOW_DEFAULTS.enabled),
|
||||||
|
thresholdChars: intInRange(config.thresholdChars, OVERFLOW_DEFAULTS.thresholdChars, 1, 10_000_000),
|
||||||
|
headLines: intInRange(config.headLines, OVERFLOW_DEFAULTS.headLines, 0, 10_000),
|
||||||
|
tailLines: intInRange(config.tailLines, OVERFLOW_DEFAULTS.tailLines, 0, 10_000),
|
||||||
|
maxSignalLines: intInRange(config.maxSignalLines, OVERFLOW_DEFAULTS.maxSignalLines, 0, 1_000),
|
||||||
|
signalPattern: signal.pattern,
|
||||||
|
signalRegex: signal.regex,
|
||||||
|
maxLineChars: intInRange(config.maxLineChars, OVERFLOW_DEFAULTS.maxLineChars, 10, 100_000),
|
||||||
|
exemptTools: stringList(config.exemptTools, OVERFLOW_DEFAULTS.exemptTools),
|
||||||
|
retentionDays: intInRange(config.retentionDays, OVERFLOW_DEFAULTS.retentionDays, 0, 3_650),
|
||||||
|
maxStoreBytes: intInRange(config.maxStoreBytes, OVERFLOW_DEFAULTS.maxStoreBytes, 1_024, 1_099_511_627_776),
|
||||||
|
maxEntryBytes: intInRange(config.maxEntryBytes, OVERFLOW_DEFAULTS.maxEntryBytes, 1_024, 1_099_511_627_776),
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,188 @@
|
|||||||
|
import { ID_PATTERN, readStoredEntry, type EntryMeta } from "./store.ts";
|
||||||
|
import type { OverflowSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
export interface QueryParams {
|
||||||
|
id: string;
|
||||||
|
search?: string;
|
||||||
|
context?: number;
|
||||||
|
lines?: { start: number; end: number };
|
||||||
|
jsonPath?: string;
|
||||||
|
maxResults?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface QueryResult {
|
||||||
|
text: string;
|
||||||
|
truncated: boolean;
|
||||||
|
error?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
function fail(text: string): QueryResult {
|
||||||
|
return { text, truncated: false, error: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
function boundOutput(text: string, limit: number): QueryResult {
|
||||||
|
if (text.length <= limit) return { text, truncated: false };
|
||||||
|
const marker = `\n[output truncated at ${limit} chars; narrow the query (smaller line range, tighter regex, or jsonPath)]`;
|
||||||
|
const keep = Math.max(0, limit - marker.length);
|
||||||
|
return { text: text.slice(0, keep) + marker, truncated: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
function metaLine(id: string, meta: EntryMeta | null): string {
|
||||||
|
if (!meta) return `[pi-overflow id=${id}]`;
|
||||||
|
return `[pi-overflow id=${id} tool=${meta.tool} ${meta.totalLines} lines / ${meta.totalBytes} bytes]`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Regexes only inspect this prefix of each stored line — a model-supplied
|
||||||
|
* pattern must not backtrack catastrophically across a megabyte-long line. */
|
||||||
|
const MATCH_SCAN_CHARS = 10_000;
|
||||||
|
|
||||||
|
function clipLine(line: string, maxLineChars: number): string {
|
||||||
|
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
|
||||||
|
}
|
||||||
|
|
||||||
|
function evalJsonPath(root: unknown, path: string): { ok: true; value: unknown } | { ok: false; error: string } {
|
||||||
|
const tokens: Array<string | number> = [];
|
||||||
|
const pattern = /(?:^|\.)([A-Za-z_$][A-Za-z0-9_$]*)|\[\s*(\d+)\s*\]|\[\s*["']([^"']+)["']\s*\]/g;
|
||||||
|
let index = 0;
|
||||||
|
let match: RegExpExecArray | null;
|
||||||
|
while ((match = pattern.exec(path)) !== null) {
|
||||||
|
if (match.index !== index) return { ok: false, error: `cannot parse jsonPath near "${path.slice(index)}"` };
|
||||||
|
index = pattern.lastIndex;
|
||||||
|
if (match[1] !== undefined) tokens.push(match[1]);
|
||||||
|
else if (match[2] !== undefined) tokens.push(Number(match[2]));
|
||||||
|
else tokens.push(match[3] as string);
|
||||||
|
}
|
||||||
|
if (index !== path.length || tokens.length === 0) {
|
||||||
|
return { ok: false, error: `cannot parse jsonPath "${path}"` };
|
||||||
|
}
|
||||||
|
let value: unknown = root;
|
||||||
|
for (const token of tokens) {
|
||||||
|
if (value === null || value === undefined) {
|
||||||
|
return { ok: false, error: `jsonPath stops at null/undefined before "${String(token)}"` };
|
||||||
|
}
|
||||||
|
if (typeof token === "number") {
|
||||||
|
if (!Array.isArray(value) || token >= value.length) {
|
||||||
|
return { ok: false, error: `index [${token}] out of range` };
|
||||||
|
}
|
||||||
|
value = value[token];
|
||||||
|
} else {
|
||||||
|
if (typeof value !== "object" || Array.isArray(value)) {
|
||||||
|
return { ok: false, error: `"${token}" is not a property of this value` };
|
||||||
|
}
|
||||||
|
const record = value as Record<string, unknown>;
|
||||||
|
if (!(token in record)) return { ok: false, error: `property "${token}" not found` };
|
||||||
|
value = record[token];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { ok: true, value };
|
||||||
|
}
|
||||||
|
|
||||||
|
function mergeRanges(hits: number[], context: number): Array<{ start: number; end: number }> {
|
||||||
|
const ranges: Array<{ start: number; end: number }> = [];
|
||||||
|
for (const hit of hits) {
|
||||||
|
const start = Math.max(1, hit - context);
|
||||||
|
const end = hit + context;
|
||||||
|
const last = ranges[ranges.length - 1];
|
||||||
|
if (last && start <= last.end + 1) {
|
||||||
|
last.end = Math.max(last.end, end);
|
||||||
|
} else {
|
||||||
|
ranges.push({ start, end });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ranges;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Execute an overflow_query against the store. Never throws on bad input —
|
||||||
|
* invalid ids, regexes, JSON, and paths all come back as clear error text.
|
||||||
|
*/
|
||||||
|
export async function runOverflowQuery(
|
||||||
|
root: string,
|
||||||
|
params: QueryParams,
|
||||||
|
settings: OverflowSettings,
|
||||||
|
): Promise<QueryResult> {
|
||||||
|
if (typeof params.id !== "string" || !ID_PATTERN.test(params.id)) {
|
||||||
|
return fail(`invalid overflow id "${String(params.id)}" — expected 12 lowercase hex chars from a [pi-overflow id=…] marker`);
|
||||||
|
}
|
||||||
|
let stored: { text: string; meta: EntryMeta | null };
|
||||||
|
try {
|
||||||
|
stored = await readStoredEntry(root, params.id);
|
||||||
|
} catch {
|
||||||
|
return fail(`no stored output for id ${params.id} (it may have been cleaned up or never stored)`);
|
||||||
|
}
|
||||||
|
const lines = stored.text.split("\n");
|
||||||
|
const head = metaLine(params.id, stored.meta);
|
||||||
|
const context = Math.min(Math.max(Math.floor(params.context ?? 2), 0), 10);
|
||||||
|
const maxResults = Math.min(Math.max(Math.floor(params.maxResults ?? 50), 1), 200);
|
||||||
|
|
||||||
|
if (typeof params.search === "string") {
|
||||||
|
let regex: RegExp;
|
||||||
|
try {
|
||||||
|
regex = new RegExp(params.search, "i");
|
||||||
|
} catch (error) {
|
||||||
|
return fail(`invalid search regex: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
|
}
|
||||||
|
const hits: number[] = [];
|
||||||
|
for (let index = 0; index < lines.length && hits.length < maxResults; index++) {
|
||||||
|
regex.lastIndex = 0;
|
||||||
|
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) hits.push(index + 1);
|
||||||
|
}
|
||||||
|
if (hits.length === 0) {
|
||||||
|
return boundOutput(`${head}\nno lines match /${params.search}/i`, settings.thresholdChars);
|
||||||
|
}
|
||||||
|
const parts = [head, `${hits.length}${hits.length >= maxResults ? "+" : ""} matching lines:`];
|
||||||
|
for (const range of mergeRanges(hits, context)) {
|
||||||
|
const end = Math.min(range.end, lines.length);
|
||||||
|
parts.push(`--- lines ${range.start}-${end} ---`);
|
||||||
|
for (let line = range.start; line <= end; line++) {
|
||||||
|
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return boundOutput(parts.join("\n"), settings.thresholdChars);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (params.lines && typeof params.lines === "object") {
|
||||||
|
const start = Math.floor(params.lines.start);
|
||||||
|
const end = Math.floor(params.lines.end);
|
||||||
|
if (!Number.isFinite(start) || !Number.isFinite(end) || start < 1 || end < start) {
|
||||||
|
return fail(`invalid line range {start: ${params.lines.start}, end: ${params.lines.end}} — need 1 <= start <= end`);
|
||||||
|
}
|
||||||
|
const clippedEnd = Math.min(end, lines.length);
|
||||||
|
if (start > lines.length) {
|
||||||
|
return fail(`start ${start} is beyond the stored ${lines.length} lines`);
|
||||||
|
}
|
||||||
|
const parts = [head, `--- lines ${start}-${clippedEnd} ---`];
|
||||||
|
for (let line = start; line <= clippedEnd; line++) {
|
||||||
|
parts.push(`L${line}: ${clipLine(lines[line - 1], settings.maxLineChars)}`);
|
||||||
|
}
|
||||||
|
return boundOutput(parts.join("\n"), settings.thresholdChars);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (typeof params.jsonPath === "string") {
|
||||||
|
let parsed: unknown;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(stored.text);
|
||||||
|
} catch (error) {
|
||||||
|
return fail(`stored output is not valid JSON: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
|
}
|
||||||
|
const result = evalJsonPath(parsed, params.jsonPath);
|
||||||
|
if (!result.ok) return fail(`jsonPath error: ${result.error}`);
|
||||||
|
let rendered: string;
|
||||||
|
try {
|
||||||
|
rendered = JSON.stringify(result.value, null, 2) ?? "undefined";
|
||||||
|
} catch {
|
||||||
|
rendered = String(result.value);
|
||||||
|
}
|
||||||
|
return boundOutput(`${head}\njsonPath ${params.jsonPath}:\n${rendered}`, settings.thresholdChars);
|
||||||
|
}
|
||||||
|
|
||||||
|
// No selector: metadata plus a bounded head/tail preview.
|
||||||
|
const headCount = Math.min(settings.headLines, lines.length);
|
||||||
|
const tailCount = Math.min(settings.tailLines, Math.max(0, lines.length - headCount));
|
||||||
|
const parts = [head, `--- lines 1-${headCount} ---`, ...lines.slice(0, headCount)];
|
||||||
|
if (tailCount > 0) {
|
||||||
|
parts.push(`… ${lines.length - headCount - tailCount} lines omitted …`);
|
||||||
|
parts.push(`--- lines ${lines.length - tailCount + 1}-${lines.length} ---`, ...lines.slice(lines.length - tailCount));
|
||||||
|
}
|
||||||
|
return boundOutput(parts.join("\n"), settings.thresholdChars);
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
/**
|
||||||
|
* Cross-extension staging for pre-bound originals. `capture.ts` (loaded
|
||||||
|
* first) stashes the raw tool_result text; `index.ts` (loaded last) consumes
|
||||||
|
* it after every other handler — RTK in particular — has run. Sharing goes
|
||||||
|
* through `globalThis[Symbol.for(...)]`, the same mechanism pi-compaction
|
||||||
|
* uses for its metrics service.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export interface StashedResult {
|
||||||
|
rawText: string;
|
||||||
|
fullOutputPath?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const STASH_KEY = Symbol.for("my-pi.pi-overflow.stash.v1");
|
||||||
|
const STASH_LIMIT = 64;
|
||||||
|
|
||||||
|
export function getStash(): Map<string, StashedResult> {
|
||||||
|
const scope = globalThis as Record<symbol, unknown>;
|
||||||
|
let stash = scope[STASH_KEY] as Map<string, StashedResult> | undefined;
|
||||||
|
if (!(stash instanceof Map)) {
|
||||||
|
stash = new Map<string, StashedResult>();
|
||||||
|
scope[STASH_KEY] = stash;
|
||||||
|
}
|
||||||
|
return stash;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function stashResult(stash: Map<string, StashedResult>, toolCallId: string, entry: StashedResult): void {
|
||||||
|
stash.delete(toolCallId);
|
||||||
|
stash.set(toolCallId, entry);
|
||||||
|
while (stash.size > STASH_LIMIT) {
|
||||||
|
const oldest = stash.keys().next().value;
|
||||||
|
if (oldest === undefined) break;
|
||||||
|
stash.delete(oldest);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,267 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { createReadStream, createWriteStream, type WriteStream } from "node:fs";
|
||||||
|
import { chmod, mkdir, open, readFile, readdir, rename, stat, unlink, writeFile } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { pipeline } from "node:stream/promises";
|
||||||
|
|
||||||
|
export interface EntryMeta {
|
||||||
|
tool: string;
|
||||||
|
argsSummary: string;
|
||||||
|
totalLines: number;
|
||||||
|
totalBytes: number;
|
||||||
|
createdAt: string;
|
||||||
|
omittedMiddle?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface StoredEntry {
|
||||||
|
id: string;
|
||||||
|
totalLines: number;
|
||||||
|
totalBytes: number;
|
||||||
|
/** Present when the source was text: the exact content written to disk. */
|
||||||
|
storedText?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const ID_PATTERN = /^[0-9a-f]{12}$/;
|
||||||
|
|
||||||
|
const TMP_MAX_AGE_MS = 3_600_000;
|
||||||
|
|
||||||
|
export function entryId(sessionId: string, toolCallId: string): string {
|
||||||
|
return createHash("sha1").update(`${sessionId}\0${toolCallId}`).digest("hex").slice(0, 12);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function ensureDir(dir: string): Promise<void> {
|
||||||
|
await mkdir(dir, { recursive: true, mode: 0o700 });
|
||||||
|
try {
|
||||||
|
await chmod(dir, 0o700);
|
||||||
|
} catch {
|
||||||
|
// Best-effort permission tightening only.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function tmpPath(path: string): string {
|
||||||
|
return `${path}.tmp-${process.pid}-${Math.random().toString(36).slice(2)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function writeFileAtomic(path: string, content: string | Buffer): Promise<void> {
|
||||||
|
const tmp = tmpPath(path);
|
||||||
|
await writeFile(tmp, content, { mode: 0o600 });
|
||||||
|
await rename(tmp, path);
|
||||||
|
}
|
||||||
|
|
||||||
|
function countNewlines(text: string): number {
|
||||||
|
let count = 0;
|
||||||
|
for (let index = 0; index < text.length; index++) {
|
||||||
|
if (text.charCodeAt(index) === 10) count++;
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function lineCount(text: string): number {
|
||||||
|
if (text.length === 0) return 0;
|
||||||
|
return countNewlines(text) + (text.endsWith("\n") ? 0 : 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
function truncateSummary(value: string, maxChars = 300): string {
|
||||||
|
return value.length > maxChars ? `${value.slice(0, maxChars)}…` : value;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function argsSummary(input: unknown): string {
|
||||||
|
try {
|
||||||
|
return truncateSummary(JSON.stringify(input) ?? "undefined");
|
||||||
|
} catch {
|
||||||
|
return "[unserializable]";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Copy `sourcePath` into `<root>/<id>.txt` without holding it all in memory.
|
||||||
|
* Files larger than `maxEntryBytes` keep the first and last halves with an
|
||||||
|
* omission marker between them.
|
||||||
|
*/
|
||||||
|
async function copyFileBounded(
|
||||||
|
sourcePath: string,
|
||||||
|
destPath: string,
|
||||||
|
maxEntryBytes: number,
|
||||||
|
): Promise<{ bytes: number; omittedMiddle: boolean }> {
|
||||||
|
const info = await stat(sourcePath);
|
||||||
|
if (!info.isFile()) throw new Error("full output path is not a file");
|
||||||
|
const size = info.size;
|
||||||
|
if (size <= maxEntryBytes) {
|
||||||
|
const tmp = tmpPath(destPath);
|
||||||
|
const out: WriteStream = createWriteStream(tmp, { mode: 0o600 });
|
||||||
|
await pipeline(createReadStream(sourcePath), out);
|
||||||
|
await rename(tmp, destPath);
|
||||||
|
return { bytes: size, omittedMiddle: false };
|
||||||
|
}
|
||||||
|
const half = Math.floor(maxEntryBytes / 2);
|
||||||
|
const omitted = size - half * 2;
|
||||||
|
const source = await open(sourcePath, "r");
|
||||||
|
try {
|
||||||
|
const head = Buffer.alloc(half);
|
||||||
|
const tail = Buffer.alloc(half);
|
||||||
|
await source.read(head, 0, half, 0);
|
||||||
|
await source.read(tail, 0, half, size - half);
|
||||||
|
const marker = Buffer.from(`\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n`);
|
||||||
|
await writeFileAtomic(destPath, Buffer.concat([head, marker, tail]));
|
||||||
|
} finally {
|
||||||
|
await source.close();
|
||||||
|
}
|
||||||
|
return { bytes: size, omittedMiddle: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function countFileLines(path: string): Promise<number> {
|
||||||
|
let count = 0;
|
||||||
|
let lastByte = -1;
|
||||||
|
let seen = false;
|
||||||
|
for await (const chunk of createReadStream(path)) {
|
||||||
|
seen = true;
|
||||||
|
const buffer = chunk as Buffer;
|
||||||
|
for (let index = 0; index < buffer.length; index++) {
|
||||||
|
if (buffer[index] === 10) count++;
|
||||||
|
}
|
||||||
|
if (buffer.length > 0) lastByte = buffer[buffer.length - 1];
|
||||||
|
}
|
||||||
|
if (!seen) return 0;
|
||||||
|
return count + (lastByte === 10 ? 0 : 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Persist an original tool output under `<root>/<id>.txt` plus `<id>.json`
|
||||||
|
* metadata. Directories are 0700, files 0600, writes are atomic. The caller
|
||||||
|
* chooses `root` (defaults come from getAgentDir() in index.ts; tests inject
|
||||||
|
* a temp directory).
|
||||||
|
*/
|
||||||
|
export async function storeEntry(
|
||||||
|
root: string,
|
||||||
|
id: string,
|
||||||
|
source: { kind: "text"; text: string } | { kind: "file"; path: string },
|
||||||
|
meta: { tool: string; args: unknown },
|
||||||
|
maxEntryBytes: number,
|
||||||
|
): Promise<StoredEntry> {
|
||||||
|
await ensureDir(root);
|
||||||
|
const textPath = join(root, `${id}.txt`);
|
||||||
|
const metaPath = join(root, `${id}.json`);
|
||||||
|
let totalLines = 0;
|
||||||
|
let totalBytes = 0;
|
||||||
|
let omittedMiddle = false;
|
||||||
|
let storedText: string | undefined;
|
||||||
|
if (source.kind === "text") {
|
||||||
|
let text = source.text;
|
||||||
|
const bytes = Buffer.byteLength(text, "utf8");
|
||||||
|
totalBytes = bytes;
|
||||||
|
if (bytes > maxEntryBytes) {
|
||||||
|
const halfChars = Math.floor(maxEntryBytes / 2);
|
||||||
|
const omitted = bytes - halfChars * 2;
|
||||||
|
text = `${text.slice(0, halfChars)}\n… ${omitted} bytes omitted (exceeded ${maxEntryBytes}-byte store limit) …\n${text.slice(text.length - halfChars)}`;
|
||||||
|
omittedMiddle = true;
|
||||||
|
}
|
||||||
|
totalLines = lineCount(text);
|
||||||
|
storedText = text;
|
||||||
|
await writeFileAtomic(textPath, text);
|
||||||
|
} else {
|
||||||
|
const copied = await copyFileBounded(source.path, textPath, maxEntryBytes);
|
||||||
|
totalBytes = copied.bytes;
|
||||||
|
omittedMiddle = copied.omittedMiddle;
|
||||||
|
totalLines = await countFileLines(textPath).catch(() => 0);
|
||||||
|
}
|
||||||
|
const entry: EntryMeta = {
|
||||||
|
tool: meta.tool,
|
||||||
|
argsSummary: argsSummary(meta.args),
|
||||||
|
totalLines,
|
||||||
|
totalBytes,
|
||||||
|
createdAt: new Date().toISOString(),
|
||||||
|
...(omittedMiddle ? { omittedMiddle: true } : {}),
|
||||||
|
};
|
||||||
|
await writeFileAtomic(metaPath, `${JSON.stringify(entry, null, 2)}\n`);
|
||||||
|
return { id, totalLines, totalBytes, ...(storedText !== undefined ? { storedText } : {}) };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function readStoredEntry(root: string, id: string): Promise<{ text: string; meta: EntryMeta | null }> {
|
||||||
|
if (!ID_PATTERN.test(id)) throw new Error(`invalid overflow id: ${id}`);
|
||||||
|
const textPath = join(root, `${id}.txt`);
|
||||||
|
const metaPath = join(root, `${id}.json`);
|
||||||
|
const text = await readFile(textPath, "utf8");
|
||||||
|
let meta: EntryMeta | null = null;
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(await readFile(metaPath, "utf8"));
|
||||||
|
if (parsed && typeof parsed === "object") meta = parsed as EntryMeta;
|
||||||
|
} catch {
|
||||||
|
meta = null;
|
||||||
|
}
|
||||||
|
return { text, meta };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete tmp files left by interrupted writes (older than one hour), entries
|
||||||
|
* older than `retentionDays`, then evict oldest-first until the store is
|
||||||
|
* under `maxStoreBytes`. All failures are swallowed — cleanup must never
|
||||||
|
* break a session.
|
||||||
|
*/
|
||||||
|
export async function cleanStore(root: string, retentionDays: number, maxStoreBytes: number): Promise<void> {
|
||||||
|
try {
|
||||||
|
const now = Date.now();
|
||||||
|
const cutoff = now - retentionDays * 86_400_000;
|
||||||
|
const names = await readdir(root);
|
||||||
|
// Stale tmp files from interrupted writes never become entries; remove
|
||||||
|
// them once they are clearly abandoned.
|
||||||
|
for (const name of names) {
|
||||||
|
if (!name.includes(".tmp-")) continue;
|
||||||
|
try {
|
||||||
|
const info = await stat(join(root, name));
|
||||||
|
if (info.isFile() && info.mtimeMs < now - TMP_MAX_AGE_MS) {
|
||||||
|
await unlink(join(root, name));
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Already gone or unreadable — nothing to do.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const ids = new Set(
|
||||||
|
names
|
||||||
|
.filter((name) => !name.includes(".tmp-"))
|
||||||
|
.map((name) => name.replace(/\.(txt|json)$/, "")),
|
||||||
|
);
|
||||||
|
const entries: Array<{ id: string; bytes: number; mtime: number }> = [];
|
||||||
|
for (const id of ids) {
|
||||||
|
let bytes = 0;
|
||||||
|
let mtime = 0;
|
||||||
|
let seen = false;
|
||||||
|
for (const suffix of ["txt", "json"] as const) {
|
||||||
|
const file = join(root, `${id}.${suffix}`);
|
||||||
|
try {
|
||||||
|
const info = await stat(file);
|
||||||
|
bytes += info.size;
|
||||||
|
mtime = Math.max(mtime, info.mtimeMs);
|
||||||
|
seen = true;
|
||||||
|
} catch {
|
||||||
|
// Missing half of a pair still counts toward cleanup.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (seen) entries.push({ id, bytes, mtime });
|
||||||
|
}
|
||||||
|
const remove = async (id: string) => {
|
||||||
|
for (const suffix of ["txt", "json"] as const) {
|
||||||
|
await unlink(join(root, `${id}.${suffix}`)).catch(() => {});
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let total = 0;
|
||||||
|
const survivors: typeof entries = [];
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (entry.mtime < cutoff) {
|
||||||
|
await remove(entry.id);
|
||||||
|
} else {
|
||||||
|
total += entry.bytes;
|
||||||
|
survivors.push(entry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (total > maxStoreBytes) {
|
||||||
|
survivors.sort((a, b) => a.mtime - b.mtime);
|
||||||
|
for (const entry of survivors) {
|
||||||
|
if (total <= maxStoreBytes) break;
|
||||||
|
await remove(entry.id);
|
||||||
|
total -= entry.bytes;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Cleanup failures never propagate.
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { open, stat } from "node:fs/promises";
|
||||||
|
import type { OverflowSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
export interface ViewMeta {
|
||||||
|
id: string;
|
||||||
|
tool: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SignalHit {
|
||||||
|
line: number;
|
||||||
|
text: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pre-collected view segments of the STORED original (never the current —
|
||||||
|
* possibly RTK- or bash-truncated — text). Line numbers here are the stored
|
||||||
|
* file's line numbers, which is exactly what overflow_query reads.
|
||||||
|
*/
|
||||||
|
export interface ViewInput {
|
||||||
|
head: string[];
|
||||||
|
signals: SignalHit[];
|
||||||
|
tail: string[];
|
||||||
|
totalLines: number;
|
||||||
|
totalBytes: number;
|
||||||
|
/** First ~thresholdChars chars of the stored content, for char fallback. */
|
||||||
|
rawHead: string;
|
||||||
|
/** Last ~thresholdChars chars of the stored content, for char fallback. */
|
||||||
|
rawTail: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Regexes only inspect this prefix of each line — catastrophic backtracking
|
||||||
|
* on a giant single-line blob is not allowed to stall the handler. */
|
||||||
|
export const MATCH_SCAN_CHARS = 10_000;
|
||||||
|
|
||||||
|
function clip(line: string, maxLineChars: number): string {
|
||||||
|
return line.length > maxLineChars ? `${line.slice(0, maxLineChars)}…` : line;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Slice a text original into the same view segments a file scan produces. */
|
||||||
|
export function collectViewFromText(text: string, settings: OverflowSettings): ViewInput {
|
||||||
|
const lines = text.split("\n");
|
||||||
|
const total = lines.length;
|
||||||
|
const headCount = Math.min(settings.headLines, total);
|
||||||
|
const tailCount = Math.min(settings.tailLines, Math.max(0, total - headCount));
|
||||||
|
const head = lines.slice(0, headCount).map((line) => clip(line, settings.maxLineChars));
|
||||||
|
const tail = tailCount > 0 ? lines.slice(total - tailCount).map((line) => clip(line, settings.maxLineChars)) : [];
|
||||||
|
const signals: SignalHit[] = [];
|
||||||
|
const regex = new RegExp(settings.signalRegex.source, "i");
|
||||||
|
for (
|
||||||
|
let index = headCount;
|
||||||
|
index < total - tailCount && signals.length < settings.maxSignalLines;
|
||||||
|
index++
|
||||||
|
) {
|
||||||
|
regex.lastIndex = 0;
|
||||||
|
if (regex.test(lines[index].slice(0, MATCH_SCAN_CHARS))) {
|
||||||
|
signals.push({ line: index + 1, text: clip(lines[index], settings.maxLineChars) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
head,
|
||||||
|
signals,
|
||||||
|
tail,
|
||||||
|
totalLines: total,
|
||||||
|
totalBytes: Buffer.byteLength(text, "utf8"),
|
||||||
|
rawHead: text.slice(0, settings.thresholdChars),
|
||||||
|
rawTail: text.slice(Math.max(0, text.length - settings.thresholdChars)),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stream the stored file line by line — never materialize it. Collects the
|
||||||
|
* first headLines lines, keeps the last tailLines lines in a ring buffer,
|
||||||
|
* and records at most maxSignalLines signal hits (line number + clipped
|
||||||
|
* text); hits landing inside the final tail range are dropped afterwards.
|
||||||
|
*/
|
||||||
|
export async function collectViewFromFile(path: string, settings: OverflowSettings): Promise<ViewInput> {
|
||||||
|
const head: string[] = [];
|
||||||
|
const tail: string[] = [];
|
||||||
|
const signals: SignalHit[] = [];
|
||||||
|
let totalLines = 0;
|
||||||
|
let remainder = "";
|
||||||
|
const regex = new RegExp(settings.signalRegex.source, "i");
|
||||||
|
|
||||||
|
const handleLine = (line: string) => {
|
||||||
|
totalLines++;
|
||||||
|
if (head.length < settings.headLines) {
|
||||||
|
head.push(clip(line, settings.maxLineChars));
|
||||||
|
} else if (signals.length < settings.maxSignalLines) {
|
||||||
|
regex.lastIndex = 0;
|
||||||
|
if (regex.test(line.slice(0, MATCH_SCAN_CHARS))) {
|
||||||
|
signals.push({ line: totalLines, text: clip(line, settings.maxLineChars) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tail.push(clip(line, settings.maxLineChars));
|
||||||
|
if (tail.length > settings.tailLines) tail.shift();
|
||||||
|
};
|
||||||
|
|
||||||
|
for await (const chunk of createReadStream(path, { encoding: "utf8" })) {
|
||||||
|
remainder += chunk;
|
||||||
|
let index = remainder.indexOf("\n");
|
||||||
|
while (index >= 0) {
|
||||||
|
handleLine(remainder.slice(0, index));
|
||||||
|
remainder = remainder.slice(index + 1);
|
||||||
|
index = remainder.indexOf("\n");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (remainder.length > 0) handleLine(remainder);
|
||||||
|
|
||||||
|
const tailStart = totalLines - tail.length + 1;
|
||||||
|
const middleSignals = signals.filter((hit) => hit.line < tailStart);
|
||||||
|
|
||||||
|
const info = await stat(path);
|
||||||
|
const take = Math.min(settings.thresholdChars, info.size);
|
||||||
|
let rawHead = "";
|
||||||
|
let rawTail = "";
|
||||||
|
if (take > 0) {
|
||||||
|
const handle = await open(path, "r");
|
||||||
|
try {
|
||||||
|
const headBuffer = Buffer.alloc(take);
|
||||||
|
const tailBuffer = Buffer.alloc(take);
|
||||||
|
await handle.read(headBuffer, 0, take, 0);
|
||||||
|
await handle.read(tailBuffer, 0, take, info.size - take);
|
||||||
|
rawHead = headBuffer.toString("utf8");
|
||||||
|
rawTail = tailBuffer.toString("utf8");
|
||||||
|
} finally {
|
||||||
|
await handle.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
head,
|
||||||
|
signals: middleSignals,
|
||||||
|
tail,
|
||||||
|
totalLines,
|
||||||
|
totalBytes: info.size,
|
||||||
|
rawHead,
|
||||||
|
rawTail,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatBytes(bytes: number): string {
|
||||||
|
if (bytes < 1024) return `${bytes} B`;
|
||||||
|
if (bytes < 1_048_576) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||||
|
if (bytes < 1_073_741_824) return `${(bytes / 1_048_576).toFixed(1)} MB`;
|
||||||
|
return `${(bytes / 1_073_741_824).toFixed(2)} GB`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function signalDescription(pattern: string): string {
|
||||||
|
const compact = pattern.replace(/\\b/g, "").replace(/[()]/g, "");
|
||||||
|
return compact.length > 60 ? `${compact.slice(0, 60)}…` : compact;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Deterministic bounded view of the stored original: head lines, signal
|
||||||
|
* lines from the middle section, and tail lines — all with the stored
|
||||||
|
* file's line numbers. Output never exceeds settings.thresholdChars and the
|
||||||
|
* tail of the stored content is always visible.
|
||||||
|
*
|
||||||
|
* Shrink order under budget pressure is tail > signals > head: head first
|
||||||
|
* collapses to min(10, headLines), signals halve stepwise to 10 then 0, then
|
||||||
|
* head/tail shrink ~35/65 with tail preferred, and char truncation of the
|
||||||
|
* stored head/tail is the final fallback.
|
||||||
|
*/
|
||||||
|
export function buildView(input: ViewInput, settings: OverflowSettings, meta: ViewMeta): string {
|
||||||
|
const budget = settings.thresholdChars;
|
||||||
|
const header =
|
||||||
|
`[pi-overflow id=${meta.id} tool=${meta.tool} ${input.totalLines} lines / ${formatBytes(input.totalBytes)}` +
|
||||||
|
` — full output stored; use overflow_query to search or read ranges]`;
|
||||||
|
|
||||||
|
const render = (headLines: string[], signals: SignalHit[], tailLines: string[]): string => {
|
||||||
|
const parts: string[] = [header];
|
||||||
|
if (headLines.length > 0) {
|
||||||
|
parts.push(`--- lines 1-${headLines.length} ---`, ...headLines);
|
||||||
|
}
|
||||||
|
if (signals.length > 0) {
|
||||||
|
parts.push(
|
||||||
|
`--- signals (lines matching ${signalDescription(settings.signalPattern)}, up to ${settings.maxSignalLines}) ---`,
|
||||||
|
...signals.map((hit) => `L${hit.line}: ${hit.text}`),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (tailLines.length > 0) {
|
||||||
|
const first = input.totalLines - tailLines.length + 1;
|
||||||
|
parts.push(`--- lines ${first}-${input.totalLines} ---`, ...tailLines);
|
||||||
|
}
|
||||||
|
return parts.join("\n");
|
||||||
|
};
|
||||||
|
|
||||||
|
const charFallback = (): string => {
|
||||||
|
const markerReserve = 64;
|
||||||
|
const bodyBudget = Math.max(64, budget - header.length - markerReserve);
|
||||||
|
const headChars = Math.floor(bodyBudget * 0.35);
|
||||||
|
const tailChars = bodyBudget - headChars;
|
||||||
|
const headText = input.rawHead.slice(0, headChars);
|
||||||
|
const tailText = input.rawTail.slice(Math.max(0, input.rawTail.length - tailChars));
|
||||||
|
const omitted = Math.max(0, input.totalBytes - headText.length - tailText.length);
|
||||||
|
return `${header}\n${headText}\n… ${omitted} chars omitted …\n${tailText}`;
|
||||||
|
};
|
||||||
|
|
||||||
|
let headList = input.head;
|
||||||
|
let signalList = input.signals;
|
||||||
|
let tailList = input.tail;
|
||||||
|
|
||||||
|
let result = render(headList, signalList, tailList);
|
||||||
|
if (result.length <= budget) return result;
|
||||||
|
|
||||||
|
// 1. Collapse head to its minimum useful window.
|
||||||
|
headList = headList.slice(0, Math.min(10, settings.headLines));
|
||||||
|
result = render(headList, signalList, tailList);
|
||||||
|
if (result.length <= budget) return result;
|
||||||
|
|
||||||
|
// 2. Halve signals stepwise until 10 remain, then drop them entirely.
|
||||||
|
while (signalList.length > 10) {
|
||||||
|
signalList = signalList.slice(0, Math.ceil(signalList.length / 2));
|
||||||
|
result = render(headList, signalList, tailList);
|
||||||
|
if (result.length <= budget) return result;
|
||||||
|
}
|
||||||
|
if (signalList.length > 0) {
|
||||||
|
signalList = [];
|
||||||
|
result = render(headList, signalList, tailList);
|
||||||
|
if (result.length <= budget) return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
// When head+tail already span every stored line there is no line-level
|
||||||
|
// slack left: go straight to char truncation so the ending survives.
|
||||||
|
if (input.head.length + input.tail.length >= input.totalLines) return charFallback();
|
||||||
|
|
||||||
|
// 3. Shrink head/tail ~35/65; tail keeps the larger share and always keeps
|
||||||
|
// its last line (the stored ending).
|
||||||
|
while (headList.length + tailList.length > 0) {
|
||||||
|
result = render(headList, signalList, tailList);
|
||||||
|
if (result.length <= budget) return result;
|
||||||
|
const headChars = headList.reduce((sum, line) => sum + line.length + 1, 0);
|
||||||
|
const tailChars = tailList.reduce((sum, line) => sum + line.length + 1, 0);
|
||||||
|
if (tailList.length > 1 && (tailChars >= headChars * (65 / 35) || headList.length === 0)) {
|
||||||
|
tailList.shift();
|
||||||
|
} else if (headList.length > 0) {
|
||||||
|
headList.pop();
|
||||||
|
} else {
|
||||||
|
tailList.shift();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Last resort: keep the stored text's own head 35% / tail 65%.
|
||||||
|
return charFallback();
|
||||||
|
}
|
||||||
@@ -0,0 +1,314 @@
|
|||||||
|
import test from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtemp, readFile, readdir, stat, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import overflowCapture from "../capture.ts";
|
||||||
|
import overflowExtension from "../index.ts";
|
||||||
|
import { getStash } from "../src/stash.ts";
|
||||||
|
import { readOverflowSettings, OVERFLOW_DEFAULTS, type OverflowSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
function harness() {
|
||||||
|
const handlers = new Map<string, Function[]>();
|
||||||
|
const tools = new Map<string, any>();
|
||||||
|
const pi: any = {
|
||||||
|
on(name: string, handler: Function) {
|
||||||
|
const list = handlers.get(name) ?? [];
|
||||||
|
list.push(handler);
|
||||||
|
handlers.set(name, list);
|
||||||
|
},
|
||||||
|
registerTool(tool: any) { tools.set(tool.name, tool); },
|
||||||
|
};
|
||||||
|
return { pi, handlers, tools };
|
||||||
|
}
|
||||||
|
|
||||||
|
function ctx() {
|
||||||
|
return { sessionManager: { getSessionId: () => "session-test" } } as any;
|
||||||
|
}
|
||||||
|
|
||||||
|
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
|
||||||
|
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
|
||||||
|
}
|
||||||
|
|
||||||
|
function textEvent(toolCallId: string, text: string, extra: Record<string, unknown> = {}) {
|
||||||
|
return {
|
||||||
|
toolCallId,
|
||||||
|
toolName: "bash",
|
||||||
|
input: { command: "run" },
|
||||||
|
content: [{ type: "text", text }],
|
||||||
|
details: {},
|
||||||
|
...extra,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runHandlers(handlers: Map<string, Function[]>, event: any, context: any) {
|
||||||
|
let current = event;
|
||||||
|
for (const handler of handlers.get("tool_result") ?? []) {
|
||||||
|
const result = await handler(current, context);
|
||||||
|
if (result) current = { ...current, ...result };
|
||||||
|
}
|
||||||
|
return current;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function makeStore() {
|
||||||
|
return mkdtemp(join(tmpdir(), "pi-overflow-test-"));
|
||||||
|
}
|
||||||
|
|
||||||
|
test("capture stashes the original before another handler shortens it; bound stage stores original but views current", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const capture = harness();
|
||||||
|
overflowCapture(capture.pi);
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const big = Array.from({ length: 3000 }, (_, i) => `orig ${i}`).join("\n");
|
||||||
|
const event = textEvent("call-1", big);
|
||||||
|
const afterCapture = await runHandlers(capture.handlers, event, ctx());
|
||||||
|
assert.equal(afterCapture, event, "capture must not modify the result");
|
||||||
|
assert.ok(getStash().has("call-1"), "capture must stash the original");
|
||||||
|
|
||||||
|
// Simulate RTK: replace content with a shortened version that is still
|
||||||
|
// over threshold.
|
||||||
|
const rtkShortened = Array.from({ length: 900 }, (_, i) => `rtk ${i}`).join("\n") + "\n" + "z".repeat(30_000);
|
||||||
|
const mid = { ...afterCapture, content: [{ type: "text", text: rtkShortened }] };
|
||||||
|
const final = await runHandlers(bound.handlers, mid, ctx());
|
||||||
|
|
||||||
|
assert.ok(!getStash().has("call-1"), "stash entry must be consumed");
|
||||||
|
const text = final.content[0].text;
|
||||||
|
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash/);
|
||||||
|
assert.ok(text.length <= settings().thresholdChars);
|
||||||
|
|
||||||
|
const id = final.details.overflow.id;
|
||||||
|
const stored = await readFile(join(root, `${id}.txt`), "utf8");
|
||||||
|
assert.ok(stored.includes("orig 0"), "store must keep the ORIGINAL, not the RTK-shortened text");
|
||||||
|
assert.ok(stored.includes("orig 2999"));
|
||||||
|
assert.equal(final.details.overflow.truncated, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("details.fullOutputPath file is used as the original", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const spill = join(root, "spill.txt");
|
||||||
|
await writeFile(spill, Array.from({ length: 5000 }, (_, i) => `spill ${i}`).join("\n"));
|
||||||
|
const event = textEvent("call-2", "x".repeat(30_000), { details: { fullOutputPath: spill } });
|
||||||
|
const final = await runHandlers(bound.handlers, event, ctx());
|
||||||
|
const stored = await readFile(join(root, `${final.details.overflow.id}.txt`), "utf8");
|
||||||
|
assert.ok(stored.includes("spill 0"));
|
||||||
|
assert.ok(stored.includes("spill 4999"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("view line numbers track the STORED file, not the current text", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
// bash spill: 5000 lines on disk, but the in-context result only carries
|
||||||
|
// the last ~100 lines (still over threshold via padding).
|
||||||
|
const spill = join(root, "spill-5000.txt");
|
||||||
|
await writeFile(
|
||||||
|
spill,
|
||||||
|
Array.from({ length: 5000 }, (_, i) => `spill line ${i + 1}`).join("\n"),
|
||||||
|
);
|
||||||
|
const current = Array.from({ length: 100 }, (_, i) => `spill line ${4901 + i}`).join("\n") + "\n" + "pad".repeat(10_000);
|
||||||
|
const event = textEvent("call-lines", current, { details: { fullOutputPath: spill } });
|
||||||
|
const final = await runHandlers(bound.handlers, event, ctx());
|
||||||
|
|
||||||
|
const text = final.content[0].text;
|
||||||
|
const id = final.details.overflow.id;
|
||||||
|
assert.match(text, /--- lines 4881-5000 ---/, "tail range must come from the stored 5000-line file");
|
||||||
|
assert.match(text, /spill line 5000/);
|
||||||
|
assert.match(text, /\[pi-overflow id=[0-9a-f]{12} tool=bash 5000 lines/);
|
||||||
|
|
||||||
|
// overflow_query with the view's tail range returns the same lines.
|
||||||
|
const tool = bound.tools.get("overflow_query");
|
||||||
|
const result = await tool.execute("q1", { id, lines: { start: 4881, end: 4883 } });
|
||||||
|
assert.match(result.content[0].text, /L4881: spill line 4881/);
|
||||||
|
assert.match(result.content[0].text, /L4883: spill line 4883/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("RTK scenario: view line numbers come from the stored original text", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const capture = harness();
|
||||||
|
overflowCapture(capture.pi);
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const original = Array.from({ length: 3000 }, (_, i) => `orig ${i + 1}`).join("\n");
|
||||||
|
await runHandlers(capture.handlers, textEvent("call-rtk", original), ctx());
|
||||||
|
// RTK rewrites the visible text but it stays over threshold.
|
||||||
|
const mid = {
|
||||||
|
...textEvent("call-rtk", ""),
|
||||||
|
content: [{ type: "text", text: "z".repeat(30_000) }],
|
||||||
|
};
|
||||||
|
const final = await runHandlers(bound.handlers, mid, ctx());
|
||||||
|
const text = final.content[0].text;
|
||||||
|
assert.match(text, /--- lines 2881-3000 ---/, "view tail must be the stored original's tail");
|
||||||
|
assert.ok(text.includes("orig 3000"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("file sources over maxEntryBytes keep real head and tail halves", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
const { storeEntry } = await import("../src/store.ts");
|
||||||
|
const src = join(root, "big.bin");
|
||||||
|
const content = "HEAD-".repeat(200) + "MID-".repeat(2000) + "TAIL-".repeat(200);
|
||||||
|
await writeFile(src, content);
|
||||||
|
const maxEntryBytes = 2000;
|
||||||
|
await storeEntry(root, "0123456789ab", { kind: "file", path: src }, { tool: "bash", args: {} }, maxEntryBytes);
|
||||||
|
const stored = await readFile(join(root, "0123456789ab.txt"), "utf8");
|
||||||
|
assert.ok(stored.startsWith("HEAD-"));
|
||||||
|
assert.ok(stored.endsWith("TAIL-"));
|
||||||
|
assert.match(stored, /bytes omitted/);
|
||||||
|
// Head and tail halves must equal the source's real bytes.
|
||||||
|
const half = Math.floor(maxEntryBytes / 2);
|
||||||
|
assert.equal(stored.slice(0, half), content.slice(0, half));
|
||||||
|
assert.equal(stored.slice(-half), content.slice(-half));
|
||||||
|
assert.ok(!stored.includes("\0".repeat(100)), "must not store zeroed content");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("interrupted-write tmp files older than an hour are cleaned", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const staleTmp = join(root, "deadbeef.txt.tmp-999-zzz");
|
||||||
|
const freshTmp = join(root, "livebeef.txt.tmp-111-aaa");
|
||||||
|
await writeFile(staleTmp, "partial");
|
||||||
|
await writeFile(freshTmp, "partial");
|
||||||
|
const old = new Date(Date.now() - 2 * 3_600_000);
|
||||||
|
const { utimes } = await import("node:fs/promises");
|
||||||
|
await utimes(staleTmp, old, old);
|
||||||
|
|
||||||
|
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
|
||||||
|
const { existsSync } = await import("node:fs");
|
||||||
|
assert.ok(!existsSync(staleTmp), "stale tmp must be removed");
|
||||||
|
assert.ok(existsSync(freshTmp), "fresh tmp must survive");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("current under threshold but original over → one-line marker appended", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const capture = harness();
|
||||||
|
overflowCapture(capture.pi);
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const big = "a".repeat(30_000);
|
||||||
|
const event = textEvent("call-3", big);
|
||||||
|
await runHandlers(capture.handlers, event, ctx());
|
||||||
|
const mid = { ...event, content: [{ type: "text", text: "short summary" }] };
|
||||||
|
const final = await runHandlers(bound.handlers, mid, ctx());
|
||||||
|
|
||||||
|
const text = final.content[0].text;
|
||||||
|
assert.ok(text.startsWith("short summary"));
|
||||||
|
assert.match(text, /\[pi-overflow id=[0-9a-f]{12}: full original output stored \(\d+ lines, .*; use overflow_query/);
|
||||||
|
assert.equal(final.details.overflow.truncated, false);
|
||||||
|
assert.ok(final.details.overflow.id);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("exempt tools and nested calls are ignored", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const big = "x".repeat(30_000);
|
||||||
|
const exempt = await runHandlers(bound.handlers, textEvent("c-ex", big, { toolName: "read" }), ctx());
|
||||||
|
assert.equal(exempt.content[0].text, big, "read results must pass through untouched");
|
||||||
|
|
||||||
|
const nested = await runHandlers(bound.handlers, textEvent("c-nest", big, { parentToolCallId: "parent" }), ctx());
|
||||||
|
assert.equal(nested.content[0].text, big, "nested calls must pass through untouched");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("image parts and structuredContent survive", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const event = textEvent("call-4", "x".repeat(30_000));
|
||||||
|
event.content.push({ type: "image", data: "b64" } as any);
|
||||||
|
(event as any).structuredContent = { rows: 3 };
|
||||||
|
const final = await runHandlers(bound.handlers, event, ctx());
|
||||||
|
assert.deepEqual(final.structuredContent, { rows: 3 });
|
||||||
|
assert.equal(final.content.at(-1).type, "image");
|
||||||
|
assert.equal(final.content.at(-1).data, "b64");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("existing details fields survive the merge", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const event = textEvent("call-5", "x".repeat(30_000), { details: { rtk: { saved: 100 }, fullOutputPath: undefined } });
|
||||||
|
const final = await runHandlers(bound.handlers, event, ctx());
|
||||||
|
assert.deepEqual(final.details.rtk, { saved: 100 });
|
||||||
|
assert.ok(final.details.overflow.id);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("files are 0600 and directories 0700", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings() });
|
||||||
|
|
||||||
|
const final = await runHandlers(bound.handlers, textEvent("call-6", "x".repeat(30_000)), ctx());
|
||||||
|
const id = final.details.overflow.id;
|
||||||
|
const dirMode = (await stat(root)).mode & 0o777;
|
||||||
|
const fileMode = (await stat(join(root, `${id}.txt`))).mode & 0o777;
|
||||||
|
const metaMode = (await stat(join(root, `${id}.json`))).mode & 0o777;
|
||||||
|
assert.equal(dirMode, 0o700);
|
||||||
|
assert.equal(fileMode, 0o600);
|
||||||
|
assert.equal(metaMode, 0o600);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("retention and total-size cleanup remove old entries", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, {
|
||||||
|
storeRoot: root,
|
||||||
|
settings: settings({ retentionDays: 1, maxStoreBytes: 100 }),
|
||||||
|
});
|
||||||
|
|
||||||
|
// An old entry (mtime in the past) must be deleted by age.
|
||||||
|
const oldId = "aaaaaaaaaaaa";
|
||||||
|
await writeFile(join(root, `${oldId}.txt`), "old");
|
||||||
|
await writeFile(join(root, `${oldId}.json`), "{}");
|
||||||
|
const past = new Date(Date.now() - 10 * 86_400_000);
|
||||||
|
const { utimes } = await import("node:fs/promises");
|
||||||
|
await utimes(join(root, `${oldId}.txt`), past, past);
|
||||||
|
await utimes(join(root, `${oldId}.json`), past, past);
|
||||||
|
|
||||||
|
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
|
||||||
|
const names = await readdir(root);
|
||||||
|
assert.ok(!names.some((n) => n.startsWith(oldId)), "old entry must be deleted");
|
||||||
|
|
||||||
|
// Fresh entries exceeding maxStoreBytes get evicted oldest-first.
|
||||||
|
for (const [i, id] of ["bbbbbbbbbbbb", "cccccccccccc", "dddddddddddd"].entries()) {
|
||||||
|
await writeFile(join(root, `${id}.txt`), "y".repeat(80));
|
||||||
|
const t = new Date(Date.now() - (3 - i) * 60_000);
|
||||||
|
await utimes(join(root, `${id}.txt`), t, t);
|
||||||
|
}
|
||||||
|
for (const handler of bound.handlers.get("session_start") ?? []) await handler({}, ctx());
|
||||||
|
const after = (await readdir(root)).filter((n) => n.endsWith(".txt"));
|
||||||
|
assert.ok(after.length <= 2, `expected eviction under maxStoreBytes, got ${after.join(",")}`);
|
||||||
|
assert.ok(!after.includes("bbbbbbbbbbbb.txt"), "oldest entry must be evicted first");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("disabled setting clears the stash and returns undefined", async () => {
|
||||||
|
const root = await makeStore();
|
||||||
|
getStash().clear();
|
||||||
|
const bound = harness();
|
||||||
|
overflowExtension(bound.pi, { storeRoot: root, settings: settings({ enabled: false }) });
|
||||||
|
getStash().set("c-off", { rawText: "x".repeat(30_000) });
|
||||||
|
const event = textEvent("c-off", "x".repeat(30_000));
|
||||||
|
const result = await runHandlers(bound.handlers, event, ctx());
|
||||||
|
assert.equal(result, event);
|
||||||
|
assert.ok(!getStash().has("c-off"));
|
||||||
|
});
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
import test from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { mkdtemp, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { runOverflowQuery } from "../src/query.ts";
|
||||||
|
import { storeEntry } from "../src/store.ts";
|
||||||
|
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
|
||||||
|
return { ...readOverflowSettings("/nonexistent/pi-overflow.json"), ...overrides };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function store(text: string, id = "0123456789ab") {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "pi-overflow-query-"));
|
||||||
|
await storeEntry(root, id, { kind: "text", text }, { tool: "bash", args: { command: "run" } }, 1_000_000);
|
||||||
|
return root;
|
||||||
|
}
|
||||||
|
|
||||||
|
test("search returns hits with merged context", async () => {
|
||||||
|
const lines = Array.from({ length: 100 }, (_, i) => (i === 9 || i === 11 || i === 50 ? `line ${i + 1} FAIL` : `line ${i + 1}`));
|
||||||
|
const root = await store(lines.join("\n"));
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 1 }, settings());
|
||||||
|
assert.ok(!result.error);
|
||||||
|
assert.match(result.text, /L10: line 10 FAIL/);
|
||||||
|
assert.match(result.text, /L12: line 12 FAIL/);
|
||||||
|
assert.match(result.text, /L51: line 51 FAIL/);
|
||||||
|
// hits at lines 10 and 12 with context 1 merge into one range 9-13
|
||||||
|
assert.match(result.text, /--- lines 9-13 ---/);
|
||||||
|
assert.match(result.text, /--- lines 50-52 ---/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("line-range retrieval returns numbered lines", async () => {
|
||||||
|
const lines = Array.from({ length: 50 }, (_, i) => `row ${i + 1}`);
|
||||||
|
const root = await store(lines.join("\n"));
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 10, end: 15 } }, settings());
|
||||||
|
assert.match(result.text, /--- lines 10-15 ---/);
|
||||||
|
assert.match(result.text, /L10: row 10/);
|
||||||
|
assert.match(result.text, /L15: row 15/);
|
||||||
|
assert.ok(!result.text.includes("row 16"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("jsonPath extracts nested values", async () => {
|
||||||
|
const doc = JSON.stringify({ snapshot: { nodes: [{ name: "a" }, { name: "b" }, { name: "c" }] } });
|
||||||
|
const root = await store(doc);
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "snapshot.nodes[2].name" }, settings());
|
||||||
|
assert.match(result.text, /"c"/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("invalid id, regex, JSON and path return clear errors without throwing", async () => {
|
||||||
|
const root = await store('{"a": {"b": 1}}');
|
||||||
|
assert.ok((await runOverflowQuery(root, { id: "../etc/passwd" }, settings())).error);
|
||||||
|
assert.ok((await runOverflowQuery(root, { id: "zzzzzzzzzzzz" }, settings())).error);
|
||||||
|
assert.ok((await runOverflowQuery(root, { id: "0123456789ab", search: "([" }, settings())).error);
|
||||||
|
const goodPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b" }, settings());
|
||||||
|
assert.ok(!goodPath.error, "valid path on valid JSON must succeed");
|
||||||
|
const badPath = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a.b.c!!" }, settings());
|
||||||
|
assert.ok(badPath.error);
|
||||||
|
const missing = await runOverflowQuery(root, { id: "ffffffffffff" }, settings());
|
||||||
|
assert.ok(missing.error);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("non-JSON content + jsonPath reports the parse failure", async () => {
|
||||||
|
const root = await store("not json at all");
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab", jsonPath: "a" }, settings());
|
||||||
|
assert.ok(result.error);
|
||||||
|
assert.match(result.text, /not valid JSON/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("output is bounded by thresholdChars and marked truncated", async () => {
|
||||||
|
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i + 1} pad ${"p".repeat(40)}`);
|
||||||
|
const root = await store(lines.join("\n"));
|
||||||
|
const result = await runOverflowQuery(
|
||||||
|
root,
|
||||||
|
{ id: "0123456789ab", lines: { start: 1, end: 5000 } },
|
||||||
|
settings({ thresholdChars: 4000 }),
|
||||||
|
);
|
||||||
|
assert.ok(result.text.length <= 4000);
|
||||||
|
assert.equal(result.truncated, true);
|
||||||
|
assert.match(result.text, /truncated/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("lines and search output clip each line to maxLineChars", async () => {
|
||||||
|
const long = `row ${"q".repeat(2_000)}`;
|
||||||
|
const root = await store(`first\n${long}\n${long} ERROR\nlast`);
|
||||||
|
const s = settings({ maxLineChars: 60 });
|
||||||
|
const lines = await runOverflowQuery(root, { id: "0123456789ab", lines: { start: 2, end: 3 } }, s);
|
||||||
|
assert.ok(lines.text.includes("q".repeat(56) + "…"));
|
||||||
|
assert.ok(!lines.text.includes("q".repeat(61)));
|
||||||
|
const search = await runOverflowQuery(root, { id: "0123456789ab", search: "ERROR", context: 0 }, s);
|
||||||
|
assert.ok(search.text.includes("q".repeat(56) + "…"));
|
||||||
|
assert.ok(!search.text.includes("q".repeat(61)));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("search only scans the first 10000 chars of each line", async () => {
|
||||||
|
const earlyHit = `hit ${"a".repeat(100)} FAIL`;
|
||||||
|
const lateHit = `${"a".repeat(10_500)} FAIL`;
|
||||||
|
const root = await store(`${earlyHit}\n${lateHit}\nplain`);
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab", search: "FAIL", context: 0 }, settings());
|
||||||
|
assert.match(result.text, /L1: hit /);
|
||||||
|
assert.ok(!result.text.includes("L2:"), "match beyond char 10000 must not be reported");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("default query returns metadata and a first-screen view", async () => {
|
||||||
|
const lines = Array.from({ length: 300 }, (_, i) => `row ${i + 1}`);
|
||||||
|
const root = await store(lines.join("\n"));
|
||||||
|
const result = await runOverflowQuery(root, { id: "0123456789ab" }, settings());
|
||||||
|
assert.ok(!result.error);
|
||||||
|
assert.match(result.text, /\[pi-overflow id=0123456789ab tool=bash \d+ lines \/ \d+ bytes\]/);
|
||||||
|
assert.match(result.text, /--- lines 1-40 ---/);
|
||||||
|
assert.ok(result.text.includes("row 300"), "tail of the stored file must appear");
|
||||||
|
});
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
import test from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { buildView, collectViewFromText } from "../src/view.ts";
|
||||||
|
import { readOverflowSettings, type OverflowSettings } from "../settings.ts";
|
||||||
|
|
||||||
|
function settings(overrides: Partial<OverflowSettings> = {}): OverflowSettings {
|
||||||
|
return {
|
||||||
|
...readOverflowSettings("/nonexistent/pi-overflow.json"),
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const meta = { id: "abc123def456", tool: "bash" };
|
||||||
|
|
||||||
|
function view(text: string, s = settings()) {
|
||||||
|
return buildView(collectViewFromText(text, s), s, meta);
|
||||||
|
}
|
||||||
|
|
||||||
|
test("below threshold remains unchanged", () => {
|
||||||
|
const out = view("line one\nline two\nline three");
|
||||||
|
assert.ok(out.includes("line one"));
|
||||||
|
assert.ok(out.includes("line three"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("above threshold includes header, line ranges, signal lines and tail", () => {
|
||||||
|
const lines: string[] = [];
|
||||||
|
for (let i = 1; i <= 1000; i++) lines.push(`row ${i} ${i === 500 ? "ERROR boom" : "ok"}`);
|
||||||
|
const out = view(lines.join("\n"));
|
||||||
|
assert.match(out, /\[pi-overflow id=abc123def456 tool=bash 1000 lines/);
|
||||||
|
assert.match(out, /--- lines 1-40 ---/);
|
||||||
|
assert.match(out, /L500: row 500 ERROR boom/);
|
||||||
|
assert.match(out, /--- lines 881-1000 ---/);
|
||||||
|
assert.ok(out.includes("row 1000"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("output never exceeds thresholdChars", () => {
|
||||||
|
const lines = Array.from({ length: 5000 }, (_, i) => `line ${i} ${"x".repeat(60)}`);
|
||||||
|
const s = settings({ thresholdChars: 2000 });
|
||||||
|
const out = view(lines.join("\n"), s);
|
||||||
|
assert.ok(out.length <= 2000, `view ${out.length} exceeds budget`);
|
||||||
|
assert.ok(out.includes("line 4999"), "tail must survive");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("few long lines still expose the ending", () => {
|
||||||
|
const lines = ["start", ...Array.from({ length: 5 }, (_, i) => `middle${i} ${"y".repeat(4000)}`), "THE_END_MARKER"];
|
||||||
|
const s = settings({ thresholdChars: 3000, headLines: 40, tailLines: 120 });
|
||||||
|
const out = view(lines.join("\n"), s);
|
||||||
|
assert.ok(out.includes("THE_END_MARKER"));
|
||||||
|
assert.ok(out.length <= 3000 + 100, `view ${out.length} far exceeds budget`);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("same input produces identical output", () => {
|
||||||
|
const lines = Array.from({ length: 800 }, (_, i) => `entry ${i} ${i % 77 === 0 ? "warning" : "fine"}`);
|
||||||
|
const text = lines.join("\n");
|
||||||
|
assert.equal(view(text), view(text));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a single line over maxLineChars is truncated", () => {
|
||||||
|
const long = "z".repeat(2000);
|
||||||
|
const out = view(`a\n${long}\nb`, settings({ maxLineChars: 50 }));
|
||||||
|
assert.ok(out.includes("z".repeat(50) + "…"));
|
||||||
|
assert.ok(!out.includes("z".repeat(51)));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("signals outrank head and survive when head+tail exceed the budget", () => {
|
||||||
|
// ~150-char lines: head 40 + tail 120 = 160 lines * 150 = ~24K plus
|
||||||
|
// overhead, over budget. Shrink order must keep some signals while head
|
||||||
|
// collapses to 10 lines before the 35/65 phase kicks in.
|
||||||
|
const lines = Array.from({ length: 400 }, (_, i) => {
|
||||||
|
const marker = i === 200 ? "ERROR detected here" : "ok";
|
||||||
|
return `line ${i + 1} ${marker} ${"x".repeat(140)}`;
|
||||||
|
});
|
||||||
|
const out = view(lines.join("\n"));
|
||||||
|
assert.match(out, /--- signals/);
|
||||||
|
assert.match(out, /L201: line 201 ERROR detected here/);
|
||||||
|
// head collapsed below the configured 40 lines
|
||||||
|
assert.match(out, /--- lines 1-(?:[0-9]|10) ---/);
|
||||||
|
assert.ok(out.includes("line 400"));
|
||||||
|
});
|
||||||
|
|
||||||
|
test("signal hits inside the tail region are not duplicated", () => {
|
||||||
|
const lines = Array.from({ length: 100 }, (_, i) => `line ${i + 1} ${"y".repeat(40)}`);
|
||||||
|
lines[99] = "line 100 tail-side ERROR";
|
||||||
|
const out = view(lines.join("\n"));
|
||||||
|
assert.ok(!out.includes("L100:"), "tail-region hit must not appear in signals");
|
||||||
|
});
|
||||||
@@ -8,7 +8,7 @@
|
|||||||
- reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。
|
- reviewer 默认跟随会话当前主模型(每次审查实时读取 `ctx.model`),经 `modelRegistry.streamSimple(...).result()` 调用;`autoReview.provider` / `autoReview.model` 必须同时设置(固定审查模型)或同时留空,只设其一视为配置错误并 `defer`。默认 `reasoning: "low"`(仅对支持推理的模型生效)、总超时 30 秒、最多 2 次尝试(仅 Provider 错误重试)、`maxOutputTokens` 4000。只把 active branch 中的直接用户消息与已识别、未取消的结构化问答作为授权证据,assistant/tool/compaction 内容不能自行授权;工具结果可替换为更短的 pi-compaction 摘要,但仍属不可信证据。同一条用户消息内完全相同的请求只缓存 `approve` 结论。
|
||||||
- `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。
|
- `config/pi-permission-policy.jsonc` 必须显式配置 `authorizerChain: ["auto-review"]`,普通未匹配请求和需要自动复核的类别声明为 `ask`;普通 pattern 必须按具体度优先(精确匹配、字面字符更多、通配符更少、pattern 更长,最后才以声明顺序破同级);`autoReview` 段只保留 `enabled`,不设 provider/model 种子值。`config/pi-permission-hard-guards.jsonc` 独立保存可由用户调整但不能被普通策略、Session grant、per-agent 规则或 reviewer 覆盖的高危 `deny`,以及 `protectedWrites` 人工写保护清单。
|
||||||
- `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。
|
- `protectedWrites` 是确定性的 human-only 写路径,不属于普通策略也不是第二个授权范围:命中目标的 `write`/`edit` 与本地 `bash` 写入(redirect-out、写操作程序参数、`bash -c`/`eval` 等包装载荷中的受保护路径)一律弹出人工确认,跳过 AutoReview、yolo 与既有 Session grant,弹窗不提供「本会话允许」选项;没有可用确认界面(无 UI、子代理转发终端)时直接拒绝。读取不受影响;`ssh_bash` 等远端 shell 工具不参与本机路径保护。
|
||||||
- 策略默认允许常规只读与低风险工具;`web_fetch`、`ctx_fetch_and_index`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。
|
- 策略默认允许常规只读与低风险工具;`web_fetch`、`skill_manage` 保持 `ask`(模型自选的 URL 可能外传数据,skill 写入会进入所有后续会话);硬防御默认只保留确定性的破坏性 Shell 命令;Git 只读管道命令放行,其余非只读操作、包管理、文件/系统/网络高风险操作与 `mcp__<server>__<tool>` 调用按唯一全局 policy 判定。项目外路径不产生独立 `external_directory` 判断,只走配置中的工具、`path` 与 Bash pattern。
|
||||||
- `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。
|
- `HardGuardPolicyLoader` 必须保持 global-only、无 SessionRules、无项目覆盖;唯一的组合包 hard-guards 文件不可用时 fail closed 到 deny-all,并报告配置问题。普通 policy 与 hard guard 配置不得合并成一份文件。
|
||||||
- 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。
|
- 内置 AutoReview 实现位于 `src/auto-review/`;同步原 AutoReview 上游时必须从源码仓库明确 tag/commit 移植,保留 `LICENSES/pi-permission-auto-review-MIT.txt`、作者和 `docs/auto-review/UPSTREAM.md`,不得恢复顶层独立扩展、npm 预编译入口或平行 gate。
|
||||||
- reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。
|
- reviewer 只能处理普通策略已判定为 `ask` 且未被 hard guard 拒绝的请求;不得绕过硬 `deny`。
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
|
- **Descend into bash control-flow bodies when enumerating command units.** `for`/`while`/`until` loops, `if`/`elif`/`else` branches, `case` items, `{ … }` brace groups, function definitions, and `!`-negated commands were previously matched only as whole-statement text, so a denied command hidden in a loop body or behind `!` (e.g. `for d in */; do find "$d" …; done`, `! rm -rf x`) bypassed every bash pattern — as did a substitution inside a value (`x=$(grep foo f)`, `export X=$(find .)`). The enumerator still emits each enclosing statement as a unit and now also emits every condition, body, and negated command — plus substitutions hosted in value lists, case subjects, assignments, declarations, and other non-command positions — so the most restrictive decision wins; pure body containers (`do …; done`, `elif`/`else` clauses, `case` items) no longer produce their own units. Hard guards share the same enumeration and now reach those nested commands too.
|
||||||
- **Replace the AutoReview text-JSON chat reviewer with a conversational tool-call reviewer.** The reviewer makes one independent model call per ask — a fixed review prompt, the rendered transcript, and a bounded JSON description of the planned action — and reaches its verdict only by calling one of two tools, `approve` or `reject`, each carrying a `reason`. A reply with no tool call counts as no verdict; multiple calls, unknown tools, and missing reasons count as invalid responses; neither is retried. An experimental TypeSafe Jev classifier rewrite explored in the working tree was dropped before release and never shipped.
|
- **Replace the AutoReview text-JSON chat reviewer with a conversational tool-call reviewer.** The reviewer makes one independent model call per ask — a fixed review prompt, the rendered transcript, and a bounded JSON description of the planned action — and reaches its verdict only by calling one of two tools, `approve` or `reject`, each carrying a `reason`. A reply with no tool call counts as no verdict; multiple calls, unknown tools, and missing reasons count as invalid responses; neither is retried. An experimental TypeSafe Jev classifier rewrite explored in the working tree was dropped before release and never shipped.
|
||||||
- **The review model follows the session by default.** With `autoReview.provider` and `autoReview.model` both unset, the reviewer calls the session's current model (`ctx.model`), resolved live per ask so a mid-session switch applies to the next request. Setting both pins a fixed reviewer; setting exactly one is a misconfiguration that warns once at `session_start` and defers every ask.
|
- **The review model follows the session by default.** With `autoReview.provider` and `autoReview.model` both unset, the reviewer calls the session's current model (`ctx.model`), resolved live per ask so a mid-session switch applies to the next request. Setting both pins a fixed reviewer; setting exactly one is a misconfiguration that warns once at `session_start` and defers every ask.
|
||||||
- **A rejection defers instead of denying.** `reject` maps to `defer` and the model's `reason` is shown in the human permission prompt, labelled as model-generated. Every review failure — timeout, provider error, malformed or truncated reply, unresolved model — also defers with a fixed system-authored note. Pressing Esc during a review aborts it and denies outright, so no human dialog appears after the user already refused.
|
- **A rejection defers instead of denying.** `reject` maps to `defer` and the model's `reason` is shown in the human permission prompt, labelled as model-generated. Every review failure — timeout, provider error, malformed or truncated reply, unresolved model — also defers with a fixed system-authored note. Pressing Esc during a review aborts it and denies outright, so no human dialog appears after the user already refused.
|
||||||
|
|||||||
@@ -21,3 +21,7 @@ AutoReview 源码已整合进 `src/auto-review/`,不再保留或加载第二
|
|||||||
## 本地维护
|
## 本地维护
|
||||||
|
|
||||||
本目录不是 submodule,不保留嵌套 `.git`、上游 `.pi` 本地状态、`dist/` 或构建产物。同步任一上游时必须从其源码仓库的明确 tag/commit 移植并与本地改动对照审查,不得整体覆盖,也不得使用 npm 预编译产物作为导入源。
|
本目录不是 submodule,不保留嵌套 `.git`、上游 `.pi` 本地状态、`dist/` 或构建产物。同步任一上游时必须从其源码仓库的明确 tag/commit 移植并与本地改动对照审查,不得整体覆盖,也不得使用 npm 预编译产物作为导入源。
|
||||||
|
|
||||||
|
## 本地改动
|
||||||
|
|
||||||
|
- Bash 命令枚举在保留整条控制流语句为独立 command unit 的基础上递归下降:`for`/`while`/`until`、`if`/`elif`/`else`、`case` 分支、`{ … }` 组合语句、函数定义与 `!` 取反命令体内的命令也按 bash pattern 逐条判定;赋值与声明值(`x=$(…)`、`export X=$(…)`)、值列表、case 词与 `[[ … ]]` 测试中的 `$(…)` 替换同样枚举,而 `do …; done`、`elif`/`else`、`case` 分支等纯容器不再单独产生 unit。藏在循环体、取反前缀或赋值载荷里的 `find`/`grep`/`rm` 等不再绕过 deny 或 hard guard。
|
||||||
|
|||||||
@@ -370,7 +370,10 @@ This holds wherever the substitution appears, not only in argument position.
|
|||||||
A substitution in a **redirect target** (`echo hi > $(rm *.txt)`, `cat < <(rm c)`, ``echo hi 2> `rm d` ``) and one in an **interpolating heredoc body** (`cat <<EOF` with `$(rm e)` in the body) are evaluated the same way.
|
A substitution in a **redirect target** (`echo hi > $(rm *.txt)`, `cat < <(rm c)`, ``echo hi 2> `rm d` ``) and one in an **interpolating heredoc body** (`cat <<EOF` with `$(rm e)` in the body) are evaluated the same way.
|
||||||
A quoted heredoc delimiter (`<<'EOF'` or `<<"EOF"`) does not interpolate, so its body is literal text and nothing in it is evaluated as a command.
|
A quoted heredoc delimiter (`<<'EOF'` or `<<"EOF"`) does not interpolate, so its body is literal text and nothing in it is evaluated as a command.
|
||||||
The enclosing command is still matched without its redirect, so a rule like `npm install` keeps matching `npm install > out.txt`.
|
The enclosing command is still matched without its redirect, so a rule like `npm install` keeps matching `npm install > out.txt`.
|
||||||
Control-flow bodies (`if`/`while`/`for`/`case`) and `{ … }` brace groups are not descended into; their contents are matched as part of the enclosing statement's text.
|
Control-flow statements (`for`/`while`/`until` loops, `if`/`elif`/`else` branches, `case` items, `{ … }` brace groups, and function definitions) are evaluated as a whole unit *and* descended into: every command in their conditions and bodies is matched against the bash patterns in addition to the enclosing statement's text.
|
||||||
|
So `for d in src docs; do find "$d" -name '*.ts'; done` evaluates both the whole loop and `find "$d" -name '*.ts'`; if `find *` is denied, the whole invocation is denied.
|
||||||
|
Non-command parts of a control-flow statement — the `for` value list, the `case` subject and patterns, `[[ … ]]` tests — are not matched as commands themselves, but substitutions nested inside them still are, so `for f in $(find …)` and `if [[ "$(rm e)" = x ]]` gate the inner command.
|
||||||
|
Negation and shell bookkeeping get the same treatment: `! grep -q x f` evaluates both `! grep -q x f` and the inner `grep -q x f`, and a substitution inside a value — `x=$(grep foo f)`, `export X=$(find .)`, `local y=$(rg z)` — is gated as a command substitution even though the assignment or declaration itself runs no command.
|
||||||
|
|
||||||
A leading environment-variable assignment prefix is stripped before matching, so the rule gates the underlying command rather than the prefix.
|
A leading environment-variable assignment prefix is stripped before matching, so the rule gates the underlying command rather than the prefix.
|
||||||
So `AWS_PROFILE=prod aws ec2 …` is matched as `aws ec2 …` — a `aws *` rule applies even though the invocation begins with `AWS_PROFILE=`.
|
So `AWS_PROFILE=prod aws ec2 …` is matched as `aws ec2 …` — a `aws *` rule applies even though the invocation begins with `AWS_PROFILE=`.
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import {
|
import {
|
||||||
EXECUTION_HOST_TYPES,
|
EXECUTION_HOST_TYPES,
|
||||||
forEachNestedExecution,
|
forEachNestedExecution,
|
||||||
|
NESTED_EXECUTION_CONTEXTS,
|
||||||
} from "#src/access-intent/bash/nested-execution";
|
} from "#src/access-intent/bash/nested-execution";
|
||||||
import type { TSNode } from "#src/access-intent/bash/parser";
|
import type { TSNode } from "#src/access-intent/bash/parser";
|
||||||
import {
|
import {
|
||||||
@@ -72,6 +73,62 @@ const COMMAND_ENUM_DESCEND = new Set([
|
|||||||
*/
|
*/
|
||||||
const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]);
|
const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compound statement node types whose bodies really run commands:
|
||||||
|
* `for`/`while`/`until` loops (tree-sitter-bash folds `until` into
|
||||||
|
* `while_statement`), `if`/`case` statements, `{ … }` brace groups
|
||||||
|
* (`compound_statement`), function definitions, and `!`-negated commands.
|
||||||
|
*
|
||||||
|
* These were once emitted whole without descending (#306 deferred that), which
|
||||||
|
* let a gated command hide inside a loop body — a `for …; do find …; done`
|
||||||
|
* matched only patterns starting with `for`. Each is now emitted whole AND
|
||||||
|
* enumerated child-by-child; a `negated_command` emits `! grep …` whole and
|
||||||
|
* then `grep …` so the `!` prefix cannot defeat a command pattern either.
|
||||||
|
*/
|
||||||
|
const CONTROL_FLOW_TYPES = new Set([
|
||||||
|
"for_statement",
|
||||||
|
"c_style_for_statement",
|
||||||
|
"while_statement",
|
||||||
|
"if_statement",
|
||||||
|
"case_statement",
|
||||||
|
"compound_statement",
|
||||||
|
"function_definition",
|
||||||
|
"negated_command",
|
||||||
|
]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Control-flow nodes that are pure body containers rather than statements in
|
||||||
|
* their own right: `do …; done`, `elif`/`else` clauses, and `case` items.
|
||||||
|
* They are descended exactly like {@link CONTROL_FLOW_TYPES} members but are
|
||||||
|
* not emitted as whole units — nobody writes a command pattern against
|
||||||
|
* `do …; done` or `a) find . ;;`, so emitting them only adds noise to prompts
|
||||||
|
* and approval suggestions.
|
||||||
|
*/
|
||||||
|
const CONTROL_FLOW_CONTAINERS = new Set([
|
||||||
|
"do_group",
|
||||||
|
"elif_clause",
|
||||||
|
"else_clause",
|
||||||
|
"case_item",
|
||||||
|
]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Named children of a control-flow node that are statements in their own
|
||||||
|
* right and get full command enumeration (whole-emit rules, wrapper
|
||||||
|
* classification, substitution descent). Any other named child — words,
|
||||||
|
* `variable_name`, a for-loop's value list, a case subject or pattern, a
|
||||||
|
* `test_command` `[[ … ]]`, a comment — contributes only the nested
|
||||||
|
* `$(…)`/`<(…)` executions it hosts.
|
||||||
|
*/
|
||||||
|
const CONTROL_FLOW_STATEMENT_TYPES = new Set([
|
||||||
|
"command",
|
||||||
|
"list",
|
||||||
|
"pipeline",
|
||||||
|
"redirected_statement",
|
||||||
|
"subshell",
|
||||||
|
...CONTROL_FLOW_TYPES,
|
||||||
|
...CONTROL_FLOW_CONTAINERS,
|
||||||
|
]);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Enumerate the command units of a bash program, in source order.
|
* Enumerate the command units of a bash program, in source order.
|
||||||
*
|
*
|
||||||
@@ -82,8 +139,19 @@ const COMMAND_ENUM_SKIP = new Set(["comment", "heredoc_end"]);
|
|||||||
* subshells (`( … )`) — emitting each inner command as its own unit *in
|
* subshells (`( … )`) — emitting each inner command as its own unit *in
|
||||||
* addition to* the enclosing command, since those inner commands really execute
|
* addition to* the enclosing command, since those inner commands really execute
|
||||||
* (#306).
|
* (#306).
|
||||||
* Control-flow bodies and `{ … }` brace groups are emitted whole without
|
* Control-flow statements (loops, `if`/`elif`/`else`, `case`, `{ … }` groups,
|
||||||
* descending (deferred).
|
* function definitions, `!` negation) are likewise emitted whole and then
|
||||||
|
* descended child-by-child: statement children (including nested control flow
|
||||||
|
* and conditions like `if grep -q …; then`) are enumerated as commands, while
|
||||||
|
* non-statement children (the for-loop value list, case subject/patterns,
|
||||||
|
* `[[ … ]]` tests, names) contribute only the nested executions they host.
|
||||||
|
* Pure body containers (`do …; done`, `elif`/`else` clauses, `case` items)
|
||||||
|
* are descended without a whole-unit emit of their own.
|
||||||
|
* Body commands keep the enclosing execution context — a loop inside `$(…)`
|
||||||
|
* runs there, a top-level loop's body runs in the current shell.
|
||||||
|
* Remaining named statements (`variable_assignment`, `declaration_command`,
|
||||||
|
* `test_command`, …) emit whole and still report the nested executions they
|
||||||
|
* host, so `x=$(grep foo f)` and `export X=$(find .)` gate the inner command.
|
||||||
*
|
*
|
||||||
* The enclosing command/subshell is always still emitted whole, so adding the
|
* The enclosing command/subshell is always still emitted whole, so adding the
|
||||||
* nested units can only ever produce a more-restrictive decision, never weaker.
|
* nested units can only ever produce a more-restrictive decision, never weaker.
|
||||||
@@ -135,9 +203,57 @@ function collectCommandsInto(
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Any other named statement (compound_statement `{ … }`, if/while/for/case,
|
if (CONTROL_FLOW_TYPES.has(node.type)) {
|
||||||
// function_definition): emit whole, do not descend — deferred (#306).
|
out.push(makeUnit(node.text, context)); // never-weaker whole emit
|
||||||
|
descendControlFlowChildren(node, context, out);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (CONTROL_FLOW_CONTAINERS.has(node.type)) {
|
||||||
|
// Pure body container — enumerate its contents, emit no unit of its own.
|
||||||
|
descendControlFlowChildren(node, context, out);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Any other named statement (variable_assignment, declaration_command,
|
||||||
|
// test_command, …) keeps the whole-emit fallback, but a substitution in its
|
||||||
|
// payload still really runs — `x=$(grep foo f)` or `export X=$(find .)` —
|
||||||
|
// so hosted executions are enumerated in addition to the whole text.
|
||||||
out.push(makeUnit(node.text, context));
|
out.push(makeUnit(node.text, context));
|
||||||
|
collectHostedCommands(node, out);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enumerate inside a control-flow statement, keeping the enclosing context.
|
||||||
|
*
|
||||||
|
* Statement children (`command`, `list`, `pipeline`, `redirected_statement`,
|
||||||
|
* `subshell`, or nested control flow such as a `do_group`)
|
||||||
|
* get full enumeration. Other named children are scanned only for nested
|
||||||
|
* executions: a `command_substitution` child directly (the `$(…)` in
|
||||||
|
* `for f in $(…)` or `case $(…) in`) is a context node in its own right — the
|
||||||
|
* strict-subtree {@link forEachNestedExecution} walk would not see it — so it
|
||||||
|
* is descended with its own context, while words, `variable_name`, test and
|
||||||
|
* expression subtrees go through the hosted-execution search.
|
||||||
|
*/
|
||||||
|
function descendControlFlowChildren(
|
||||||
|
node: TSNode,
|
||||||
|
context: BashCommandContext | undefined,
|
||||||
|
out: BashCommand[],
|
||||||
|
): void {
|
||||||
|
for (let i = 0; i < node.childCount; i++) {
|
||||||
|
const child = node.child(i);
|
||||||
|
if (!child?.isNamed) continue;
|
||||||
|
if (CONTROL_FLOW_STATEMENT_TYPES.has(child.type)) {
|
||||||
|
collectCommandsInto(child, context, out);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const nestedContext = NESTED_EXECUTION_CONTEXTS.get(child.type);
|
||||||
|
if (nestedContext) {
|
||||||
|
descendCommandChildren(child, nestedContext, out);
|
||||||
|
} else {
|
||||||
|
collectHostedCommands(child, out);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function makeUnit(
|
function makeUnit(
|
||||||
|
|||||||
@@ -0,0 +1,179 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
resetWarmBashParser,
|
||||||
|
warmBashParser,
|
||||||
|
} from "#src/access-intent/bash/parser";
|
||||||
|
import { parseBashCommandsSync } from "#src/access-intent/bash/sync-commands";
|
||||||
|
|
||||||
|
describe("collectCommands — control-flow descent", () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
resetWarmBashParser();
|
||||||
|
await warmBashParser();
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
resetWarmBashParser();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a for-loop's body commands in addition to the whole statement", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync('for d in a b; do find "$d" -name x; done'),
|
||||||
|
).toEqual([
|
||||||
|
{ text: 'for d in a b; do find "$d" -name x; done' },
|
||||||
|
{ text: 'find "$d" -name x' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a command substitution in the for value list as a nested context", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync('for f in $(find . -name x); do echo "$f"; done'),
|
||||||
|
).toEqual([
|
||||||
|
{ text: 'for f in $(find . -name x); do echo "$f"; done' },
|
||||||
|
{ text: "find . -name x", context: "command_substitution" },
|
||||||
|
{ text: 'echo "$f"' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits the condition and body commands of a redirected while loop", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync('while read l; do grep x "$l"; done < list'),
|
||||||
|
).toEqual([
|
||||||
|
{ text: 'while read l; do grep x "$l"; done' },
|
||||||
|
{ text: "read l" },
|
||||||
|
{ text: 'grep x "$l"' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits an if's condition, branch, and else commands", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync("if grep -q x f; then echo y; else rm z; fi"),
|
||||||
|
).toEqual([
|
||||||
|
{ text: "if grep -q x f; then echo y; else rm z; fi" },
|
||||||
|
{ text: "grep -q x f" },
|
||||||
|
{ text: "echo y" },
|
||||||
|
{ text: "rm z" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a case item's commands", () => {
|
||||||
|
expect(parseBashCommandsSync("case $x in a) find . ;; esac")).toEqual([
|
||||||
|
{ text: "case $x in a) find . ;; esac" },
|
||||||
|
{ text: "find ." },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a command substitution in the case subject as a nested context", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync("case $(rm x) in a) find . ;; esac"),
|
||||||
|
).toEqual([
|
||||||
|
{ text: "case $(rm x) in a) find . ;; esac" },
|
||||||
|
{ text: "rm x", context: "command_substitution" },
|
||||||
|
{ text: "find ." },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("descends a brace group", () => {
|
||||||
|
expect(parseBashCommandsSync("{ grep x f; }")).toEqual([
|
||||||
|
{ text: "{ grep x f; }" },
|
||||||
|
{ text: "grep x f" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("descends a function definition's body", () => {
|
||||||
|
expect(parseBashCommandsSync("f() { grep x f; }")).toEqual([
|
||||||
|
{ text: "f() { grep x f; }" },
|
||||||
|
{ text: "{ grep x f; }" },
|
||||||
|
{ text: "grep x f" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("descends nested loops", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync("for a in 1; do for b in 2; do find .; done; done"),
|
||||||
|
).toEqual([
|
||||||
|
{ text: "for a in 1; do for b in 2; do find .; done; done" },
|
||||||
|
{ text: "for b in 2; do find .; done" },
|
||||||
|
{ text: "find ." },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("descends a c-style for loop", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync("for ((i=0;i<3;i++)); do echo i; done"),
|
||||||
|
).toEqual([
|
||||||
|
{ text: "for ((i=0;i<3;i++)); do echo i; done" },
|
||||||
|
{ text: "echo i" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits no container units for a clean loop body", () => {
|
||||||
|
expect(parseBashCommandsSync('for d in a b; do echo "$d"; done')).toEqual([
|
||||||
|
{ text: 'for d in a b; do echo "$d"; done' },
|
||||||
|
{ text: 'echo "$d"' },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not emit a [[ ]] test as a separate command unit", () => {
|
||||||
|
expect(parseBashCommandsSync("if [[ -f x ]]; then echo y; fi")).toEqual([
|
||||||
|
{ text: "if [[ -f x ]]; then echo y; fi" },
|
||||||
|
{ text: "echo y" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("catches a substitution hosted inside a [[ ]] test", () => {
|
||||||
|
expect(
|
||||||
|
parseBashCommandsSync('if [[ "$(rm x)" = y ]]; then echo y; fi'),
|
||||||
|
).toEqual([
|
||||||
|
{ text: 'if [[ "$(rm x)" = y ]]; then echo y; fi' },
|
||||||
|
{ text: "rm x", context: "command_substitution" },
|
||||||
|
{ text: "echo y" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("collectCommands — assignment, declaration, and negation", () => {
|
||||||
|
beforeEach(async () => {
|
||||||
|
resetWarmBashParser();
|
||||||
|
await warmBashParser();
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
resetWarmBashParser();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a substitution in a plain assignment value as a nested context", () => {
|
||||||
|
expect(parseBashCommandsSync("x=$(grep foo f)")).toEqual([
|
||||||
|
{ text: "x=$(grep foo f)" },
|
||||||
|
{ text: "grep foo f", context: "command_substitution" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a substitution in an export declaration value", () => {
|
||||||
|
expect(parseBashCommandsSync("export X=$(find .)")).toEqual([
|
||||||
|
{ text: "export X=$(find .)" },
|
||||||
|
{ text: "find .", context: "command_substitution" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a substitution in a local declaration inside a function body", () => {
|
||||||
|
expect(parseBashCommandsSync("f() { local y=$(rg z); }")).toEqual([
|
||||||
|
{ text: "f() { local y=$(rg z); }" },
|
||||||
|
{ text: "{ local y=$(rg z); }" },
|
||||||
|
{ text: "rg z", context: "command_substitution" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a negated command's inner command as a normal unit", () => {
|
||||||
|
expect(parseBashCommandsSync("! grep -q x f")).toEqual([
|
||||||
|
{ text: "! grep -q x f" },
|
||||||
|
{ text: "grep -q x f" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits a negated condition's inner command inside an if", () => {
|
||||||
|
expect(parseBashCommandsSync("if ! grep -q x f; then :; fi")).toEqual([
|
||||||
|
{ text: "if ! grep -q x f; then :; fi" },
|
||||||
|
{ text: "! grep -q x f" },
|
||||||
|
{ text: "grep -q x f" },
|
||||||
|
{ text: ":" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -14,10 +14,13 @@ import { BashProgram } from "#src/access-intent/bash/program";
|
|||||||
import { resolveBashCommandCheck } from "#src/handlers/gates/bash-command";
|
import { resolveBashCommandCheck } from "#src/handlers/gates/bash-command";
|
||||||
import { pathFlavorForPlatform } from "#src/path/path-flavor";
|
import { pathFlavorForPlatform } from "#src/path/path-flavor";
|
||||||
import { PathNormalizer } from "#src/path-normalizer";
|
import { PathNormalizer } from "#src/path-normalizer";
|
||||||
|
import { PermissionResolver } from "#src/permission-resolver";
|
||||||
import type { ScopedPermissionResolver } from "#src/permission-resolver";
|
import type { ScopedPermissionResolver } from "#src/permission-resolver";
|
||||||
import type { PermissionState } from "#src/types";
|
import { SessionRules } from "#src/session-rules";
|
||||||
|
import type { DenyWithReason, PermissionState } from "#src/types";
|
||||||
|
|
||||||
import { makeCheckResult } from "#test/helpers/handler-fixtures";
|
import { makeCheckResult } from "#test/helpers/handler-fixtures";
|
||||||
|
import { createInMemoryManager } from "#test/helpers/manager-harness";
|
||||||
|
|
||||||
/** Decision strength ordering: deny (2) > ask (1) > allow (0). */
|
/** Decision strength ordering: deny (2) > ask (1) > allow (0). */
|
||||||
const STRENGTH: Record<PermissionState, number> = {
|
const STRENGTH: Record<PermissionState, number> = {
|
||||||
@@ -146,3 +149,103 @@ describe("bash command gate — nested execution hosts do not weaken", () => {
|
|||||||
expect(await decide("cat <<'EOF'\n$(rm x)\nEOF", resolver)).toBe("allow");
|
expect(await decide("cat <<'EOF'\n$(rm x)\nEOF", resolver)).toBe("allow");
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The same totality property for control-flow bodies.
|
||||||
|
*
|
||||||
|
* A `for`/`while`/`if`/`case`/function was once emitted whole without
|
||||||
|
* descending, so `for d in …; do find …; done` only ever matched patterns
|
||||||
|
* starting with `for` — a `find*: deny` rule was bypassed. The enumerator now
|
||||||
|
* emits the whole statement AND its body commands, so hosting a gated command
|
||||||
|
* inside control flow cannot weaken its decision.
|
||||||
|
*/
|
||||||
|
describe("bash command gate — control-flow bodies do not weaken", () => {
|
||||||
|
/** Real pattern matching over a bash map: `*` allow, the rest deny-with-reason. */
|
||||||
|
function makePolicyResolver(
|
||||||
|
bash: Record<string, PermissionState | DenyWithReason>,
|
||||||
|
): ScopedPermissionResolver {
|
||||||
|
const manager = createInMemoryManager({
|
||||||
|
global: { permission: { "*": "allow", bash } },
|
||||||
|
});
|
||||||
|
return new PermissionResolver(manager, new SessionRules());
|
||||||
|
}
|
||||||
|
|
||||||
|
async function checkWith(
|
||||||
|
command: string,
|
||||||
|
resolver: ScopedPermissionResolver,
|
||||||
|
) {
|
||||||
|
const program = await BashProgram.parse(
|
||||||
|
command,
|
||||||
|
new PathNormalizer(pathFlavorForPlatform(process.platform), "/cwd"),
|
||||||
|
);
|
||||||
|
return resolveBashCommandCheck(command, program.commands(), undefined, resolver);
|
||||||
|
}
|
||||||
|
|
||||||
|
it("denies the reported repro: find inside a for-loop command substitution", async () => {
|
||||||
|
const resolver = makePolicyResolver({
|
||||||
|
"*": "allow",
|
||||||
|
"find*": { action: "deny", reason: "use the find tool" },
|
||||||
|
"grep*": { action: "deny", reason: "use the grep tool" },
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await checkWith(
|
||||||
|
'cd /Users/sia/code/my-pi && ls -1 extensions/ && echo "=== LOC by ext ===" && for d in pi-*/; do n=$(find "$d" -name \'*.ts\' -not -path \'*/node_modules/*\' -not -path \'*/dist/*\' -not -path \'*/test*\' | xargs wc -l 2>/dev/null | tail -1 | awk \'{print $1}\'); echo "$d $n"; done && echo "=== extensions ts ===" && wc -l extensions/*.ts | tail -3',
|
||||||
|
resolver,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(result.state).toBe("deny");
|
||||||
|
expect(result.command).toBe(
|
||||||
|
'find "$d" -name \'*.ts\' -not -path \'*/node_modules/*\' -not -path \'*/dist/*\' -not -path \'*/test*\'',
|
||||||
|
);
|
||||||
|
expect(result.matchedPattern).toBe("find*");
|
||||||
|
expect(result.reason).toBe("use the find tool");
|
||||||
|
expect(result.commandContext).toBe("command_substitution");
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
'for d in a b; do find "$d" -name x; done',
|
||||||
|
'while read l; do grep x "$l"; done < list',
|
||||||
|
"if grep -q x f; then echo y; else echo n; fi",
|
||||||
|
"case $x in a) find . ;; esac",
|
||||||
|
"{ grep x f; }",
|
||||||
|
"f() { grep x f; }",
|
||||||
|
"for a in 1; do for b in 2; do find .; done; done",
|
||||||
|
"x=$(grep foo f)",
|
||||||
|
"export X=$(find .)",
|
||||||
|
"f() { local y=$(rg z); }",
|
||||||
|
"! grep -q x f",
|
||||||
|
"if ! grep -q x f; then :; fi",
|
||||||
|
])("denies a gated command inside `%s`", async (command) => {
|
||||||
|
const resolver = makePolicyResolver({
|
||||||
|
"*": "allow",
|
||||||
|
"find*": "deny",
|
||||||
|
"grep*": "deny",
|
||||||
|
"rg*": "deny",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect((await checkWith(command, resolver)).state).toBe("deny");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("denies a hard-guard payload hidden in a loop body", async () => {
|
||||||
|
const resolver = makePolicyResolver({
|
||||||
|
"*": "allow",
|
||||||
|
"rm -rf*": "deny",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = await checkWith("for f in a; do rm -rf /; done", resolver);
|
||||||
|
expect(result.state).toBe("deny");
|
||||||
|
expect(result.matchedPattern).toBe("rm -rf*");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("allows a control-flow statement whose body commands are allowed", async () => {
|
||||||
|
const resolver = makePolicyResolver({
|
||||||
|
"*": "allow",
|
||||||
|
"find*": "deny",
|
||||||
|
"grep*": "deny",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(
|
||||||
|
(await checkWith('for d in a b; do echo "$d"; done', resolver)).state,
|
||||||
|
).toBe("allow");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import {
|
|||||||
autoReviewConfigFileSchema,
|
autoReviewConfigFileSchema,
|
||||||
normalizeAutoReviewConfig,
|
normalizeAutoReviewConfig,
|
||||||
} from "#src/auto-review/config";
|
} from "#src/auto-review/config";
|
||||||
|
import { stripJsonComments } from "#src/config-loader";
|
||||||
import { unifiedConfigSchema } from "#src/config-schema";
|
import { unifiedConfigSchema } from "#src/config-schema";
|
||||||
|
|
||||||
/** The bundle's authoritative policy seed, as shipped. */
|
/** The bundle's authoritative policy seed, as shipped. */
|
||||||
@@ -16,10 +17,10 @@ const SEED_PATH = fileURLToPath(
|
|||||||
|
|
||||||
function readSeed(): Record<string, unknown> {
|
function readSeed(): Record<string, unknown> {
|
||||||
const raw = readFileSync(SEED_PATH, "utf8");
|
const raw = readFileSync(SEED_PATH, "utf8");
|
||||||
const stripped = raw
|
// Parse with the same string-aware stripper the loader uses: a naive regex
|
||||||
.replace(/^\s*\/\/.*$/gm, "")
|
// strips `/*`-looking runs inside string values (e.g. a `'src/**/*.ts'`
|
||||||
.replace(/\/\*[\s\S]*?\*\//g, "");
|
// glob in a deny reason) and corrupts the JSON.
|
||||||
return JSON.parse(stripped) as Record<string, unknown>;
|
return JSON.parse(stripJsonComments(raw)) as Record<string, unknown>;
|
||||||
}
|
}
|
||||||
|
|
||||||
describe("bundled policy seed", () => {
|
describe("bundled policy seed", () => {
|
||||||
|
|||||||
@@ -2,9 +2,10 @@
|
|||||||
|
|
||||||
- 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。
|
- 扩展入口是 `pi-rtk-optimizer/index.ts`,主要实现位于 `pi-rtk-optimizer/src/`。
|
||||||
- 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。
|
- 组合包运行环境为 Node.js 22.19 或更高版本;本子目录开发验证还需要 npm、Bun 和项目声明的开发依赖。
|
||||||
- RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合和兜底截断;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。
|
- RTK 默认只压缩非搜索输出,包括本地 `bash` 与远端 `ssh_bash` 的 ANSI 清理、测试聚合、构建过滤、Git 压缩、Lint 聚合;兜底截断默认关闭,新结果的展示截断由 pi-overflow 统一负责;`ssh_bash` 只复用输出处理,远端命令不得进入 RTK rewrite。
|
||||||
- RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。
|
- RTK 命令改写默认关闭(`commandRewritingEnabled` 默认 `false`);默认安装不依赖系统 `rtk` CLI,只有用户主动开启命令改写时才需要 `rtk rewrite`,其命令支持策略由已安装的 `rtk rewrite` 决定,不在扩展内重复维护规则。
|
||||||
- FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。
|
- FFF override 独占搜索工具与搜索输出;RTK 不得处理本地或远端搜索工具的调用/结果(含 `grep` 工具结果),也不得通过默认命令改写接管 `rg`、`grep`、`find`、`fd`。
|
||||||
|
- RTK 不处理带 `parentToolCallId` 的嵌套调用(codemode 脚本内调用),否则会丢失 bash 的 structuredContent。
|
||||||
- `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。
|
- `readCompaction.enabled`、`sourceCodeFilteringEnabled` 和 `smartTruncate.enabled` 当前均默认 `false`,源码读取保持原样;未经用户明确决定,不因节省上下文而改变这些默认值。若以后开启 read 压缩,优先考虑 `readCompaction + smartTruncate`,源码过滤仍独立评估;必须保留精确 `offset/limit` 读取、短文件和行锚点的完整性。
|
||||||
- read 压缩属于有损能力。任何默认值调整都必须同时说明哪些内容可能被省略、精确读取如何恢复原文,以及对 `edit` 文本匹配和排障证据的影响。
|
- read 压缩属于有损能力。任何默认值调整都必须同时说明哪些内容可能被省略、精确读取如何恢复原文,以及对 `edit` 文本匹配和排障证据的影响。
|
||||||
- 工具输出压缩可能损失证据。排障和审计相关改动应优先保证原始输出可恢复,并覆盖锚点完整性和截断边界。
|
- 工具输出压缩可能损失证据。排障和审计相关改动应优先保证原始输出可恢复,并覆盖锚点完整性和截断边界。
|
||||||
|
|||||||
@@ -154,7 +154,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
|
|||||||
|--------|------|---------|-------|-------------|
|
|--------|------|---------|-------|-------------|
|
||||||
| `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled |
|
| `outputCompaction.smartTruncate.enabled` | boolean | `false` | — | Enable smart line-based truncation for read output when read compaction is enabled |
|
||||||
| `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation |
|
| `outputCompaction.smartTruncate.maxLines` | number | `220` | 40–4000 | Maximum lines after smart truncation |
|
||||||
| `outputCompaction.truncate.enabled` | boolean | `true` | — | Enable hard character truncation |
|
| `outputCompaction.truncate.enabled` | boolean | `false` | — | Enable hard character truncation (off in this bundle — pi-overflow owns display truncation) |
|
||||||
| `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output |
|
| `outputCompaction.truncate.maxChars` | number | `12000` | 1000–200000 | Maximum characters in final output |
|
||||||
|
|
||||||
### Source Code Filtering Levels
|
### Source Code Filtering Levels
|
||||||
@@ -195,7 +195,7 @@ When `read` output uses Pi hashline/anchor prefixes, the compactor treats each a
|
|||||||
"maxLines": 220
|
"maxLines": 220
|
||||||
},
|
},
|
||||||
"truncate": {
|
"truncate": {
|
||||||
"enabled": true,
|
"enabled": false,
|
||||||
"maxChars": 12000
|
"maxChars": 12000
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,3 +13,4 @@
|
|||||||
|
|
||||||
- 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。
|
- 默认只负责非搜索工具输出压缩;RTK 命令改写收敛为可选能力(`commandRewritingEnabled` 默认 `false`)。
|
||||||
- `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。
|
- `readCompaction`、`sourceCodeFiltering`、`smartTruncate` 默认关闭,源码读取保持原样。
|
||||||
|
- `outputCompaction.truncate` 兜底截断默认关闭(组合包 `config/rtk-optimizer.json`),新结果的展示截断由 pi-overflow 统一负责;RTK 只保留语义压缩。
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user