feat(pi-condense): vendor source and add summarizer retries

2026-10-26 10:45:22 +00:00 · 2026-08-31 15:13:41 +08:00
parent 8002d7c213
commit e80959cc3f
94 changed files with 21327 additions and 185 deletions
@@ -0,0 +1,211 @@
+# Configuration reference
+
+Full `contextPrune` settings, commands, footer widget states, spilled outputs,
+and the summarizer-model-by-plan table. See the [README](../README.md) for the
+6 knobs most people touch and the conceptual model; see
+[PRUNING.md](../PRUNING.md) for the algorithm behind each setting.
+
+## Full settings JSON
+
+Settings live under the `contextPrune` key in `<agent-dir>/settings.json` (i.e. pi's own settings file). `<agent-dir>` is `$PI_CODING_AGENT_DIR` if set, otherwise `~/.pi/agent`. Each pi preset gets its own settings, so you can run different summarizer models per preset. Missing fields use the maintained my-pi defaults in memory; explicit values are preserved and extension load does not rewrite the file.
+
+```json
+{
+  "contextPrune": {
+    "enabled": true,
+    "showPruneStatusLine": true,
+    "summarizerModel": "default",
+    "summarizerThinking": "default",
+    "summarizerIdleTimeoutMs": 90000,
+    "summarizerMaxTimeoutMs": 300000,
+    "summarizerMaxRetries": 2,
+    "summarizerRetryBaseDelayMs": 1000,
+    "summarizerRetryMaxDelayMs": 8000,
+    "pruneOn": "agent-message",
+    "batchingMode": "turn",
+    "quietOversizedSkips": false,
+    "minBatchChars": 1000,
+    "recoveryGraceTurns": 3,
+    "protectedTools": [],
+    "protectedPaths": ["**/skills/**/*.md"],
+    "dedupByContentHash": true,
+    "autoBudgetThreshold": 0.735,
+    "spillThreshold": 65536,
+    "spillPreviewBytes": 2048,
+    "budgetTurnDelta": 0.04,
+    "chainCompression": {
+      "enabled": true,
+      "rollingWindow": 3,
+      "stripFinalAssistantThinking": true,
+      "fuseRangeSummary": true
+    }
+  }
+}
+```
+
+## Every key
+
+| Key | Values | Default | Notes |
+|---|---|---|---|
+| `enabled` | `true` / `false` | `true` | Master switch |
+| `showPruneStatusLine` | `true` / `false` | `true` | Footer widget + queued-turn notifications |
+| `summarizerModel` | `"default"` or `"provider/model-id"` | `"default"` | `default` = your active pi model. See [Choosing a summarizer model](#choosing-a-summarizer-model) |
+| `summarizerThinking` | `default`/`off`/`minimal`/`low`/`medium`/`high`/`xhigh` | `default` | Provider-specific reasoning effort knob |
+| `pruneOn` | `agent-message` / `on-demand` | `agent-message` | Trigger mode - see README Architecture section |
+| `batchingMode` | `turn` / `agent-message` | `turn` | How coarse each summary is (independent of `pruneOn`) |
+| `quietOversizedSkips` | `true` / `false` | `false` | Silences `skipped-oversized` / `skipped-trivial` info notifications |
+| `minBatchChars` | non-negative integer, `0` disables | `1000` | Pre-flush guard - batches smaller than this skip the LLM entirely |
+| `recoveryGraceTurns` | non-negative integer (user-turn-groups), `0` disables | `3` | After a `context_tree_query` recovery, render that tool's output verbatim for this many user-turn-groups before re-stubbing it. Enforced at render time (Phase 1 + chain-compression eligibility), not at capture time. See [PRUNING.md § What Pruning Does](../PRUNING.md#what-pruning-does) |
+| `summarizerIdleTimeoutMs` | non-negative integer (ms), `0` disables | `90000` | Abort a summarizer attempt after this much silence. Resets on every stream event and also bounds time-to-first-token. A timeout is transient and can consume retry slots. |
+| `summarizerMaxTimeoutMs` | non-negative integer (ms), `0` disables | `300000` | Hard ceiling on one summarizer attempt. Each retry is independently bounded. |
+| `summarizerMaxRetries` | integer `0`-`10` | `2` | Transient retries after the initial attempt. `2` means three total attempts. With a distinct summarizer model, the final slot is reserved for session-model fallback. `0` reports the first failure immediately and disables fallback rescue. |
+| `summarizerRetryBaseDelayMs` | non-negative integer (ms) | `1000` | Delay before retry 1; subsequent waits double. `0` makes retries immediate. |
+| `summarizerRetryMaxDelayMs` | non-negative integer (ms) | `8000` | Per-wait cap for exponential retry backoff. `0` makes retries immediate. |
+| `protectedTools` | `string[]` | `[]` | Never-pruned tool names (e.g. `["todowrite","todoread"]`). When a protected tool's chain is range-compressed, its output is preserved verbatim inside the `<compressed-chain>` block as `<protected-output>` - protected outputs are never lost. |
+| `protectedPaths` | `string[]` | `["**/skills/**/*.md"]` | Globs matched against a tool call's `args.path`; matching outputs are never pruned (same semantics as `protectedTools`, including `<protected-output>` relocation in compressed chains). Already-summarized matching reads are repaired on the next turn; chain-compressed ones are not. Set `[]` to disable. |
+| `dedupByContentHash` | `true` / `false` | `true` | Re-reads of identical (toolName, content) skip the LLM and alias the original |
+| `autoBudgetThreshold` | fraction `0`-`1`, or `null` | `0.735` | Token-budget auto-flush at 73.5% of the window or 300k tokens, whichever comes first, regardless of `pruneOn`. `null` disables it. See [Token-budget auto-flush](#token-budget-auto-flush) |
+| `spillThreshold` | positive integer | `65536` | Minimum chars (`resultText.length`) for a single tool result to be spilled eagerly to a sidecar file at capture time rather than waiting for normal summarization. Non-positive / invalid values fall back to the default; to effectively disable spilling, set it above any result you expect. See [Spilled outputs](#spilled-outputs) |
+| `spillPreviewBytes` | non-negative integer | `2048` | Head preview (bytes) kept inline in the stub and index record for a spilled result. Full body is on disk. |
+| `budgetTurnDelta` | fraction `0`-`1` | `0.04` | Force a flush when one turn grows context by at least 4%, ORed with `autoBudgetThreshold`. The fraction is measured against `min(context window, 300k)`. |
+| `chainCompression.enabled` | `true` / `false` | `true` | Master toggle for chain-level range compression |
+| `chainCompression.rollingWindow` | positive integer | `3` | Keep this many most-recent closed chains raw; compress older ones |
+| `chainCompression.stripFinalAssistantThinking` | `true` / `false` | `true` | Strip thinking blocks from the kept final text-only assistant when compressing |
+| `chainCompression.fuseRangeSummary` | `true` / `false` | `true` | Fuse a compressed chain's per-batch summaries into one cohesive LLM summary (one extra summarizer call per multi-batch span); off keeps the per-batch concatenation |
+| `purgeErrors.enabled` | `true` / `false` | `true` | Replace failed toolCall argument bodies with compact stubs after cooldown |
+| `purgeErrors.cooldownTurns` | positive integer | `2` | Turns to wait after a tool error before purging its argument body |
+| `purgeErrors.minArgChars` | non-negative integer | `500` | Only purge arg bodies at least this many characters long |
+
+See [PRUNING.md § Chain Compression](../PRUNING.md#chain-compression) and [PRUNING.md § Error Purge](../PRUNING.md#error-purge) for the full algorithms.
+
+The three pre-flush features (`minBatchChars`, `protectedTools`, `dedupByContentHash`) are explained in [PRUNING.md § Pre-flush Pipeline & Safeguards](../PRUNING.md#pre-flush-pipeline--safeguards). They run BEFORE any summarizer LLM call and can each drop a batch outright while still advancing the prune frontier.
+
+### Token-budget auto-flush
+
+When `autoBudgetThreshold` is set to a value in `(0, 1]`, the extension checks context usage at the end of every tool-using turn. If `tokens` reaches `min(300_000, threshold * contextWindow)`, ALL pending batches are flushed immediately - regardless of `pruneOn` mode. This is an **additional** trigger layered on top of `pruneOn`, not a replacement.
+
+- `0.8` means 80% of the context window - it is a **fraction**, not a percentage. `0.8 != 80`.
+- **The trigger point is capped at 300,000 tokens** (`MAX_BUDGET_WINDOW` in `src/budget.ts`). Without it, an advertised 1M window makes every setting unreachable: `0.9` would mean 900k tokens, so a session ends before pruning ever fires. The ceiling never binds on a model advertising 300k or less - a 256k window at `0.9` still fires at 230.4k - so only huge-window models change behavior, and they flush earlier.
+- `budgetTurnDelta` applies the same ceiling in the other shape: growth is measured against `min(context window, 300k)`, because a 300k ceiling on a single turn's *growth* could never bind.
+- The trigger is a no-op when `tokens` is `null` (right after a provider-side compaction); it resumes once usage is known again.
+- Editable live via `/pruner settings` (row "Auto-flush at context %", presets Off / 60 / 70 / 80 / 90%).
+- Default `null` = off.
+
+Inspired by DCP's `maxContextLimit` nudging; simplified to a single threshold that forces a flush rather than separate nudge/force levels.
+
+### Spilled outputs
+
+Single tool results larger than `spillThreshold` chars are written to `<session-dir>/<sessionId>-blobs/<key>.txt` at capture time and replaced in context with a short stub (tool name, byte size, head preview, file path). `<key>` is the sanitized occurrence key `id@resultTimestamp` (`@` becomes `_`, so `bash_23@1150` -> `bash_23_1150.txt`), not the bare `toolCallId` - keying on the bare id alone would let two occurrences of one reused provider id share a path and the second overwrite the first's blob. Recovery reads the `spillPath` persisted on the record rather than re-deriving it from the id, so a bare-id-named sidecar (no occurrence timestamp) still resolves. See [PRUNING.md § Occurrence Identity](../PRUNING.md#occurrence-identity). The full body is recoverable via the native `read` tool at the embedded path (offset/limit supported) or via `context_tree_query` by id, which falls back to the inline preview if the sidecar is missing. Moving a session `.jsonl` without its `-blobs/` directory loses only the giant-blob recovery path; bodies under `spillThreshold` stay inline in the index entry as usual.
+
+### Choosing a summarizer model
+
+The `default` setting reuses whatever model you have active in pi - convenient but wasteful, since summary writing doesn't need a top-tier coding model. Picking the smallest/fastest model on your plan saves both latency and cost.
+
+If the configured summarizer model suffers a transient outage while your active pi model is healthy, pi-condense automatically falls back to the session model for the duration (with a one-time notice) and probes the configured model back every few minutes - no configuration needed.
+
+| Plan | Suggested summarizer |
+|---|---|
+| OpenAI / Codex / Copilot | `openai/gpt-4.1-mini`, `google/gemini-2.5-flash`, `xai/grok-3-fast` |
+| OpenRouter | `openrouter/qwen/qwen3-30b-a3b` (cheap MoE) |
+| Anthropic direct | `anthropic/claude-haiku-3-5` |
+| Google AI direct | `google/gemini-2.5-flash` |
+
+Set it from the slash command (saves immediately):
+
+```bash
+/pruner model openai/gpt-4.1-mini
+/pruner thinking low
+# or both in one go:
+/pruner model openai/gpt-4.1-mini:low
+```
+
+### Summarizer timeouts and retries
+
+Every summarizer attempt is bounded by two independent timers:
+
+- **Idle timeout** (`summarizerIdleTimeoutMs`, default 90s) resets on every
+  stream event, including reasoning/`thinking` events. It also bounds
+  time-to-first-token.
+- **Total-duration ceiling** (`summarizerMaxTimeoutMs`, default 300s) is a hard
+  upper bound for one attempt.
+
+Timeouts and provider failures are classified as transient. The initial call is
+followed by at most `summarizerMaxRetries` retries (default 2), with exponential
+backoff starting at `summarizerRetryBaseDelayMs` and capped by
+`summarizerRetryMaxDelayMs` (`1s → 2s` under maintained defaults). The UI
+reports an error only after the retry budget is exhausted. Each attempt has its
+own idle and ceiling timers, so the worst-case duration can span multiple attempt
+ceilings; user abort remains immediate, including during backoff.
+
+Authentication failures and unusable empty/length-truncated summaries do not
+consume retries. With a distinct `summarizerModel`, the final retry slot switches
+to the active session model and feeds the sticky fallback controller. Set
+`summarizerMaxRetries` to `0` to disable retries and fallback rescue. Both timeout
+fields can independently be set to `0`.
+
+## Commands
+
+| Command | Effect |
+|---|---|
+| `/pruner` | Interactive picker over all subcommands |
+| `/pruner settings` | Settings overlay (toggle / cycle every option) |
+| `/pruner on` / `off` | Enable / disable pruning |
+| `/pruner status` | Show mode, model, trigger, cumulative stats, and a `--- context ---` block (see below) |
+| `/pruner stats` | Detailed cumulative summarizer token/cost stats |
+| `/pruner model [id[:thinking]]` | Get / set summarizer model (and optionally thinking level) |
+| `/pruner thinking [level]` | Get / set summarizer reasoning effort |
+| `/pruner prune-on [mode]` | Get / set trigger mode |
+| `/pruner batching [mode]` | Get / set batching granularity (`turn` / `agent-message`) |
+| `/pruner protected-tools [names]` | Show or edit the never-pruned tool allowlist (comma- or space-separated; `none` clears) |
+| `/pruner protected-paths [globs]` | Show or edit the never-pruned path globs (`none` clears) |
+| `/pruner min-batch-chars [n]` | Show or set the pre-flush trivial-batch threshold (`0` disables) |
+| `/pruner recovery-grace [n]` | Show or set the post-recovery verbatim grace window, in user-turn-groups (`0` disables) |
+| `/pruner dedup [on\|off\|status]` | Toggle pre-flush content-hash dedup |
+| `/pruner tree` | Foldable browser of pruned tool calls; `Ctrl-O` opens the full summary in an overlay |
+| `/pruner compact` | Retroactively compress every eligible closed chain (bypasses `rollingWindow`) |
+| `/pruner now` | Flush pending batches immediately with a multi-row progress widget above the input |
+| `/pruner help` | Full help text |
+
+## Tools surfaced to the LLM
+
+**`context_tree_query`** - always available when the extension is loaded. Pruned summaries end with short refs like `Summarized tool refs: \`t1\`, \`t2\`. Use \`context_tree_query\` with these refs to retrieve the original full outputs.` The model passes those refs (or full `toolCallId`s) and gets back the original tool result text from the session index. Each per-tool bullet in the summary also carries its own inline `` `tN` `` ref, so recovering a specific tool is a single hop; the footer still lists every ref as a fallback. Content-hash-deduped duplicates resolve to the original's record automatically.
+
+Short refs (`tN`) always resolve 1:1 and are the primary, recommended path; a reused `toolCallId` returns every matching occurrence instead of silently picking one, including occurrences deduplicated to an earlier record. See [PRUNING.md § Occurrence Identity](../PRUNING.md#occurrence-identity) for multi-occurrence recovery mechanics.
+
+## Footer status widget
+
+A footer widget shows the current state, controlled by `showPruneStatusLine`:
+
+Every rendered state is prefixed with a single leading `|` divider so the segment stays visually isolated in the shared footer regardless of where other extensions' status segments land (load-order independent); there is no trailing divider - the footer's own space-join between segments already provides one.
+
+- `| prune: OFF` - disabled
+- `| prune: ON` - enabled, no flushes yet
+- `| prune: ON . 92.0k->14.0k (-85%)` - enabled; live reclaim ratio (estimated tokens before->after, percent reduction). Updates on every `pruneMessages` call.
+- `| prune: 3 pending` - batches queued, waiting for the trigger
+- `| prune: summarizing...` - flush in progress
+- `| prune: ON . 92.0k->14.0k (-85%) . diag u1/o2` - same as above, plus a self-hiding diagnostic segment (`u`/`m`/`o` counters, each hidden at zero). See [PRUNING.md § Diagnostics](../PRUNING.md#diagnostics).
+- `| prune: ON . think 3.2k . gap 5.1k . chain 41%` - a context-pressure suffix, shown only when `frontierGapTokens > 0` (nothing to append when the frontier is fully caught up). All three are chars/4 token estimates, same convention as the reclaim ratio above:
+  - `think` - `openCycleThinkingTokens`: thinking-block tokens trapped in the still-open cycle (after the last text-only assistant reply), not yet eligible for pruning.
+  - `gap` - `frontierGapTokens`: summarization-eligible tool-result tokens sitting past the persisted prune frontier.
+  - `chain` - `largestChainSharePct`: the largest single chain's (or the open segment's) share of total branch chars.
+  See [PRUNING.md § Single-chain sessions](../PRUNING.md#single-chain-sessions) for exact definitions.
+- `| prune: recovered pending (reload)` - shown at `agent_end` when a session reload found recoverable unflushed work but no new turn re-queued it. See [PRUNING.md § Reload rearm](../PRUNING.md#reload-rearm) for what "recovered" means and when it actually flushes.
+
+Setting `showPruneStatusLine: false` hides the widget and silences the queued-turn notice; pruning still runs.
+
+The status line does not show cost. Full token/cost detail is available via `/pruner stats`. The extension also emits cumulative session cost on the `cost:external` pi.events channel for external aggregators - see [README § External cost channel](../README.md#external-cost-channel).
+
+### `/pruner status` context block
+
+When context metrics are available, `/pruner status` appends a `--- context ---` block below the settings summary:
+
+```
+--- context ---
+thinking:     3.2k tokens (open segment)
+chain share:  41%
+frontier gap: 5.1k tokens
+rearmed:      yes
+```
+
+- `thinking` / `chain share` / `frontier gap` are the same `openCycleThinkingTokens` / `largestChainSharePct` / `frontierGapTokens` metrics shown in the footer suffix above, formatted as full labels here instead of a compact suffix.
+- `rearmed: yes` appears only while the reload-rearm flag is armed (cleared the moment any flush attempt runs). See [PRUNING.md § Reload rearm](../PRUNING.md#reload-rearm).