my-pi/pi-condense/doc/specs/2026-07-09-summarizer-call-timeout.md

Summarizer call timeout (idle + ceiling)

Date: 2026-07-09 Status: proposed Area: src/summarizer.ts, src/types.ts, src/config.ts, src/commands.ts, docs

Context

runOnce in src/summarizer.ts drives a stream() call with exactly one escape hatch: options.signal (the user's Esc). It has no time budget. The stream loop for await (const event of responseStream) and the trailing await responseStream.result() both block indefinitely if the provider accepts the connection but stalls (no bytes, no close — common under provider congestion).

The automatic flush paths (turn_end, message_end, agent_end in index.ts) call flushPending without a signal, and flushPending awaits summarization inline. So a stalled summarizer call hangs the whole agent turn with no user-visible cause and no recovery.

Measured on this laptop (32,122 real assistant-generation latency samples across all sessions; dominant model claude-opus-4-8), a legitimate summarizer call is not "a handful of seconds":

Summarizer output size	median	p95	p99	observed max
~800 tok (median call)	12.9s	29.3s	40.7s	331s
~1.7-2k tok (p90-95)	26.6s	44.2s	55.4s	290s
~2.8k tok (p99)	40.7s	62.7s	76.3s	256s
~5k tok (max)	64s	92.5s	124.6s	208s

Per-call output-token distribution (11,092 samples from consecutive context-prune-stats deltas): median 798, p95 2032, p99 2779 tokens.

Implication: a flat total-duration timeout that avoids false-aborting healthy calls would have to sit at ~200s+, and under the outage-fallback retry (below) a hang would then burn ~2x that before giving up. That is not an acceptable "short" bound.

Problem

A stalled summarizer connection hangs the agent turn indefinitely. There is no timeout anywhere in the call chain, and the automatic flush paths pass no abort signal, so nothing can cancel it.

Idea

Bound every summarizer call with a two-part timeout inside runOnce, driven by a local AbortController combined with the caller's options.signal:

1. Idle (inactivity) timeout — primary mechanism. A healthy stream emits text_start / text_delta / text_end events continuously (sub-second inter-token gaps). A true hang emits nothing. Arm a timer for summarizerIdleTimeoutMs before the loop (this covers time-to-first-token) and reset it on every stream event. If it fires, abort the stream. Because it measures silence, not total time, it never false-aborts a legitimately long-but-flowing generation regardless of output size.
2. Total-duration ceiling — backstop. A single timer for summarizerMaxTimeoutMs armed once at call start, never reset. Catches the pathological "slow-drip-forever" stream that keeps emitting a token every N < idle seconds but never completes. Generous by design: it only clips calls beyond the observed legitimate p99.

Both are configurable; 0 disables either independently.

A timeout classifies as transient so it flows through the existing outage-fallback state machine in runSummarization (option B): when a distinct summarizerModel is configured, a primary that times out gets the one existing session-model retry (itself bounded by the same idle+ceiling timeout). This is not a new retry loop — it reuses the machine that already exists for provider outages. When summarizerModel is "default" (no distinct fallback), the single attempt times out, warns, and drops the batch (retried on the next natural flush; frontier does not advance).

A timeout is surfaced to the user via ctx.ui.notify(..., "warning") with timeout-specific wording, distinct from the existing "error"-severity API-failure notice.

Values (grounded in the data above)

Setting	Default	Rationale
summarizerIdleTimeoutMs	20000 (20s)	Healthy inter-token gaps are sub-second; the only real spike is TTFT under congestion (~<3s, occasionally ~10s). 20s gives comfortable headroom over TTFT while catching a silent hang fast. Values < 15s risk false-aborting TTFT spikes.
summarizerMaxTimeoutMs	180000 (180s / 3 min)	Clears every observed p99 (max p99 was 124.6s) with headroom; only clips pathological multi-minute outliers (208-331s completed calls, i.e. provider degradation). Pure backstop.

Worst-case wall-clock for a genuine silent hang under option B: idle (primary stalls, ~20s) + idle (session-model retry also stalls, ~20s) = ~40s, then one warning + drop. The 180s ceiling only participates when a stream is dribbling (never goes fully idle) yet never finishes.

Design

Config (src/types.ts)

Add two fields to ContextPruneConfig and DEFAULT_CONFIG:

/**
 * Idle (inactivity) timeout for a single summarizer stream call, in ms.
 * Reset on every received stream event; armed before the first event so it
 * also bounds time-to-first-token. If no event arrives within this window the
 * call is aborted and classified transient (feeds the outage-fallback retry).
 * 0 disables the idle timer. Default 20000.
 */
summarizerIdleTimeoutMs: number;
/**
 * Total-duration ceiling for a single summarizer stream call, in ms. Armed
 * once at call start, never reset — a hard upper bound that catches a stream
 * that keeps dribbling events but never completes. Same transient/warning
 * handling as the idle timeout. 0 disables the ceiling. Default 180000.
 */
summarizerMaxTimeoutMs: number;

DEFAULT_CONFIG: summarizerIdleTimeoutMs: 20000, summarizerMaxTimeoutMs: 180000.

Add two preset arrays for the settings overlay (strings, "0" = disabled sentinel), mirroring the existing *_PRESETS pattern:

export const IDLE_TIMEOUT_PRESETS = [
  { value: "0", label: "0 (disabled)" },
  { value: "10000", label: "10s" },
  { value: "20000", label: "20s (default)" },
  { value: "45000", label: "45s" },
  { value: "90000", label: "90s" },
];
export const MAX_TIMEOUT_PRESETS = [
  { value: "0", label: "0 (disabled)" },
  { value: "120000", label: "120s" },
  { value: "180000", label: "180s (default)" },
  { value: "300000", label: "300s" },
  { value: "600000", label: "600s" },
];

Normalization (src/config.ts)

In normalize(), clamp both like the existing minBatchChars guard: finite number >= 0 → Math.floor(value), else the default. 0 is a valid (disabling) value, so the guard is >= 0, not > 0.

runOnce (src/summarizer.ts) — the core change

• Extend the existing transient variant with an optional timedOut field — it keeps message (do not drop it or add a new variant):

  | { kind: "transient"; message: string; timedOut?: boolean }
  

• Read the budgets from config at the top of runOnce:

  const idleMs = config.summarizerIdleTimeoutMs;
  const maxMs = config.summarizerMaxTimeoutMs;
  let timedOut = false;
  let timeoutKind: "idle" | "ceiling" | null = null;
  let idleTimerId: ReturnType<typeof setTimeout> | null = null;
  let ceilingTimerId: ReturnType<typeof setTimeout> | null = null;
  

• combineSignals is a small local helper (inline in summarizer.ts), tolerant of an absent caller signal:

  function combineSignals(...signals: (AbortSignal | undefined)[]): AbortSignal | undefined {
    const present = signals.filter((s): s is AbortSignal => !!s);
    if (present.length === 0) return undefined;
    if (present.length === 1) return present[0];
    return AbortSignal.any(present); // Node 20+; host runtime is node 24.5.0
  }
  

  Build a local AbortController (timeoutController) and pass stream(..., { signal: combineSignals(options.signal, timeoutController.signal), ... }).
• Idle timer: a helper bumpIdle() that clearTimeout(idleTimerId) and, when idleMs > 0, re-arms idleTimerId = setTimeout(() => { timedOut = true; timeoutKind = "idle"; timeoutController.abort(); }, idleMs). Call bumpIdle() once before the loop (covers TTFT), then inside the loop on every received event — not only text_*. Reasoning-capable summarizer models (summarizerThinking wires reasoningEffort) emit thinking_start/thinking_delta/thinking_end (pi-ai types.d.ts:275-287); bumping on any event keeps a legitimately-reasoning-but-quiet stream alive, so extended reasoning never false-aborts. reportTextProgress stays gated to text_* events (unchanged).
• Ceiling timer: ceilingTimerId = setTimeout(() => { timedOut = true; timeoutKind ??= "ceiling"; timeoutController.abort(); }, maxMs) when maxMs > 0, armed once at call start.
• Clear both timers in a finally (clearTimeout(idleTimerId); clearTimeout(ceilingTimerId);) so no timer leaks after the call returns or throws.
• Classification (centralize so no transient site escapes it): a timeout abort can surface either as a throw in the catch OR as response.stopReason === "error" (pi-ai's error event, types.d.ts:307). At every transient return site — the stopReason === "error" branch and the catch — set timedOut/timeoutKind into the outcome: if timedOut && !options.signal?.aborted, return { kind: "transient", timedOut: true, message: <worded by timeoutKind> } — e.g. `summarizer ${modelLabel(model)} stalled (no output for ${idleMs / 1000}s)` for idle, `summarizer ${modelLabel(model)} exceeded ${maxMs / 1000}s ceiling` for ceiling. A simple way: compute the timeout message once from timeoutKind and prefer it whenever timedOut is set, at both sites.
• User abort is unchanged: options.signal?.aborted still short-circuits and re-throws so flushPending restores state silently. The timeout controller aborting does not set options.signal.aborted (it is a separate controller), so a timeout is never misclassified as a user abort.

runSummarization (src/summarizer.ts) — warning severity

Replace the bare notifyError sites with a helper that picks severity from the outcome. Wording is path-neutral — runSummarization is shared by summarizeBatch and summarizeRange, and a timed-out range fusion neither "drops a batch" nor "retries next flush" (it falls back to per-batch concat at render, see below), so the message must not assert batch semantics:

const notifyFailure = (o: { message: string; timedOut?: boolean }) =>
  ctx.ui.notify(
    o.timedOut
      ? `pi-condense: ${o.message}; summarizer call abandoned`
      : `pruner: summarization failed: ${o.message}`,
    o.timedOut ? "warning" : "error",
  );

Threaded through:

• Legacy no-fallback path: case "transient": notifyFailure(r); return null;
• Fallback path, fallback-only fail: notifyFailure(r).
• Fallback path, both-down: notifyFailure(r2.kind === "transient" || r2.kind === "auth" ? r2 : r) — preserves today's exact message-selection (the auth arm must stay: a session-model auth failure after a primary timeout should surface the auth error, not the stale timeout message) and carries whichever timedOut flag applies.
• Fallback rescue (primary times out, session model succeeds): no failure notice; the existing "enter" fallback warning (summarizer model X failing, using session model Y until it recovers, already "warning" severity) fires via emit. A timeout is treated exactly like any transient here. Scope note: the "every timeout gets timeout-specific wording" guarantee applies to non-rescued timeouts only; a rescued primary timeout is reported solely as the generic fallback-entry warning. This is intentional — the rescue path is not a failure and does not warrant a second, redundant notice.

No change to the FallbackController itself: a timeout is a transient, and the controller already handles transient primary → session retry → sticky-until-probe. Option B falls out of the existing machine for free.

Commands / UI (src/commands.ts)

• Imports: export IDLE_TIMEOUT_PRESETS / MAX_TIMEOUT_PRESETS from src/types.ts and import them in src/commands.ts alongside the existing MIN_BATCH_CHARS_PRESETS etc.
• SettingsList entries: add two entries in the settings overlay (placed after recoveryGraceTurns), each using its preset array via values: PRESETS.map((p) => p.value) with the same currentValue-in-cycle fallback as minBatchChars (fall back to the default preset value when the stored value is not in the cycle). Description text states the disabling sentinel (0) and the idle-vs-ceiling distinction.
• onChange wiring (required): every setting has an explicit id-matched branch in the overlay's onChange handler; a SettingsList entry without one renders but never persists. Add branches for summarizerIdleTimeoutMs and summarizerMaxTimeoutMs that parseInt the new value, clamp >= 0 (fall back to the default on NaN/negative), assign to newConfig, and refresh the description — mirroring the minBatchChars branch.
• /pruner status: append two lines alongside the existing model/thinking lines, formatted idle timeout: 20s / max timeout: 180s, rendering disabled when the value is 0.
• Out of scope: dedicated /pruner idle-timeout / /pruner max-timeout subcommands. The settings overlay + settings.json are the surfaces; a bespoke subcommand each is YAGNI. Add later if asked.

Error handling and edge cases

• Idle disabled (0), ceiling set: only the ceiling timer arms. The ceiling setTimeout is armed at call start and fires on schedule regardless of stream activity (JS timers are not blocked by a pending await), so even a stream that never yields its first event is still caught — at the ceiling (up to 180s), not fast. Accepted; it is the user's explicit choice. With both disabled there is no first-token bound at all (see next).
• Both disabled (0/0): exact current behavior (no timeout), including no first-token bound — a stream that never yields hangs indefinitely, the pre-feature bug. Preserved deliberately as the escape hatch for anyone who hits false aborts.
• Reasoning summarizer models: because the idle timer resets on thinking_* events too (not just text_*), a model doing long extended reasoning before emitting text keeps the idle timer alive and is not false-aborted. Residual risk only if a provider streams reasoning with inter-event gaps beyond summarizerIdleTimeoutMs; mitigation is to raise it.
• User Esc during a bounded call: options.signal fires; the combined signal aborts the stream; the existing options.signal?.aborted checks re-throw the abort so flushPending restores state. Not classified as a timeout (no warning).
• Timeout fires after the stream already completed: the finally clears both timers on the normal-completion path before they can fire; a race where the timer fires between result() resolving and cleanup is harmless (the outcome is already built; aborting a settled stream is a no-op).
• Parallel batches (summarizeBatches): each runOnce owns its own timeout controller, so one batch timing out does not abort its siblings; each is independently classified.
• Range fusion (summarizeRange) and per-batch both route through runSummarization → runOnce, so both inherit the timeout with no extra code. On a range-fusion timeout, runSummarization returns null; compressEligible then proceeds without rangeSummaryText, and the renderer falls back to the per-batch concatenation (per ChainCompressionEntry.rangeSummaryText semantics). The chain is still compressed — only the fused summary is skipped. This is why the warning wording is batch-neutral.
• Timer leak: finally clears both setTimeout handles on every exit path (success, unusable, transient, throw).

Testing approach

bun test src/ — extend src/summarizer-wiring.test.ts (its mock.module swap of stream and note-capturing ctx are the right harness). Mock-harness change required: the current stub is stream: (model: any) => streamImpl(model), which drops the options/signal argument, so a hanging stream cannot observe the abort. Change it to forward all args — stream: (...args: any[]) => streamImpl(...args) — and type streamImpl as (model, input, opts) so helpers can read opts.signal. Add a hangingStream(signal) helper whose async iterator awaits a promise that rejects when signal aborts, and whose result() likewise never resolves until abort. Drive with a tiny summarizerIdleTimeoutMs (e.g. 20) so tests run fast.

Cases:

1. Idle timeout → transient + warning. summarizerModel: "default", hanging stream, summarizerIdleTimeoutMs: 20. Assert summarizeBatch returns null, one note at level "warning" whose text mentions stalling.
2. Ceiling timeout → transient + warning. Idle disabled (0), tiny summarizerMaxTimeoutMs, a stream that dribbles events forever (never idles). Assert ceiling warning fires.
3. Option B: primary idle-times-out, session model rescues. Distinct summarizerModel, controller present; streamImpl returns a hanging stream for PRIMARY and an okStream for SESSION. Assert result is the session summary, "enter" fallback warning present, no failure notice.
4. Both time out → both-down warning. Distinct model; both models hang. Assert null + onBothDown-path warning at "warning" level.
5. User abort is not a timeout. Pre-aborted options.signal; assert the call throws (propagated abort) and no "warning" note is pushed.
6. Disabled (0/0) → no timer, unchanged behavior. Existing ok/err wiring tests still pass with the new fields defaulted; add one asserting an okStream succeeds with both timeouts 0.

Documentation impact

• Feature / user-facing docs introduced: none.
• Materially amended existing docs: doc/configuration.md and CHANGELOG.md (one entry). README.md settings table lists only a subset, so no change there. The exact doc/configuration.md deltas:
  • Two rows in the "Every key" table:

    | `summarizerIdleTimeoutMs` | non-negative integer (ms), `0` disables | `20000` | Abort a summarizer stream call after this much silence (no stream event). Resets on every event, so it never false-aborts a flowing generation; catches a stalled connection fast. A timeout feeds the same outage-fallback retry as a provider error. `0` = no idle bound. |
    | `summarizerMaxTimeoutMs` | non-negative integer (ms), `0` disables | `180000` | Hard ceiling on total duration of a single summarizer stream call. Backstop for a stream that dribbles forever without going idle. Generous by design (clears the observed p99). `0` = no ceiling. |
    

  • Two keys added to the "Full settings JSON" sample (after summarizerThinking): "summarizerIdleTimeoutMs": 20000, and "summarizerMaxTimeoutMs": 180000,.
  • A short "Summarizer timeouts" subsection near the existing outage-fallback note (line ~98): idle-vs-ceiling distinction, that a timeout classifies as transient and feeds the same session-model fallback retry, and that both tries are timeout-bounded.
• Derived / memory docs invalidated: none. (AGENTS.md project-layout and customType tables are unaffected — no new session entry types, events, or src/*.ts files; the change is confined to existing files.)