feat(compaction): support SubAPI native remote compaction

- support sub-api/openai-responses through the public Responses endpoint
- preserve explicit Authorization and support header-only authentication
- retain Codex JWT requirements and native compaction failure fallback
- restore SubAPI checkpoints and preserve portable model-switching summaries
- add provider, authentication and checkpoint lifecycle regression tests
- document compatibility requirements and the Sub2API protocol reference
This commit is contained in:
云服务部-叶林立
2026-09-07 16:25:48 +08:00
parent 6e7effa792
commit ff1e7400da
9 changed files with 431 additions and 12 deletions
+3 -1
View File
@@ -108,7 +108,9 @@
- AutoReview 只对完成的 `allow` / `deny` 计数,`defer` 不计;达到 500 次后持久化 pending,并只在 `agent_settled` 拉起无工具、无扩展、无项目上下文的隔离后台 AgentSession。建议必须经过 schema 与确定性安全过滤、以 same-session follow-up 交给主 Agent,并在用户明确确认前禁止自动修改任一策略文件;shutdown 时保留 pending 供下一 Session 继续。
- `pi-lsp` 的 TypeScript/JavaScript 后端由根包固定依赖提供,`extensions/lsp.ts` 使用当前 Node 可执行文件直接启动包内 `typescript-language-server` CLI,不得依赖或调用 VS Code GUI;Kotlin/JDT LS 仍使用组合包部署的 `kotlin-lsp --stdio` 与 `jdtls` 配置,根包不安装这两个系统可执行文件,便捷脚本可在用户逐项明确确认后通过 Homebrew 安装 Kotlin LSP、Java 21 和 JDT LS。
- 终端环境配置只属于便捷脚本:Kitty 可选安装后可通过官方 kitten 启用 Solarized Dark;Oh My Zsh 使用不切换 shell 的 unattended 安装;Powerlevel10k 将仓库内置的 `config/p10k.zsh`(Rainbow/ASCII 单行紧凑主题)部署到 `~/.config/my-pi/p10k.zsh`(遵循 `XDG_CONFIG_HOME`),不得覆盖用户自己的 `~/.p10k.zsh`。`.zshrc` 中脚本拥有的内容必须使用 `# >>> my-pi:<id> >>>` / `# <<< my-pi:<id> <<<` 受管块,更新时只替换块内内容;标记不完整或重复时拒绝修改。实际修改前必须创建带时间戳的备份。`git`、`zsh-autosuggestions`、`zsh-syntax-highlighting` 在安装流程中逐项询问后才安装或启用。
- Codex 远程压缩只对 `openai-codex` 生效,默认在 turn boundary 达到 90% 时触发;官方 Hippo Extension 在已初始化项目的 session start 注入上下文、过滤捕获工具错误,并在 session shutdown 执行 `hippo sleep`。
- 原生远程压缩限定为 `openai-codex` + `openai-codex-responses` 与 `sub-api` + `openai-responses`,默认在 turn boundary 达到 effective 下界 90% 时触发。`pi-compaction/src/native-provider.ts` 统一资格与传输选择:Codex 保留原 endpoint/JWT/account ID;SubAPI 从显式 `baseUrl` 解析公共 `/responses`,使用配置的 API key 和请求头,不解析 ChatGPT account ID、不硬编码网关域名,也不调用独立 `/responses/compact`。两者复用 projection、`compaction_trigger`、SSE 结果、native checkpoint 恢复和 portable 模型切换摘要;SubAPI 部署须支持原生 compaction v2。普通聊天支持不代表压缩支持,失败回退 Pi,不自动改写用户模型配置。
- SubAPI 原生压缩认证必须保留非空显式 `Authorization`(大小写不敏感)的优先级;没有该头时才使用非空 API key 生成 Bearer,均缺失时可接受非空 `cf-aig-authorization` 代理认证头。不得要求 header-only 配置额外提供假 API key,也不得生成 `Bearer undefined`;空白或缺失凭据在网络请求前拒绝并回退 Pi。Codex 仍必须提供有效 JWT,不能用请求头绕过。
- 官方 Hippo Extension 在已初始化项目的 session start 注入上下文、过滤捕获工具错误,并在 session shutdown 执行 `hippo sleep`。
- Hippo Extension 直接使用 PATH 中的官方全局 `hippo` CLI;`install.sh` 只在 CLI 缺失且用户明确确认后安装 `config/hippo-memory-version` 指定的 npm 版本,不得扫描项目、选择初始化目录或执行 `hippo init`。用户必须在自己选择的项目目录中手动初始化。
- `update.sh` 只升级已经安装且属于 npm 全局包的 Hippo CLI;必须从 `pi update` 后的组合包读取目标版本,先用 `npm view` 确认,再比较本地版本,版本相同不得重新安装,CLI 缺失或来源未知时保留现状。
- `uninstall.sh` 只在根包移除成功后询问是否卸载 npm 全局 Hippo CLI,默认拒绝;不得删除项目 `.hippo/`、全局记忆或旧 Hermes 数据。
+2
View File
@@ -352,6 +352,8 @@ compaction 和排队 follow-up 完成前不会提前通知;同时监听 `pi-pe
唯一配置位于 Pi agent `settings.json` 的 `piCompaction` 字段,默认 soft/hard/target 比例为 72%/90%/55%,扩展只在内存中归一化且不写回配置。`/compaction-status` 显示 raw/effective/reclaimed token、置信度、segment 和后台任务。完整配置、Session version 1 格式和双上游来源见 [`pi-compaction/README.md`](pi-compaction/README.md) 与 [`pi-compaction/UPSTREAM.md`](pi-compaction/UPSTREAM.md)。
原生远程压缩同时支持 `openai-codex` + `openai-codex-responses` 和 `sub-api` + `openai-responses`,使用触发时当前会话模型。SubAPI 保留现有模型配置,从 `baseUrl` 调用公共 `/responses` 并使用 `compaction_trigger` 流式协议;优先保留非空显式 `Authorization`,否则使用 API key,也支持仅凭非空 `cf-aig-authorization` 的代理认证。缺失或空白凭据在请求前拒绝。SubAPI 不要求客户端解析 ChatGPT account ID,也不调用独立 `/responses/compact`;Codex 的 JWT 要求保持不变。网关部署须支持 Sub2API 原生压缩 v2;请求失败沿用 Pi 回退,切换模型或 Provider 时使用 portable summary。
本实现不读取或迁移旧 `contextPrune`、`context-prune-*` 或旧 Codex native checkpoint;切换后应创建新 Session。它与 Context Mode 职责互补:Context Mode 防止大型原始内容进入会话,Pi Compaction 管理已经进入会话的长期历史。
### Codex fast mode
+7
View File
@@ -1,5 +1,12 @@
# Changelog
## Unreleased
- Add native remote compaction v2 for `sub-api` with `openai-responses`, using the configured public Responses URL and gateway authentication.
- Restore SubAPI native checkpoints on subsequent requests and preserve the existing Codex transport, portable model-switching summary and failure fallback.
- Cover provider eligibility, URL/authentication handling and both providers' native checkpoint lifecycles with offline tests.
- Preserve explicit SubAPI Authorization headers over API keys, support header-only Authorization/proxy authentication, and reject missing credentials without weakening Codex JWT requirements.
## 1.0.0
- Introduce one incremental Context Authority for provider requests, threshold decisions, UI metrics, and native compaction.
+20 -1
View File
@@ -7,7 +7,7 @@ High-performance unified context projection and compaction for Pi.
- incrementally index completed tool calls;
- replace large tool results with deterministic bounded evidence stubs;
- refine large stubs with non-blocking semantic summaries;
- expose one effective context projection to normal provider requests and Codex native compaction;
- expose one effective context projection to normal provider requests and Codex/SubAPI native compaction;
- cancel Pi's raw threshold compaction until the effective lower-bound reaches the configured threshold;
- retain original tool evidence through `context_tree_query`;
- publish read-only effective metrics for the footer and context observer.
@@ -47,6 +47,25 @@ Set `piCompaction` in the Pi agent `settings.json`:
The extension never writes settings. Invalid or missing fields are normalized in memory.
## Native providers
Native compaction is enabled by `piCompaction.native.enabled` and uses the model selected in the session when compaction starts. It does not select a separate summarization model.
| Provider | API | Native transport |
| --- | --- | --- |
| `openai-codex` | `openai-codex-responses` | Existing Codex Responses endpoint and JWT/account-ID authentication |
| `sub-api` | `openai-responses` | Public Responses endpoint and resolved gateway headers or API key |
For SubAPI, keep the existing `models.json` API as `openai-responses`. The adapter requires an explicit HTTP(S) `baseUrl`: for example, `https://gateway.example/v1` becomes `https://gateway.example/v1/responses`; an existing `/responses` suffix is not duplicated. It preserves configured headers and merges the native compaction beta feature, without decoding the API key or adding a ChatGPT account ID. No gateway hostname or credentials are bundled.
SubAPI preserves a non-empty explicit `Authorization` header (case-insensitively) ahead of the API key. Without it, a non-blank API key supplies Bearer authentication; if no key is available, a non-empty `cf-aig-authorization` header also supports proxy-managed authentication. Header-only credentials do not create a dummy Bearer token. Missing or blank credentials fail before any network request and return control to Pi. Official Codex still requires its API key/JWT even when explicit authentication headers are present.
Both transports send the current projection as ResponseItems, append `compaction_trigger`, and parse the SSE compaction item. SubAPI must support Sub2API native remote compaction v2 on its public Responses endpoint. This is not a client for the separate `/v1/responses/compact` endpoint; supporting only that endpoint or ordinary Responses chat is insufficient.
Request interception also restores SubAPI native checkpoints into subsequent Responses input. Checkpoints retain a bounded portable summary for switching models/providers, and their version 1 format is unchanged. Native authentication, transport or response failures return control to Pi's normal compaction fallback. Other provider/API pairs and `native.enabled: false` retain the existing portable-summary path. Setting `native.enabled` to false does not discard an already-created native checkpoint.
The gateway must have a compatible deployed version, model and available upstream account. Offline tests cover the protocol and lifecycle; they do not establish live gateway availability. No settings migration, provider rename or Pi core modification is required.
## Session format
The only custom entry type is `pi-compaction`, version 1. Segment summary entries contain hashes, summaries, and source references; they never duplicate complete raw tool output. Native checkpoints use Pi compaction-entry details with the same kind/version.
+8
View File
@@ -21,3 +21,11 @@ This package is maintained directly in `my-pi` and combines two separately licen
- License: `LICENSES/pi-codex-compaction-MIT.txt`
No npm tarball, npm cache, `node_modules`, build output, or nested Git metadata is used as an import source.
## Sub2API protocol reference
- Official repository: https://github.com/Wei-Shaw/sub2api
- Reference commit: `ab99d56e9626e6cd731592dae8553c9758a0efa2`
- Protocol evidence: `backend/internal/service/openai_compact_body_signal.go` identifies native remote compaction v2 as Responses plus `stream: true` and `compaction_trigger`, and handles the Codex beta feature.
- The local `src/native-provider.ts` adapter is independently implemented against that protocol. No Sub2API source was imported; the existing Codex serializer, stream parser and checkpoint format are reused.
- This reference describes upstream capability, not a guarantee about any particular deployment, model or account. The separate Responses compact endpoint is not used by this adapter.
+7 -10
View File
@@ -14,21 +14,19 @@ import {
type SessionEntry,
} from "@earendil-works/pi-coding-agent";
import { DEFAULT_CONFIG, loadConfig } from "./src/config.js";
import { buildNativeCompactionTransport, isNativeCompactionModel } from "./src/native-provider.js";
import {
buildCodexHeaders,
buildCompactionRequestBody,
buildReplacementHistory,
buildToolPayload,
callRemoteCompaction,
findNativeCheckpoint,
isJsonObject,
isOpenAICodexModel,
mergeFeatureHeader,
messagesToResponseItems,
modelKey,
NATIVE_COMPACTION_KIND,
NATIVE_COMPACTION_VERSION,
resolveCodexResponsesUrl,
stripInputFromPayload,
type JsonObject,
type NativeCompactionDetails,
@@ -364,12 +362,12 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
});
pi.on("before_provider_headers", (event, ctx) => {
if (isOpenAICodexModel(ctx.model)) setFeatureHeader(event.headers);
if (isNativeCompactionModel(ctx.model)) setFeatureHeader(event.headers);
});
pi.on("before_provider_request", (event, ctx) => {
const model = ctx.model;
if (!model || !isOpenAICodexModel(model) || !isJsonObject(event.payload)) return undefined;
if (!model || !isNativeCompactionModel(model) || !isJsonObject(event.payload)) return undefined;
const runtime = runtimeFor(ctx);
const sessionId = ctx.sessionManager.getSessionId();
payloads.set(sessionId, { modelKey: modelKey(model), payload: stripInputFromPayload(event.payload) });
@@ -401,7 +399,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
const model = ctx.model;
if (!model) return undefined;
if (!runtime.config.native.enabled || !isOpenAICodexModel(model)) {
if (!runtime.config.native.enabled || !isNativeCompactionModel(model)) {
try {
const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
if (!auth.ok) return undefined;
@@ -438,7 +436,7 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
try {
const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model);
if (!auth.ok || !auth.apiKey) throw new Error(auth.ok ? "Codex authentication unavailable" : auth.error);
if (!auth.ok) throw new Error(auth.error);
const currentCheckpoint = checkpoint.status === "valid" ? checkpoint.checkpoint.details : undefined;
const input = codexInput(model, runtime, snapshot, currentCheckpoint);
const sessionId = ctx.sessionManager.getSessionId();
@@ -451,10 +449,9 @@ export default function piCompactionExtension(pi: ExtensionAPI): void {
tools: buildToolPayload(pi.getAllTools(), pi.getActiveTools()),
sessionId,
});
if (ctx.hasUI) ctx.ui.notify("pi-compaction: creating Codex native checkpoint…", "info");
if (ctx.hasUI) ctx.ui.notify(`pi-compaction: creating ${model.provider} native checkpoint...`, "info");
const remote = await callRemoteCompaction({
url: resolveCodexResponsesUrl(model.baseUrl),
headers: buildCodexHeaders({ apiKey: auth.apiKey, headers: cleanHeaders(auth.headers), sessionId }),
...buildNativeCompactionTransport({ model, apiKey: auth.apiKey, headers: cleanHeaders(auth.headers), sessionId }),
body,
signal: event.signal,
});
+66
View File
@@ -0,0 +1,66 @@
import {
buildCodexHeaders,
isJsonObject,
isOpenAICodexModel,
mergeFeatureHeader,
resolveCodexResponsesUrl,
type Model,
} from "./native-compaction.js";
export function isNativeCompactionModel(model: unknown): model is Model<any> {
return isOpenAICodexModel(model) || (
isJsonObject(model) && model.provider === "sub-api" && model.api === "openai-responses"
);
}
export function buildNativeCompactionTransport(params: {
model: Pick<Model<any>, "provider" | "api" | "baseUrl">;
apiKey?: string;
headers?: Record<string, string>;
sessionId: string;
}): { url: string; headers: Headers } {
if (isOpenAICodexModel(params.model)) {
if (!params.apiKey) throw new Error("Codex native compaction authentication requires an API key.");
return {
url: resolveCodexResponsesUrl(params.model.baseUrl),
headers: buildCodexHeaders({ ...params, apiKey: params.apiKey }),
};
}
if (!isNativeCompactionModel(params.model)) {
throw new Error("Native compaction is not supported for this provider and API.");
}
if (!params.model.baseUrl?.trim()) {
throw new Error("SubAPI native compaction requires an explicit baseUrl.");
}
let url: URL;
try {
url = new URL(params.model.baseUrl.trim());
} catch {
throw new Error("SubAPI native compaction requires a valid HTTP(S) baseUrl.");
}
if (!["https:", "http:"].includes(url.protocol) || url.username || url.password || url.hash) {
throw new Error("SubAPI native compaction requires an HTTP(S) baseUrl without credentials or a fragment.");
}
const path = url.pathname.replace(/\/+$/, "");
url.pathname = path.endsWith("/responses") ? path : `${path}/responses`;
// Sub2API accepts native compaction v2 through its public Responses endpoint.
// The gateway owns upstream Codex credentials; honor the client's resolved authentication.
const headers = new Headers(params.headers);
if (!headers.get("authorization")?.trim()) {
if (params.apiKey?.trim()) {
headers.set("authorization", `Bearer ${params.apiKey}`);
} else if (!headers.get("cf-aig-authorization")?.trim()) {
throw new Error("SubAPI native compaction authentication requires an API key or a non-empty Authorization or cf-aig-authorization header.");
}
}
headers.set("content-type", "application/json");
headers.set("accept", "text/event-stream");
headers.set("OpenAI-Beta", "responses=experimental");
headers.set("originator", "pi");
headers.set("user-agent", "pi-compaction");
headers.set("session-id", params.sessionId);
headers.set("x-client-request-id", params.sessionId);
headers.set("x-codex-beta-features", mergeFeatureHeader(headers.get("x-codex-beta-features")));
return { url: url.toString(), headers };
}
@@ -101,3 +101,205 @@ test("model switching after a native checkpoint projects the portable checkpoint
assert.equal(projected.messages[1].content, "continue");
await h.handlers.get("session_shutdown")![0]!({}, ctx);
});
for (const provider of ["sub-api", "openai-codex"]) {
test(`${provider} native compaction uses the selected model and restores its checkpoint`, async (t) => {
const h = harness();
piCompaction(h.pi);
const branch: any[] = [{
type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Remember the test decision.", timestamp: 1 },
}];
const ctx = context(branch);
ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
id: "selected-model", input: ["text"], contextWindow: 100_000,
baseUrl: provider === "sub-api" ? "https://gateway.example/v1" : "https://codex.example/codex",
};
const jwt = Buffer.from(JSON.stringify({
"https://api.openai.com/auth": { chatgpt_account_id: "test-account" },
})).toString("base64url");
const apiKey = provider === "sub-api" ? "ordinary-test-key" : `header.${jwt}.signature`;
ctx.modelRegistry.getApiKeyAndHeaders = async (model: unknown) => {
assert.equal(model, ctx.model);
return { ok: true, apiKey, headers: { "x-test-route": "preserved" } };
};
const requests: Array<{ url: string; headers: Headers; body: any }> = [];
t.mock.method(globalThis, "fetch", async (url: string, init: RequestInit) => {
requests.push({ url: String(url), headers: new Headers(init.headers), body: JSON.parse(String(init.body)) });
const events = [
{ type: "response.output_item.done", item: { type: "compaction", encrypted_content: "opaque-checkpoint" } },
{ type: "response.completed", response: { usage: { input_tokens: 12, output_tokens: 3 } } },
];
return new Response(events.map((event) => `data: ${JSON.stringify(event)}\n\n`).join(""), {
headers: { "content-type": "text/event-stream" },
});
});
try {
await h.handlers.get("context")![0]!({ messages: branch.map((entry) => entry.message) }, ctx);
const headers: Record<string, string> = { "x-codex-beta-features": "existing-feature" };
await h.handlers.get("before_provider_headers")![0]!({ headers }, ctx);
assert.match(headers["x-codex-beta-features"]!, /existing-feature/);
assert.notEqual(headers["x-codex-beta-features"], "existing-feature");
await h.handlers.get("before_provider_request")![0]!({ payload: {
model: "stale-cached-model", input: [{ role: "user", content: "STALE-RAW-HISTORY" }],
store: true, stream: false,
} }, ctx);
const result = await h.handlers.get("session_before_compact")![0]!({
reason: "manual", branchEntries: branch, signal: new AbortController().signal,
preparation: { firstKeptEntryId: "u1" },
}, ctx);
assert.ok(result?.compaction, "native compaction should return a checkpoint");
assert.equal(requests.length, 1);
const request = requests[0]!;
assert.equal(request.url, provider === "sub-api"
? "https://gateway.example/v1/responses" : "https://codex.example/codex/responses");
assert.equal(request.headers.get("authorization"), `Bearer ${apiKey}`);
assert.equal(request.headers.get("x-test-route"), "preserved");
assert.equal(request.headers.get("chatgpt-account-id"), provider === "sub-api" ? null : "test-account");
assert.equal(request.body.model, "selected-model");
assert.equal(request.body.stream, true);
assert.equal(request.body.store, false);
assert.deepEqual(request.body.input.at(-1), { type: "compaction_trigger" });
assert.match(JSON.stringify(request.body.input), /Remember the test decision/);
assert.doesNotMatch(JSON.stringify(request.body.input), /STALE-RAW-HISTORY/);
const details = result.compaction.details;
assert.equal(details.checkpoint, "native");
assert.equal(details.modelKey, `${provider}:${ctx.model.api}:selected-model`);
assert.ok(details.replacementHistory.some((item: any) => item.encrypted_content === "opaque-checkpoint"));
assert.ok(details.portableSummary.length > 0);
branch.push(
{ type: "compaction", id: "c1", parentId: "u1", ...result.compaction },
{ type: "message", id: "u2", parentId: "c1", message: { role: "user", content: "Continue after compaction.", timestamp: 2 } },
);
const projected = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
assert.doesNotMatch(JSON.stringify(projected.messages), /opaque-checkpoint/);
const payload = await h.handlers.get("before_provider_request")![0]!({
payload: { model: ctx.model.id, input: [], messages: [], previous_response_id: "stale-response" },
}, ctx);
assert.ok(payload.input.some((item: any) => item.encrypted_content === "opaque-checkpoint"));
assert.match(JSON.stringify(payload.input), /Continue after compaction/);
assert.equal(payload.previous_response_id, undefined);
assert.equal(payload.messages, undefined);
ctx.model = { ...ctx.model, id: "another-model" };
const switched = await h.handlers.get("context")![0]!({ messages: [] }, ctx);
assert.match(JSON.stringify(switched.messages), /Remember the test decision/);
assert.equal(await h.handlers.get("before_provider_request")![0]!({ payload: { input: [] } }, ctx), undefined);
} finally {
await h.handlers.get("session_shutdown")![0]!({}, ctx);
}
});
}
test("SubAPI native failures and cancellation return control to Pi without a checkpoint", async (t) => {
const h = harness();
piCompaction(h.pi);
const branch: any[] = [{
type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Test only.", timestamp: 1 },
}];
const ctx = context(branch);
ctx.model = { provider: "sub-api", api: "openai-responses", id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1" };
ctx.modelRegistry.getApiKeyAndHeaders = async () => ({ ok: true, apiKey: "test-key" });
const responses = [
new Response('{"error":{"message":"unsupported"}}', { status: 400 }),
new Response('data: {"type":"response.completed","response":{}}\n\n', { headers: { "content-type": "text/event-stream" } }),
];
let calls = 0;
t.mock.method(globalThis, "fetch", async (_url: string, init: RequestInit) => {
init.signal?.throwIfAborted();
calls++;
return responses.shift()!;
});
try {
for (let attempt = 0; attempt < 2; attempt++) {
assert.equal(await h.handlers.get("session_before_compact")![0]!({
reason: "manual", branchEntries: branch, preparation: { firstKeptEntryId: "u1" },
signal: new AbortController().signal,
}, ctx), undefined);
}
const controller = new AbortController();
controller.abort();
assert.equal(await h.handlers.get("session_before_compact")![0]!({
reason: "manual", branchEntries: branch, preparation: { firstKeptEntryId: "u1" }, signal: controller.signal,
}, ctx), undefined);
assert.equal(calls, 2);
assert.equal(branch.some((entry) => entry.type === "compaction"), false);
} finally {
await h.handlers.get("session_shutdown")![0]!({}, ctx);
}
});
test("other providers and SubAPI Chat Completions keep their provider requests unchanged", async () => {
const h = harness();
piCompaction(h.pi);
const ctx = context();
try {
for (const model of [
{ provider: "other", api: "openai-responses", id: "test" },
{ provider: "sub-api", api: "openai-completions", id: "test" },
]) {
ctx.model = model;
const headers: Record<string, string> = {};
await h.handlers.get("before_provider_headers")![0]!({ headers }, ctx);
assert.equal(headers["x-codex-beta-features"], undefined);
assert.equal(await h.handlers.get("before_provider_request")![0]!({ payload: { input: [] } }, ctx), undefined);
}
} finally {
await h.handlers.get("session_shutdown")![0]!({}, ctx);
}
});
test("native runtime delegates authentication to each provider without blocking SubAPI header-only auth", async (t) => {
const cases: Array<{ name: string; provider?: string; auth: any; succeeds: boolean; authorization?: string; gatewayAuthorization?: string }> = [
{ name: "explicit header", auth: { ok: true, apiKey: "dummy-key", headers: { Authorization: "Bearer explicit" } }, succeeds: true, authorization: "Bearer explicit" },
{ name: "header only", auth: { ok: true, headers: { authorization: "Bearer header-only" } }, succeeds: true, authorization: "Bearer header-only" },
{ name: "proxy header only", auth: { ok: true, headers: { "CF-AIG-Authorization": "Bearer proxy" } }, succeeds: true, gatewayAuthorization: "Bearer proxy" },
{ name: "blank header", auth: { ok: true, headers: { Authorization: " " } }, succeeds: false },
{ name: "missing credentials", auth: { ok: true }, succeeds: false },
{ name: "registry failure", auth: { ok: false, error: "credentials unavailable" }, succeeds: false },
{ name: "Codex header only", provider: "openai-codex", auth: { ok: true, headers: { Authorization: "Bearer header-only" } }, succeeds: false },
{ name: "Codex invalid JWT", provider: "openai-codex", auth: { ok: true, apiKey: "invalid", headers: { Authorization: "Bearer header-only" } }, succeeds: false },
];
for (const scenario of cases) {
await t.test(scenario.name, async (t) => {
const h = harness();
piCompaction(h.pi);
const branch: any[] = [{
type: "message", id: "u1", parentId: null,
message: { role: "user", content: "Synthetic authentication test.", timestamp: 1 },
}];
const ctx = context(branch);
const provider = scenario.provider ?? "sub-api";
ctx.model = {
provider, api: provider === "sub-api" ? "openai-responses" : "openai-codex-responses",
id: "test-model", input: ["text"], baseUrl: "https://gateway.example/v1",
};
ctx.modelRegistry.getApiKeyAndHeaders = async () => scenario.auth;
let calls = 0;
t.mock.method(globalThis, "fetch", async (_url: string, init: RequestInit) => {
calls++;
const headers = new Headers(init.headers);
assert.equal(headers.get("authorization"), scenario.authorization ?? null);
assert.equal(headers.get("cf-aig-authorization"), scenario.gatewayAuthorization ?? null);
const events = [
{ type: "response.output_item.done", item: { type: "compaction", encrypted_content: "test-checkpoint" } },
{ type: "response.completed", response: {} },
];
return new Response(events.map((event) => `data: ${JSON.stringify(event)}\n\n`).join(""), {
headers: { "content-type": "text/event-stream" },
});
});
try {
const result = await h.handlers.get("session_before_compact")![0]!({
reason: "manual", branchEntries: branch, preparation: { firstKeptEntryId: "u1" },
signal: new AbortController().signal,
}, ctx);
assert.equal(Boolean(result?.compaction), scenario.succeeds);
assert.equal(calls, scenario.succeeds ? 1 : 0);
} finally {
await h.handlers.get("session_shutdown")![0]!({}, ctx);
}
});
}
});
+116
View File
@@ -0,0 +1,116 @@
import test from "node:test";
import assert from "node:assert/strict";
import { buildCodexHeaders, mergeFeatureHeader } from "../src/native-compaction.ts";
import { buildNativeCompactionTransport, isNativeCompactionModel } from "../src/native-provider.ts";
const subApi = {
provider: "sub-api",
api: "openai-responses",
id: "gpt-test",
baseUrl: "https://gateway.example/v1",
};
const params = { model: subApi, apiKey: "test-api-key", sessionId: "session-test" };
test("native compaction eligibility is limited to the supported provider/API pairs", () => {
assert.equal(isNativeCompactionModel(subApi), true);
assert.equal(isNativeCompactionModel({ provider: "openai-codex", api: "openai-codex-responses" }), true);
for (const model of [
null, {},
{ ...subApi, provider: "another-gateway" },
{ ...subApi, api: "openai-completions" },
{ ...subApi, api: "openai-codex-responses" },
{ provider: "openai-codex", api: "openai-responses" },
]) assert.equal(isNativeCompactionModel(model), false);
});
test("SubAPI uses its public Responses URL and API key without Codex account parsing", () => {
const { url, headers } = buildNativeCompactionTransport({
...params,
headers: { "x-gateway-route": "test", "x-codex-beta-features": "existing-feature" },
});
assert.equal(url, "https://gateway.example/v1/responses");
assert.equal(headers.get("authorization"), "Bearer test-api-key");
assert.equal(headers.has("chatgpt-account-id"), false);
assert.equal(headers.get("x-gateway-route"), "test");
assert.equal(headers.get("content-type"), "application/json");
assert.equal(headers.get("accept"), "text/event-stream");
assert.equal(headers.get("session-id"), "session-test");
assert.equal(headers.get("x-client-request-id"), "session-test");
assert.equal(headers.get("x-codex-beta-features"), mergeFeatureHeader("existing-feature"));
});
test("SubAPI URL normalization preserves prefixes and avoids duplicating responses", () => {
for (const [baseUrl, expected] of [
["https://gateway.example/v1/", "https://gateway.example/v1/responses"],
["https://gateway.example/v1/responses", "https://gateway.example/v1/responses"],
["https://gateway.example/v1/responses/", "https://gateway.example/v1/responses"],
["https://gateway.example/prefix/v1?route=test", "https://gateway.example/prefix/v1/responses?route=test"],
["http://localhost:8080", "http://localhost:8080/responses"],
]) {
assert.equal(buildNativeCompactionTransport({ ...params, model: { ...subApi, baseUrl } }).url, expected);
}
});
test("invalid SubAPI endpoints and unsupported providers fail without a Codex fallback", () => {
for (const baseUrl of [undefined, "", " ", "/v1", "invalid", "file:///tmp/test", "https://user:secret@gateway.example/v1", "https://gateway.example/v1#fragment"]) {
assert.throws(() => buildNativeCompactionTransport({ ...params, model: { ...subApi, baseUrl } }), /SubAPI/);
}
assert.throws(() => buildNativeCompactionTransport({
...params, model: { ...subApi, provider: "other" },
}), /not supported/);
});
test("official Codex transport retains its URL, JWT account ID and headers", () => {
const payload = Buffer.from(JSON.stringify({
"https://api.openai.com/auth": { chatgpt_account_id: "test-account" },
})).toString("base64url");
const codexParams = {
...params,
model: { provider: "openai-codex", api: "openai-codex-responses", id: "gpt-test" },
apiKey: `header.${payload}.signature`,
headers: { "x-custom": "preserved" },
};
const transport = buildNativeCompactionTransport(codexParams);
assert.equal(transport.url, "https://chatgpt.com/backend-api/codex/responses");
assert.deepEqual([...transport.headers], [...buildCodexHeaders(codexParams)]);
assert.equal(transport.headers.get("chatgpt-account-id"), "test-account");
assert.throws(() => buildNativeCompactionTransport({ ...codexParams, apiKey: "ordinary-key" }), /account ID/);
});
test("SubAPI preserves explicit Authorization ahead of its API key, case-insensitively", () => {
for (const name of ["Authorization", "authorization", "AUTHORIZATION"]) {
const { headers } = buildNativeCompactionTransport({
...params, headers: { [name]: "Bearer explicit-token" },
});
assert.equal(headers.get("authorization"), "Bearer explicit-token");
}
});
test("SubAPI accepts header-only authentication without synthesizing a bearer token", () => {
for (const name of ["Authorization", "CF-AIG-Authorization"]) {
const { headers } = buildNativeCompactionTransport({
...params, apiKey: undefined, headers: { [name]: "Bearer header-only-token" },
});
assert.equal(headers.get(name), "Bearer header-only-token");
if (name === "CF-AIG-Authorization") assert.equal(headers.has("authorization"), false);
}
});
test("SubAPI rejects absent or blank credentials but can fall back to a real API key", () => {
const candidates: Record<string, string>[] = [{}, { Authorization: " " }, { "cf-aig-authorization": " " }, { "x-api-key": "not-supported" }];
for (const headers of candidates) {
for (const apiKey of [undefined, "", " "]) {
assert.throws(() => buildNativeCompactionTransport({ ...params, apiKey, headers }), /authentication/i);
}
}
const result = buildNativeCompactionTransport({ ...params, headers: { Authorization: " " } });
assert.equal(result.headers.get("authorization"), "Bearer test-api-key");
});
test("official Codex still requires an API key even with explicit header credentials", () => {
assert.throws(() => buildNativeCompactionTransport({
...params, apiKey: undefined,
model: { provider: "openai-codex", api: "openai-codex-responses" },
headers: { Authorization: "Bearer header-only-token" },
}), /Codex.*authentication/i);
});