my-pi/pi-permission-system/docs/retro/0547-zod-config-schema-single-source.md

issue, issue_title

issue	issue_title
547	Include a JSON Schema definition in pi-permissions.jsonc for completions and easier configuration

Retro: #547 — Include a JSON Schema definition in pi-permissions.jsonc for completions and easier configuration

Stage: Planning (2026-07-06T00:00:00Z)

Session summary

Planned issue #547 (third-party, filed by JasonLandbridge). Discovered the requested JSON Schema already exists (schemas/permissions.schema.json), the config already accepts $schema, and completions technically already work — the real defects are a stale hosted URL (points at the pre-monorepo gotgenes/pi-permission-system fork) and a hand-maintained schema that drifts from the TS types and hand-rolled loader guards. Through three ask_user rounds the operator chose the "Full" direction: adopt zod (pin 4.4.3, the production latest) as the single source of truth, derive the Draft 2020-12 JSON Schema from it, and route the loader's runtime validation through it with strict, breaking semantics (reject a malformed field with a clear description, fail-closed to ask). Wrote a 6-step lift-and-shift TDD plan and committed it.

Observations

• Third-party issue → did not skip the ask_user gate; the operator materially reshaped scope (zod over the issue's TypeBox suggestion; runtime validation, not schema-only; stricter/breaking with clear errors).
• The operator interjected two directives mid-plan: use the current production zod (4.4.3, confirmed via pnpm view, since the npm shim is blocked) and lean on schema composability without over-abstraction — folded both into the design (bottom-up composable schemas mirroring the existing $defs, reused: "ref").
• Key design guards captured: no .default() in the parse schema (defaults belong post-merge in normalizePermissionSystemConfig, else global/project override semantics break); allow $schema explicitly under strictObject; fail-closed to ask on reject is the security-critical invariant and gets its own test.
• markdownDescription is not confirmed to auto-copy from z.toJSONSchema (colinhacks/zod#5272 is open) — the Step 2 parity test decides whether .meta() suffices or the override callback is needed.
• Scope interaction with open roadmap Step 8 (#532): #547 removes the two config-only guards (normalizeOptionalStringArray, normalizeOptionalPositiveInt) that #532 meant to keep, and keeps the domain guards #532 meant to move — noted as a Non-Goal, updating architecture.md line 807/906 rather than completing #532.
• No follow-up issues filed: deeper normalize.ts/policy-loader.ts guard cleanup is deferred to existing #532; per-agent frontmatter validation left as an Open Question (not speculative-filed).
• Release: ship independently — #547 is not in the architecture roadmap.
• Freshness gate is a vitest parity test (runs in the existing pnpm -r run test CI job), so no ci.yml edit is needed.

Stage: Implementation — TDD (2026-07-06T17:26:46Z)

Session summary

Implemented all 6 planned TDD steps across 6 commits: added the composable zod schema module (config-schema.ts) as the single source of truth, generated permissions.schema.json from it (fixing the stale $id URL), routed the config-file loader through strict safeParse (breaking, fail-closed), removed the two superseded config-only guards, derived the config types from zod, and updated docs/ADR/migration/skill. Package tests went 2283 → 2293 (+10 net: +22 from config-schema.test.ts, +1 fail-closed test, −13 from removed guard tests). All deterministic gates pass; pre-completion reviewer returned WARN (one finding, fixed).

Observations

• Two unplanned but necessary deviations, both honoring the plan's scope. (1) normalizeUnifiedConfig turned out to be dual-purpose — it also validated per-agent .md frontmatter (which carries non-config keys like name/model). A strict strictObject would have rejected those and silently dropped agent permission blocks. Fix: policy-loader.ts now extracts only the permission block via the exported tolerant normalizeFlatPermissionValue, so config files are strict while agent frontmatter stays tolerant (the plan's stated boundary). (2) Legacy-file validation issues are suppressed in loadAndMergeConfigs so the move-it migration message stays the clean, actionable signal.
• The generated schema is not byte-identical to the hand-maintained one — zod emits anyOf-of-consts (not oneOf) and adds a safe-int maximum on integers. Functionally equivalent for editors; the parity test snapshots the generator output for freshness rather than matching the old file. markdownDescription, examples, default, and per-value descriptions all survive via .meta() (no override callback needed).
• gen:schema chains biome format because JSON.stringify(…, null, 2) collapses differently than biome (single-element arrays); chaining keeps the committed file deterministic and lint-clean.
• No .default() in the parse schema — defaults stay in normalizePermissionSystemConfig post-merge, preserving global-vs-project override semantics (a plan guard that held up).
• Type derivation was safe — expectTypeOf(...).toEqualTypeOf(...) in config-schema.test.ts (enforced by tsc, since tsconfig includes test) proved the z.infer types equal the hand-written ones before the lift-and-shift, so all ~58 consumers compiled untouched.
• Pre-completion reviewer: WARN — the only finding was a missing README Documentation-table row for the new docs/migration/strict-config-validation.md; added it (amended into the docs commit) and re-verified lint. Everything else PASS, including explicit tests for the fail-closed ask fallback and legacy-file suppression.
• #532 interaction — removed the two config-only guards it planned to keep; updated architecture.md's Step 8 note and the ADR to reflect the shrunk target without closing #532.
• zod 4.4.3 installed cleanly, no minimumReleaseAgeExclude needed; tarball verified to ship src/config-schema.ts + schema + migration doc and exclude scripts/, test/, and the internal ADR.

Stage: Final Retrospective (2026-07-06T18:00:00Z)

Session summary

One continuous session carried #547 through planning, TDD implementation, and ship. Shipped pi-permission-system v19.0.0 (major, breaking): zod is now the single source of truth for the config-file shape, the JSON Schema and config types both derive from it, the stale hosted $id/$schema URLs are repointed to the monorepo, and config-file validation is strict and fail-closed. Six implementation commits, +10 net tests, pre-completion WARN fixed, CI green, release-please PR #550 merged, issue closed.

Observations

What went well

• Discovery-first planning reframed the issue. The request was "add a JSON Schema," but investigation found the schema already existed and completions already worked; the real defects were a stale hosted URL and schema/type/loader drift. Reframing avoided building a redundant artifact and produced a higher-value single-source-of-truth outcome.
• Throwaway exploration scripts before committing to the schema shape. Two disposable explore-zod.ts runs revealed that reused: "ref" produced ugly __schemaN $defs and that .meta({ id }) alone yields clean $defs — and that markdownDescription/examples/default all pass through .meta() (no override needed). This is the testing skill's "write a disposable exploratory script first" rule paying off directly.
• Caught a latent regression during implementation. normalizeUnifiedConfig was dual-purpose (config files and per-agent frontmatter); a naive strict-ification would have silently dropped agent permission blocks (frontmatter carries name/model/etc.). Grepping the callers surfaced it, and the fix kept agent frontmatter tolerant while config files went strict.
• Proactively checked biome warnings (exit 0). After removing test blocks, an unused it import lingered as a biome warning — which pnpm run lint passes and the pre-completion reviewer's error-gated checks would miss. Explicitly inspecting biome check warnings caught it before commit.
• Incremental verification, no feedback-loop gap. pnpm run check / the affected test file / pnpm run lint ran after every TDD step, and the full suite + fallow dead-code + a pnpm pack tarball inspection ran at the end. expectTypeOf(...).toEqualTypeOf(...) (enforced by tsc) de-risked the ~58-consumer type lift-and-shift before it landed.
• Ship-flow rule adherence under an in-progress release check. The release-please PR reported UNSTABLE with a check still IN_PROGRESS; per the ship prompt I waited and re-polled rather than falling back to gh pr merge while the check ran, then merged clean via release_pr_merge.

What caused friction (agent side)

• other (path typo) — one Edit used an absolute path missing the pi-packages/packages/ segment, tripping the permission gate. Impact: one rejected call + retry, no rework.
• missing-context — did not anticipate that biome formats committed JSON, so the first gen:schema output failed lint (biome collapses single-element arrays differently than JSON.stringify). Impact: one fix cycle; resolved by chaining biome format --write into gen:schema so the generated file is deterministic and lint-clean.

What caused friction (user side)

• None. The operator's two mid-planning interjections (zod over TypeBox; pin the latest production version + favor schema composability) were well-timed and materially improved the design; they could not have come earlier since the operator did not yet know the schema already existed.

Diagnostic details

• Model-performance correlation — one subagent dispatch: pre-completion-reviewer on anthropic/claude-sonnet-5 (274s, 53 tool uses) for judgment-heavy review (acceptance criteria, design review, doc staleness, Mermaid render). Appropriate model for the work; no mismatch.
• Escalation-delay tracking — no rabbit holes; the longest same-target sequence was the two deliberate zod-exploration script runs, which were investigative, not stuck.
• Unused-tool detection — web_search + fetch_content (zod v4 toJSONSchema docs) and disposable scripts covered the unfamiliar-library risk; no Explore/Plan subagent was warranted (full context was already in-session).
• Feedback-loop gap analysis — none: verification ran incrementally after each step rather than only at the end.

Changes made

1. Added this Final Retrospective stage entry to packages/pi-permission-system/docs/retro/0547-zod-config-schema-single-source.md. No prompt, AGENTS.md, or skill changes: the operator chose retro-only, since the gen:schema script already self-documents the biome format chaining and the other candidate lessons duplicate existing guidance.