Rendered from specs/F26-compat-gate.md — edit it there, not here.

estimate2 ptsS2–4 hCompatibility verdict layered on the F15 diff — sized by its band. Advisory only — estimated by the project's AI coding agent (Claude Code) on 2026-07-20 from the implementation (304 LOC across 3 files); not part of the maturity score. Rubric: S13.

value3.5 valueNicheRICE 1.76CI reach 1 (under 5% of users) · impact 4 (removes a manual workflow) · confidence 0.8 (domain-reasoned) A CI compatibility gate is decisive for the few teams that publish versioned schemas and invisible to everyone else. Advisory only — a judgement about worth, not a measurement, estimated by the project's AI coding agent (Claude Code) on 2026-07-25; not part of the maturity score. RICE divides this score by the S13 effort estimate (2 pts). Rubric: S16.

F26 — Backward-Compatibility Verdict & CI Gate

Overview

F15 classifies every change between two schema versions as breaking / non-breaking / informational / unclassified, but stops at showing the diff — it explicitly deferred "CI enforcement" to a companion. This spec is that companion. It turns the classification into a single verdict — "is the new schema backward-compatible with documents valid under the old one?" — surfaces that verdict in the diff command, and ships a headless CLI so the same check can gate a pull request in CI (fail the build when a schema change would break existing consumers).

User Stories

• As a schema publisher, I want one yes/no answer — is this change safe to ship? — not a list I have to interpret every time.
• As a platform team, I want a CI step that fails a PR when a schema change is breaking, so an incompatible schema never merges by accident.
• As a reviewer, I want the diff's summary to lead with the verdict, so I judge compatibility before reading the details.

Functional Requirements

Verdict

• F26-FR-01 A pure function MUST reduce a set of F15 DiffEntry values to a verdict: the per-kind counts and a boolean compatible. By default compatible is true exactly when there are no breaking changes.
• F26-FR-02 A strict mode MUST additionally treat any unclassified change as incompatible (the classifier could not prove safety), so a strict CI gate errs on the side of caution.
• F26-FR-03 The verdict MUST map to a process exit code for CI: 0 when compatible, 1 when breaking changes make it incompatible, and — in strict mode with no breaking changes but ≥ 1 unclassified change — a distinct 2 ("compatibility unknown"), so a pipeline can treat "unknown" differently from "breaking" if it wants.
• F26-FR-04 A renderer MUST produce a report that leads with the verdict headline (e.g. ⛔ NOT backward-compatible — 2 breaking change(s) or ✅ Backward-compatible) followed by F15's grouped change list, conveying severity by text label, never colour alone (S06).

Surfaces

• F26-FR-05 The F15 diff command's summary notification MUST lead with the verdict verb (backward-compatible vs not), reusing the same pure verdict so the editor and CI never disagree.
• F26-FR-06 A headless CLI (scripts/schema-compat.mjs, run via npm run schema:compat -- <old> <new> [--strict] [--json]) MUST compare two schema files (JSON/JSONC/YAML), print the verdict report (or JSON with --json), and exit with the F26-FR-03 code — reusing the same classifier and verdict modules as the extension, with no duplicated logic and no vscode import. It MUST print a usage message and exit non-zero on bad arguments or an unreadable/unparseable file.

Non-Functional Requirements

• F26-NFR-01 The verdict and report renderer (schemaCompat) MUST be a pure, vscode-free module with ≥ 80 % unit-test coverage (Article V), building only on F15's schemaDiff. The CLI wrapper is a thin entry script (like the other scripts/*.mjs), not unit-tested, and must add no new runtime dependency.
• F26-NFR-02 The comparison MUST run in-process with no network or file access inside the pure module (the CLI does its own reads); it reuses F15's in-process classifier (S03) and never throws on malformed input — a parse failure is reported by the CLI as a non-zero exit, not a stack trace.

Out of Scope

• Deep comparison across external $ref targets (bundle via F14 first) — the same limitation F15 documents.
• A hosted GitHub Action wrapper — the npm script is the CI primitive; wiring it into a specific workflow is the consumer's choice (an example is documented).
• Changing F15's classification rules; F26 only aggregates them.

Acceptance Criteria

1. Adding a name to required yields a verdict with compatible: false, exit code 1, and a report headline naming 1 breaking change.
2. Adding an optional property yields compatible: true, exit code 0.
3. A change the classifier reports as unclassified yields exit 0 by default but exit 2 under --strict.
4. npm run schema:compat -- old.json new.json prints the verdict and exits with the matching code; a missing file prints usage/error and exits non-zero.
5. The diff command's notification leads with "backward-compatible" or "NOT backward-compatible", consistent with the CLI's verdict for the same pair.

Relation to Existing Specs

• Directly extends F15 (schema diff) — consumes its DiffEntry output and reuses diffSchemas/renderReport; this is the "CI enforcement" F15 listed as out of scope.
• Honours S03 (in-process, no new latency/deps) and S06 (text-first severity). S05: nothing leaves the machine; the CLI only reads the two files it is given.

Requirement traceability

Status of each requirement in specs/traceability.json. Test coverage is auto-discovered from [ID] tags in test titles and is not listed here.

Requirement	Status	Implementation	Note
F26-FR-01	implemented	src/schemaCompat.ts	compatibilityVerdict reduces DiffEntry counts; compatible when no breaking
F26-FR-02	implemented	src/schemaCompat.ts	strict mode also fails on unclassified changes
F26-FR-03	implemented	src/schemaCompat.ts	verdictExitCode: 0 compatible / 1 breaking / 2 strict-unknown
F26-FR-04	implemented	src/schemaCompat.ts	renderCompatReport leads with verdictHeadline, then F15 grouped list
F26-FR-05	implemented	src/SchemaDiffCommand.ts, src/schemaCompat.ts	diff command summary leads with the verdict verb
F26-FR-06	manual	scripts/schema-compat.mjs, src/schemaCompat.ts	headless CLI: npm run schema:compat; reuses out/ modules, verdict exit codes; verified manually
F26-NFR-01	implemented	src/schemaCompat.ts	pure vscode-free verdict/renderer building only on schemaDiff; unit-tested
F26-NFR-02	implemented	src/schemaCompat.ts	in-process, no I/O in the pure module, never throws

Documented in

Documentation sections tagged spec:F26 (or one of its requirement ids) — 118 of ~364 expected words (32%). The expectation comes from this spec's evaluated complexity of 9.1 (8 documentable requirements — 6 FR, 2 NFR — weighted by kind and by each definition's length).

• JSON Schema: Diff Against Baseline/guide/commands · 118 words

Changes over time

• 0cff56f2026-07-18feat(F26): backward-compatibility verdict & CI gate for schema diff