Rendered from specs/F26-compat-gate.md — edit it there, not here.

estimate2 ptsS2–4 hCompatibility verdict layered on the F15 diff — sized by its band. Advisory only — estimated by the project's AI coding agent (Claude Code) on 2026-07-20 from the implementation (304 LOC across 3 files); not part of the maturity score. Rubric: S13.

value3.5 valueNicheRICE 1.76CI reach 1 (under 5% of users) · impact 4 (removes a manual workflow) · confidence 0.8 (domain-reasoned) A CI compatibility gate is decisive for the few teams that publish versioned schemas and invisible to everyone else. Advisory only — a judgement about worth, not a measurement, estimated by the project's AI coding agent (Claude Code) on 2026-07-25; not part of the maturity score. RICE divides this score by the S13 effort estimate (2 pts). Rubric: S16.

F26 — Backward-Compatibility Verdict & CI Gate

Overview

F15 classifies every change between two schema versions as breaking / non-breaking / informational / unclassified, but stops at showing the diff — it explicitly deferred "CI enforcement" to a companion. This spec is that companion. It turns the classification into a single verdict — "is the new schema backward-compatible with documents valid under the old one?" — surfaces that verdict in the diff command, and ships a headless CLI so the same check can gate a pull request in CI (fail the build when a schema change would break existing consumers).

User Stories

• As a schema publisher, I want one yes/no answer — is this change safe to ship? — not a list I have to interpret every time.
• As a platform team, I want a CI step that fails a PR when a schema change is breaking, so an incompatible schema never merges by accident.
• As a reviewer, I want the diff's summary to lead with the verdict, so I judge compatibility before reading the details.

Functional Requirements

Verdict

• F26-FR-01 A pure function MUST reduce a set of F15 DiffEntry values to a verdict: the per-kind counts and a boolean compatible. By default compatible is true exactly when there are no breaking changes.
• F26-FR-02 A strict mode MUST additionally treat any unclassified change as incompatible (the classifier could not prove safety), so a strict CI gate errs on the side of caution.
• F26-FR-03 The verdict MUST map to a process exit code for CI: 0 when compatible, 1 when breaking changes make it incompatible, and — in strict mode with no breaking changes but ≥ 1 unclassified change — a distinct 2 ("compatibility unknown"), so a pipeline can treat "unknown" differently from "breaking" if it wants.
• F26-FR-04 A renderer MUST produce a report that leads with the verdict headline (e.g. ⛔ NOT backward-compatible — 2 breaking change(s) or ✅ Backward-compatible) followed by F15's grouped change list, conveying severity by text label, never colour alone (S06).

Surfaces

• F26-FR-05 The F15 diff command's summary notification MUST lead with the verdict verb (backward-compatible vs not), reusing the same pure verdict so the editor and CI never disagree.
• F26-FR-06 A headless CLI (scripts/schema-compat.mjs, run via npm run schema:compat -- <old> <new> [--strict] [--json]) MUST compare two schema files (JSON/JSONC/YAML), print the verdict report (or JSON with --json), and exit with the F26-FR-03 code — reusing the same classifier and verdict modules as the extension, with no duplicated logic and no vscode import. It MUST print a usage message and exit non-zero on bad arguments or an unreadable/unparseable file.
• F26-FR-07 The verdict MUST also be available passively, as a CodeLens on the schema document's first line reading the breaking-change count against the file's Git HEAD version (e.g. "2 breaking changes vs HEAD"), whose command opens the full report (F15). A schema with no Git baseline, or with no changes, MUST show no lens rather than an empty one. The lens MUST be governed by jsonschema.compat.codeLens (default true) and MUST compute from the same classifier as the command — never a second implementation.
• F26-FR-08 Lens computation MUST NOT block typing: it MUST be debounced and MUST reuse the previous result while a recomputation is in flight.

Non-Functional Requirements

• F26-NFR-01 The verdict and report renderer (schemaCompat) MUST be a pure, vscode-free module with ≥ 80 % unit-test coverage (Article V), building only on F15's schemaDiff. The CLI wrapper is a thin entry script (like the other scripts/*.mjs), not unit-tested, and must add no new runtime dependency.
• F26-NFR-02 The comparison MUST run in-process with no network or file access inside the pure module (the CLI does its own reads); it reuses F15's in-process classifier (S03) and never throws on malformed input — a parse failure is reported by the CLI as a non-zero exit, not a stack trace.

Out of Scope

• Deep comparison across external $ref targets (bundle via F14 first) — the same limitation F15 documents.
• A hosted GitHub Action wrapper — the npm script is the CI primitive; wiring it into a specific workflow is the consumer's choice (an example is documented).
• Changing F15's classification rules; F26 only aggregates them.

Acceptance Criteria

1. Adding a name to required yields a verdict with compatible: false, exit code 1, and a report headline naming 1 breaking change.
2. Adding an optional property yields compatible: true, exit code 0.
3. A change the classifier reports as unclassified yields exit 0 by default but exit 2 under --strict.
4. npm run schema:compat -- old.json new.json prints the verdict and exits with the matching code; a missing file prints usage/error and exits non-zero.
5. The diff command's notification leads with "backward-compatible" or "NOT backward-compatible", consistent with the CLI's verdict for the same pair.

Relation to Existing Specs

• Directly extends F15 (schema diff) — consumes its DiffEntry output and reuses diffSchemas/renderReport; this is the "CI enforcement" F15 listed as out of scope.
• Honours S03 (in-process, no new latency/deps) and S06 (text-first severity). S05: nothing leaves the machine; the CLI only reads the two files it is given.

History

• 2026-09-02 — Added F26-FR-07/08: the verdict as a passive CodeLens on the schema's first line, counting breaking changes against Git HEAD. Debounced and computed in the background from the same classifier as the command, so typing is never blocked; no baseline or no changes means no lens rather than an empty one.

Requirement traceability

Status of each requirement in specs/traceability.json. Test coverage is auto-discovered from [ID] tags in test titles and is not listed here.

Requirement	Status	Implementation	Note
F26-FR-01	implemented	src/schemaCompat.ts	compatibilityVerdict reduces DiffEntry counts; compatible when no breaking
F26-FR-02	implemented	src/schemaCompat.ts	strict mode also fails on unclassified changes
F26-FR-03	implemented	src/schemaCompat.ts	verdictExitCode: 0 compatible / 1 breaking / 2 strict-unknown
F26-FR-04	implemented	src/schemaCompat.ts	renderCompatReport leads with verdictHeadline, then F15 grouped list
F26-FR-05	implemented	src/SchemaDiffCommand.ts, src/schemaCompat.ts	diff command summary leads with the verdict verb
F26-FR-06	manual	scripts/schema-compat.mjs, src/schemaCompat.ts	headless CLI: npm run schema:compat; reuses out/ modules, verdict exit codes; verified manually
F26-FR-07	implemented	src/CompatCodeLensProvider.ts, package.json	debounced CodeLens reusing the F15/F26 classifier; countBreaking unit-tested
F26-FR-08	implemented	src/CompatCodeLensProvider.ts, package.json	debounced CodeLens reusing the F15/F26 classifier; countBreaking unit-tested
F26-NFR-01	implemented	src/schemaCompat.ts	pure vscode-free verdict/renderer building only on schemaDiff; unit-tested
F26-NFR-02	implemented	src/schemaCompat.ts	in-process, no I/O in the pure module, never throws

Documented in

Documentation sections tagged spec:F26 (or one of its requirement ids) — 415 of ~455 expected words (91%). The expectation comes from this spec's evaluated complexity of 11.4 (10 documentable requirements — 8 FR, 2 NFR — weighted by kind and by each definition's length).

• EvolveREADME.md · 83 words
• JSON Schema: Diff Against Baseline/guide/commands · 118 words
• jsonschema.compat.codeLens Type: boolean · Default: true/guide/configuration · 42 words
• Guarding: schema tests/guide/lifecycle · 172 words

Changes over time

• 56cbbde2026-09-02feat(specs): schema tests, refactorings, outline, agent tools & AI authoring
• 0cff56f2026-07-18feat(F26): backward-compatibility verdict & CI gate for schema diff