Skip to content

Spec insights

What the spec corpus looks like in aggregate: how many requirements exist and in what state, how well they are tested and documented, and what the advisory effort and value estimates add up to.

Every number — and every chart — is derived at build time from the files in specs/ — the same source the matrix reads, so the two can never disagree. Counted facts and advisory estimates are kept in separate sections on purpose: an estimate is a judgement, and mixing the two would lend it an authority it has not earned.

The counted section charts each spec's requirements by status in the same colors as the status badges. The advisory section plots customer value against effort for every scored feature: the S16 tier bands sit behind the dots and the diagonal guides mark constant RICE, so value-per-effort reads as slope. The per-spec RICE ranking itself lives on the matrix page, whose RICE column sorts and carries a proportional bar — this page deliberately does not repeat it.

Two further charts join artifacts the repository already keeps but never plotted together: each S13 estimate's committed size snapshot against the size of that code today, which surfaces the estimates that have aged out of their band, and the S08 demo registry against the value ranking, which surfaces the features nothing demonstrates.

Recording what broke

Conventional Commits already marks which commits are fixes; what it never captured is what they fixed. Every fix: commit now names the requirement it repaired in a trailer, checked by the same commit-msg hook that runs commitlint:

fix(auth): treat an unauthenticated GitHub 404 as auth-required

Fixes: F07-FR-03

The hook rejects a fix: with no trailer, a malformed id, or an id that is not in the traceability matrix. Dependency patches are exempt — use the deps scope, since patching an advisory repairs no requirement of this project. There is no other escape hatch: if a fix repairs behaviour no requirement describes, the requirement gets written first, because behaviour here is specified before it is built.

Attribution was not backfilled onto existing history, and it is not inferred from the files a commit touched. That inference was tried and measured: the 8 fix commits in the history touch files claimed by 29 different specs, correlating 0.89 with raw commit churn — it measures which files are shared, not which requirements break. Counts come from git log on each docs build, and the count of unattributed fixes is always shown beside them so an empty column reads as "not recorded yet" rather than "never breaks".

Test strength

Line coverage is gated at 80% on all four axes and every spec clears it, sitting between 90% and 99% — the gate working, but also a measurement with no range left to rank anything by. The mutation score replaces it as the test-strength axis: it mutates the source and asks whether the tests notice, so it measures whether tests assert rather than merely execute.

Because a full mutation run takes orders of magnitude longer than the commit gate, it is never part of npm run verify. Refresh it on demand:

bash
npm run test:mutation   # runs Stryker, writes reports/mutation/mutation.json
npm run mutation:score  # distils it into mutation-score.json

That committed artifact stores the mutant tallies per file alongside the derived score, so the number can be audited and recomputed rather than taken on trust, and it records the date it was generated — the score is a snapshot of code that keeps moving, so the charts show its age. If the artifact is absent the site reports the score as not measured; a missing measurement is never drawn as a zero.

Counted facts

Everything in this section is a count taken from the spec files and the traceability matrix at build time.

47specs28 feature · 19 system
612requirements13.0 per spec on average
67%carry a test tag407 of 612 requirements
43%mean doc coverage5 of 47 specs fully documented

Requirements by status

444 implemented168 manual

One bar per spec, segments colored like the badges above; bar length compares requirement counts across specs. Hover a segment for its exact count.

Requirement lifecycle

How long requirements sit specified before they are built, from the S11 stamps. These are deliberately not backfilled: the matrix was populated in one commit long after most requirements were written, so a git-derived date would record when the matrix learned of a requirement and pass it off as when the requirement was specified.

47lifecycle-tracked565 of 612 predate stamping
0specified, not yet builtnothing outstanding
0median days to buildacross 47 completed
same day46
1–3 days1

Where the fixes land

Every fix: commit names the requirement it repaired in a Fixes: trailer (S19), enforced by the commit hook. Counts are read from git log on each build, so they cannot drift from the history.

13attributed fixesof 81 fix commits
68unattributedpredate the convention, or exempt
3specs with a recorded fixof 47

Advisory estimates

These are judgements, not measurements. Effort comes from the S13 rubric and value from S16; neither feeds the maturity score, which is built from observable facts only.

310story points total6.6 mean across 47 specs
275.8customer value total9.9 mean across 28 features
4.40best RICEF02 · Live (Debounced) Preview Updates
27.5highest valueF03 · JSON / YAML Validation

Value against effort

Each dot is a scored feature spec (dots sharing the same estimates are merged — hover for the spec ids). Height is the S16 value score with its tier bands; the diagonals are constant RICE, so steeper-than-a-guide means more value per point. The full per-spec numbers live in the matrix, sortable by RICE.

MarginalNicheModerateHighCritical RICE 0.5 RICE 1 RICE 2 RICE 405100102030 effort (story points, S13) customer value (S16) F01 — value 25 · 13 pts · RICE 1.92F01F02 — value 13.2 · 3 pts · RICE 4.40F02F03 — value 27.5 · 8 pts · RICE 3.44F03F04 — value 17.6 · 8 pts · RICE 2.20F05, F25 — value 5.3 · 3 pts · RICE 1.76F06, F08 — value 10.6 · 8 pts · RICE 1.32F07 — value 5.8 · 8 pts · RICE 0.72F09 — value 6 · 2 pts · RICE 3.00F10 — value 13.2 · 5 pts · RICE 2.64F11 — value 5.3 · 8 pts · RICE 0.66F12 — value 15.4 · 8 pts · RICE 1.92F13 — value 12 · 5 pts · RICE 2.40F14, F15, F22 — value 7.0 · 8 pts · RICE 0.88F16 — value 7.2 · 8 pts · RICE 0.90F17 — value 7.2 · 3 pts · RICE 2.40F18 — value 8.6 · 8 pts · RICE 1.08F19 — value 5.3 · 5 pts · RICE 1.06F20 — value 10.6 · 5 pts · RICE 2.11F21 — value 10.6 · 3 pts · RICE 3.52F23 — value 1.4 · 3 pts · RICE 0.45F24 — value 4.8 · 5 pts · RICE 0.96F26 — value 3.5 · 2 pts · RICE 1.76F27 — value 7.7 · 3 pts · RICE 2.56F28 — value 15.4 · 13 pts · RICE 1.18

Are the effort estimates still current?

Each estimate committed the implementation size it was written against (S13). Plotted against the size today, anything on the diagonal has not moved; anything above it has grown since. The horizontal lines are the rubric's own LOC band boundaries, so a dot that has crossed one is an estimate whose band no longer matches its code — the same specs npm run check:spec-effort warns about, from the same function.

This chart deliberately does not plot effort points against implementation size: points are derived from that size by the band table, so the two cannot help but agree, and reading the agreement as calibration would be circular.

S13 bands 1504008001,5002,5004,000 unchanged since estimated 005005001,5001,5003,0003,0005,0005,000 lines when the estimate was written lines today F01 Schema Preview Panel — estimated at 1,820 lines (band 8), 2,201 today (band 8)F02 Live (Debounced) Preview Updates — estimated at 916 lines (band 5), 1,156 today (band 5)F03 JSON / YAML Validation — estimated at 1,813 lines (band 8), 1,813 today (band 8)F04 Schema–File Binding — estimated at 1,878 lines (band 8), 1,970 today (band 8)F05 Visual Form-Based Schema Editor — estimated at 262 lines (band 2), 262 today (band 2)F06 Schema Inference from Data Files — estimated at 2,057 lines (band 8), 2,162 today (band 8)F07 Remote Schema Authentication — estimated at 1,169 lines (band 5), 1,196 today (band 5)F08 Local Schema Cache — estimated at 2,155 lines (band 8), 2,191 today (band 8)F09 Preview Configuration: File and Settings — estimated at 75 lines (band 1), 1,471 today (band 5) — consider re-estimatingF09F10 Inline Schema Binding via `$schema` Field — estimated at 1,093 lines (band 5), 1,080 today (band 5)F11 TOML File Support — estimated at 2,369 lines (band 8), 2,474 today (band 8)F12 Schema Catalog & Registry Binding — estimated at 1,467 lines (band 5), 1,476 today (band 5)F13 `$ref` Navigation & Hover — estimated at 948 lines (band 5), 1,090 today (band 5)F14 Schema Bundling & Dereferencing — estimated at 1,336 lines (band 5), 1,465 today (band 5)F15 Schema Diff & Breaking-Change Detection — estimated at 1,399 lines (band 5), 1,517 today (band 8) — consider re-estimatingF15F16 Sample Data Generation — estimated at 1,788 lines (band 8), 1,788 today (band 8)F17 Schema Linting — estimated at 634 lines (band 3), 656 today (band 3)F18 Code Generation (Schema → Types) — estimated at 2,592 lines (band 13), 2,701 today (band 13)F19 TOML Schema IntelliSense — estimated at 1,023 lines (band 5), 1,036 today (band 5)F20 Workspace Validation Report — estimated at 1,658 lines (band 8), 1,798 today (band 8)F21 Validation Quick Fixes — estimated at 631 lines (band 3), 631 today (band 3)F22 Draft Migration — estimated at 1,253 lines (band 5), 1,371 today (band 5)F23 Schema Coverage (unused-in-data lens) — estimated at 799 lines (band 3), 812 today (band 5) — consider re-estimatingF23F24 `$ref` Dependency Graph View — estimated at 868 lines (band 5), 1,256 today (band 5)F25 Nearest-Match Ranking for `enum` Quick Fixes — estimated at 408 lines (band 3), 408 today (band 3)F26 Backward-Compatibility Verdict & CI Gate — estimated at 304 lines (band 2), 304 today (band 2)F27 Standalone Command-Line Interface — estimated at 775 lines (band 3), 781 today (band 3)F28 Preview Scroll Sync — estimated at 2,440 lines (band 8), 2,440 today (band 8)S01 Webview Security — estimated at 582 lines (band 3), 809 today (band 5) — consider re-estimatingS01S02 Workspace Trust Integration — estimated at 1,650 lines (band 8), 1,995 today (band 8)S03 Performance and Resource Management — estimated at 3,039 lines (band 13), 3,457 today (band 13)S04 Reliability and Offline Behaviour — estimated at 454 lines (band 3), 454 today (band 3)S05 Privacy and Data Collection — estimated at 769 lines (band 3), 796 today (band 3)S06 Accessibility — estimated at 979 lines (band 5), 1,220 today (band 5)S07 Documentation Traceability — estimated at 2,601 lines (band 13), 2,874 today (band 13)S08 End-to-End Testing — estimated at 2,656 lines (band 13), 2,944 today (band 13)S09 CI Workflow Scoping — estimated at 272 lines (band 2), 2,001 today (band 8) — consider re-estimatingS09S10 Spec Visualization on the Docs Site — estimated at 1,705 lines (band 8), 3,500 today (band 13) — consider re-estimatingS10S11 Traceability Matrix Schema & Generated Types — estimated at 4,822 lines (band 21), 6,241 today (band 21)S12 Maturity Scorecard Visualization on the Docs Site — estimated at 2,093 lines (band 8), 2,428 today (band 8)S13 Spec Effort Estimation (Advisory) — estimated at 2,090 lines (band 8), 2,372 today (band 8)S14 Delivery Performance (DORA Metrics) — estimated at 198 lines (band 2), 2,603 today (band 13) — consider re-estimatingS14S15 Cross-Platform Tooling (Development & Deployment) — estimated at 385 lines (band 2), 1,639 today (band 8) — consider re-estimatingS15S16 Feature Value Estimation (Advisory) — estimated at 1,985 lines (band 8), 2,139 today (band 8)S17 Concurrent Verify Gate — estimated at 846 lines (band 5), 1,007 today (band 5)S18 Test Strength (Mutation Score Publication) — estimated at 2,109 lines (band 8), 2,269 today (band 8)S19 Defect Attribution — estimated at 1,535 lines (band 8), 1,542 today (band 8)

8 of 47 estimates have aged out of their bandF09, F15, F23, S01, S09, S10, S14, S15. Drift is a prompt to re-estimate, never a failure: the checker warns and the build stays green.

Value against test strength

Mutation score (S18) rather than line coverage: every spec clears the 80% coverage gate and sits in a 90–99% band, so coverage ranks nothing. Mutation testing asks whether the tests assert, which still varies. A feature toward the bottom-right is worth a lot and thinly tested. Scores measured 2026-07-26; they age as the code moves.

0% 25% 50% 75% 100% 60% break 0102030 customer value (S16) mutation score F01 Schema Preview Panel — value 25.0, mutation score 61.1% F02 Live (Debounced) Preview Updates — value 13.2, mutation score 53.7% F02F03 JSON / YAML Validation — value 27.5, mutation score 64.1% F04 Schema–File Binding — value 17.6, mutation score 66.4% F06 Schema Inference from Data Files — value 10.6, mutation score 61.4% F07 Remote Schema Authentication — value 5.8, mutation score 68.5% F08 Local Schema Cache — value 10.6, mutation score 66.5% F09 Preview Configuration: File and Settings — value 6.0, mutation score 57.5% F10 Inline Schema Binding via `$schema` Field — value 13.2, mutation score 64.4% F11 TOML File Support — value 5.3, mutation score 64.4% F12 Schema Catalog & Registry Binding — value 15.4, mutation score 68.6% F13 `$ref` Navigation & Hover — value 12.0, mutation score 61.1% F14 Schema Bundling & Dereferencing — value 7.0, mutation score 58.5% F15 Schema Diff & Breaking-Change Detection — value 7.0, mutation score 55.2% F16 Sample Data Generation — value 7.2, mutation score 56.8% F17 Schema Linting — value 7.2, mutation score 60.8% F18 Code Generation (Schema → Types) — value 8.6, mutation score 60.5% F19 TOML Schema IntelliSense — value 5.3, mutation score 61.6% F20 Workspace Validation Report — value 10.6, mutation score 64.9% F21 Validation Quick Fixes — value 10.6, mutation score 77.7% F22 Draft Migration — value 7.0, mutation score 62% F23 Schema Coverage (unused-in-data lens) — value 1.4, mutation score 58% F24 `$ref` Dependency Graph View — value 4.8, mutation score 53.2% F25 Nearest-Match Ranking for `enum` Quick Fixes — value 5.3, mutation score 76.4% F26 Backward-Compatibility Verdict & CI Gate — value 3.5, mutation score 55.8% F27 Standalone Command-Line Interface — value 7.7, mutation score 67.9% F28 Preview Scroll Sync — value 15.4, mutation score 64%

1 feature scoring 10 or better falls below the 60% break thresholdF02 (53.7%). The gate measures the whole suite at once, so a spec below the line is invisible to it while the total stays above. 1 scored feature has no mutated implementation file and so no score.

Which features have never been demonstrated

Feature specs by customer value, marked with whether an end-to-end demo exercises them — read from S08's demo registry, the same file the GIF pipeline and release-time change detection use. Whether a demo exists is a counted fact; only the ordering is an estimate. 19 of 47 specs carry one.

01020F03F03 JSON / YAML Validation — value 27.5· demo recordedF01F01 Schema Preview Panel — value 25.0· demo recordedF04F04 Schema–File Binding — value 17.6· demo recordedF12F12 Schema Catalog & Registry Binding — value 15.4· no demo no demo F28F28 Preview Scroll Sync — value 15.4· demo recordedF02F02 Live (Debounced) Preview Updates — value 13.2· demo recordedF10F10 Inline Schema Binding via `$schema` Field — value 13.2· demo recordedF13F13 `$ref` Navigation & Hover — value 12.0· demo recordedF06F06 Schema Inference from Data Files — value 10.6· demo recordedF08F08 Local Schema Cache — value 10.6· no demo no demo F20F20 Workspace Validation Report — value 10.6· demo recordedF21F21 Validation Quick Fixes — value 10.6· demo recordedF18F18 Code Generation (Schema → Types) — value 8.6· demo recordedF27F27 Standalone Command-Line Interface — value 7.7· no demoF16F16 Sample Data Generation — value 7.2· demo recordedF17F17 Schema Linting — value 7.2· demo recordedF14F14 Schema Bundling & Dereferencing — value 7.0· demo recordedF15F15 Schema Diff & Breaking-Change Detection — value 7.0· no demoF22F22 Draft Migration — value 7.0· demo recordedF09F09 Preview Configuration: File and Settings — value 6.0· demo recordedF07F07 Remote Schema Authentication — value 5.8· demo recordedF05F05 Visual Form-Based Schema Editor — value 5.3· demo recordedF11F11 TOML File Support — value 5.3· no demoF19F19 TOML Schema IntelliSense — value 5.3· no demoF25F25 Nearest-Match Ranking for `enum` Quick Fixes — value 5.3· no demoF24F24 `$ref` Dependency Graph View — value 4.8· no demoF26F26 Backward-Compatibility Verdict & CI Gate — value 3.5· no demoF23F23 Schema Coverage (unused-in-data lens) — value 1.4· no demo customer value (S16) — highest first

The gap is not evenly spread: 2 features scoring 10 or better have no demoF12 (15.4), F08 (10.6).

Value tiers

Critical2
High6
Moderate12
Niche7
Marginal1

Effort spread

S14
M8
L20
XL4
XXL1

Released under the MIT License.