Measuring and Gating Requirement Coverage

Run coverage profiles to find which requirements are reachable from your boundary nodes, gate CI on uncovered items, and turn gaps into tracked review sessions.

Coverage profiles answer a single question: is every requirement reachable from a defined boundary? You declare one or more profiles in your xspec config, each naming a boundary (e.g., test files) and a target set (e.g., leaf requirement nodes), then run xspec coverage to see which targets are covered, which are not, and which are intentionally ignored.

Running coverage

xspec coverage              # run all configured profiles
xspec coverage tested       # run one profile by name
xspec coverage tested --check  # CI gate: exits 1 if any node is uncovered

Coverage reads the graph automatically — you do not need to run xspec build first. Read commands auto-refresh from sources.

Reading the output

$ xspec coverage
profile tested
  required: 3, covered: 2, uncovered: 1, ignored: 5
  covered:
    specs/AUTH.mdx#auth.login.valid
      path: test/auth.test.ts#testValidLogin -> specs/AUTH.mdx#auth.login.valid
    specs/AUTH.mdx#auth.login.invalid
      path: test/auth.test.ts#testInvalidLogin -> specs/AUTH.mdx#auth.login.invalid
  uncovered:
    specs/AUTH.mdx#auth.lockout
  ignored:
    specs/AUTH.mdx: root node; non-leaf under targets: "leaves"
    specs/AUTH.mdx#auth: non-leaf under targets: "leaves"
    specs/OVERVIEW.mdx#overview: coverage="none"

The summary line breaks down into four counts:

Field	Meaning
required	Nodes that must be covered (in target set, not ignored)
covered	Required nodes reachable from at least one boundary node
uncovered	Required nodes with no path from any boundary node
ignored	Nodes excluded from coverage — structural parents or tagged coverage="none"

Every covered node shows one shortest covering path so you can trace exactly which boundary node satisfies it.

CI gating with --check

Pass --check to make xspec coverage exit with code 1 if any required node is uncovered. A typical CI sequence looks like this:

1. Build the graph

   xspec build
   
2. Validate consistency

   xspec check
   
3. Gate on coverage

   xspec coverage tested --check
   

You can gate on multiple profiles in the same pipeline by repeating the command for each profile name, or by running xspec coverage --check to check all profiles at once.

Direct vs transitive mode

Each profile declares a traversal mode that controls what counts as a covering path:

• direct — the boundary node itself must reference the requirement in a single edge. No intermediaries allowed. Use this for test code that should cite specs explicitly.
• transitive — chains of any length are allowed. A test can cover a design spec that in turn covers a product requirement. Use this for higher-level design or architecture profiles.

Running both modes as separate profiles is cheap and gives you layered visibility: tested (direct, test files) catches regressions in explicit citation, while designed (transitive, design specs) ensures architectural coverage.

Practical tips

Default to leaves as your target. Parent nodes are structural containers; leaf nodes are the testable statements. The built-in "leaves" target selector picks them automatically and keeps parent nodes in the ignored count rather than the uncovered count.

• Scope profiles with tags. Use targetTags: ["critical"] to build a profile that hard-gates only your highest-priority requirements. This lets you enforce strict coverage on a subset without requiring 100 % coverage on everything.

• Multiple profiles are cheap. Each profile is a separate named configuration entry. Running tested (direct) and designed (transitive) together costs little and shows two complementary views in one command.

• Turn gaps into a checklist. Once you have a list of uncovered nodes, convert it into a tracked review session:

  xspec review create --coverage tested --name coverage-gaps
  

  This creates a session with one item per uncovered node. Resolve each item as you add coverage, and the session tracks your progress. See Running Staged Review Sessions for details.