Skip to content

CI: gate main on openspec validate --all --strict #105

Closed

Description

@theCodeDrift

What

No workflow runs openspec validate. Add one so a spec that fails --strict can't reach main.

What exists today

There are two OpenSpec workflows, and neither validates spec content:

  • pr-check-openspec.yml: the archive gate. Checks that a change directory got archived, on the tip PR of a stack only.
  • openspec-rot.yml: a daily sweep of main for unarchived change debris.

Both are about change lifecycle. Nothing checks that the specs in openspec/specs/ are well-formed.

Why it matters

Because nothing enforces it, spec rot accumulates silently and only gets found by hand. Three examples turned up recently, all pre-existing on main:

  • openspec/specs/cli-help/spec.md carried an orphaned duplicate of a requirement that had lost both its ### Requirement: header and its opening code fence. A documentation template parsed as real ## Goal / ## Preconditions headings, which ended the Requirements section and hid every requirement below it from the parser. We only found it because it made an unrelated openspec archive fail.
  • openspec/specs/cli-rules/spec.md fails --strict with roughly 18 errors: requirements with no #### Scenario: block, and requirements whose text contains neither SHALL nor MUST.
  • openspec/specs/cli-update-engine/spec.md fails because it has zero requirements. It's a tombstone for a decommissioned capability, retained "for historical reference".

Suggested shape

A job running pnpm openspec validate --all --strict on pull requests and on pushes to main.

It has to be repo-wide (--all). The failures above live in specs that a given PR doesn't touch, so a changed-files-only check would never have surfaced them.

Blocked until the specs are clean

Turning the gate on while cli-rules and cli-update-engine fail would make it red from day one, which is how a check gets ignored. A cleanup PR addressing both is in flight on top of the current stack. Land that first, then add the gate.

Activity

  1. theCodeDrift commented on Aug 15, 2026

    @theCodeDrift
    MemberAuthor

    Follow-up from the cleanup in #106: --strict alone isn't enough. It misses a whole class of rot.

    A second ## silently truncates a spec

    Anything after a second ## heading inside the requirements section stops being parsed as a requirement. What's below it isn't invalid, it's invisible. Measured across openspec/specs/:

    Spec Requirements in file Visible to the parser --strict verdict
    cli-rules 33 23 failing (for unrelated reasons)
    infrastructure 20 1 passing
    skills 7 1 passing

    infrastructure and skills were green the whole time. 35 requirements went unread while the validator reported success. That's worse than a red check: a passing gate is an active claim that the spec was read.

    Same shape as the cli-help defect found earlier, where a lost opening code fence turned a documentation template's ## Goal / ## Preconditions into real headings and hid every requirement below them.

    So the gate wants two checks

    1. pnpm openspec validate --all --strict (the original ask)
    2. Every ### Requirement: sits under ## Requirements

    The second is a few lines and catches what the first structurally cannot:

    for spec in pathlib.Path('openspec/specs').glob('*/spec.md'):
        lines = spec.read_text().split('\n')
        total = sum(1 for l in lines if l.startswith('### Requirement:'))
        visible, inreq = 0, False
        for l in lines:
            if l.startswith('## '):
                inreq = l.strip() == '## Requirements'
            elif l.startswith('### Requirement:') and inreq:
                visible += 1
        if total != visible:
            fail(f"{spec}: {total - visible} requirement(s) hidden by a '##' heading")

    #106 brings all 23 specs to 0 failures with every requirement visible, so both checks are green today and the gate can go on without landing red.

    Topical grouping inside a requirements section still works, just as a bold lead-in line rather than a heading. #106 converts ## API Contract, ## Build Tooling, ## Distribution and friends that way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions