feat(tests): add a static skill quality gate with a shrink-only baseline

.github/workflows/validate-pr.yml

@@ -57,6 +57,23 @@ jobs:
         # the tests/ files import yaml.
         run: python3 -m pip install --user pytest pyyaml
 
+      - name: Reject a skill-quality baseline that grew
+        # tests/skill-quality-baseline.json may only shrink. The pytest gate
+        # enforces "skill <= its entry"; this step enforces "entry <= what the
+        # base branch had" so a PR cannot raise or add a ceiling to pass.
+        # The working tree is the PR's test merge commit, so it already holds any
+        # slimming that landed on the base branch, and OLD could be base.sha or
+        # the merge-base. The merge-base is used to match the dist/ and CHANGELOG
+        # steps.
+        run: |
+          set -e
+          MERGE_BASE="$(git merge-base "${{ github.event.pull_request.base.sha }}" "${{ github.event.pull_request.head.sha }}")"
+          if git show "${MERGE_BASE}:tests/skill-quality-baseline.json" > /tmp/base-baseline.json 2>/dev/null; then
+            python3 tests/skill_quality.py --check-ratchet /tmp/base-baseline.json
+          else
+            echo "notice: no baseline at the merge-base; skipping ratchet diff" >&2
+          fi
+
       - name: Validate the Kiro Power (powers/konductor/)
         # Users import the Power from the tagged release matching VERSION
         # (github.com/aws-solutions/konductor/tree/v<VERSION>/powers/konductor),

AGENTS.md

@@ -65,6 +65,20 @@ instructions.
   convert a description from one form to the other for consistency alone. Fix a description
   that gives no activation condition at all. You may broaden an existing condition (for example,
   add a disjunct) when the skill's actual scope grew, but do not churn the form otherwise.
+- Reference material a skill needs only sometimes (catalogues, long templates, worked examples)
+  lives in `skills/<name>/references/*.md`, one level deep, Markdown only. `SKILL.md` keeps the
+  instructions and must name each reference file with a cue saying when to read it (for example
+  "Read `references/aws-best-practices.md` before Stage 3"). `make test-py` runs the checks
+  listed in the docstring of `tests/skill_quality.py`. Among them, it fails on a frontmatter
+  `name` that does not match the directory, a description that is too short or gives no
+  activation condition, a missing link target, an unclosed code fence, an unmentioned reference
+  file, a reference mention without a read cue, and any size over the thresholds in
+  `tests/skill-quality-baseline.json` (currently 500 lines, 32 KiB and 512 description
+  characters) that the baseline does not carry. The baseline may only shrink.
+  `make test-py` only checks each skill against its entry; CI rejects a raised or added entry,
+  and so does `python3 tests/skill_quality.py --print-baseline` when you regenerate. Renaming a
+  skill counts as adding an entry, so bring an oversized skill under every threshold before you
+  rename it.
 - Orchestrators are named `konductor`, `konductor-mux-orchestrator`, and `konductor-cmux-orchestrator`; 
   every specialist uses a `k-*` name. Skills are unprefixed unless avoiding a known collision.
 - **License headers:** All code files must carry an SPDX short-form identifier as the very first line

CHANGELOG.md

@@ -8,6 +8,12 @@ not per individual commit.
 
 ## [Unreleased]
 
+### Added
+
+- `make test-py` now checks every skill's frontmatter, description, size, code fences, and
+  `references/` links against a shrink-only baseline in `tests/skill-quality-baseline.json`.
+  `python3 tests/skill_quality.py --print-baseline` refuses to write a baseline that grew.
+
 ### Changed
 
 - Claude Code subagents now preload 1 to 4 skills each (90 KB in total, down from 1.8 MB); the rest

tests/skill-quality-baseline.json

@@ -0,0 +1,69 @@
+{
+  "_comment": "Shrink-only ratchet read by tests/test_skill_quality.py. An entry is the exact measured value of a dimension that is over its threshold. Lower or delete an entry when you slim a skill; never raise one; never add one for an existing skill unless the threshold above was lowered in the same change. Renaming a skill adds an entry, so bring a renamed skill under every threshold first. Regenerate with: python3 tests/skill_quality.py --print-baseline",
+  "skills": {
+    "cost-estimation": {
+      "lines": 667
+    },
+    "cypress-test-implementation": {
+      "bytes": 53895,
+      "lines": 1944
+    },
+    "dynamodb-design": {
+      "bytes": 75539,
+      "lines": 2195
+    },
+    "dynamodb-validation": {
+      "bytes": 64678,
+      "lines": 1903
+    },
+    "e2e-test-strategy": {
+      "bytes": 46255,
+      "lines": 1595
+    },
+    "humanize-writing": {
+      "bytes": 38522,
+      "lines": 567
+    },
+    "iam-policy-design": {
+      "bytes": 39567,
+      "lines": 969
+    },
+    "infra-validation": {
+      "bytes": 49149,
+      "lines": 1346
+    },
+    "policy-validation": {
+      "bytes": 36839,
+      "lines": 968
+    },
+    "security-test-generation": {
+      "lines": 736
+    },
+    "smithy-modeling": {
+      "lines": 603
+    },
+    "smithy-validation": {
+      "lines": 574
+    },
+    "sop-state-management": {
+      "bytes": 32890
+    },
+    "test-coverage-analysis": {
+      "bytes": 69262,
+      "lines": 2064
+    },
+    "ui-text-validation": {
+      "lines": 570
+    }
+  },
+  "thresholds": {
+    "bytes": 32768,
+    "description_chars": 512,
+    "lines": 500
+  },
+  "tolerance": {
+    "bytes": 1024,
+    "description_chars": 50,
+    "lines": 50
+  }
+}