feat(workflows): add the peer-advisory-drift reusable check - #17
Conversation
Dependabot audits what is installed. Nothing audits what a package declares it supports, so a peerDependencies range keeps admitting a vulnerable version long after the advisory lands — the range never changes, the advisory database does. The drift arrives with no commit, no pull request and no alert. Scheduled rather than event-driven for that reason: the degradation happens between releases, when nobody is reading the file. Reports one row per declared range with the floor to move to, opens a tracking issue, rewrites it as findings change, and closes it once every range is clear.
There was a problem hiding this comment.
Pull request overview
Adds a reusable GitHub Actions workflow (peer-advisory-drift) that audits declared dependency ranges (e.g. peerDependencies) against GitHub’s advisory database, and maintains a self-healing tracking issue when drift is detected.
Changes:
- Introduces a reusable workflow that queries GitHub’s advisory GraphQL API and reports vulnerable version-range intersections in the job summary.
- Creates/updates/closes a labeled tracking issue based on drift presence, with optional
fail-on-driftbehavior. - Installs
semverin a temporary runner directory to avoid mutating the audited workspace.
Suppressed comments (1)
.github/workflows/peer-advisory-drift.yml:262
drift-countis documented as the number of declared ranges that admit a vulnerable version, but the script currently writesfindings.length(number of advisory matches). This makes the reusable workflow output (and the optional fail message) report the wrong unit.
appendFileSync(process.env.GITHUB_OUTPUT, `drift-count=${findings.length}\n`)
…byte The query stopped at first: 100 with no pagination. That is not a theoretical cap: `next` returns 159 advisories today, so the audit silently dropped a third of them. Measured on a manifest declaring next ^16.0.0 -- 37 matches and a worst severity of "high" before, 39 and "critical" after. Under-reporting is the one failure mode a scheduled detector cannot have, because it reads as clean. The page loop is bounded and THROWS on the bound rather than returning a partial result. The group key also carried a literal NUL byte where a space belonged. It did not change behaviour -- the key stayed unique -- but it made the file binary to grep and truncated it under awk, which is a trap for whoever edits this next.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
.github/workflows/peer-advisory-drift.yml:284
drift-countis documented as the number of declared ranges that admit a vulnerable version, but the script writesdrift-count=${findings.length}(advisory matches). This makes the output misleading (e.g., “39” instead of “3”) and can confuse callers consuming the reusable workflow output.
appendFileSync(process.env.GITHUB_STEP_SUMMARY, `## Peer advisory drift\n\n${report}\n`)
appendFileSync(process.env.GITHUB_OUTPUT, `drift-count=${findings.length}\n`)
appendFileSync(process.env.GITHUB_OUTPUT, `skipped-count=${skipped.length}\n`)
.github/workflows/peer-advisory-drift.yml:190
- The script can fetch the same package’s vulnerability list multiple times if a package appears in more than one audited field (the comment above explicitly calls this out). This increases runtime and risks hitting GitHub GraphQL rate limits unnecessarily; cache
vulnerabilitiesFor(name)per package name and reuse it across declarations.
for (const vuln of await vulnerabilitiesFor(name)) {
…package drift-count is documented as the number of declared ranges that admit a vulnerable version, but it was emitting the advisory total -- 39 where the maintainer has one line to edit. It now emits the range count, and the total moves to a new advisory-count output. The advisory list is also cached per package. A package declared in more than one audited field was fetched once per declaration; measured with next in two fields, 4 round trips become 2. Cached only after the last page, so a partial result is never served as the whole answer.
|
Picking up the two suppressed comments from the last review — both were right, both fixed in 7a7bd95.
|
| GraphQL round trips | |
|---|---|
next in one field |
2 |
next in two fields, before |
4 |
next in two fields, after |
2 |
Two rather than one because next spans two pages — the pagination fix from the sibling thread.
The cache is populated only after the final page, never mid-loop. Storing a partial page set would hand every later caller a short list as though it were complete, which is the same silent-under-report failure the pagination fix exists to remove.
Verified after both changes
nest-queue still reports drift-count=0 advisory-count=0, and the next case reports 1 / 39 with ^16.2.11 as the floor.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (4)
.github/workflows/peer-advisory-drift.yml:333
- The tracking issue is closed/skipped when
DRIFT_COUNT=0, even ifskipped-count>0(unaudited ranges). If the issue is meant to be the primary signal, includeSKIPPED_COUNTin the “no drift” condition so unaudited declarations still open/update the issue.
if [ "$DRIFT_COUNT" -eq 0 ]; then
.github/workflows/peer-advisory-drift.yml:126
SEVERITY_THRESHOLD.toUpperCase()will throw if the caller passes an empty string (or omits the env var), and an emptyDEPENDENCY_TYPESwill silently result infields=[](no declarations audited). Add defensive defaults/validation so a misconfigured caller fails loudly instead of appearing “clean”.
const RANK = { LOW: 0, MODERATE: 1, HIGH: 2, CRITICAL: 3 }
const threshold = RANK[process.env.SEVERITY_THRESHOLD.toUpperCase()] ?? 0
const manifestPath = process.env.MANIFEST_PATH
const fields = process.env.DEPENDENCY_TYPES.split(',').map((f) => f.trim()).filter(Boolean)
.github/workflows/peer-advisory-drift.yml:329
gh label create … 2>/dev/null || truehides real failures (e.g. missing permissions / API errors), which makes scheduled-run breakages harder to diagnose. Prefer a create-or-edit flow that keeps unexpected errors visible.
gh label create "$LABEL" --color B60205 \
--description "A declared range admits a version with a published advisory" \
2>/dev/null || true
.github/workflows/peer-advisory-drift.yml:320
- The audit script writes
skipped-count, but the issue logic only considersDRIFT_COUNT. If a repo only has unaudited (non-semver) ranges, this workflow will close/avoid the tracking issue even though the run explicitly found uncovered declarations. ExportSKIPPED_COUNTso the issue logic can keep the signal visible.
This issue also appears on line 333 of the same file.
env:
GH_TOKEN: ${{ github.token }}
LABEL: ${{ inputs.issue-label }}
DRIFT_COUNT: ${{ steps.audit.outputs.drift-count }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
Dependabot audits what is installed. Nothing audits what a package declares it supports — and that is where the rot happens.
A library pinned to a current
@nestjs/commonin its own devDependencies is invisible to Dependabot while itspeerDependenciesrange still admits a version with a published advisory. The consumer resolving that range gets to pick the low end.The failure mode is that the declared range never changes and becomes wrong anyway. The advisory database is what moves. A range audited clean today starts admitting a vulnerable version the day a new advisory is published — no commit, no PR, no alert. That is why this is scheduled and not a release-checklist item: the degradation happens between releases, when nobody is reading the file.
This is not hypothetical. It is exactly how
nest-notificationwent from "the only repo with peer ranges verified against the advisory database" to admitting a vulnerable@nestjs/common, with nobody touching the file.What it reports
One row per declared range, because that is the unit of repair — one edit fixes every advisory it admits. Run against
nest-authtoday:peerDependencies@nestjs/common^11.0.0^11.0.16peerDependencies@nestjs/core^11.0.0^11.1.18peerDependenciesnext^16.0.0^16.2.11Grouping matters here: ungrouped,
nextalone produced 37 rows that all resolve to the same single edit, burying the one number the reader needs. Full advisory lists stay in collapsed<details>."Raise floor to" is the highest first-patched version among the matches, since that is what clears everything below it — not the first one in the list.
Behaviour
workspace:*,file:../x) are reported as unaudited rather than skipped silently — a silent skip is how a dependency quietly stops being covered.fail-on-driftis off by default. The issue is the signal; a permanently red scheduled run on the default branch just trains people to ignore it.Verification
Not desk-checked — actually executed against real manifests:
nest-queue(floors already raised)nest-auth(^11.0.0on both)^11.0.16/^11.1.18/^16.2.11workspace:*andfile:../xTwo defects were found and fixed by running it rather than reading it:
NODE_PATHdoes not apply to ESM resolution. The script importedsemverand died withERR_MODULE_NOT_FOUNDon the first run. Fixed by writing the script beside the installednode_modulessoimportresolves by walking up from the file.>= 1.2.0, < 1.4.1— comma-separated. semver wants a space-separated conjunction; parsed as-is the comma reads as a union and matches far more than it should.Supply chain
semveris installed into a throwaway$RUNNER_TEMPdirectory with--ignore-scripts, never into the caller's workspace — this job must not perturb the tree it audits, and a pnpm workspace would reject a stray npm install at its root anyway. Only GitHub-owned actions are used, so the org's third-party allow-list is unaffected.semver rather than a hand-rolled comparison because range intersection is the kind of logic that looks trivial and is not: prerelease tails,
||unions, and the>=x, <yform all have to be handled the way the package managers handle them.v1tag movedCallers pin
bymaxone/.github/....yml@v1. After merge,v1has to be moved to include this commit or every caller fails with "workflow not found". Say the word and I will move it.Ready for your review — not merging it.