Skip to content

feat: expose the analysis outcome and make reporting configurable - #3

Merged
javier-godoy merged 6 commits into
masterfrom
feature/wip-action-required
Sep 15, 2026
Merged

javier-godoy merged 6 commits into
masterfrom
feature/wip-action-required

Conversation

@javier-godoy

@javier-godoy javier-godoy commented Sep 9, 2026

Copy link
Copy Markdown
Member

Refs FlowingCode/AddonsInternal#99

WIP commits are a valid, expected state of an unfinished branch, but they were reported
exactly like a genuinely invalid commit message, so the same red ❌ meant two different
things. Rather than settle that here, this branch exposes what the analysis found and lets
the caller decide how to report it (see FlowingCode/GithubActions#7).

Changes

refactor: carry the sha of every commit alongside its message The commits of a pull request were mapped to their message and everything else was discarded, and the commits of a push carry their identifier as id rather than sha. Both are now mapped to a message and a sha, so a commit can be identified and not only read. It also logs which branch was taken — the push payload, the pull request, or neither — so that a run which analysed nothing says why.
feat: expose the result of the analysis through outputs The action reported its findings only through the log and its own outcome, so a caller could not tell what was found. It now sets results, describing every commit as JSON: its sha, its header, its level, and the reason when it is invalid.
feat: report invalid commit messages as annotations core.error instead of core.info for an invalid message, so the offending commit and the reason are visible on the pull request itself rather than only in the job log. The abbreviated digest precedes the header, which does not identify the commit on its own when the same message appears more than once.
feat: add enforce input for callers that report the outcome themselves An invalid commit message and a WIP commit both fail the check with the same red ❌, though only one of them is a defect, and a step cannot block a merge without looking like a failure. enforce: false makes the action a pure analyzer — it produces results, exits successfully and writes no annotations — so that a caller can report WIP commits as a check run of their own, which can conclude action_required and be both.
fix: classify scoped and breaking WIP commits as WIP isWIP matched the literal WIP: prefix, but validateCommitMessage accepts WIP(scope): and WIP!: as well, so those commits were reported as valid. The type is now parsed off the header. The bug predates this branch, where it only skewed the action's own outcome; results turns it into a contract that callers gate merges on.
build: rebuild the dist bundle Rebuilt from the sources above.

Interface

Name Type Description
results output The result for every commit, as a JSON array of {sha, header, level, reason}, where level is valid, wip or invalid.
enforce input Whether the action reports the outcome and fails on what it found. Default true.
invalid message WIP commit annotations
enforce: true (default) ❌ fails ❌ fails one per invalid commit, none for WIP
enforce: false analysed only analysed only none

A value other than true or false fails the action rather than being read as the default,
so a misspelling is not silently a policy. The comparison is exact, so TRUE and True are
errors too, and so is an empty value: action.yml supplies the default whenever the action is
called as one, so a present-but-empty enforce was written by the caller — an unset workflow
input interpolated into it, say — and taking it for the default would be the same silent
policy. The input is absent only when the bundle runs outside Actions, which is the one case
the built-in default covers. enforce is one boolean rather than a mode enumeration because
there are only two behaviours left to choose between: whether the action reports, or the
caller does.

results is the only output: has-errors and has-wip are derivable from it, and the
Semantic Versioning level is already exported as the SEMVER_LEVEL environment variable. It
is set before the check fails, so it can be read from an if: always() step, and on every
path — including the ones where the commits cannot be retrieved or enforce is invalid, where
it is an empty array. Those paths still fail even with enforce: false, because not being able
to analyse anything is a failure of the action itself rather than a finding about the commits.

The default changes nothing. With enforce: true a WIP commit still fails, exactly as on
master, so merging this branch moves no existing caller. FlowingCode/AddonsInternal#99 asks for
a pull request that is blocked and not shown as a failure, and a step cannot be both: its
conclusion comes from its exit code. Only a check run can, by concluding action_required, and
only a caller can create one. So the answer to #99 is enforce: false plus that check run —
which is what FlowingCode/GithubActions#7 does — and not a default that would leave a caller
without one merging unfinished code behind a green tick.

README.md gains a Work in Progress section documenting both. It states that the option
exists and why, and leaves the caller-side workflow to the action that already implements it
rather than carrying a snippet of its own.

Verification

npm test passes (3/3), with the WIP case extended to cover WIP(scope): and WIP!:. The
built bundle was also run against synthetic payloads:

payload enforce outcome
one valid, one WIP unset (default) ::error::🚧 Work-in-Progress (WIP) commits found. They must be squashed before rebasing or merging., exit 1
one valid, one WIP false no annotation, exit 0
valid + invalid + WIP true ::error::🚩 9f8e7d6 broken message : The commit header is not formatted according to Conventional Commits., then ::error::🚫 According to the…, exit 1, SEMVER_LEVEL=2, and the sha of all three commits in results
valid + invalid + WIP false the same line as plain log output, no ::error::, exit 0
one valid, one WIP yes ::error::🚫 The enforce input must be true or false, not "yes"., exit 1
one valid, one WIP empty string ::error::🚫 The enforce input must be true or false, not ""., exit 1, with results set to [] and SEMVER_LEVEL to 0

and the three paths through extractCommits now log which one was taken:

event log
pull_request ℹ️ Read 6 commit(s) from the pull request.
push ℹ️ Read 1 commit(s) from the push payload.
issue_comment ℹ️ No commits to check: the "issue_comment" event has neither a push payload nor a pull request.

The last two were indistinguishable before: a push carrying no commits and an event the action
does not read both produced an empty results and a green check, with nothing in the log to
tell them apart.

End to end in javier-godoy/test-repo

Every pull request below was re-run against this branch. Two of them call the action directly,
so its own reporting is exercised; the rest run
FlowingCode/GithubActions/.github/workflows/check-commits.yml@feature/wip-action-required
the caller in FlowingCode/GithubActions#7, which calls this branch with enforce: false and
renders every conclusion itself. check-commits / check-commits is the job's own check run,
which GitHub creates and which carries no message; semver-alignment and wip-commits are
created by the caller and carry the verdict.

The annotations carry the abbreviated digest ahead of the header: the caller reads sha
from results, so the commit that must be amended is named rather than only described.

results being ordered and per-commit is what lets the caller go further than this action
could. It tells a branch that may be squashed at merge time — exactly one non-WIP commit
followed by WIP commits — apart from one the author has to consolidate, purely from the order
and the levels, and the action knows nothing about that policy. Compare
javier-godoy/test-repo#11 and javier-godoy/test-repo#25 with javier-godoy/test-repo#24 below.

The action reporting for itself (default enforce)

javier-godoy/test-repo#26 — invalid commit messages — ❌
Check Conclusion Title
check-commits ❌ failure
[failure] 🚫 According to the Flowing Code Commit Message Guidelines, some of the commit messages are not valid.
[failure] 🚩 5a71654 note what this branch is for : The commit header is not formatted according to Conventional Commits.
[failure] 🚩 23effa8 call the action directly with the default report : The commit header is not formatted according to Conventional Commits.

Both annotations are the action's own, and each names its commit.

javier-godoy/test-repo#27 — a WIP commit — ❌
Check Conclusion Title
check-commits ❌ failure
[failure] 🚧 Work-in-Progress (WIP) commits found. They must be squashed before rebasing or merging.

The default is unchanged: a WIP commit still fails, and no annotation is written for it.

A caller that reports the outcome (enforce: false)

javier-godoy/test-repo#13 — valid commit message — ✅
Check Conclusion Title
check-commits / check-commits ✅ success
semver-alignment ✅ success Consistent: PATCH change
wip-commits ✅ success No Work-in-Progress (WIP) commits

No annotations.

javier-godoy/test-repo#12 — invalid commit message — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ⚪ neutral The semantic versioning level is unknown
wip-commits ✅ success No Work-in-Progress (WIP) commits
[failure] 1 commit message does not follow the guidelines
[failure] 3d91269 broken commit message without a type : The commit header is not formatted according to Conventional Commits.

One of the two commit messages parsed, but the level is unknown rather than the NONE of the
one that did: only a valid commit raises the level, so the commit that could not be parsed
might have described a breaking change. The version is not verified against a level derived
from a subset of the commits.

javier-godoy/test-repo#11 — one WIP commit, squashable at merge time — ⚠️
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ✅ success Consistent: NONE change
wip-commits ⚠️ action_required 🚧 Must squash WIP commit
[failure] 1 Work-in-Progress (WIP) commit must be squashed on merge
[warning] 🚧 A Work-in-Progress (WIP) commit was found. The branch can be squashed on merge, so the author does not need to consolidate it.

The action itself did not fail for the WIP commit: the warning is the action's, the failure is
the caller's, and wip-commits is what says which problem it was. The history here is one
non-WIP commit followed by a single WIP one, so the caller asks for a Squash and Merge rather
than sending the author back to rebase.

javier-godoy/test-repo#25 — two WIP commits after one non-WIP — ⚠️
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ✅ success Consistent: NONE change
wip-commits ⚠️ action_required 🚧 Must squash WIP commits
[failure] 2 Work-in-Progress (WIP) commits must be squashed on merge
[warning] 🚧 2 Work-in-Progress (WIP) commits were found. The branch can be squashed on merge, so the author does not need to consolidate it.

Still squashable with more than one WIP commit: the shape that matters is one non-WIP commit
first, not how many WIP commits follow it.

javier-godoy/test-repo#24 — every commit is a WIP commit — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ✅ success Consistent: NONE change
wip-commits ❌ failure 🚧 Must consolidate WIP commit
[failure] 1 Work-in-Progress (WIP) commit must be consolidated by the author
[warning] 🚧 A Work-in-Progress (WIP) commit was found. It must be consolidated by the author before merging.

There is no non-WIP commit for the WIP one to be squashed into, so this is a failure rather
than action_required — contrast webiny#11 and webiny#25, which differ from this one only in the order and
the levels the caller reads out of results.

javier-godoy/test-repo#14 — breaking change without a version bump — ❌ on two checks
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ❌ failure 🚫 MAJOR version required
wip-commits ❌ failure 🚧 Must consolidate WIP commit
[failure] 1 Work-in-Progress (WIP) commit must be consolidated by the author — 🚫 MAJOR version required
[warning] 🚧 A Work-in-Progress (WIP) commit was found. It must be consolidated by the author before merging.

Two independent problems, each on its own check run, and the job's failure message names both.

javier-godoy/test-repo#17 — every commit message invalid — ❌, level unknown
Check Conclusion Title
check-commits / check-commits ❌ failure
semver-alignment ⚪ neutral The semantic versioning level is unknown
wip-commits ✅ success No Work-in-Progress (WIP) commits
[failure] 2 commit messages do not follow the guidelines
[failure] 3b62ec4 note what this branch is for : The commit header is not formatted according to Conventional Commits.
[failure] c577a04 point check-commits at the feature branch : The commit header is not formatted according to Conventional Commits.

The action exits 0 under enforce: false and the caller fails the job. No message could be
parsed, so the level is unknown rather than NONE and the version is not verified against it.

From a fork

javier-godoy/test-repo#19, javier-godoy/test-repo#20 and javier-godoy/test-repo#21 are opened
from FlowingCode/test-repo2. They confirm the one limitation of leaving the reporting to the
caller: a pull_request from a fork gets a read-only GITHUB_TOKEN whatever the
permissions: block asks for, so checks.create answers 403 there and only the job's own
check run exists. The annotations are workflow commands emitted by the caller and still appear,
and the job still fails, so the outcome is reported either way — only the checks that carry it
separately are missing.

javier-godoy/test-repo#19 — fork, invalid commit message — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
[failure] 1 commit message does not follow the guidelines
[failure] b3bf7fb fork pr with an invalid commit message : The commit header is not formatted according to Conventional Commits.
javier-godoy/test-repo#20 — fork, two WIP commits after one non-WIP — ❌
Check Conclusion Title
check-commits / check-commits ❌ failure
[failure] 2 Work-in-Progress (WIP) commits must be squashed on merge
[warning] 🚧 2 Work-in-Progress (WIP) commits were found. The branch can be squashed on merge, so the author does not need to consolidate it.

The same shape as webiny#25, and the caller reaches the same verdict — but wip-commits cannot be
created, so nothing carries the action_required conclusion and the pull request shows only a
plain failure.

javier-godoy/test-repo#21 — fork, valid commit messages — ✅
Check Conclusion Title
check-commits / check-commits ✅ success

No annotations.

Not covered end to end: no test pull request carries a WIP(scope): or WIP!: commit, so the
classification fix above rests on the unit tests rather than on a run.

🤖 Generated with Claude Code

@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 9 times, most recently from 890ad04 to 0f2f7a0 Compare September 10, 2026 12:33
@javier-godoy javier-godoy changed the title feat: add fail-on-wip input and has-wip output feat: expose the analysis outcome and make reporting configurable Sep 10, 2026
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch from 0f2f7a0 to 9cc1e86 Compare September 10, 2026 17:42
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 2 times, most recently from b59c6d6 to df3ebd4 Compare September 10, 2026 17:58
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 13 times, most recently from 7ac4231 to b7ce6ac Compare September 15, 2026 12:41
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch 2 times, most recently from 17b2193 to 6b04ae7 Compare September 15, 2026 13:50
The commits of a pull request were mapped to their message and everything else
was discarded, and the commits of a push carry their identifier as id rather
than sha. Both are now mapped to a message and a sha, so that a caller can
identify a commit and not only read it.
The action reported its findings only through the log and its own outcome,
so a caller could not tell what was found. It now also sets results, which
describes every commit as JSON: its sha, its header, whether it is valid, WIP
or invalid, and the reason when it is invalid.

results is set before the check fails, so that it can be read from a step
that runs on failure, and on the path where the commits cannot be retrieved,
where it is empty. It is the only output: has-errors and has-wip are
derivable from results, and the Semantic Versioning level is already
exported as the SEMVER_LEVEL environment variable.
core.error creates an annotation on the check run, so the offending commit
and the reason why it is invalid are visible on the pull request itself,
instead of only in the job log.

The abbreviated digest precedes the header, which does not identify the commit
on its own when the same message appears more than once in a pull request, and
leaves nothing to look the commit up by.
The action reported the outcome the only way it knew, and a caller could not ask
for anything else: an invalid commit message and a Work-in-Progress commit both
failed the check with the same red X, though only one of them is a defect.

A WIP commit is now listed in the results output as a level of its own, so that a
caller can report it as a check of its own and the red X is left to mean what it
used to mean alone. The action itself still fails on a WIP commit when it reports,
because a step cannot both block the merge and avoid the red X: telling the two
apart is the caller's to do. An invalid commit message still fails.

enforce says whether the action reports at all. With false it becomes a pure
analyzer: it produces the result for every commit, exits successfully and writes
no annotations, so that the caller can render the conclusions, the wording and
the annotations itself, without those being compiled into this action. A value
other than true or false fails the action rather than being read as the default,
so that a misspelling is not silently a policy. The comparison is exact, so TRUE
is an error too.
isWIP matched the literal "WIP:" prefix, but validateCommitMessage accepts
WIP(scope): and WIP!: as well, so those commits were reported as valid. The
results output published by this branch turns that into a contract callers
gate merges on, so the type is parsed off the header rather than prefix-matched.
Rebuilt from the current sources.
@javier-godoy
javier-godoy force-pushed the feature/wip-action-required branch from 6b04ae7 to 357dfe7 Compare September 15, 2026 14:01
@javier-godoy
javier-godoy marked this pull request as ready for review September 15, 2026 15:26
@javier-godoy
javier-godoy merged commit 3e790d0 into master Sep 15, 2026
@javier-godoy
javier-godoy deleted the feature/wip-action-required branch September 15, 2026 16:08
@github-project-automation github-project-automation Bot moved this from To Do to Pending release in Flowing Code Addons Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Pending release

Development

Successfully merging this pull request may close these issues.

1 participant