feat: expose the analysis outcome and make reporting configurable - #3
Merged
Merged
Conversation
This was referenced Sep 9, 2026
javier-godoy
force-pushed
the
feature/wip-action-required
branch
9 times, most recently
from
September 10, 2026 12:33
890ad04 to
0f2f7a0
Compare
javier-godoy
force-pushed
the
feature/wip-action-required
branch
from
September 10, 2026 17:42
0f2f7a0 to
9cc1e86
Compare
javier-godoy
force-pushed
the
feature/wip-action-required
branch
2 times, most recently
from
September 10, 2026 17:58
b59c6d6 to
df3ebd4
Compare
javier-godoy
force-pushed
the
feature/wip-action-required
branch
13 times, most recently
from
September 15, 2026 12:41
7ac4231 to
b7ce6ac
Compare
javier-godoy
force-pushed
the
feature/wip-action-required
branch
2 times, most recently
from
September 15, 2026 13:50
17b2193 to
6b04ae7
Compare
The commits of a pull request were mapped to their message and everything else was discarded, and the commits of a push carry their identifier as id rather than sha. Both are now mapped to a message and a sha, so that a caller can identify a commit and not only read it.
The action reported its findings only through the log and its own outcome, so a caller could not tell what was found. It now also sets results, which describes every commit as JSON: its sha, its header, whether it is valid, WIP or invalid, and the reason when it is invalid. results is set before the check fails, so that it can be read from a step that runs on failure, and on the path where the commits cannot be retrieved, where it is empty. It is the only output: has-errors and has-wip are derivable from results, and the Semantic Versioning level is already exported as the SEMVER_LEVEL environment variable.
core.error creates an annotation on the check run, so the offending commit and the reason why it is invalid are visible on the pull request itself, instead of only in the job log. The abbreviated digest precedes the header, which does not identify the commit on its own when the same message appears more than once in a pull request, and leaves nothing to look the commit up by.
The action reported the outcome the only way it knew, and a caller could not ask for anything else: an invalid commit message and a Work-in-Progress commit both failed the check with the same red X, though only one of them is a defect. A WIP commit is now listed in the results output as a level of its own, so that a caller can report it as a check of its own and the red X is left to mean what it used to mean alone. The action itself still fails on a WIP commit when it reports, because a step cannot both block the merge and avoid the red X: telling the two apart is the caller's to do. An invalid commit message still fails. enforce says whether the action reports at all. With false it becomes a pure analyzer: it produces the result for every commit, exits successfully and writes no annotations, so that the caller can render the conclusions, the wording and the annotations itself, without those being compiled into this action. A value other than true or false fails the action rather than being read as the default, so that a misspelling is not silently a policy. The comparison is exact, so TRUE is an error too.
isWIP matched the literal "WIP:" prefix, but validateCommitMessage accepts WIP(scope): and WIP!: as well, so those commits were reported as valid. The results output published by this branch turns that into a contract callers gate merges on, so the type is parsed off the header rather than prefix-matched.
Rebuilt from the current sources.
javier-godoy
force-pushed
the
feature/wip-action-required
branch
from
September 15, 2026 14:01
6b04ae7 to
357dfe7
Compare
javier-godoy
marked this pull request as ready for review
September 15, 2026 15:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs FlowingCode/AddonsInternal#99
WIPcommits are a valid, expected state of an unfinished branch, but they were reportedexactly like a genuinely invalid commit message, so the same red ❌ meant two different
things. Rather than settle that here, this branch exposes what the analysis found and lets
the caller decide how to report it (see FlowingCode/GithubActions#7).
Changes
refactor: carry the sha of every commit alongside its messageidrather thansha. Both are now mapped to a message and a sha, so a commit can be identified and not only read. It also logs which branch was taken — the push payload, the pull request, or neither — so that a run which analysed nothing says why.feat: expose the result of the analysis through outputsresults, describing every commit as JSON: itssha, its header, its level, and the reason when it is invalid.feat: report invalid commit messages as annotationscore.errorinstead ofcore.infofor an invalid message, so the offending commit and the reason are visible on the pull request itself rather than only in the job log. The abbreviated digest precedes the header, which does not identify the commit on its own when the same message appears more than once.feat: add enforce input for callers that report the outcome themselvesenforce: falsemakes the action a pure analyzer — it producesresults, exits successfully and writes no annotations — so that a caller can report WIP commits as a check run of their own, which can concludeaction_requiredand be both.fix: classify scoped and breaking WIP commits as WIPisWIPmatched the literalWIP:prefix, butvalidateCommitMessageacceptsWIP(scope):andWIP!:as well, so those commits were reported asvalid. The type is now parsed off the header. The bug predates this branch, where it only skewed the action's own outcome;resultsturns it into a contract that callers gate merges on.build: rebuild the dist bundleInterface
results{sha, header, level, reason}, wherelevelisvalid,wiporinvalid.enforcetrue.enforce: true(default)enforce: falseA value other than
trueorfalsefails the action rather than being read as the default,so a misspelling is not silently a policy. The comparison is exact, so
TRUEandTrueareerrors too, and so is an empty value:
action.ymlsupplies the default whenever the action iscalled as one, so a present-but-empty
enforcewas written by the caller — an unset workflowinput interpolated into it, say — and taking it for the default would be the same silent
policy. The input is absent only when the bundle runs outside Actions, which is the one case
the built-in default covers.
enforceis one boolean rather than a mode enumeration becausethere are only two behaviours left to choose between: whether the action reports, or the
caller does.
resultsis the only output:has-errorsandhas-wipare derivable from it, and theSemantic Versioning level is already exported as the
SEMVER_LEVELenvironment variable. Itis set before the check fails, so it can be read from an
if: always()step, and on everypath — including the ones where the commits cannot be retrieved or
enforceis invalid, whereit is an empty array. Those paths still fail even with
enforce: false, because not being ableto analyse anything is a failure of the action itself rather than a finding about the commits.
The default changes nothing. With
enforce: truea WIP commit still fails, exactly as onmaster, so merging this branch moves no existing caller. FlowingCode/AddonsInternal#99 asks for
a pull request that is blocked and not shown as a failure, and a step cannot be both: its
conclusion comes from its exit code. Only a check run can, by concluding
action_required, andonly a caller can create one. So the answer to #99 is
enforce: falseplus that check run —which is what FlowingCode/GithubActions#7 does — and not a default that would leave a caller
without one merging unfinished code behind a green tick.
README.mdgains a Work in Progress section documenting both. It states that the optionexists and why, and leaves the caller-side workflow to the action that already implements it
rather than carrying a snippet of its own.
Verification
npm testpasses (3/3), with the WIP case extended to coverWIP(scope):andWIP!:. Thebuilt bundle was also run against synthetic payloads:
enforce::error::🚧 Work-in-Progress (WIP) commits found. They must be squashed before rebasing or merging., exit 1falsetrue::error::🚩 9f8e7d6 broken message : The commit header is not formatted according to Conventional Commits., then::error::🚫 According to the…, exit 1,SEMVER_LEVEL=2, and the sha of all three commits inresultsfalse::error::, exit 0yes::error::🚫 The enforce input must be true or false, not "yes"., exit 1::error::🚫 The enforce input must be true or false, not ""., exit 1, withresultsset to[]andSEMVER_LEVELto0and the three paths through
extractCommitsnow log which one was taken:pull_requestℹ️ Read 6 commit(s) from the pull request.pushℹ️ Read 1 commit(s) from the push payload.issue_commentℹ️ No commits to check: the "issue_comment" event has neither a push payload nor a pull request.The last two were indistinguishable before: a push carrying no commits and an event the action
does not read both produced an empty
resultsand a green check, with nothing in the log totell them apart.
End to end in javier-godoy/test-repo
Every pull request below was re-run against this branch. Two of them call the action directly,
so its own reporting is exercised; the rest run
FlowingCode/GithubActions/.github/workflows/check-commits.yml@feature/wip-action-required—the caller in FlowingCode/GithubActions#7, which calls this branch with
enforce: falseandrenders every conclusion itself.
check-commits / check-commitsis the job's own check run,which GitHub creates and which carries no message;
semver-alignmentandwip-commitsarecreated by the caller and carry the verdict.
The annotations carry the abbreviated digest ahead of the header: the caller reads
shafrom
results, so the commit that must be amended is named rather than only described.resultsbeing ordered and per-commit is what lets the caller go further than this actioncould. It tells a branch that may be squashed at merge time — exactly one non-WIP commit
followed by WIP commits — apart from one the author has to consolidate, purely from the order
and the levels, and the action knows nothing about that policy. Compare
javier-godoy/test-repo#11 and javier-godoy/test-repo#25 with javier-godoy/test-repo#24 below.
The action reporting for itself (default
enforce)javier-godoy/test-repo#26 — invalid commit messages — ❌
check-commitsBoth annotations are the action's own, and each names its commit.
javier-godoy/test-repo#27 — a WIP commit — ❌
check-commitsThe default is unchanged: a WIP commit still fails, and no annotation is written for it.
A caller that reports the outcome (
enforce: false)javier-godoy/test-repo#13 — valid commit message — ✅
check-commits / check-commitssemver-alignmentwip-commitsNo annotations.
javier-godoy/test-repo#12 — invalid commit message — ❌
check-commits / check-commitssemver-alignmentwip-commitsOne of the two commit messages parsed, but the level is unknown rather than the
NONEof theone that did: only a valid commit raises the level, so the commit that could not be parsed
might have described a breaking change. The version is not verified against a level derived
from a subset of the commits.
javier-godoy/test-repo#11 — one WIP commit, squashable at merge time —⚠️
check-commits / check-commitssemver-alignmentwip-commitsThe action itself did not fail for the WIP commit: the warning is the action's, the failure is
the caller's, and
wip-commitsis what says which problem it was. The history here is onenon-WIP commit followed by a single WIP one, so the caller asks for a Squash and Merge rather
than sending the author back to rebase.
javier-godoy/test-repo#25 — two WIP commits after one non-WIP —⚠️
check-commits / check-commitssemver-alignmentwip-commitsStill squashable with more than one WIP commit: the shape that matters is one non-WIP commit
first, not how many WIP commits follow it.
javier-godoy/test-repo#24 — every commit is a WIP commit — ❌
check-commits / check-commitssemver-alignmentwip-commitsThere is no non-WIP commit for the WIP one to be squashed into, so this is a
failureratherthan
action_required— contrast webiny#11 and webiny#25, which differ from this one only in the order andthe levels the caller reads out of
results.javier-godoy/test-repo#14 — breaking change without a version bump — ❌ on two checks
check-commits / check-commitssemver-alignmentwip-commitsTwo independent problems, each on its own check run, and the job's failure message names both.
javier-godoy/test-repo#17 — every commit message invalid — ❌, level unknown
check-commits / check-commitssemver-alignmentwip-commitsThe action exits 0 under
enforce: falseand the caller fails the job. No message could beparsed, so the level is unknown rather than
NONEand the version is not verified against it.From a fork
javier-godoy/test-repo#19, javier-godoy/test-repo#20 and javier-godoy/test-repo#21 are opened
from FlowingCode/test-repo2. They confirm the one limitation of leaving the reporting to the
caller: a
pull_requestfrom a fork gets a read-onlyGITHUB_TOKENwhatever thepermissions:block asks for, sochecks.createanswers 403 there and only the job's owncheck run exists. The annotations are workflow commands emitted by the caller and still appear,
and the job still fails, so the outcome is reported either way — only the checks that carry it
separately are missing.
javier-godoy/test-repo#19 — fork, invalid commit message — ❌
check-commits / check-commitsjavier-godoy/test-repo#20 — fork, two WIP commits after one non-WIP — ❌
check-commits / check-commitsThe same shape as webiny#25, and the caller reaches the same verdict — but
wip-commitscannot becreated, so nothing carries the
action_requiredconclusion and the pull request shows only aplain failure.
javier-godoy/test-repo#21 — fork, valid commit messages — ✅
check-commits / check-commitsNo annotations.
Not covered end to end: no test pull request carries a
WIP(scope):orWIP!:commit, so theclassification fix above rests on the unit tests rather than on a run.
🤖 Generated with Claude Code