Repository navigation
Slice 128: the attribution and sign-off checks say what they check - #106
Merged
Merged
Conversation
.github/workflows/gate.yml carries a step named "DCO sign-off present on
every commit" whose body is:
git log --format=%B BASE..HEAD | grep -q '^Signed-off-by: '
It concatenates every message in the range and greps for one sign-off
anywhere, so a range of five commits where one is signed passes. The step is
named "on every commit"; it checks "at least one in the range".
AC1 builds a range with one signed and one unsigned commit and asserts the
current shape accepts it, which it does. AC2 asserts the workflow no longer
carries that shape, which it still does, in both legs.
test/dco_test.exs does check every non-merge commit over the whole history on
every leg, so the practical exposure is small. The defect is that a name is a
claim and this one is false.
Signed-off-by: Ayla Croft <aylacroft@proton.me>
…y what they check) Opened after seven pull request descriptions (#99 to #105) carried an assistant attribution and nothing objected. A correction first, because it shaped the scope. I had reported that the merge commits on main lacked DCO sign-off and called it a rule violation. That was wrong. Slice 120 decided the opposite deliberately and test/dco_test.exs records the reasoning: a sign-off certifies the right to submit work, a forge's merge commit introduces none, it is authored by whoever pressed the button, and standard DCO tooling skips merges for that reason. Both enforcers use --no-merges and agree. This slice does not change that. What it does change is three checks that did not do what they were named. 1. The workflow step named "DCO sign-off present on every commit" concatenated every message in the range and grepped for ONE sign-off anywhere, so five commits with one signed passed it. It now asks per commit, prints each failure, and is renamed to "DCO sign-off on every commit that carries work", which is what it does. Both legs. Exposure was small because dco_test.exs checks the whole history on every leg; the defect was in the telling, and a name is a claim. 2. plan_check rule 8 consulted unsigned_merge_exempt, a list holding one merge sha, from inside a --no-merges loop where a merge sha can never appear. The exemption had never matched and never could, while telling a reader that merges were checked and one was excused. Removed rather than made live: making it live would have reversed slice 120's decision by accident. 3. Nothing looked at a pull request description. The hook covers commit messages and plan_check covers history; both look at the tree, and a description is not in it. Added as a step in both legs, with the body passed through env rather than interpolated into the script, because a description is attacker-controlled text and ${{ }} in a run: is a script injection. Rule 8 also now checks attribution on every commit including merges, because gh pr merge --body writes a merge body and nothing was reading it. Demonstrated against a planted merge: the new rule names it, the old rule recovered from ad8c22f reports nothing at all. Found by the gate while closing this and fixed here: CHANGELOG.md had no entry for slice/127, caught by its own census, and two unused imports left by 127's rename. Signed-off-by: Ayla Croft <aylacroft@proton.me>
The new pull request description check failed this very pull request: the description quoted the literal trailer in order to explain the defect, and the check cannot tell a description of the pattern from an instance of it. That is the check working. The description was rewritten to name the trailer rather than contain it, rather than the check being taught an exception, because an exception for 'text that is only describing the pattern' is a way around it. Signed-off-by: Ayla Croft <aylacroft@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opened after seven pull request descriptions (#99 to #105) carried the assistant's generation trailer and nothing objected.
A correction first, because it set the scope
I had reported that the merge commits on
mainlacked DCO sign-off and called it a rule violation. That was wrong. Slice 120 decided the opposite deliberately, andtest/dco_test.exsrecords the reasoning: a sign-off certifies the right to submit work, a forge's merge commit introduces none, it is authored by whoever pressed the button, and standard DCO tooling skips merges for exactly that reason. Both enforcers use--no-mergesand agree with each other.This slice does not change that. It changes three checks that did not do what they were named.
1. A name that was a claim, and it was false
Concatenates every message in the range and greps for one sign-off anywhere. Five commits with one signed passes a step called "on every commit".
Exposure was small, because
test/dco_test.exschecks every non-merge commit over the whole history on every leg. The defect was in the telling, which is the part this project says it cares about: a name is a claim.Now asks per commit, prints each failure, renamed to "DCO sign-off on every commit that carries work". Both legs, with the
--no-mergeschoice justified in a comment againsttest/dco_test.exsrather than left implicit.2. An exemption that could never be reached
plan_check.shrule 8 consultedunsigned_merge_exempt— a list holding one merge sha — from inside a--no-mergesloop, where a merge sha can never appear. It had never matched and never could, while telling every reader of that file that merge commits were checked and one was excused.Removed rather than made live. Making it live would have reversed slice 120's decision by accident, which is a thing a cleanup should never do.
3. Nothing looked where the rule was actually broken
The hook covers commit messages, plan_check covers history. Both look at the tree. A pull request description is not in the tree, is the first thing a reader sees, and is where the rule broke seven times in a day.
Added in both legs. The body goes through
env, not into the script —${{ }}interpolated into arun:is a script injection, and a description is attacker-controlled text.Rule 8 also now checks attribution on every commit including merges, because
gh pr merge --bodywrites a merge body and nothing was reading it.Demonstrated against planted violations
FAIL commit c6881a5...: no Signed-off-by lineFAIL commit 78136e9...: attribution: 3:Co-Authored-By: ...And the same planted merge against the old rule, recovered from
ad8c22f:The old rule was blind to it.
Found by the gate while closing this, and fixed here
CHANGELOG.mdhad no entry forslice/127, caught by its own census: "the file states it is derived from the tags; add the entry rather than the exception." I had merged and tagged 127 without one.For the reviewer
The PR-description check cannot fail a pull request opened before it existed, so #99 to #105 were corrected by hand and recorded in NOTES.md rather than caught. Nothing under
lib/changed.Note: this description deliberately does not quote the literal strings the new check matches. The check is not clever enough to tell a description of the pattern from an instance of it, and making it clever enough would be a way to get around it.