M13: covering what was already claimed - #17
Merged
Merged
Conversation
M12 made the suites continuous. This is about what they were continuously running, which was less than the milestones above claim. The restart policy firing at all; the backoff doubling through 200ms, 400ms, 800ms; on-abnormal treating a signal death as abnormal; a watchdog miss taking the restart policy; %N and %u against a running machine; oxctl reload; and output = "null" actually discarding. Each is asserted by a milestone that says "Verified:", and each was — once, by hand, on somebody's machine. flaky.toml is the whole story in one file. It has existed since M2, its comment describes the doubling exactly, and it was never reachable from `default`, so nothing ever started it. A unit file that documents a test nobody runs is worse than no unit file, because it reads as coverage. The watchdog one is the most interesting. ARCHITECTURE argues at length that a watchdog miss must take the restart policy where an oxctl stop overrides it, and calls that asymmetry the point of a watchdog. The only unit that ever missed a watchdog declared restart = "no", so only the half that changes nothing had run. Nothing was broken. All seven worked the first time they were asked to, which is worth saying plainly: this bought guarding, not fixing. 38 checks per architecture, 40 against the distribution image. Deferred: the backoff reset after a sustained Active period, which M2 also claims. Telling "reset" from "never incremented" needs an assertion about a counter's history, and substring matching over a log cannot make one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M12 made the suites continuous. This is about what they were
continuously running, which turned out to be less than the milestones
above claim.
Seven behaviours had no coverage at all. Each is asserted by a milestone
that says "Verified:", and each was — once, by hand, on the machine of
whoever wrote it.
on-abnormaltreating a signal death as abnormal.%Nand%u, against a running machine.oxctl reload.output = "null"actually discarding.flaky.tomlis the whole story in one fileIt has existed since M2. Its comment describes the doubling exactly:
And it was never reachable from
default, so nothing ever started it. Aunit file that documents a test nobody runs is worse than no unit file,
because it reads as coverage.
The watchdog one is the most interesting
ARCHITECTURE argues at length that a watchdog miss must take the restart
policy where an
oxctl stopoverrides it, and calls that asymmetry "thepoint of a watchdog". The only unit that ever missed a watchdog declared
restart = "no"— so only the half that changes nothing had run.Nothing was broken
All seven worked the first time they were asked to. That is the honest
result and worth saying plainly: this milestone bought guarding, not
fixing. The code was right; nothing was watching it.
Verified
38 checks per architecture, 40 against the real Alpine userspace, 10 in
the container.
Deferred: the backoff reset after a sustained
Activeperiod, which M2also claims. Telling "reset" from "never incremented" needs an assertion
about a counter's history, and substring matching over a log cannot make
one — it needs a different kind of test than this suite is.