The assertions are std/test

CHANGELOG.md

@@ -2,6 +2,20 @@
 
 ## v0.1.0 (unreleased)
 
+### Changed
+
+- **The assertions are `std/test`.** twill 1.11 ships the assertions the test
+  runner already assumed, and names spool's `is_none_i64` and `equal_opt_i64`
+  as the workaround its `fail(name, why)` replaces: both recorded a failure as
+  `check(name, false)` and threw away the reason. Every suite imports
+  `std/test` as `t` now, the three sites that used those two helpers are a
+  `match` whose `None` or `Some` arm calls `t.fail` with what was actually
+  there, and `tests/harness.tw` keeps one predicate, `before`, and nothing
+  that counts. `report` returns the status instead of calling `exit`, and
+  prints its summary in the shape `twill test` reads, so the runner shows the
+  counts beside each file: 207 assertions across six suites, where before it
+  showed none. `docs/needs.md` entry 15 records it.
+
 First cut of spool, the package manager for twill, written in twill.
 
 It runs, and it fetches. `init`, `list`, `remove` and `add` do their whole job,

README.md

@@ -36,12 +36,12 @@ twill on every push rather than gating on the prose in this file.
 
 ```
 $ twill test tests
-ok    tests/lockfile_test.tw
-ok    tests/manifest_test.tw
-ok    tests/resolve_test.tw
-ok    tests/semver_test.tw
-ok    tests/sha256_test.tw
-ok    tests/ui_test.tw
+ok    tests/lockfile_test.tw  (54 passed, 0 failed)
+ok    tests/manifest_test.tw  (44 passed, 0 failed)
+ok    tests/resolve_test.tw  (37 passed, 0 failed)
+ok    tests/semver_test.tw  (33 passed, 0 failed)
+ok    tests/sha256_test.tw  (28 passed, 0 failed)
+ok    tests/ui_test.tw  (11 passed, 0 failed)
 
 6 file(s): 6 passed, 0 failed
 ```

docs/needs.md

@@ -5,7 +5,7 @@ the list of language and runtime features the source uses that `mode systems`
 did not provide, with the file that needs each one and what spool did in the
 meantime.
 
-This is now a record rather than a queue. **All fourteen entries below are
+This is now a record rather than a queue. **All fifteen entries below are
 delivered.** Thirteen were delivered as of twill 1.7.1, the release `spool.toml`
 and CI pin, and entry 1 -- the process interface, the one thing between spool
 and fetching a package -- landed after it. Verified by fetching: against a git
@@ -346,3 +346,34 @@ this subset will ask.
   passing `Arr` and `Dict` values into `add_deps` to be filled in.
 - Does a `while` loop's condition see bindings introduced in its body on the
   next iteration? spool assumes normal scoping.
+
+### 15. Assertions in the toolchain
+
+**Would improve:** `tests/`
+**Status:** **delivered** (twill 1.11), and taken up. This entry is written
+after the fact, because skein's entry 16 said the ask for a `std/test` "should
+be its own entry when someone writes it" and nobody had.
+
+`twill test` arrived in 1.8 and solved discovery; the assertions each file
+called were still `tests/harness.tw`, the copy every repository in the
+ecosystem carried. twill 1.11's `std/test` is those assertions in the
+toolchain, and its own header names spool's `is_none_i64` and `equal_opt_i64`
+as the workaround its `fail(name, why)` exists to replace: both recorded a
+failure as `check(name, false)` and threw away the reason. The three call
+sites are a `match` now whose wrong arm calls `t.fail` with what was actually
+there, and `tests/harness.tw` keeps one predicate, `before`, and nothing that
+counts.
+
+The visible change is the summary line. The copy printed `lockfile: 54 passed,
+0 failed`, which the runner could not read because it looks for the word
+`passed` followed by a number, so `twill test` showed each file with no counts
+beside it. `std/test` prints `lockfile passed 54 failed 0` and then `OK` or
+`FAILED`, and the runner shows `(54 passed, 0 failed)` next to every file: 207
+assertions across the six. The `exit(1)` is gone with it; `report` returns
+the status, which is why the CI job is unchanged.
+
+One rule the move fixed in place, so the next reader does not relearn it: a
+helper module that imports `std/test` gets its own instance and its own
+counter, and a failure it records never reaches the suite's `report`. That
+was measured in skein with a probe that came back green with a failure in it.
+Anything that counts lives in the suite, not in a helper.

tests/harness.tw

@@ -1,68 +1,19 @@
 mode systems
 
-# A test harness, in twill.
-#
-# There is no test runner in the toolchain yet, so a test file is a program:
-# it calls these, and `report` exits non-zero if anything failed. That is
-# primitive and it is enough to be a CI gate the day `mode systems` runs.
+# What is left of the test harness once the assertions are twill 1.11's
+# `std/test`: one predicate two suites use to check that a rendering is in
+# order. It records nothing. A helper here that imported `std/test` would get
+# its own instance of the module and its own counter, and a failure it recorded
+# would never reach the suite's `report`; that was measured, in skein, with a
+# probe that came back green with a failure in it. So this file holds nothing
+# that counts, and a suite passes `before` to `t.check`.
 #
 # Test names are written as sentences, because a failing test's name is the
 # only documentation anyone reads at the moment it fails.
 
 import "../src/strutil.tw" as s
 
-struct Counter {
-  passed: I64,
-  failed: I64,
-}
-
-let TESTS = Counter { passed: 0, failed: 0 }
-
-fn check(name: Str, ok: Bool) {
-  if ok {
-    TESTS.passed = TESTS.passed + 1
-  } else {
-    TESTS.failed = TESTS.failed + 1
-    print("FAIL " + name)
-  }
-}
-
-fn equal_str(name: Str, got: Str, want: Str) {
-  if got == want {
-    TESTS.passed = TESTS.passed + 1
-  } else {
-    TESTS.failed = TESTS.failed + 1
-    print("FAIL " + name)
-    print("  got:  " + got)
-    print("  want: " + want)
-  }
-}
-
-fn equal_i64(name: Str, got: I64, want: I64) {
-  equal_str(name, s.i64_to_str(got), s.i64_to_str(want))
-}
-
-# The searches in strutil, manifest and lockfile return Opt[I64] rather than an
-# index-or-minus-one. These three keep the unwrapping out of the tests, so what
-# a test asserts stays on one line.
-
-fn equal_opt_i64(name: Str, got: Opt[I64], want: I64) {
-  match got {
-    Some(v) => equal_i64(name, v, want),
-    None => check(name, false),
-  }
-}
-
-fn is_none_i64(name: Str, got: Opt[I64]) {
-  match got {
-    Some(_) => check(name, false),
-    None => check(name, true),
-  }
-}
-
-# before reports whether `first` occurs before `second` in `hay`. A needle that
-# is absent fails, which is stricter than the comparison this replaced: there a
-# missing needle was -1 and still sorted before everything.
+# Whether `first` appears before `second` in `hay`. False when either is absent.
 fn before(hay: Str, first: Str, second: Str) -> Bool {
   match s.index_of(hay, first) {
     Some(a) => match s.index_of(hay, second) {
@@ -72,10 +23,3 @@ fn before(hay: Str, first: Str, second: Str) -> Bool {
     None => false,
   }
 }
-
-fn report(suite: Str) {
-  print(suite + ": " + s.i64_to_str(TESTS.passed) + " passed, " + s.i64_to_str(TESTS.failed) + " failed")
-  if TESTS.failed > 0 {
-    exit(1)
-  }
-}

tests/lockfile_test.tw

@@ -1,6 +1,7 @@
 mode systems
 
-import "harness.tw" as t
+import "std/test" as t
+import "harness.tw" as h
 import "../src/strutil.tw" as s
 import "../src/lockfile.tw" as lockfile
 
@@ -38,7 +39,7 @@ fn render_sorts_packages_regardless_of_input_order() {
   let b = lockfile.render(lockfile.Lock { packages: [entry("mid"), entry("zeta"), entry("alpha")], err: "" })
   t.equal_str("input order does not change the rendered lockfile", a, b)
   t.check("packages come out in name order",
-    t.before(a, "alpha", "mid") and t.before(a, "mid", "zeta"))
+    h.before(a, "alpha", "mid") and h.before(a, "mid", "zeta"))
 }
 
 fn render_is_byte_for_byte_stable_across_repeated_calls() {
@@ -115,8 +116,14 @@ fn parse_rejects_unknown_keys() {
 
 fn find_looks_up_a_package_by_name() {
   let l = lockfile.Lock { packages: [entry("a"), entry("b")], err: "" }
-  t.equal_opt_i64("find locates a locked package", lockfile.find(l, "b"), 1)
-  t.is_none_i64("find reports a package that is not locked", lockfile.find(l, "missing"))
+  match lockfile.find(l, "b") {
+    Some(i) => t.equal_i64("find locates a locked package", i, 1),
+    None => t.fail("find locates a locked package", "find returned None"),
+  }
+  match lockfile.find(l, "missing") {
+    Some(i) => t.fail("find reports a package that is not locked", "find returned Some(" + str(i) + ")"),
+    None => t.check("find reports a package that is not locked", true),
+  }
 }
 
 fn an_empty_lockfile_renders_and_parses() {

tests/manifest_test.tw

@@ -1,6 +1,7 @@
 mode systems
 
-import "harness.tw" as t
+import "std/test" as t
+import "harness.tw" as h
 import "../src/strutil.tw" as s
 import "../src/manifest.tw" as manifest
 
@@ -135,7 +136,7 @@ fn render_sorts_dependencies_and_round_trips() {
     ],
   }
   let out = manifest.render(m)
-  t.check("render sorts dependencies", t.before(out, "alpha", "zebra"))
+  t.check("render sorts dependencies", h.before(out, "alpha", "zebra"))
   let back = parsed(manifest.parse(out))
   t.equal_str("render is stable across a round trip", manifest.render(back), out)
 }

tests/resolve_test.tw

@@ -5,7 +5,7 @@ mode systems
 # network or the filesystem, which is the whole reason resolve.tw is separate
 # from vendor.tw.
 
-import "harness.tw" as t
+import "std/test" as t
 import "../src/strutil.tw" as s
 import "../src/manifest.tw" as manifest
 import "../src/resolve.tw" as resolve

tests/semver_test.tw

@@ -1,6 +1,6 @@
 mode systems
 
-import "harness.tw" as t
+import "std/test" as t
 import "../src/semver.tw" as semver
 
 fn parse_accepts_a_plain_triple_and_a_leading_v() {
@@ -74,7 +74,10 @@ fn highest_matching_picks_the_greatest_version_satisfying_every_constraint() {
   }
 
   let impossible = [semver.parse_constraint("^1.0.0"), semver.parse_constraint("^2.0.0")]
-  t.is_none_i64("^1 and ^2 together match nothing", semver.highest_matching(versions, impossible))
+  match semver.highest_matching(versions, impossible) {
+    Some(at) => t.fail("^1 and ^2 together match nothing", "matched " + semver.to_str(versions[at])),
+    None => t.check("^1 and ^2 together match nothing", true),
+  }
 }
 
 parse_accepts_a_plain_triple_and_a_leading_v()

tests/sha256_test.tw

@@ -7,7 +7,7 @@ mode systems
 # delusions. `std/hash` is still imported for `padded_len`, which is where the
 # block boundaries are stated.
 
-import "harness.tw" as t
+import "std/test" as t
 import "../src/strutil.tw" as s
 import "std/hash" as sha
 import "../src/pkghash.tw" as pkghash

tests/ui_test.tw

@@ -1,6 +1,6 @@
 mode systems
 
-import "harness.tw" as t
+import "std/test" as t
 import "../src/ui.tw" as ui
 import "std/term/caps" as cp
 import "std/term/theme" as th