Skip to content

pos-module-tests: the JSON formatter iterates an empty array, so every run reports 0 assertions and no per-test results #32

Description

Summary

pos-module-tests/modules/tests/public/views/partials/tests/show_js.liquid declares tests_array and then iterates that same empty array instead of contracts. The loop body never executes, so every run reports total_assertions: 0 and tests: [], whether it passed or failed.

The fix already exists in the archived standalone repo. Platform-OS/pos-module-tests has for contract in contracts at that line. This monorepo copy still has the pre-fix version, so anyone installing from the registry gets the bug.

Current code in this repo

assign tests_array = []
for contract in tests_array          # <- iterates the empty array declared above
  assign total_assertions = total_assertions | plus: contract.total
  assign test_result = { "name": contract.test_path, ... }
  assign tests_array << test_result
endfor
...
assign result.total_tests = contracts.size    # correct, so the count looks healthy
assign result.total_assertions = total_assertions   # always 0
assign result.tests = tests_array                   # always []

Observed

{"success":true,"total_tests":17,"total_assertions":0,"total_errors":0,"duration_ms":281,"tests":[]}

17 contracts were built and every one was discarded.

After changing the loop to iterate contracts

total_tests: 3   total_assertions: 20   tests: 3 entries with per-test names and counts

Why this matters more than a cosmetic count

total_tests and total_errors survive the bug, so a suite still passes and fails correctly, and that is exactly what makes it dangerous: a test whose assertions never fire is indistinguishable from one that passes. We hit that. A set of assertions in our suite were passing the wrong argument name, so they never evaluated, and CI reported green for weeks. With per-test assertion counts visible, that is caught immediately, because a test reporting 0 assertions stands out.

It also blocks any CI guard on assertion counts, since such a guard would fail every green run.

Fix

One line, and it matches what the archived repo already has:

-  for contract in tests_array
+  for contract in contracts

Environment

  • tests module 1.3.4 from partners.platformos.com
  • pos-cli 6.2.4

Happy to open a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions