fix(bin): stop fm-crew-state from false-failing healthy active validations

[RajeshRajendiran]

Intent

Fix bin/fm-crew-state.sh to correctly report healthy running no-mistakes validations instead of false-failing.

Changes:

1. Added nm_coarse_status_is_active() treating pending and running as active in the runs-list lookup
2. Added pending case in coarse status handling with detail 'validating (pending)'
3. Added stale-run override: when axi status reports failed but runs-list shows active for same branch+head, report working instead of failed
4. Added CI_READY_LOG_SUPERSEDED variable to suppress ci-ready shortcut when the run provably cannot be the one that appended a checks-green status-log line
5. Changed stale-record override detail string to 'validating (stale-record override)' for audibility
6. Extended CI_READY_LOG_SUPERSEDED to cover coarse running rows (no step detail means we cannot verify status-log's checks-green belongs to current run)

Tests updated: test_coarse_running_run_has_no_ci_log_and_suppresses_ci_ready verifies coarse running suppresses ci-ready when no step detail is available. All 46 existing tests pass.
shellcheck -x passes cleanly on changed scripts.

What Changed

• Added nm_coarse_status_is_active() (treating pending and running as active) and a pending case in the coarse-status handler so a freshly created, not-yet-started run reports working with detail validating (pending) instead of an unrecognized status.
• Added a stale-record override: when axi status reports failed but the branch's newest runs-list row for the same head is still active, the verdict flips to working with detail validating (stale-record override), so a superseded failed record no longer false-fails a healthy crew while genuine terminal failures still surface.
• Introduced CI_READY_LOG_SUPERSEDED to suppress the checks green ci-ready shortcut both for the stale-record override and for coarse pending/running rows (which carry no step detail), preventing a green status-log line left by an earlier run from misreporting done; updated the FM_CREW_STATE_RUNS_LIMIT doc note and added/updated tests covering these paths.

Risk Assessment

✅ Low: The change is well-bounded, carefully scoped to the failed-verdict path, preserves the newest-row and green-not-demoted invariants, and is covered by behavioral regression tests reproducing the false-fail and masking scenarios.

Testing

shellcheck -x passes on the changed script; the entire fm-crew-state test suite (60 tests, including the 7 new ones for pending/running coarse rows, stale-record override, genuine-failure safety, checks-green non-demotion, and superseded ci-ready suppression) passes; and a reproducible hermetic driver exercising the real bin/fm-crew-state.sh shows the base commit false-failing a healthy stale-record+active-run crew as state: failed while the target commit correctly reports state: working · validating (stale-record override). New-test regression coverage was confirmed by running the new assertions against the base bin and observing the expected failures. No actionable issues found.

Evidence: Stale-record override before/after (real fm-crew-state.sh verdict)

Source: Stale-record override before/after (real fm-crew-state.sh verdict)

===== BEFORE FIX (base commit 87681a4) ===== state: failed · source: run-step · run failed ===== AFTER FIX (target commit ca42337) ===== state: working · source: run-step · validating (stale-record override)

===== BEFORE FIX (base commit 87681a4) =====
state: failed · source: run-step · run failed

===== AFTER FIX (target commit ca42337) =====
state: working · source: run-step · validating (stale-record override)

Evidence: Reproducible hermetic before/after driver script

Source: Reproducible hermetic before/after driver script

#!/usr/bin/env bash
# End-to-end demonstration of the fm-crew-state.sh false-fail fix.
# Drives the REAL bin/fm-crew-state.sh (and the base-commit version) through a
# minimal hermetic fake `no-mistakes` / `tmux` / `herdr` toolbin - the same
# technique the test suite uses - so no real no-mistakes install is required.
# Prints the helper's actual verdict for the buggy (base) and fixed (target)
# versions side by side for the stale-record-override scenario.
set -u

PROJ="$PWD"
while [ "$PROJ" != "/" ] && [ ! -x "$PROJ/bin/fm-crew-state.sh" ]; do
  PROJ="$(dirname "$PROJ")"
done
[ -x "$PROJ/bin/fm-crew-state.sh" ] || { echo "no project root found" >&2; exit 2; }

WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT

# --- minimal hermetic fakes (mirror tests/fm-crew-state.test.sh) -------------
make_fakebin() {  # <dir>
  local fb="$1/fakebin"; mkdir -p "$fb"
  cat > "$fb/no-mistakes" <<'SH'
#!/usr/bin/env bash
set -u
case "${1:-}" in
  axi) shift
       case "${1:-}" in
         status) shift
                 if [ "${1:-}" = --run ]; then printf '%s\n' "${FM_FAKE_AXI_STATUS_RUN:-}"
                 else printf '%s\n' "${FM_FAKE_AXI_STATUS:-}"; fi ;;
         logs)   printf '%s\n' "${FM_FAKE_CI_LOGS:-}" ;;
       esac ;;
  runs) printf '%s\n' "${FM_FAKE_RUNS_LIST:-}" ;;
esac
exit 0
SH
  cat > "$fb/tmux" <<'SH'
#!/usr/bin/env bash
case "${1:-}" in
  display-message) printf '%%1\n' ;;
  capture-pane)    printf 'all quiet\n> \n' ;;
esac
exit 0
SH
  cat > "$fb/herdr" <<'SH'
#!/usr/bin/env bash
case "${1:-}" in
  status) [ "${2:-}" = --json ] && { printf '{"client":{"version":"0.7.1","protocol":14},"server":{"running":true}}\n'; exit 0; } ;;
  server) exit 0 ;;
esac
exit 0
SH
  chmod +x "$fb/no-mistakes" "$fb/tmux" "$fb/herdr"
  printf '%s\n' "$fb"
}

run_failed() {  # <branch>  -> TOON `axi status` for a terminal FAILED run
  cat <<EOF
run:
  id: "01RUN"
  branch: $1
  status: completed
  head: "${FM_FAKE_RUN_HEAD:-abc1234}"
  pr: ""
  findings: none
outcome: failed
EOF
}

write_meta() {  # <file> <lines...>
  local f=$1; shift
  : > "$f"
  for ln in "$@"; do printf '%s\n' "$ln" >> "$f"; done
}

run_scenario() {  # <bin> <label>
  local bin=$1 label=$2 d short fb
  d="$WORK/$label"; mkdir -p "$d/state" "$d/wt"
  git -C "$d/wt" init -q
  git -C "$d/wt" commit -q --allow-empty -m init
  git -C "$d/wt" checkout -q -b fm/feat-override
  FM_FAKE_RUN_HEAD=$(git -C "$d/wt" rev-parse HEAD); export FM_FAKE_RUN_HEAD
  short=$(git -C "$d/wt" rev-parse --short=7 HEAD)
  fb=$(make_fakebin "$d")
  write_meta "$d/state/feat-override.meta" \
    "window=fm:fm-feat-override" "worktree=$d/wt" "kind=ship"
  FM_FAKE_AXI_STATUS="$(run_failed fm/feat-override)"; export FM_FAKE_AXI_STATUS
  FM_FAKE_RUNS_LIST="$(cat <<EOF
  running    fm/feat-override ${short}  2026-08-01 09:30
  failed     fm/feat-override ${short}  2026-08-01 09:00
EOF
)"; export FM_FAKE_RUNS_LIST
  echo "===== $label ====="
  PATH="$fb:$PATH" FM_STATE_OVERRIDE="$d/state" "$bin" feat-override
  echo
}

# Base version placed INSIDE bin/ so its `source "$SCRIPT_DIR/..."` libs resolve.
BASE_BIN="$PROJ/bin/.demo-base-fm-crew-state.sh"
git -C "$PROJ" show 87681a40777bb061ef923ef98b494cd7ef6054b6:bin/fm-crew-state.sh > "$BASE_BIN"
chmod +x "$BASE_BIN"

run_scenario "$BASE_BIN"                "BEFORE FIX (base commit 87681a4)"
run_scenario "$PROJ/bin/fm-crew-state.sh" "AFTER FIX (target commit ca42337)"

rm -f "$BASE_BIN"

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

• shellcheck -x bin/fm-crew-state.sh
• bash tests/fm-crew-state.test.sh (full suite, 60 ok / all fm-crew-state tests passed)
• bash /tmp/no-mistakes-evidence/01M0ED5CSW6TF5E95NG50WFZDZ/demo-stale-override.sh (hermetic before/after of stale-record override driving the real bin/fm-crew-state.sh through fake no-mistakes/tmux/herdr)
• Regression check: ran the new test assertions against the base-commit bin/fm-crew-state.sh with a non-fatal fail() and observed failures for a pending coarse row is a live run, pending coarse row names its own detail, coarse running without step detail -> working, coarse running keeps run-step source, coarse running must not inherit a stale ci-ready log, stale terminal axi status must not false-fail a healthy run, a pending replacement run is live

✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning

• ⚠️ linter found issues (exit code 1)

✅ **Push** - passed

✅ No issues found.

[RajeshRajendiran]

Closing: captain dropped this task. The fix is validated but not merging at this time.