fix(middleman): resolve broadcast failures through the verifier

[miguel502]

A broadcast with no clean answer was written Failure with no hash: invisible to the verifier forever, though the tx may have landed.

sendTransaction now derives the hash locally (sha256 of the bytes it broadcasts, as matchTxInBlock does), so every outcome carries one.

It also stops flattening errors: dedup is success, a deterministic CheckTx rejection is rejected, timeouts and resets are unknown.

ExecuteTransaction anchors hash + heights BEFORE broadcasting; an unknown outcome stays pending for the chain to settle.

[miguel502]

PR description — fixes #339

Summary

A broadcast that returned no clean answer was recorded as a permanent Failure with no hash. The
verifier sweep selects on status = pending AND hash IS NOT NULL AND executionHeight IS NOT NULL,
so such a row was never revisited: if the node had accepted the transaction before the connection
dropped, it still landed on chain while our database said it failed — forever, with no supplier
rows ever created for it.

The broadcaster no longer decides the outcome. It anchors the transaction (hash + heights), sends
it, and lets the verifier settle it against the chain. Only a rejection the node states
deterministically, on the first attempt, short-circuits to Failure.

Against the reporter's suggested direction

#	Reporter recommended	What shipped	Verdict
1	Derive the tx hash locally — sha256(txBytes) uppercased, same derivation as matchTxInBlock	deriveTxHash exported from @igniter/pocket, used by the anchor and every non-success outcome; rejects non-hex input rather than hashing truncated bytes	Taken as written
2	Persist the hash, leave the tx pending, let the tri-state verifier reach the verdict	persistBroadcastAnchor writes hash + executionHeight + timeoutHeight; the unknown-outcome path writes no status	Taken as written
3	Short-circuit to Failure only for genuinely deterministic rejections; stop flattening transport errors and rejections	Done, then narrowed twice: sdk codes 20/32 are not definitive (32 also means already landed), and a rejection is only trusted on attempt 1 (a retry answers about the world the first attempt created)	Taken, narrowed further
4	Copy the provider's shape: sign → persist hash → broadcast, with if (txn.hash) → re-broadcast	Persist-before-broadcast adopted. Re-broadcast-on-re-entry not adopted — it would in fact be safe under the new classification (19 → dedup success, 32 → indeterminate); the real asymmetry is that the provider can re-sign and middleman cannot, since the wallet holds the key. Cost: a run dying between anchor and broadcast doesn't re-send, so the verifier fails it ~30 blocks later	Diverged
5	Delete the dead if (!result) branch	Removed from the new flow; retained verbatim inside legacyFlow, which must not change for Temporal replay	Taken as written

Where #3 was narrowed further

Two cases look like hard rejections and are not. Treating either as terminal recreates this issue:

• CheckTx codes 20 and 32, scoped to the sdk codespace since cosmos codes are
  codespace-scoped. Code 32 fires when the sequence is already consumed — i.e. the transaction
  landed — as well as when a predecessor is still in the mempool. Code 20 is a full mempool.
• Any rejection on a retry. The activity retries up to 3 times, so a re-broadcast is a normal
  path. If the first attempt landed the tx, the node answers about the world that tx created:
  insufficient funds now that the stake deducted the balance, or a consumed sequence. A rejection
  is only trusted on attempt 1; on a retry it goes to the verifier.

Where #4 diverged, and the cost

Re-broadcasting on re-entry would in fact be safe under the new classification — an in-mempool
repeat answers code 19 (dedup → success) and a landed one answers code 32 (indeterminate). The
real asymmetry with the provider is narrower than ordered-vs-unordered: the provider can re-sign
its own transactions, middleman cannot, because the wallet holds the key.

Cost of not adopting it: a run that dies between the anchor write and a successful broadcast does
not re-send, so the verifier settles it Failure once coverage passes timeoutHeight — a correct
verdict reached slowly, for a transaction never actually transmitted. Adopting re-broadcast on
re-entry is the natural follow-up; it is left out here so the behavioural switch is made
deliberately rather than as a side effect of this fix.

Correction to the issue text

The issue says the middleman never got the #308 broadcast/verify split. It got half — the
if (transaction.hash) handoff guard and the "verification moved to the verifier" docstring were
already there. What was missing was persisting the hash before broadcasting.

The blocker that had to be fixed first

Recommendation 2 could not have worked as written. parseSignerAndSequence decoded the signed
payload as base64; middleman stores it as hex. Because hex digits are valid base64
characters the mistake never threw — it produced garbage bytes, TxRaw.decode failed, and the
catch returned {sequence: null, timeoutHeight: null} for every transaction since v0.13.0.

Both consumers were affected, so decideVerification computed
orderedRequiredCoverage = POSITIVE_INFINITY and its hash: absent → failure branch was
unreachable. No middleman transaction could reach a failure verdict. Success verdicts were
unaffected (a tx found on chain settles on the hash alone), which is why the stranded population is
only transactions that were broadcast and never landed — and why this surfaced as one user report
rather than a flood.

Measured against the real function with a real TxRaw:

hex payload parsed as base64 (production): {"sequence":null,"timeoutHeight":null}
same payload parsed as hex:                {"sequence":7,"timeoutHeight":1030}

Without this, the change would have moved the defect rather than removed it: from "wrongly marked
failed, quickly" to "stuck pending, silently, forever". It now has the test file it never had,
including a two-sided assertion that the encodings are not interchangeable.

Also included

• The rejection path claims the row through claimTerminalTransition's CAS before running any
  effect, and stands down if it loses. Anchoring pre-broadcast makes the row visible to the
  verifier while the broadcaster is still retrying, and the verifier can reach the opposite
  verdict (success on goal-state alone, when a sibling tx staked the same operator). Releasing the
  addresses afterwards is unrecoverable: the provider's markStaked requires state = Delivered,
  so a release landing first turns the verifier's success effect into a silent no-op and the key
  can be re-delivered to another delegator while the first is staked on-chain.

  This deliberately inverts the effects-before-CAS invariant documented on
  claimTerminalTransition, which assumes both racers reach the same decision. The
  expired-before-broadcast branch keeps the original order — its row has no hash, so no competing
  writer exists.

• The broadcast-rejection exit releases the provider's addresses. It was the only terminal exit
  that did not, unlike the expired path and the verifier's apply-failure.

• The "transaction is not pending" guard returns instead of throwing. A plain Error from
  workflow code does not fail the run — the SDK turns it into an unhandled rejection, failing the
  workflow task, which the server retries forever while the run sits RUNNING and
  ALLOW_DUPLICATE_FAILED_ONLY blocks a replacement. Pre-existing line; anchoring before the
  broadcast widened the window from microseconds to the whole broadcast phase.

• Deterministic handling for an unbroadcastable payload. The anchor returns null for an empty
  or non-hex payload and the workflow terminalizes it, instead of failing the run and letting the
  10s dispatcher relaunch it forever.

• patched('execute-transaction-command-sequence-v2'). The workflow's command sequence changed
  (activity inserted, one dropped, terminal branches reordered). Temporal replays history rather
  than resuming, and Igniter is upgraded by restarting the process, so a run open at that moment
  would mismatch, fail the workflow task, and retry forever — a silent permanent wedge needing a
  manual terminate. The gate keeps pre-upgrade runs on the old path. Those runs still get the
  substance of the fix, because activities are not version-pinned: the fixed sendTransaction
  returns a hash on every outcome, so the old !result.transactionHash branch no longer fires.

Tests

35 new tests across three files.

• packages/pocket/src/sendTransaction.test.ts (17) — the error ladder, codespace scoping, and the
  invariant the anchor rests on: the bytes hashed are the bytes broadcast, asserted against what is
  handed to cosmjs, with the derivation cross-checked against node:crypto.
• apps/middleman-workflows/src/workflows/ExecuteTransaction.test.ts (17) — every outcome branch,
  ordering assertions via invocationCallOrder, the CAS-lost path, and the legacy flow.
• apps/middleman-workflows/src/activities/parseSignerAndSequence.test.ts (4) — the hex/base64
  regression, both directions.

Full gate green: lint 11/11, build 16/16, test 23/23, tsc --noEmit clean, no-console guard clean.