Implement Zawrs and wait for wfi, wrs.nto and wrs.sto in one place

[davidharrishmc]

What

Adds the Zawrs wait-on-reservation-set instructions, wrs.nto and wrs.sto, enabled on rv64gc and gated by ZAWRS_SUPPORTED elsewhere. It also rebuilds the wfi wait so that wfi, wrs.nto and wrs.sto share one wait mechanism. This supersedes #1923, #1933 and #1935 and carries their tests. #1926, now merged, is kept: FlushWCause = TrapM.

Wait design

A waiting instruction stays in M. WaitM stalls the whole pipeline through StallW, as an LSU stall does, so the instruction in W keeps forwarding. One flop, WaitedM, records that the instruction in M has waited, and one counter of waiting cycles sets the time limits. The counter holds while the instruction stays in M and clears when M advances.

• hazard: StallWCause |= WaitM & ~FlushWCause, FlushWCause = TrapM, StallM = StallW
• trap: ValidIntsM = (Committed | WaitedM) ? 0 : EnabledIntsM. InterruptM no longer gates on wfi.
• Removed: WFIStallM, WFIInterruptedM, StallMCause, LatestUnstalledW, wfiW and the ~wfiM | wfiW interrupt gate.

	wfi	wrs.nto	wrs.sto
Waits while	no locally enabled interrupt is pending	the reservation is valid and no interrupt is pending	same as wrs.nto
Enabled interrupt already pending	trap on the instruction itself	same	same
Woken by an interrupt	retires; an enabled interrupt is taken with mepc = pc + 4	same	same
No reservation	—	completes at once	completes at once
Time limit	below M with TW = 1, or U mode with S: illegal instruction after 2^WFI_TIMEOUT_BIT cycles, mepc = the wfi, not retired	below M with TW = 1: illegal instruction after 2^WRSNTO_TIMEOUT_BIT; otherwise waits	completes after 2^WRSSTO_TIMEOUT_BIT

The limits are 2^14, 2^13 and 2^12 cycles on every configuration. The counter is sized by WAIT_TIMEOUT_BIT, the largest limit in use.

With a single hart, no other agent can write the reservation set. Once a wrs holds a reservation, only an interrupt or a timeout ends the wait. The committed ReservationValidW is exported from lrsc.sv. Hypervisor support and hstatus.VTW are out of scope.

Bugs fixed

• Stale operand after a woken wfi (hazard: stall Writeback while a wfi waits #1933). W was flushed every waiting cycle, so forwarding stopped. li a0,42; wfi; addi a1,a0,0 gave the old a0. The same happens to an lr result after a wrs.
• Interrupt taken one instruction late (trap: take an interrupt enabled before a wfi on the wfi itself #1935). An interrupt already pending, then enabled by a CSR write or xRET just before a wfi, was taken after the wfi retired. It is now taken on the wfi, as intr_mip_mie_xret_csrwr requires.
• wfi both retires and traps (Flush a trapping wfi in the Writeback stage #1926, already merged; kept). A trapping wfi retired, counting in minstret, and also became mepc.
• Spurious trap on the next instruction (privdec: gate the WFI timeout with wfiM #1923). A wfi woken in the cycle its count reached the limit made the next instruction take an illegal-instruction trap.
• Counter reset during a stall (Record trap CSRs and trap HPM events only when the trap is taken (~StallW) #1930). The counter no longer depends on TrapM, so it cannot change while a trap waits for ~StallW.
• Wrong mcause. A TW timeout coinciding with an enabled interrupt wrote mcause = 5 without the interrupt bit.

Also in this PR

PrivilegedM already requires funct3 = 0 and rd = 0, so the redundant rdzeroM terms on the fence encodings are dropped. The illegal encodings still trap.

Testing

New self-checking tests in tests/coverage, all listed in coverage64gc:

• From the superseded PRs: wfitimeoutnext, wfiBackToBack, wfiForward, csrwfiInt.
• New:
  • wfitimeoutint: a TW timeout racing an enabled interrupt.
  • wfiTW: a TW wait woken before the limit; a timed-out wfi that does not retire; U mode.
  • wrsWait: no reservation, the short timeout, the TW limit, U mode with TW = 0, an interrupt already pending, and forwarding across the wait.

test	main	Zawrs alone	this PR
wfitimeoutnext	fail	pass	pass
wfiBackToBack, wfiForward, csrwfiInt	fail	fail	pass
wfitimeoutint	fail	fail	pass
wfiTW, wfitimeout	pass	pass	pass
wrsWait	fail	fail (stale lr result)	pass

ACT priv suites (Zawrs, Interrupts, Sm, S, U, Ssstrict, Zicntr, Exceptions) were built with Zawrs enabled and all pass:

• rv64gc: 366/366
• rv32gc: 372/372

ImperasDV lockstep:

• 0 mismatches on the ACT Zawrs and Interrupts ELFs and on wrsWait, wfiBackToBack, wfiForward, csrwfiInt and wfitimeout.
• wfiTW, wfitimeoutint and wfitimeoutnext are waived in regression-wally. ImperasDV traps a TW = 1 wfi below M at once, while Wally waits a bounded time. Both are legal: mstatus_tw_always_illegal allows the first and mstatus_tw_op the second.

lint-wally is clean, and buildroot Linux boots to the login prompt.

Synthesis, sky130 rv64gc full core at 200 MHz, compared with the Zawrs-only version:

• Total area is unchanged within noise.
• Hazard + privdec + trap area is 8.6% smaller.
• No path through StallW, StallM, FlushW, StallD or InterruptM gets slower.
• The interrupt-pending-to-stall path improves from 5.77 to 5.58 ns.

Note for the reviewer

The ACT test-generation config for cvw-rv64gc must declare Zawrs, or the Ssstrict illegal-encoding sweep expects wrs.nto and wrs.sto to trap. Two files need changing:

• the UDB yaml: add Zawrs and set ZAWRS_NTO_IS_NOP: false
• sail.json: set Zawrs.supported: true

Supersedes #1923, #1933 and #1935.

🤖 Generated with Claude Code

[davidharrishmc]

Split in two: the wfi wait fix is #1941 (against main), and Zawrs is #1942 (against rva23, with #1941's wait commit cherry-picked first, so review only its Zawrs commit).