Summary
PR #587's report catalog declares screenshots for three manual protocol tests, but python-keepkey's screenshot hook runs only through callback_ButtonRequest(). These tests intentionally use call_raw() / direct transport reads so they can control per-word, per-character, and per-roll state; they never dispatch the callback and therefore capture zero frames.
Both fresh CI runs at firmware head 087615a7b reproduce the same per-test audit failure:
test_msg_resetdevice::test_reset_device: declared 1, captured 0test_msg_recoverydevice_cipher::test_nopin_nopassphrase: declared 1, captured 0test_msg_resetdevice::test_reset_device_dice: declared 2, captured 0
The Python tests themselves pass; the failure is that the release evidence claims screens without producing them.
Security/CI impact
The OLED report is a release sign/see artifact. Excluding these tests or dismissing the gate would allow seed-word, recovery-cipher, or dice confirmations to change without visual evidence while the catalog still claims coverage.
Required remediation
- Add an explicit settled capture API for manual protocol flows.
- Bind captures to verified device state, not arbitrary test timing:
- first seed word after
read_reset_word();
- first cipher grid after
read_recovery_cipher();
- dice entry after the expected DiceRoll ButtonRequest;
- dice digest only after the device digest matches the injected roll string.
- Preserve the declared counts (1, 1, and 2); do not exclude these tests from the audit.
- Verify the screenshot audit and both full/bitcoin-only integration matrices.
Example failed job: https://github.com/BitHighlander/keepkey-firmware/actions/runs/32925086390/job/98049125585
Summary
PR #587's report catalog declares screenshots for three manual protocol tests, but python-keepkey's screenshot hook runs only through
callback_ButtonRequest(). These tests intentionally usecall_raw()/ direct transport reads so they can control per-word, per-character, and per-roll state; they never dispatch the callback and therefore capture zero frames.Both fresh CI runs at firmware head
087615a7breproduce the same per-test audit failure:test_msg_resetdevice::test_reset_device: declared 1, captured 0test_msg_recoverydevice_cipher::test_nopin_nopassphrase: declared 1, captured 0test_msg_resetdevice::test_reset_device_dice: declared 2, captured 0The Python tests themselves pass; the failure is that the release evidence claims screens without producing them.
Security/CI impact
The OLED report is a release sign/see artifact. Excluding these tests or dismissing the gate would allow seed-word, recovery-cipher, or dice confirmations to change without visual evidence while the catalog still claims coverage.
Required remediation
read_reset_word();read_recovery_cipher();Example failed job: https://github.com/BitHighlander/keepkey-firmware/actions/runs/32925086390/job/98049125585