[Regression] Arrow Keys for on screen keyboard don't work for GH Copilot CLI

[carlos-zamora]

Windows Terminal version

1.26.901

Windows build number

10.0.26603.0 on ARM Surface Pro

Other Software

GitHub Copilot 1.0.29

Steps to reproduce

Open copilot CLI
Open on screen keyboard
(optional) /model to open a TUI
try to use arrow keys to move cursor or selected items

Expected Behavior

cursor should move or selection should be updated as if you used a physical keyboard

Actual Behavior

no input detected

other keys work fine (ie ESC, typing, etc)
physical keyboard works fine

REGRESSION: WT 1.24.10921 (Stable) works fine. This is potentially related to KKP? That's a big change in input handling that happened recently

[DHowett]

@carlos-zamora can you grab a debug tap trace for the arrow keys vs. the OSK arrow keys?

[carlos-zamora]

OSK:

␛[38;2;188;68;167m█␣␛[39m␛[38;2;22;198;12m▘▝␛[39m␛[38;2;188;68;167m␣█␛[39m␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;188;68;167m│␛[39m␛[26;15H␛[?2026l␛[57419u␛[57420u␛[57417u␛[57418u␛[O

Physical Keys:

␛[38;2;188;68;167m│␛[39m␛[57;15H␛[?2026l␛[A␛[?2026h␛[53;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␣␣1.␣Stored␣in␣your␣account,␣only␣for␣you␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[55;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␛[38;2;97;214;214m❯␣3.␣Stored␣on␣this␣device␣only␛[39m␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[57;15H␛[?2026l␛[B␛[?2026h␛[53;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␛[38;2;97;214;214m❯␣1.␣Stored␣in␣your␣account,␣only␣for␣you␛[39m␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[55;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␣␣3.␣Stored␣on␣this␣device␣only␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[57;15H␛[?2026l␛[D␛[?2026h␛[57;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␣␣␛[38;2;134;134;134mApply␣to:␣This␣session␛[39m␣␣␛[38;2;134;134;134mThis␣repository␛[39m␣␣␛[7m␛[38;2;97;214;214m[All␣repositories]␛[39m␛[27m␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[57;46H␛[?2026l␛[C␛[?2026h␛[57;1H␛[2K␛[38;2;90;90;90m│␛[39m␣␣␣␛[38;2;134;134;134mApply␣to:␣␛[39m␛[7m␛[38;2;97;214;214m[This␣session]␛[39m␛[27m␣␣␛[38;2;134;134;134mThis␣repository␛[39m␣␣␛[38;2;134;134;134mAll␣repositories␛[39m␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␛[38;2;90;90;90m│␛[39m␛[57;15H␛[?2026l

[DHowett]

Parsing out the parts I think I saw:

OSK - ␛[57419u␛[57420u␛[57417u␛[57418u
Physical - ␛[A␛[B␛[C␛[D

Yeah, that's not great. @lhecker - we seem to be mis-encoding the arrow keys from the on-screen keyboard¹ in KKP

Footnotes

1. which has always been "weird" ↩

[lhecker]

Logging HWND messages when pressing Up-Arrow once, reveals that we receive 2 WM_KEYDOWN messages with the following wParam/lParam...

For a regular Keyboard:

0x26 = VK_UP ✅
0x01480001 = repeat:1 ✅ scan_code:0x48 ✅ extended_key:1 ✅ (!!!)

0x26 = VK_UP ✅
0xC1480001 = repeat:1 ✅ scan_code:0x48 ✅ extended_key:1 ✅ previous_key_state:1 ⚠️ transition_state:1 ⚠️

Side note: "The transition state" is publicly documented to be always 0 for WM_KEYDOWN. And yet, somehow, it's set to 1 for us. That's on top of us receiving two events for some reason? Maybe it's an artefact of us using ToUnicode?

In any case, for the Touch Keyboard:

0x26 = VK_UP ✅
0x09990001 = repeat:1 ✅ scan_code:0 ❌ extended_key:0 ❌

0x26 = VK_UP ✅
0xC0000001 = repeat:1 ✅ scan_code:0 ❌ extended_key:0 ❌ previous_key_state:1 ⚠️ transition_state:1 ⚠️

The lack of ENHANCED_KEY in dwControlKeyState dictates that this is a keypad key. KKP in turn dictates that keypad and non-keypad keys get different sequences. The lack of scan codes is another issue, technically (the codes are also well defined). This is an issue that goes beyond WT, because all of this is publicly well defined: https://learn.microsoft.com/en-us/windows/win32/inputdev/about-keyboard-input#scan-codes (the 0xE0.. prefix to keys indicates that it is an extended key.)

In short, we should file this bug internally and make sure it's prioritized and rolled out quickly.