Audit the answered state, and say what has not been tested - #34
Merged
Conversation
The WCAG audit loaded each page and ran axe against it. On the chat page that is an empty shell: a heading, an input and a send button. Everything a reader spends time in — the answer, result tables, the image gallery and its data-derived alt text, citations, the response identifier and its copy button, the feedback controls — only exists after a question has been answered, and none of it was covered. That is also the surface most likely to be wrong, because it is assembled from model output rather than written by hand. Add a fifth target, '/ (answered)', which stubs the chat response, drives the real UI to submit a question, and audits the resulting DOM. The stub is served through the real SSE parser and the real renderers, so axe sees the production DOM; only the words are fixed. Stubbing rather than asking live keeps the pass running where there is no ELM credential, and keeps the DOM stable so a violation cannot appear and disappear on its own. It found three contrast failures on its first run, both fixed here: the tag line in a result table row (#777 on #0a0a0a, 4.42:1) and the 'Response ID:' label (#6f6f6f, 3.94:1). Both render as small text, so WCAG 2.2 AA wants 4.5:1. The accessibility statement claimed ten things without saying how any of them were checked, and two — live regions for streamed content, and alternative text on images — were about the state that was never audited. Rewrite it to say what is covered automatically, what the automation cannot establish, and what has not been tested at all. A statement that lists its gaps is worth more than one that implies a pass, and the Regulations ask for an accurate statement rather than a clean one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The WCAG audit loaded each page and ran axe against it. On the chat page that is
an empty shell — a heading, an input and a send button. Everything a reader
actually spends time in only exists after a question has been answered: the
answer text, result tables, the image gallery and its data-derived alt text,
inline citations, the response identifier and its copy button, the feedback
controls. None of it was covered, and it is the surface most likely to be wrong,
because it is assembled from model output rather than written by hand.
What changed
A fifth audit target,
/ (answered). It stubs the chat response, drives thereal UI to submit a question, and audits the resulting DOM. The stub is served
through the real SSE parser and the real renderers, so axe sees the production
DOM — only the words are fixed.
Stubbed rather than live for two reasons: CI does not always have an ELM
credential, so a real question would skip or fail there and the pass would
quietly stop covering the answered state; and an accessibility check wants a
stable DOM, or a violation appears and disappears without the interface having
changed.
Two contrast fixes it found on its first run. The tag line in a result table
row was
#777on#0a0a0a(4.42:1) and theResponse ID:label was#6f6f6f(3.94:1). Both render as small text, so WCAG 2.2 AA wants 4.5:1. Both are now
#8a8a8a.The accessibility statement, rewritten. It claimed ten things without saying
how any were checked, and two of them — live regions for streamed content, and
alternative text on images — were precisely about the state that was never
audited. It now says what is covered automatically, what the automation cannot
establish (whether data-derived alt text is useful, whether a live region
actually announces well), and adds a "What we have not yet tested" section
naming screen-reader, voice-recognition and 400%-reflow testing as outstanding
before public launch.
The Regulations ask for an accurate statement, not a clean one, and a statement
that lists its gaps is worth more than one that implies a pass.
Verification
node scripts/a11y-audit.mjs— 0 machine-detectable violations across allfive targets. The answered pass exercises 31 rules against 27 for the shell.
next lint— no new warnings.node --test tests/unit/*.test.mjs— 1277 pass, 0 fail.