[a11y] Use JSON as the source of truth for the component conformance reports #49239

Summary

The accessibility scorecard script (scripts/a11yScorecard.mjs) uses regex to read the per-component reports (packages/mui-material/src//accessibility.md). This approach is fragile. This issue proposes structured data as the source of truth. The script then generates the Markdown reports from that data.

This follow-up comes from the review on #48925: #48925 (review)

Part of #14187.

Current state

Each component report is hand-written Markdown. The script parses these parts with regex:

• The count table (| ✅ Supports | 23 |).
• The criterion headings (#### 1.4.3 Contrast (Minimum) · AA) and the testing groups (### ⚙️ Automated).
• The status line tokens (🚩 · ✅ Supports · ○ Author).
• The ## Known gaps bullets.
• The Pass: line.
• The evidence citations. Four different phrases mark the same thing: Covered by, Confirmed by, axe-core, a11y.json.

From this data, the script writes four outputs:

• The Reports table in packages/mui-material/src/accessibility.md.
• docs/data/material/getting-started/accessibility/scorecard.json.
• packages/mui-material/src/manual-testing.md.
• The summary table on the public conformance page.

Problems

• A small change to the wording or format can break the parser silently. For example, the parser can miss a row or read a wrong token.
• The count table repeats data that the criteria already contain. Nothing checks that the two agree.
• The evidence check depends on free prose. It needs four patterns for one concept.
• The script mixes reading, parsing, validation, and writing in one 550-line file. This makes it hard to review.

Proposal

1. Add one structured file for each component, for example /accessibility.json. Each criterion has these fields:
   • number, name, level
   • conformance, responsibility, group (Manual, Hybrid, or Automated)
   • flagged
   • evidence: a list of { type: 'axe' | 'unit' | 'playwright' | 'review', ref }
   • pass, steps, notes
   • inheritedFrom (for group components such as RadioGroup and ToggleButtonGroup)
2. Generate each accessibility.md report from its data file. Do not calculate the count table by hand.
3. Replace the evidence regex with a schema rule: an Automated criterion must have at least one evidence entry that is not of type review.
4. Validate each data file against a JSON schema in --check.
5. Split the script into modules: read, validate, roll up, render, and write.

Open questions

• Format. Long prose (notes and testing steps) is hard to edit and review inside JSON strings. Options:
  • JSON with Markdown strings.
  • YAML.
  • Markdown with structured front matter for each criterion. The prose stays in the Markdown body.
• Migration. Do we convert all 12 reports in one PR, or one component per PR?
• Consumers. Must the scorecard.json shape stay stable for the docs page?

Scope

• 12 component reports (Accordion, AccordionSummary, Avatar, Button, Checkbox, LinearProgress, Radio, RadioGroup, Switch, TextField, ToggleButton, ToggleButtonGroup).
• scripts/a11yScorecard.mjs and its CI step.

We start this work after the a11y stack (#48915 → #48926) merges, because each report lives in a different layer of the stack.

Search keywords: