Repository navigation
fix: declarative monitoring evidence, wizard Compliance step and CI failure-masking guard - #963
Merged
Merged
Conversation
chore: synchronize main history into development after #944
Label external monitoring plans in manifests and Annex IV, warn on inert exporter declarations, and preserve unresolved endpoint references with explicit precedence and null handling. Align CLI/web wizards, six site locales, EN/TR references and QMS guidance; add regression coverage. Resolves CR-A01-007, TECH-T09-003, DOC-D1B01-004, DOC-D3B04A-002, DOC-BLEGAL-048, DOC-D1B05-003, DOC-D4B02-003, DOC-D4B09-008, DOC-D4B09-009, DOC-BFIELD-017, DOC-D3B04A-014, DOC-BFIELD-031. Fixes #93 Co-authored-by: Codex <noreply@openai.com>
Add defaulted_fields and endpoint_redacted to the evidence projection, mask literal URLs with the shared scheme/host policy, and validate unresolved ASCII env names. Emit exporter warnings at file load rather than each pipeline validation. Keep schema defaults and existing input types. Align CLI/browser env parsing, omit disabled browser plans, expose the external check interval, and add Node execution/schema regressions to CI and the canonical gauntlets. Expand export-to-verifier and warning-count coverage, plus EN/TR input, migration, and evidence examples. Resolves review gaps in CR-A01-007, DOC-BFIELD-031, DOC-D3B04A-014, TECH-T09-003, DOC-BFIELD-017, DOC-BLEGAL-048, DOC-D4B09-009. Refs #93, #946 Co-authored-by: Codex <noreply@openai.com>
Identify monitoring as Annex IV point 9 / Article 72(3), and risk management as point 5, in the touched EN/TR reference tables. Distinguish metadata summaries from the full external monitoring plan and remove stale wizard Article 12+17 labels from current user-facing references. Replace nonexistent webhook_failed audit events with forgelm.webhook WARNING diagnostics, correct allow_private_destinations, and qualify lifecycle notifications and evaluation trends in compliance summaries. The remaining Annex IV mapping is tracked by #539; broader verifier completeness and audit-reference behavior remain in #573 and #658. Resolves DOC-D3B04B-013 and remaining monitoring documentation surfaces of DOC-D1B01-004, DOC-D3B04A-002, DOC-D4B02-003, DOC-D4B09-008. Fixes #231 Refs #93, #539, #946 Co-authored-by: Codex <noreply@openai.com>
Describe HF offline flags separately from operator configuration of reporting, API judges, and webhooks. Private-destination opt-in permits additional destinations and does not restrict delivery to internal hosts. Keep the EN/TR manual tables and release note aligned. Addresses the documentation surface of CR-MISC-004; broader offline enforcement remains tracked in #74 and #100. Refs #93, #100, #946 Co-authored-by: Codex <noreply@openai.com>
Record expected defaulted_fields directly in the export test matrix and cover a whitespace-only endpoint_env without an explicit endpoint. Blank env references leave endpoint defaulted; an explicitly blank endpoint remains an operator-supplied absence. Preserve serializer behavior. Refs #93, #946 Co-authored-by: Codex <noreply@openai.com>
Require absolute HTTP(S) literal plan URLs without echoing invalid input. Project evidence with stdlib URL parsing to preserve ports and IPv6 brackets while removing userinfo and secret-bearing tails; keep HTTP log masking unchanged. Default wizard drift answers to No, emit the browser's selected none exporter explicitly, and clear corrected field errors. Preserve schema defaults and model_fields_set provenance. Keep new deferred-locale keys on English fallback. Replace the inactive-export test's obsolete constructor boundary with load_config, cover CLI warning count and null input rejection, and compare 256 generated YAML cases against real ForgeConfig values. Document evidence file membership, input persistence, library reconstruction, flag semantics and compatibility changes. Findings: CR-A01-007, DOC-BFIELD-031, TECH-T09-003, DOC-D4B09-009; review R2-B-1, R2-B-2, R2-B-3, R2-B-4, R2-C2-1, R2-A2-4. Validation: full canonical gauntlet, 5169 passed / 22 skipped, 29 Node tests, 31 guards; 8/8 targeted JavaScript mutations killed in isolated temporary copies. Refs #93, #946, #948 Co-authored-by: Codex <noreply@openai.com>
Remove the nonexistent webhook.enabled setting introduced in the air-gap table. Disable delivery by omitting the block or leaving URL sources unset. Explain that API teachers still need network access and are not disabled by --offline. Update rendered ForgeConfig error examples and document the full root-model scope of raw-input hiding. Correct remaining risk references to Annex IV point 5, compute resources to point 2(c), and performance metrics to points 2(g)/4. Mark partial mappings and remaining legal-completeness work explicitly with tracking issues; preserve verifier behavior. Align the touched Turkish drift terminology. Findings: DOC-D3B04B-013, CR-MISC-004, DOC-D3B02-002, DOC-D1B05-003; review R2-D2-1 and validation-diagnostic/Annex-label follow-ups. Validation: full canonical gauntlet passed; rebuilt EN/TR manuals and verified bilingual structure, code blocks, schema snippets, source references and links. Refs #93, #231, #74, #100, #539, #573, #946 Co-authored-by: Codex <noreply@openai.com>
Reflow the touched EN/TR monitoring, validation, air-gap and partial Annex IV paragraphs to the documentation standard's approximately 120-column width. Preserve all words and code blocks; verify the diff changes whitespace only. Addresses the consolidated CodeRabbit prose-format finding at #946 (comment). Validation: bilingual spine/code-block parity, anchors, self-contained manual links, schema/YAML snippets, generated manual rebuild/check, and diff hygiene. Runtime and tests are unchanged from the full 5169-test canonical pass. Refs #93, #946 Co-authored-by: Codex <noreply@openai.com>
fix: restore the declarative monitoring evidence contract
STEP_RENDERERS['compliance'] called appendDetail, which only the dataset, training and operations renderers declared as local helpers. Every visit threw a ReferenceError after the risk select and before the Article 9 risk-assessment textareas, the vulnerable-groups row and the benchmark and LLM-judge controls, so strict tiers could never pass the step and the controller's counter, YAML preview and saved state stopped for it. Declare appendDetail once at the wizard's shared scope, taking the state explicitly, and delete the three local copies. Extract the VM + DOM-shim seam into tests/site/wizard_harness.cjs so every browser-wizard test runs the shipped renderers through one shim, and add render smoke coverage for every step across all risk tiers, both detail modes and every optional gate. The new tests fail on the previous wizard.js (12/12, "appendDetail is not defined") and pass now. Finding: F-W1-SITE-01. Validation: node --test tests/site/*.test.cjs (43 pass); headless Chromium audit of 10 scenarios x 9 steps on the served site: no exceptions, counter / YAML preview / persisted step consistent on every step, and a keyboard-only high-risk Compliance flow advances to Operations once filled. Fixes #879 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Text, number, slider, select and textarea captions were sibling <label> elements without `for`, and their controls had no id. Chromium named the text inputs after their placeholders and left selects, number inputs and sliders unnamed; clicking a caption focused nothing. Only the monitoring interval and the wrapped checkbox labels were associated. Give every captioned control a stable, unique id derived from the state field it edits (wizard-field-<kebab-field>) and point its caption at it with `for`. Each step renders a field at most once, so step and accordion re-renders rebuild the same ids. The LoRA method caption names its group of method buttons through role="group" + aria-labelledby, since a <label> cannot name buttons. Checkbox labels keep wrapping their toggles. Add DOM-level regressions that render every step across every conditional branch: each control's caption, id uniqueness and stability, re-rendering toggles and accordions, wrapped checkboxes and the group name. They fail on the previous wizard.js and on swapped, missing or duplicated ids. Validation: node --test tests/site/*.test.cjs (47 pass). Headless Chromium audit of 10 scenarios x 9 steps (all risk tiers, beginner and expert) on the served site, EN and TR: 375/375 controls labelled, accessible names from label[for] (none empty or placeholder-derived), real-mouse caption clicks focus their control, Tab reaches every control, no duplicate ids or exceptions; the first invalid field announces its caption. Fixes #950 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
testing.md forbids `|| true` in CI, but the only failure-masking check,
test_no_gauntlet_short_circuits_on_failure, read the three documented
gauntlets and only for the literal `|| true`. Appending `|| true` to the
validation job's `node --test` step left all 16 wiring tests green, and so
did `|| :`, `|| exit 0`, `| tee` without pipefail, `set +e`, a step-level
`continue-on-error`, and `cmd && echo ok` followed by another line, whose
failure bash's errexit ignores. The gauntlet check itself accepted
`{ pytest tests/ || true; }` once all three documents carried it.
Add tools/check_ci_failure_masking.py. A small POSIX-shell lexer and
parser read every workflow and composite-action `run:` block and the three
gauntlets, so comments, quoted prose, `python -c` programs and
here-documents are never commands. For every required command - pytest,
node --test, ruff, mypy, the forgelm CLI and the tools/ guards - it proves
a non-zero exit fails the step: no `||` fallback that can exit 0, no pipe
without pipefail, no negation, background, command substitution,
conditional branch, loop that may skip it or shell function, no errexit
gap, no continue-on-error, and a shell it can model. A no-op `|| true`
anywhere else is refused, except bandit/pip-audit immediately followed by
their severity helper reading the same report.
Wire it into ci.yml's validate job and all three gauntlets (31 -> 32),
document the rules in testing.md, and add tests with negative mutations
of the real workflows and gauntlets, including the issue's reproduction.
Mutating the guard (10 variants) turns its tests red.
Validation: guard clean on the checkout (176 required invocations across
5 workflows and 3 gauntlets); tests/test_guard_wiring.py and the new
tests (119 pass); ruff; doc-count, source-path, anchor and parity guards.
Fixes #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…copy Make the verifiable corrections in the monitoring-plan and webhook copy that #946 rewrote in all six locales, ahead of the native review #948 asks for. - DE/ES named the provider's Article 72 obligation with terms the AI Act does not use. Use its own: "Beobachtung nach dem Inverkehrbringen" and "vigilancia poscomercialización" (FR already used "surveillance après commercialisation"), as worded in Art. 72 on the Commission's AI Act Service Desk. - FR/ES dropped "metadata" from the hint, blurring ForgeLM's optional record with the provider's system; restore it. - DE used both "Überwachung" and "Monitoring" for the external stack; keep the site's "Monitoring". FR and TR mixed curly and straight apostrophes within a value; use the straight form the rest of each locale uses. The ZH webhook text alone used ASCII punctuation and "endpoint"; use the full-width forms and 端点 of the surrounding copy. - TR docs: replace a leftover "post-market monitoring" with "pazar sonrası izleme", "EU Genel Veri Koruma Tüzüğü" with "AB ...", and the curly apostrophes #946 introduced. Record Yasası versus Tüzük, post-market monitoring and drift in the localization glossary. The new validation and interval keys keep their English fallback, per the deferred-tier policy. Pin the metadata / Art. 72 / Annex IV point 9 distinction, the official terms and the punctuation in the wizard tests; both new tests fail on the previous translations. Validation: node --test tests/site/*.test.cjs (49 pass); site chrome, bilingual spine, TR-mirror link, self-contained manual, anchor and site claims guards. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opus review of baf98de (no blockers); verified fixes: - DE cites annex points as "Nummer" (Art. 43(1): "Anhang III Nummer 1"): "Annex IV Nummer 9". FR/ES gain the article a sentence subject needs ("l'Annex IV", "el Annex IV"), as the site already writes elsewhere. - DE/FR/ES give "YAML" the gender each locale uses everywhere else (das / le / el YAML) in both webhook toggle strings. - TR: restore the wizard's informal register in the Operations step description, write "metadata'sını" like the rest of the TR copy, use "aktarım sistemi" for the metrics exporter in both configuration references, "AB" instead of "EU" in two QMS SOPs, and drop the English "monitoring" from the compliance page's Article 9 card. - Glossary: describe how TR pages actually name the EU AI Act, widen the "kayma" note to the threshold shift the guides also call that, and add the metrics-exporter row. - Tests: accept the forms a native review may choose (Anhang IV, l'annexe IV, 附件四, Ek IV; 上市后监测; piyasaya arz sonrası izleme) and ignore citations such as 17(1)(h) and code tokens such as endpoint_env in the Chinese checks; name them for what they check. Both still fail on the translations before baf98de. - CHANGELOG: write for users, name the interval key, and limit the Chinese and apostrophe claims to the copy actually changed. Left for the native-review packet on #948: Anhang IV localisation, FR apostrophe typography, ES variety, ZH site-wide punctuation, citation styles, and the EN "detailed plan" wording of Annex IV point 9. Validation: node --test tests/site/*.test.cjs (49 pass); site chrome, bilingual spine and code-block, TR-mirror link, schema-drift, YAML snippet and anchor guards. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opus review of 71beb85 and d03d851 (no blockers); verified fixes: - Keyboard activation of a control that re-renders the step (gate and feature toggles, the risk select, option cards) dropped focus to <body>, so revealing conditional fields cost a keyboard user their place. All such controls now carry stable ids (checkboxes keep their wrapping label; cards use controlId(field) + '-' + option) and render() refocuses the rebuilt control with the same id; a step change finds no such id. - "required for risk tier" badges ran into the caption in the accessible name ("…evaluationrequired…" on locked toggles); a text space keeps them separate words whatever the label's CSS display. - appendDetail now builds every per-field detail paragraph (makeToggle and the max-loss row built theirs inline); comments no longer claim accordions re-render. - Tests: render every step after a failed Next (error rows), assert no aria-label/aria-labelledby hides a caption, require ids on every control that re-renders the step, pin the badge spacing, share hasClass/isElement through the harness. A planted undefined call in an error branch, an aria-label override, a removed badge space and a removed checkbox id now each fail the suite. - Docs: the YAML round-trip test, not the shared harness, picks the Python interpreter; test-module counts 128 -> 129 in CONTRIBUTING.md and the EN/TR architecture reference. Validation: node --test tests/site/*.test.cjs (51 pass). Headless Chromium with real key events: Space on the benchmark, judge, webhook and GaLore toggles, ArrowDown on the risk select and Enter on a trainer card keep focus, and Tab then reaches the revealed field; Next on an invalid step still focuses its first invalid field; locked-toggle and meta-input names read "... required for risk tier". Full audit (10 scenarios x 9 steps): 375/375 labelled, no duplicate ids or exceptions. Refs #950, #879 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Opus review of b16f23d (no blockers, four majors), every shape checked against bash 5.2 in a container with always-failing stub commands: - `exit $?` / bare `exit` re-raise only directly after `||`; inside a group `$?` is the previous command's status, so `|| { echo …; exit $?; }` (which testing.md recommended) masked the failure. An earlier `exit` that may leave 0 now counts, and only a literal code or a status captured first (`|| { rc=$?; …; exit "$rc"; }`) re-raises. Docs fixed. - Nested shells parse flag clusters that take an option name (`bash -euo pipefail -c '…'` was skipped entirely, which let a step disable the guard itself), read `bash ci/x.sh` scripts from the repository, and flag scripts piped into a shell. - errexit is modelled as bash applies it: a `{ … }` group's status after a failure ignored inside it does not end the shell, `set -e` cannot undo an ignored context, and `set +e` in an item counts before its command (also via `command`, `builtin` and `shopt -o`). - `.github/workflows/*.yaml` are scanned (also by test_guard_wiring.py), and `uses:` of a local composite action that runs a required command must not carry continue-on-error. - Fail closed on what it cannot follow: `${{ }}` literals that paste shell syntax or supply the command, required commands named behind unknown wrappers (`$PY -m pytest`), EXIT/ERR traps that exit 0, `eval`, `for` loops without a fixed word list, and substitutions inside `${…}`, `[[ … ]]`, unquoted here-documents, nested shells and `find -exec`. `coverage run`, `stdbuf`, `xvfb-run`, `uv run --with …` and `env --chdir` are modelled. - Parse `2>&1` after compounds, `(( … ))`, C-style `for`, `;&` and `$(case …)`; unparseable steps fail closed only when they name a required command. Deep nesting, malformed YAML and non-mapping shapes are findings, not tracebacks; has_required is memoised. - Docs no longer claim completeness: the guard accepts only shapes it can prove (capture-and-reraise, PIPESTATUS, wait and every-branch forms are rejected and documented); custom shells are bash, sh or dash; the continue-on-error rationale comment is a review rule. code-review.md now agrees with testing.md on continue-on-error. Validation: 133 differential cases vs bash 5.2: 0 false negatives, 0 crashes (18 documented conservative rejections); 180 guard/wiring tests; 11 of 12 guard mutations fail the suite (the remaining one is equivalent on realistic input); guard clean on the checkout (176 invocations). Refs #951 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntroller Address the second wizard review round: - The auto-revert toggle only refreshed the YAML preview, so "Max acceptable loss" stayed hidden after turning it on and stayed visible after turning it off until something else re-rendered the step. The Compliance crash fixed in #879 had hidden this. makeToggle now takes a `reveals` flag and re-renders for toggles that show or hide fields. - render() keeps the caret, not only focus, when another tab's change re-renders the field being edited. - `.wizard-slider` sets `outline: none`; sliders now show a `:focus-visible` ring. - A controller harness (bootWizard) runs initWizard against a modal skeleton, so step bookkeeping (#879), focus restoration, Next's focus on the first invalid field and the storage handler are tested. Render tests cover the compliance toggles' detail rows, the error rows after a failed Next, and toggles that reveal fields. Removing any of these behaviours fails a test. - testing.md lists the real-browser checks a wizard markup, focus or style change still needs, since the DOM shim cannot see them. Verified in Chrome for Testing 156 (headless, over CDP): each revealing toggle and the auto-revert toggle keeps focus and Tab reaches the field it reveals; single-letter type-ahead on the risk select re-renders and keeps focus; the slider ring shows on keyboard focus; a storage event keeps focus and caret; every Tab stop of all nine steps, in two scenarios at 1280 and 375 px, shows a focus ring and no exceptions. Refs #950, #879 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Annex IV point 9 asks for "a detailed description of the system in place to evaluate the AI system performance in the post-market phase ..., including the post-market monitoring plan"; "detailed" describes the description, not the plan. The English and Turkish monitoring hints now say so. The German, French, Spanish and Chinese hints keep "detailed plan" until their native review. - The German monitoring hint used "dokumentieren", the verb Art. 72(1) gives the provider's own duty. It now "deklariert", like the English hint and the rest of the German monitoring copy. - Two edited Turkish paragraphs spelled "metadatası"; the rest of the Turkish copy writes "metadata'sı". - The Turkish site cited "Regulation (EU) 2024/1689" in English; it now uses the glossary's "(AB) 2024/1689 sayılı Tüzük". One Turkish reference page wrote "AB YZ Yasası"; it now says "EU AI Act", the form every other Turkish reference page uses. - The glossary said *kayma* is used only for a schema or threshold shift, but the manuals also use it for a preference shift. The row now lists all three uses. - The Chinese webhook toggle hint used an ASCII comma beside the delivery note's full-width punctuation in the same row; both are full-width now, and the test covers both strings. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Traps. `$?` is the status a trap was entered with only in the action's
first command; after `echo bye`, `exit $?` exits 0. The guard accepted
`trap 'echo bye; exit $?' EXIT` and rejected the correct
`trap 'rc=$?; …; exit $rc' EXIT`. It now models both, strips `trap --`,
follows functions the action calls (passing on whether `$?` still holds
the entry status), and rejects a required command inside a trap and an
action taken from a variable.
Repository scripts. `./ci/x.sh` (with its shebang's options), `source`
and `.` are followed like `bash ci/x.sh`; a file outside the repository,
such as `.venv/bin/activate`, is left alone. Findings name the script.
Redefinitions and dead ends. A function, alias or `hash -p` that takes
over a required command, or one the analysis relies on (`exit`, `false`,
`set`, a wrapper or a shell), is rejected, as is bash's
`command_not_found_handle` and a required command after an unconditional
`exit` that can pass or an `exec`.
Wrappers and lexing. `uvx`, `uv tool run`, `pipx run`, `env -S`,
`ionice` and `taskset` keep the program's status; unknown wrappers are
caught wherever they put the program, while commands that never run
their arguments (`grep`, `mkdir`, `rm`, ...) are not. `((pytest) ||
true)` is two subshells, not arithmetic; substitutions inside
arithmetic, array assignments and C-style `for` headers are seen;
`case` counts only in command position; `time { … }` parses;
`set -- +e` sets parameters; `set $OPT`, `shopt -uo errexit` and
`eval 'set +e'` count as turning errexit off. `|| true` is refused in
command substitutions and trap actions too.
False positives. An installer piped from `curl`/`wget`, `sh -s`,
`bash -n`, and a `${{ }}` path prefix such as
`${{ runner.temp }}/venv/bin/pip` are no longer findings.
Workflows. Composite actions are found under any `.github` directory
and wherever a `uses: ./path` points, and a missing one is a finding.
Steps and jobs with a literal `if: false` count for nothing. A matrix
value that pastes shell syntax is rejected. Unreadable files, missing
gauntlet documents and YAML the loader cannot construct are findings
instead of tracebacks. `ci.yaml` keeps the inventory check, and the
first closing fence ends a gauntlet block.
Tests pin every case above. 26 shapes are also run under `bash -e`
with a failing `pytest` stub when bash 5 is available (as on GitHub's
Linux runners); all 26 agree with bash 5.2.21. A further 4,500
grammar-fuzzed scripts produced no false negative or crash against
bash 5.2, and 80 realistic CI snippets raise one finding: an unguarded
`pytest --collect-only | tail -1`. testing.md now names the two
scanner/helper pairs the exception accepts and the guard's limits.
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The wiring test checks only that a gauntlet `|| true` sits inside a
`{ …; }` group, and its message told contributors to "wrap it as
'{ cmd || true; }'" whatever the command. The failure-masking guard now
rejects `{ pytest tests/ || true; }`, so the hint led straight into the
next failure. It now says only a scanner followed by its severity helper
may use `|| true`, and the docstring names the guard that checks the
rest.
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The real-repository tests copy the workflows and gauntlet documents into a temporary root, mutate them and expect a finding. Two of them checked only the exit status, so a finding the copy alone produced, such as a repository script a workflow runs but the copy lacks, would have satisfied them however the mutation was handled. The copy helper now asserts that the copied tree scans exactly like the checkout, and those tests check the finding their mutation should cause. Refs #951 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three-document gauntlet test looked for the document name and the masking message anywhere in the output, so one document's finding could satisfy the check for another. Each document now needs its own finding line naming it and the masked `pytest tests/`. Refs #951 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The owner, a native Turkish speaker, settled the Turkish items of the native-review packet: - post-market monitoring is "piyasaya arz sonrası izleme", after the Turkish legal term for placing on the market, in place of "pazar sonrası izleme"; - an Annex IV point is "Annex IV §9", like the existing "Annex IV §1" strings, because "madde" means Article and sits next to "Madde 72"; - the metrics exporter stays "exporter", like endpoint and webhook, in place of "aktarım sistemi"; - a monitoring alert is "alarm", a webhook's chat message is "bildirim" and a log warning is "uyarı"; all three used to be "uyarı", and one card used the English "alert'ler"; - Turkish uses the straight apostrophe; six wizard error messages used ’. The site strings, the Turkish guides and references, and the glossary in localization.md follow these. A test pins them, and each of four reverted strings fails it. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ates The model-id and dataset-name input handlers meant to re-render once a field Next had flagged became valid, but they computed the "before" state with fieldHasError() after storing the new value. It always equalled the "after" state, so `prevErr && !stillErr` never held and the red border, aria-invalid="true" and the error row stayed until something else re-rendered the step. Each handler now keeps the error row it rendered and, once the value validates, drops the class, sets aria-invalid="false" and removes the row in place, as the Compliance fields do. No re-render means the step is not rebuilt or re-announced, and focus and caret stay put. A value that is still invalid keeps its error. A controller test covers both fields; disabling either check, or the aria-invalid update, fails it. Verified in Chrome for Testing 156 by typing into both fields after a failed Next. Fixes #952 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each quickstart step is a grid with `grid-template-columns: auto 1fr`. A `1fr` track cannot shrink below the min-content width of what it holds, so the longest `white-space: pre` line of a step's terminal block widened the column instead of scrolling inside `.code-block-body`. From 320 to 768 px wide, quickstart.html laid out about 990 px wide and the menu button sat off screen; at 1280 px the fourth step card was 893 px wide in an 880 px container while the others were 810 px. `minmax(0, 1fr)` lets the column shrink, as the wizard's grids already do. Checked in Chrome for Testing 156 at 320, 375, 414, 768 and 1280 px: the page is never wider than the viewport, the menu button stays on screen, every step card has the same width, and long lines scroll inside their code block. index.html is unchanged. Fixes #953 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n module The lexer, the parser, the command-tree dataclasses and the two tree walkers move from tools/check_ci_failure_masking.py to tools/_shell_syntax.py, with public names (Simple, Script, parse, children, ...). The guard imports them the way check_bilingual_code_blocks.py imports check_bilingual_parity.py. The guard drops from 2,345 to 1,613 lines; the module is not a CI guard, so the guard inventory stays at 32. No behaviour change: the guard reports the same 176 invocations across 8 surfaces, and its tests pass unchanged. Refs #951 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checked against bash 5.2, the guard missed these masking shapes:
- an ERR trap whose bare `exit` follows another command (on ERR bash exits
with that command's status; only EXIT keeps the entry status), and an
ERR trap that turns errexit off;
- a tool runner given a package spec (`uvx ruff@0.8.0`, `pipx run
ruff==0.8`), `python -m coverage run` and `conda run`;
- `ruff --exit-zero` and `-e`;
- substitutions in a compound command's redirections (`done < <(cmd)`, a
here-document on a loop) and in `case` patterns, which the parser
dropped;
- `set -n`; a backgrounded `set -e` read as turning errexit on; `$'`
inside double quotes read as a quote; a captured status assigned again
before `exit "$rc"`; `source /dev/stdin <<EOF` and bash's lone `-`;
- scripts under `$GITHUB_WORKSPACE` or a step's `working-directory:`, and
a repository script run by path that it could not find;
- `continue-on-error` reaching a required command through a nested or
unusually spelled local action, `if: false` on a step or job that runs
one, and a workflow set without ci.yml.
It rejected correct shapes: `if ! cmd; then ...; exit 1; fi` and
`if cmd; then ...; else ...; exit 1; fi`, a `for` loop over a glob (it runs
at least once without nullglob), `exec {fd}>file`, `pytest --version |
head`, `python -m py_compile tools/...`, and installers read through
`bash -` or `bash <(curl ...)`.
Its caches were keyed by id() while trap actions were freed mid-scan, so
one script could get different verdicts; parsed trees now live until the
scan ends. Odd names (too long, a NUL byte, a symlink loop) and a relative
or symlinked --root give findings or work instead of tracebacks, finding
paths use forward slashes on every platform, and matrix values are read
within a budget.
The scanner exception's helper may take new options: it only has to be
given the report. A finding names the command whose failure skips a
required one (`cd x && pytest`), and line numbers say what they count in.
The bash comparison test pins 52 shapes, expects exit status 1 from the
safe ones and the named finding for the masked ones, skips on Windows,
and fails on CI's Linux runners if bash 5 is missing. testing.md lists
the rules and states the scope: accidental masking, not deliberate
evasion; `$CMD`, make/tox, docker exec/ssh, `eval "$(...)"` and remote
actions are beyond it.
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
code-review.md now says "required command", as testing.md and the guard do. coding.md's `|| true` row points at the one sanctioned exception, and documentation.md no longer counts four doc guards when more enforce it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A sibling tab's update re-rendered the wizard from storage without the open sections (`openAccordions` was missing from the defaults, so loadState dropped it) and without the steps this tab had flagged (`attemptedSteps` is wiped on load). The Compliance step's sections closed, its errors vanished and focus fell to the page behind the dialog. The open sections are now part of the saved state, so they survive a reload too, and the flagged steps stay in their tab: they are no longer saved, which would make two tabs overwrite each other in turn. After a re-render focus returns to the rebuilt control without scrolling, section headers have ids so focus returns to them as well, and a control the update removed leaves focus in the dialog. A flagged field's error now follows its value in place through one helper, used by all eleven validated fields: it goes once the value is valid and names the check the value fails now, so `abc` in the model field reads "format", not "required". The GRPO reward-model tip showed its `<code>` tags as text in every language; tip text now turns `<code>` spans into code elements, still without parsing the copy as HTML. Tests: the controller walks the steps with Next instead of seeding the step it checks, and covers the sibling-tab update, reloads and focus fallback; every gate shows and hides its fields; a new test checks that each rebuilt control shows the state it was built from, against the scenarios plus every boolean inverted and non-default choices. The harness scenarios now cover every option card, its header lists the DOM shim's limits, and caption keys must exist in English and Turkish. Refs #557, #683, #952 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On phones compliance.html laid out 347-375 px wide: its article cards used `1fr` tracks that grew to their longest file path, and inline code in prose could not break. The tracks are now `minmax(0, 1fr)`, evidence paths and inline code wrap, and the contact grid, the quickstart spotlight stages and the use-case rows get the same tracks. At 320 and 375 px no page in any of the six languages is wider than the screen. The wizard's locked toggles were squeezed to half their width beside a long caption, and their "required" badge had no style outside the cards. The switch keeps its size, and the badge is the same pill in captions, in a colour that reads at its size. A focused invalid field kept only its red border, so focus was hard to see; it now gets an outline. On the light theme the focus colour was 2.3:1 on white and the error text 3.8:1; they use the deep ember (5.2:1) and the darker red the contact form uses (6.5:1). Refs #953 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
English now cites Annex IV points as §N, as Turkish does and as most English pages and the code already did: the verify-annex-iv field table and its output, the ISO 27001 and statement-of-applicability rows, the user manual and the site hint. A code comment that mapped the performance metrics to §7 now names §2(g) / §4, as the verify module does. Annex IV §9 asks for a detailed description of the post-market monitoring system, its plan included. The configuration pages, the compliance overview and the German, French, Spanish and Chinese site hints said it "includes the detailed plan". German, French and Spanish now follow the Regulation's own wording; the Chinese line is listed for native review on #948. Turkish follows the glossary where it still did not: the incident SOP and the GPU-cost page call monitoring and cost alerts "alarm" (the glossary row now names threshold alerts), two pages say "alarm ver", "trend izleme" is one term, the Act is "EU AI Act" in full, and the site fixes a possessive and a comma. French sends "alertes", and the Chinese webhook card uses full-width punctuation throughout. A new test pins the glossary across the Turkish docs, reading the public Turkish trees by name; the site tests pin the §9 wording in every locale and the Turkish corrections. The test-module count is 130. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntry The locale entry carried six languages and eight topics; it is now three entries: the deferred-locale monitoring copy, what Annex IV §9 asks for, and the Turkish terminology. New entries cover the cross-tab state fix, the in-place error messages, the GRPO tip's code, the narrow-screen layouts and the wizard's contrast; the guard entry names its new rules and its scope; English Annex IV citations move to §N under Changed; and the #879 entry carries its finding id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cross-tab handler skips a closed wizard, so a tab whose wizard was closed while another tab saved progress kept its older copy, and reopening the dialog wrote that copy back over the newer answers. Opening the dialog now adopts the saved state first, through the same helper the cross-tab handler uses, keeping this tab's flagged steps. The test harness now models the dialog's `hidden` attribute and close button, so the controller tests can close and reopen it; the reopen test fails without the fix. The CHANGELOG says every validated field shares the in-place error update; there are nineteen, served by eleven call sites. Refs #683 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Checked against bash 5.2, the shapes this branch began to accept let
some failures through:
- an `if` gate whose passing branch runs `exit 0`, or whose failing
branch calls a function or `exec`s before its `exit 1`: no branch may
now leave early, a function call included;
- a `for` loop whose body can `continue` or `break` past its required
command, glob or not; the finding now names it;
- a repository script fed by `curl` (`curl … | bash ci/x.sh`): only a
script operand that is itself `<(curl …)` counts as a download.
And some failures it never saw: `exit 0`, `exec` or `set -n` inside a
sourced or `eval`'d script now stops the step as it does at top level;
`ruff -ne` and other combined flags count as `--exit-zero`; and a
captured status changed by `$((rc=0))` or `let rc--` no longer
re-raises. `working-directory: ${{ github.action_path }}` in a composite
action is resolved instead of reported.
Tests pin each rule, including the gate's branch rules, loop variables,
arithmetic, a here-document's `$'`, and a matrix budget that now fails
without the budget; five more shapes join the bash comparison.
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reopened wizard adopted the saved state whenever there was one, but when storage refuses writes it still holds the tab's old copy, so the answers typed since were replaced by older ones. The tab now remembers the stored text it is in step with (what it last saved, loaded or adopted) and adopts only when storage holds something newer; a test reopens the wizard with writes failing. Refs #683 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At 320 and 375 px the German, French and Spanish home pages laid out up to 401 px wide (a button label that could not wrap, pipeline stages sized to their longest word), the German privacy and terms titles did the same, and the quickstart spotlight cut off German and Turkish text. Buttons now wrap their labels on narrow screens, headings and pipeline stages wrap long words, and the pipeline and spotlight columns shrink. No page is wider than the screen at 320 or 375 px in any site language, and nothing changes at 768 or 1280 px. The CHANGELOG now says so, describes the Annex IV citation change by what it touches (the verify module's descriptions are not printed), and no longer credits the glossary with "trend izleme"; the Turkish-docs test reads nested folders and pins "trend izleme". Refs #953, #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ments
The lexer read the inside of `NAME=(…)` and of arithmetic as text that
is only expanded, skipping single-quoted parts, but it did not know
double quotes there: in `msgs=("don't" "$(pytest || true)")` the
apostrophe opened a quote that never closed, so the substitution, which
bash runs and whose failure it masks, was never read. Double-quoted
parts are now read as such. The shape joins the bash comparison.
Also from review: the scope in `testing.md` and the guard's docstring
now say which job conditions are beyond the guard (any but `if: false`,
which it reports); the `|| true` row of `coding.md`'s table escapes its
pipes, which split the row; and the Turkish-docs test fails when a
scanned tree has no Turkish page, instead of dropping it silently (it
no longer scans docs/roadmap/, which has none; roadmap-tr.md is read at
the top level).
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The discovery test now also expects product_strategy-tr.md, so renaming either top-level Turkish page fails it instead of narrowing the scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Read a substitution inside single quotes in a double-quoted `${…}` or a
here-document, where bash keeps the quotes and runs it, and read a lone
carriage return as part of a word, as bash does.
- Scan trap actions for substitutions, check DEBUG and RETURN traps that can
exit or turn errexit off, and reject an EXIT/ERR trap that runs `exec`.
- Stop at an `exit 0`, `exec` or `set -n` that bash always reaches: behind `!`
or `x=1 &&`, in a `{ … }` group or in a function the script calls. A
fallback group that can leave through `eval` or a function before its
`exit 1` masks the failure.
- A captured status changed through `eval`, `command read`, a nameref,
`getopts` or `wait -p` no longer counts as re-raised, and `exit ²` is no
exit code; it crashed the guard.
- Follow `source "$PWD/…"`, refuse a script named from `$(dirname "$0")` that
it cannot find, and report a script that runs itself instead of failing on
the recursion.
- Reject `if: failure()`, `if: cancelled()` and `if: ${{ !success() }}` on a
step or job that runs a required command, and `ruff --fix-only`; accept
`continue-on-error: ${{ false }}`.
- Accept a `continue` or `break` after the required command or inside a
nested loop only, and name what disqualifies a rejected `if` gate.
- Every accepted shape in the tests must name a required command, a test
covers the wiring inventory's `.yaml` workflows, symlink tests skip where
symlinks need privileges, and 75 shapes are now checked against bash 5.2.
Refs #951
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rror back - A sibling tab's update kept this tab's open sections only when both tabs had the same ones open: open sections were saved and adopted with the other tab's answers. They now stay with their tab, like the flagged steps, and are never saved, so two tabs cannot overwrite each other's. - A validated field whose corrected value is broken again shows its error again, in place and where a render puts it; the row used to stay away until the next Next. - Another tab clearing all site storage resets the wizard, and a saved step that is not a step number starts it over instead of leaving the pane blank, with Reset out of reach. - Drop a dead parameter, and the comments about a storage flag nothing read and reduced-motion handling the script never did. - Tests cover all 19 validated fields, every control saving what it sets, the sections Next and their headers open, focus outside the step left alone and a refocus without scrolling, storage resets, the Reset button, every detail paragraph a renderer names, the custom-model card, the whole `<code>` detail line, and every free-text field showing a value of its own. Refs #950, #952, #683 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as code - `overflow-wrap: anywhere` on inline code, article evidence and spotlight snippets applied at every width, so on desktop a Chinese "、/system", which has no break opportunity before the slash, split as "/syst|em". It now applies on phones only, and wider screens keep code whole as before. - At 320 px a terminal block's copy button was clipped: the step card takes less padding on phones, and a code block's header wraps its button below a long translated label. - The light theme's section-header focus ring was 2.26:1, as the fields' was; it takes the deep ember shade too (5.18:1). - Four page strings with `<code>` or `&` were set as plain text, so the tags showed in every language on the home, features, quickstart and compliance pages. They use `data-i18n-html` now, and a test keeps markup out of the strings a page shows as plain text. Refs #953, #950 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… claim `technical_documentation` holds the ForgeLM version, the generation time and the known limitations, yet the verifier's field table, the writer's comment and the EN/TR reference and user-manual tables called it §3-5, "design and development methodology". Annex IV §1(c) asks for software versions and §3 for capabilities and limitations. The Annex IV page said §7 of the bundle is `performance_metrics`. Annex IV §7 lists the harmonised standards applied; the bundle has no key for the §8 copy of the EU declaration of conformity, which the provider, not the deployer, draws up under Article 47. The verifier discards the descriptions, so no output changes. Refs #539 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ring copy - Eight "Ek IV" become "Annex IV", as the glossary decided, and two "piyasaya sunulmuş" become "piyasaya arz edilmiş", the legal term for placing on the market. - The glossary test now also keeps out "Ek IV", "piyasaya sunul…", a drift, threshold, cost or webhook "uyarı", other spellings of the Act, and any word for the Regulation but "Tüzük". Its docstring and the CHANGELOG no longer claim that it pins every decision: one that needs judgement in general prose stays with review. - The monitoring test pins, in every locale, the plan Annex IV §9 includes, the high-risk providers Art. 72 binds and that the declaration alone does not discharge it, plus French "alertes" and the YAML's article in French and Spanish. Its comment cites §9, and the ForgeConfig check gets 120 s on a loaded host. Refs #948 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #539 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…i-masking-locale-copy Fix the wizard's Compliance step and field labels, guard CI against failure masking, correct monitoring copy
|
Sorry @cemililik, your pull request is larger than the review limit of 150,000 diff characters |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage |  |
Docstring coverage is 29.74% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 232 functions across 21 files. (59 skippe… | Write docstrings for the functions missing them to satisfy the coverage threshold. |
-
Check name Status Explanation Title check The title clearly summarizes the main changes: declarative monitoring evidence, browser wizard updates, and the CI failure-masking guard. Description check The description is comprehensive and covers the summary, related issues, change details, type, testing results, checklist, breaking changes, and follow-ups. It does not include a separate Changes head… Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Full details: Description check
-
Explanation
The description is comprehensive and covers the summary, related issues, change details, type, testing results, checklist, breaking changes, and follow-ups. It does not include a separate Changes heading, but the Summary section provides the required change details.
-
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 29.74% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 232 functions across 21 files. (59 skipped: 58 unsupported, 1 too large.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
✨ Finishing Touches: 💡 1
-
📝 Generate docstrings: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- 🤖 Coding Agent task started for unit test generation.
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Warning
Some tools did not complete. Review the errors below.
🔧 ast-grep (0.45.3)
-
site/js/translations.js
-
ast-grep timed out on this file
-
Comment @coderabbitai help to get the list of available commands.
|
Starting docstring generation with the Coding Agent. The task will commit the result to the current branch. |
|
Open the task to resolve the delivery issue or retry. |
|
Note Unit test generation is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it. Generating unit tests... This may take up to 20 minutes. |
|
🤖 Coding Agent task started for unit test generation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promote two reviewed pull requests from
developmenttomain. This is not a release:pyproject.tomlstays at0.11.1rc1, the CHANGELOG entries stay under[Unreleased], and no tag follows. Like #934, it bringsmainup to the reviewed integration branch.monitoring.*declares an external plan labelleddeclarative_only: true, so evidence no longer suggests that ForgeLM monitors anything; config validation, JSON evidence, the CLI and browser wizards and the EN/TR docs agree.tools/check_ci_failure_masking.py, with its shell parser intools/_shell_syntax.py), corrects the monitoring copy in six languages, applies the owner's Turkish terminology, and keeps the site within a phone's width.Source:
6401de10e41f4917e54e5dc6edb4239f591f032a. Base at preparation:85a4d9b, which is also the merge base, somainholds no commit thatdevelopmentlacks. #945, the earlier synchronization ofmainintodevelopment, adds no tree changes. 81 files, +7,641/−607.Related issues
Closed when they merged into
development: #93 and #231 (through #946); #879, #950, #951, #952 and #953 (through #954).Partly resolved and still open; each pull request names the findings it covers:
technical_documentationlabel and the Annex IV page's §7 sentence (parts of CR-A07-062 and DOC-BLEGAL-018); the Art. 9 citations remain.Follow-ups opened during #954: #955, #956, #957, #958, #959, #960, #961 and #962.
Notes for reviewers
Main: preserve branch historyruleset requires. Afterwards, bringmainback intodevelopmentwith a synchronization pull request, as chore: synchronize main history into development after #944 #945 did, sodevelopmenthas the merge commit too.[Unreleased]carries two Breaking entries from fix: restore the declarative monitoring evidence contract #946.monitoring.endpoint_envmust be a trimmed ASCII identifier, and a non-blankmonitoring.endpointmust be an absolute HTTP(S) URL; configs that relied on the looser parsing now fail validation. They reach users with the next release. Underrelease.md's pre-1.0 rule a breaking change needs at least a minor bump, so that release should be0.12.0rather than0.11.1.Type
Testing
pytest tests/passes: CI on6401de1reports 5,658 passed, 22 skipped on Python 3.10, 3.11, 3.12 and 3.13.ruff check .andruff format --check .pass (CIlint).validatejob (the 32 guards and the config dry run) andusermanuals-validatepass on6401de1.5111d58, thedevelopmenthead before Fix the wizard's Compliance step and field labels, guard CI against failure masking, correct monitoring copy #954.development.Checklist
docs/standards/coding.md[Unreleased]🤖 Generated with Claude Code
Summary by CodeRabbit
Changed
Improvements