fix: declarative monitoring evidence, wizard Compliance step and CI failure-masking guard

.github/workflows/ci.yml

@@ -161,6 +161,11 @@ jobs:
     steps:
       - uses: actions/checkout@v5
 
+      - uses: actions/setup-node@v7
+        with:
+          node-version: "24"
+          package-manager-cache: false
+
       - uses: actions/setup-python@v6
         with:
           python-version: "3.11"
@@ -181,6 +186,23 @@ jobs:
       - name: Import-origin guard
         run: python3 tools/check_import_origin.py --strict
 
+      # testing.md forbids `|| true` in CI, but only the documented gauntlets
+      # were checked, and only for the literal `|| true`: appending it to the
+      # Node test command below left every wiring test green. This guard
+      # parses every run: block and the three gauntlets with a small shell
+      # lexer, and fails when a required command (pytest, node --test, ruff,
+      # mypy, the forgelm CLI, a tools/ guard) can fail without failing its
+      # step: an `||` fallback that can exit 0, a pipe without pipefail,
+      # set +e, continue-on-error, or a failure errexit ignores inside an
+      # &&/|| list or a { } group. bandit and pip-audit keep `|| true` only
+      # when the next command is their severity helper reading the same
+      # report.
+      - name: CI failure-masking guard (strict)
+        run: python3 tools/check_ci_failure_masking.py --strict
+
+      - name: Browser wizard regression tests
+        run: node --test tests/site/*.test.cjs
+
       - name: CLI smoke checks
         run: |
           forgelm --version

AGENTS.md

@@ -85,7 +85,7 @@ ForgeLM/
 │                            # export, inference, results, utils, __main__,
 │                            # _pypdf_normalise, _script_sanity,
 │                            # _strip_pattern
-├── tests/                   # 128 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current)
+├── tests/                   # 130 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current)
 ├── tools/                   # CI guards: check_anchor_resolution,
 │                            # check_bilingual_parity, check_cli_help_consistency,
 │                            # check_field_descriptions, check_no_analysis_refs,
@@ -166,12 +166,14 @@ Default workflow for a non-trivial change:
    ```bash
    python3 tools/check_import_origin.py --strict && \
      ruff format . && ruff check . && pytest tests/ && \
+     node --test tests/site/*.test.cjs && \
      python3 -m forgelm --config config_template.yaml --dry-run && \
      python3 -m mypy --strict --follow-imports=silent forgelm/__init__.py forgelm/_version.py tests/typing/public_surface_probe.py && \
      python3 tools/check_field_descriptions.py --strict forgelm/config.py && \
      python3 tools/check_http_discipline.py && \
      python3 tools/check_no_mutation_artifacts.py --strict && \
      python3 tools/check_strict_json_writers.py --strict && \
+     python3 tools/check_ci_failure_masking.py --strict && \
      python3 tools/check_bilingual_parity.py --strict && \
      python3 tools/check_bilingual_code_blocks.py --strict && \
      python3 tools/check_anchor_resolution.py --strict && \
@@ -215,7 +217,7 @@ Default workflow for a non-trivial change:
    does not cover the `tools/check_*.py` guards that import `forgelm`
    with `sys.path[0] == tools/`.
 
-   All 31 must pass — the exact set `.github/workflows/ci.yml`
+   All 32 must pass — the exact set `.github/workflows/ci.yml`
    runs, held there by `tests/test_guard_wiring.py`, which now compares the
    two inventories in **both** directions (the usermanual-schema-drift guard —
    `check_usermanual_schema_drift.py --strict` — validates that every
@@ -346,6 +348,25 @@ Default workflow for a non-trivial change:
    in-repo `blob/main/<path>` URL is checked by stripping the prefix, so
    converting a link to absolute form cannot trade a PyPI 404 for a
    universal one.
+   The CI failure-masking guard (`check_ci_failure_masking.py --strict`)
+   closes the gap the gauntlet's own `|| true` check left open. That check,
+   `test_no_gauntlet_short_circuits_on_failure` in
+   `tests/test_guard_wiring.py`, read only the three gauntlet blocks (in
+   CLAUDE.md, AGENTS.md and CONTRIBUTING.md), and only for the literal
+   `|| true`, so appending `|| true` to `ci.yml`'s `node --test` step left
+   every wiring test green — as did `|| :`, `|| exit 0`, `| tee` without
+   `pipefail`, `set +e`, a step-level `continue-on-error`, or
+   `cmd && echo ok` followed by another line (bash's errexit ignores a
+   failure inside an `&&` / `||` list). The guard parses every workflow
+   `run:` block and this block with a small POSIX-shell parser
+   (`tools/_shell_syntax.py`), so comments, quoted prose and `python -c`
+   programs are never read as commands, and checks that a failing required
+   command — `pytest`, `node --test`, `ruff`, `mypy`, the `forgelm` CLI or a
+   `tools/` guard — still fails its step. It accepts only the shapes it
+   models, so keep required commands plain. It catches accidental masking,
+   not deliberate evasion; `docs/standards/testing.md` lists what is beyond
+   it. The bandit / pip-audit `|| true` exception holds only when the very
+   next command is the severity helper reading the same report.
 
 ## Branches, issues and the project board
 

CHANGELOG.md

@@ -4,8 +4,40 @@ All notable changes to ForgeLM are documented here.
 
 ## [Unreleased]
 
+### Breaking
+
+- Tighten `monitoring.endpoint_env` to trimmed ASCII identifiers; URLs, `$`/`env:` prefixes,
+  internal whitespace, and non-ASCII names now fail validation. Blank names remain absent.
+  This changes acceptance of previously valid YAML; use a variable name, such as `MONITORING_PLAN_URL`.
+- Require non-blank `monitoring.endpoint` literals to be absolute HTTP(S) URLs with a host
+  and valid port; schemeless addresses are rejected instead of silently becoming `<malformed-url>`
+  in evidence. Use `https://monitor.example.com:8443/plan` or an `endpoint_env` name (#93).
+
 ### Changed
 
+- Record monitoring default provenance in `defaulted_fields` and mark literal URL projection with
+  `endpoint_redacted` (also true when the URL already contains only scheme/host/port). Literal
+  endpoints retain scheme/host/port in JSON evidence, including IPv6 brackets; userinfo, path,
+  query and fragment are removed. Env references remain unresolved. Absent endpoints record `null`
+  instead of the legacy `"$None"`, and non-empty `endpoint_env` takes precedence.
+  Monitoring metadata is exported to `compliance_report.json` and, when emitted,
+  `annex_iv_metadata.json`; the flattened `training_manifest.yaml` summary does not contain it.
+  Monitoring declarations remain optional in ForgeLM at every risk tier, including strict tiers;
+  providers’ Article 72 obligations apply independently. Treat legacy monitoring blocks without
+  `declarative_only` as declarations; their default provenance was not recorded (#93).
+- Default CLI/browser wizard drift-alert answers to No / unchecked; only declare alerts when
+  the operator confirms them. Preserve the schema's `true` default with `defaulted_fields`
+  provenance. Both wizards explicitly record the selected `none` exporter (#93).
+- Accept case-insensitive `env:` references with surrounding whitespace, lowercase letters and
+  initial underscores in both CLI wizard monitoring and webhook inputs; reject malformed names.
+- Cite Annex IV points as `§N` throughout, as the Turkish pages and most English ones already did:
+  the EN/TR `verify-annex-iv` reference and user-manual field tables (`Annex IV §2(c)`, `§9`,
+  `§2(g) / §4`, `§5`), the ISO 27001 mappings, the wizard hint, and the verify module's own field
+  table and comments (refs #948).
+- Omit Pydantic `input_value` / `input_type` excerpts from all rendered `ForgeConfig` validation
+  errors, preserving field paths and reasons. `MonitoringConfig` also applies this protection
+  standalone; other standalone submodels retain their existing diagnostics (#93).
+
 - Redesign the root README around a GPU-free first run, a concise capability overview,
   optional evidence controls, and clear contribution and documentation paths. Reuse the
   site's ForgeLM mark and distinguish the Apache-licensed code from CC-BY-SA seed datasets.
@@ -15,6 +47,93 @@ All notable changes to ForgeLM are documented here.
 
 ### Fixed
 
+- Use the Regulation's own Article 72 terms for post-market monitoring in the German and Spanish
+  wizard copy ("Beobachtung nach dem Inverkehrbringen", "vigilancia poscomercialización"), and restore
+  the "metadata" the French and Spanish hints had dropped, so every locale again separates ForgeLM's
+  optional record from the provider's monitoring system and the Annex IV material. Correct citation,
+  article and gender slips in the touched German, French and Spanish strings, and use one German term
+  for the external stack. The new validation and interval keys keep their English fallback; native
+  review of the deferred locales remains open (refs #948).
+- Say what Annex IV §9 asks for: a detailed description of the post-market monitoring system, its
+  plan included. The wizard hint in every locale and the EN/TR configuration and compliance pages said
+  it "includes the detailed plan"; German, French and Spanish now follow the Regulation's wording, and
+  the Chinese line awaits native review. French webhook copy says "alertes", and the Chinese webhook
+  card uses full-width punctuation and 端点 throughout (refs #948).
+- Label the Annex IV `technical_documentation` key §1(c) / §3, the software version and the known
+  limitations it holds, instead of §3-5 "design and development methodology", in the verifier's field
+  table, the writer's comment and the EN/TR reference and user-manual tables. The Annex IV page no
+  longer calls `performance_metrics` the bundle's §7: no key holds the copy of the EU declaration of
+  conformity that §8 asks for, which the provider draws up under Article 47. Only descriptions and
+  comments change, here and in the `§N` citations above: no output of the verifier or the writer does
+  (refs #539).
+- Apply the owner's Turkish terminology across the site and the Turkish docs: "piyasaya arz sonrası
+  izleme" (and "piyasaya arz" for placing on the market), "AB" and the Regulation as "(AB) 2024/1689
+  sayılı Tüzük", "Annex IV §N" (never "Ek IV"), "exporter" kept in English, "alarm" for a monitoring
+  or threshold alert, "bildirim" for a webhook message and "uyarı" for a log warning, "EU AI Act" in
+  full, and the straight apostrophe throughout; "trend izleme", the user manual's term, replaces
+  "trend takibi". The `localization.md` glossary records the owner's decisions; tests pin them in the
+  site strings and keep their superseded spellings out of the Turkish docs (refs #948).
+- Name every browser-wizard field by its visible caption. Text, number, slider, select and textarea
+  captions now point at their control through unique, stable `id`/`for` pairs, so screen readers
+  no longer announce a placeholder or nothing, and clicking a caption focuses its field. The LoRA
+  method caption names its group of method buttons; checkbox labels keep wrapping their toggles.
+  Keyboard focus now stays on the toggle, select, option card or section header that re-renders a
+  step, and on the field being edited, caret included, when another tab's change re-renders it,
+  without scrolling; a control the re-render removes leaves focus in the dialog instead of the page
+  behind it. "Required for risk tier" badges read as a separate word, and sliders show a keyboard
+  focus ring (#950).
+- Keep the browser wizard's open sections and flagged errors when another tab updates it. The update
+  replaced this tab's state with the saved one, which holds neither, so the Compliance step's sections
+  closed and its errors vanished. Open sections and flagged steps now stay with their tab and are never
+  saved, so two open tabs cannot overwrite each other's. A wizard reopened in a tab whose copy is older
+  starts from the newer saved answers instead of writing its own copy over them; another tab clearing
+  all site storage resets it like the wizard's own Reset; and a saved step that is not a step number
+  starts the wizard over instead of leaving it blank (refs #557, #683).
+- Render the browser wizard's Compliance step again. It called a helper that only three other step
+  renderers declared, so every visit threw a `ReferenceError` before the Article 9 risk-assessment,
+  benchmark and LLM-judge controls appeared, and strict risk tiers could not complete the step.
+  The helper is now shared, and every step has DOM-level render coverage. Turning auto-revert on or
+  off now shows or hides its loss threshold at once (F-W1-SITE-01, #879).
+- Clear the browser wizard's error on a model id or dataset name as soon as the corrected value
+  validates. The input handlers read the "before" state after storing the new value, so the red
+  border, `aria-invalid="true"` and the error message stayed until something else re-rendered the
+  step. The error now clears in place, keeping focus and the caret in the field, and comes back in
+  place when a corrected value is broken again; while the value is invalid the message names the
+  check it fails now (`abc` reads "format", not "required"). All 19 validated fields share this
+  behaviour (#952).
+- Show the code in the browser wizard's GRPO reward-model tip as code; its `<code>` tags appeared as
+  text in every language. The same happened on the home, features, quickstart and compliance pages,
+  whose strings with `<code>` or `&amp;` were set as plain text; they now render as markup, and a test
+  keeps markup out of the strings a page shows as plain text.
+- Stop `quickstart.html` from scrolling sideways on phones and tablets. A step's grid column could
+  not shrink below its longest terminal line, so from 320 to 768 px wide the page laid out about
+  990 px wide, with the menu button off screen; on desktop one step card overflowed its container.
+  The column can now shrink, so a long line scrolls inside its code block, which already allowed it
+  (#953).
+- Stop the site from laying out wider than a phone. `compliance.html`'s article cards grew to their
+  longest file path and long inline code could not break; on the German, French and Spanish home
+  pages a button label and the pipeline stages, and on the German privacy and terms pages the title,
+  did the same, the quickstart spotlight cut off its German and Turkish text, and at 320 px a terminal
+  block's copy button was clipped. Columns now shrink (`minmax(0, 1fr)`), headings wrap long words,
+  and on phones inline code may break anywhere and buttons wrap their labels; wider screens keep code
+  whole. The contact grid and use-case rows get the same columns. At 320, 375, 768 and 1280 px no page
+  is wider than the screen or cuts content off in any of the six site languages; on desktop only the
+  quickstart steps, whose long lines now scroll inside their code blocks, and the compliance cards'
+  equal columns change.
+- Make the browser wizard easier to read and operate: a focused invalid field gets an outline instead
+  of keeping only its red border, the light theme's focus colour (2.3:1 on white) on fields, sliders
+  and section headers and its error text (3.8:1) give way to darker shades that pass, locked toggles
+  keep their size beside long captions,
+  and the "required for risk tier" badge is styled as a pill.
+- Restore the monitoring-plan evidence contract: `monitoring.*` declares an external plan,
+  labelled `declarative_only: true`; ForgeLM runs no exporter, drift detector, alert, or scheduler.
+  Warn once per config-file load for an enabled exporter declaration, without repeating during
+  pipeline stage validation. Align CLI/web wizards and EN/TR guidance, correct Annex IV monitoring
+  point 9 / risk-management point 5 labels, and verify exported declarations end to end (#93).
+- Correct webhook troubleshooting in the site and EN/TR manuals: delivery failures appear as
+  `forgelm.webhook` warnings, never `webhook_failed` audit events; use the actual
+  `allow_private_destinations` setting (#231). Clarify that `--offline` sets HF offline flags;
+  reporting, API judges, and webhooks require separate disconnected-run configuration (refs #100).
 - Rework the homepage terminal into a first-run guide with separately copyable installation and
   config-generation commands, an optional training step, and readable code in both themes and on
   narrow screens, with interface copy in all six site languages. Keep the dated CPU recording and
@@ -156,6 +275,18 @@ All notable changes to ForgeLM are documented here.
 
 ### Added
 
+- **`tools/check_ci_failure_masking.py`** (32nd CI guard) fails CI when a required command's failure
+  would not fail its step. The only failure-masking check read the three documented gauntlets, and
+  only for the literal `|| true`, so appending `|| true` to the validation job's `node --test` step left
+  CI green. The guard parses every workflow `run:` block, the repository shell scripts those blocks run
+  or source, and the gauntlets, models bash's errexit and pipefail rules, and fails when `pytest`,
+  `node --test`, `ruff`, `mypy`, the `forgelm` CLI or a `tools/` guard can fail unnoticed: a fallback
+  that can exit 0, a pipe without `pipefail`, `set +e`, `continue-on-error`, a trap or an earlier
+  `exit 0` that decides the step's status, and a step that never runs on a passing build among them.
+  It accepts only the shapes it models, so keep required commands plain; `docs/standards/testing.md`
+  lists them, and what is beyond a guard that targets accidental masking. Bandit and pip-audit keep
+  `|| true` only when the next command is their severity helper reading the same report. Its shell
+  parser lives in `tools/_shell_syntax.py` (#951).
 - Add a recorded CPU validation, data audit, and audit-log verification sample with EN/TR provenance
   and limitations; link it from the README and website, expose installation and dry-run quickstart
   commands in the hero, and make contributor entry points visible.