Repository navigation
docs: compliance and privacy wording, SBOM and Article 12/14, and the six-language site copy, described as the code behaves - #1000
Merged
Conversation
…of duties
The human-approval gate records who approved (FORGELM_OPERATOR, else the OS user) next to who trained, and writes
both to the audit log. It does not compare the two, and both identities are self-asserted. Several documents said
otherwise:
- the ISO 27001 mapping ("enforces trainer != approver") and the SOC 2 mapping ("approval gate operator != trainer");
- the Statement of Applicability row for A.8.2 ("non-trainer operator");
- the deployer guide, the change-management SOP and the user manual ("dual-controlled", "two distinct operator IDs
prove segregation of duties");
- the deployer guide's checklist, which said F-compliance-110 raises ConfigError if require_human_approval is missing
for a strict risk tier. It only checks that the safety evaluation is enabled; the CLI wizard turns the approval gate
on for those tiers, hand-written YAML is not checked.
The same documents already said the right thing elsewhere (the SoA row for A.5.3, access_control.md, the approve
reference); this brings the rest in line. English and Turkish. The design spec in docs/design is a record of intent and
is left as written; the website copy that says "dual-controlled" is a separate change.
Refs #73, #99, #90
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The page promised more than the code delivers, in the place a data-protection officer would read first: - "By the time data lands in JSONL, identifiable subjects are removed" and "no specific person's data is recoverable from the model. The audit report confirms this." ForgeLM masks eight identifier formats (email, phone, card, IBAN and the TR/DE/FR/US national IDs); names, addresses and free text are not detected, the patterns are tuned to miss borderline matches, and nothing measures what a trained model remembers. - "Detect memorisation: the audit step flags rows that overlap with known PII patterns" described PII detection as if it were memorisation detection. - The audit report was said to attribute rows to source documents and to carry a SHA-256 manifest. It does neither (it records counts and summaries, the audit's own metadata, and an optional Croissant card) and is not written to the audit log. - `retention` was said to "auto-delete originals" and to "record the intended TTL in the audit log". The horizons are only reported by `forgelm purge --check-policy`, which always exits 0. - An `ingestion: pii_mask:` YAML block and an `international_transfers.enabled` setting were shown; neither exists (the schema-drift guard skips unknown top-level keys, which is how they passed). The access and erasure sections now point at what exists: `forgelm reverse-pii` to find a subject's identifier in the files you give it, `forgelm purge --row-id` to remove rows (both write to the audit log), and the raw sources searched separately. English and Turkish. A reader can no longer cite this page for "personal data is removed before training" or "the model cannot reproduce a person's data". The page now says what the detectors cover and that coverage is partial. Refs #101 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…the model and exits 0 Exit 0 was documented as "every gate passed", a classifier load failure as halting the pipeline with no final_model/, and classification errors as unsafe. In code an evaluation that produces no verdict (evaluation_completed=False: classifier not loaded, probes file missing or empty, most verdicts unreadable) keeps the model and exits 0 (4 with the approval gate) regardless of auto_revert. The behaviour is unchanged; making it fail closed is a breaking exit-code change left for a major version. Refs #87 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ot release assets The supply-chain reference, the deployer guide, the change-management SOP and the user-manual pages told an auditor to run `gh release download ... --pattern 'sbom-*'` and to re-emit the SBOM from the git tag and diff it against the one attached to the release. publish.yml only uploads the twelve files as workflow-run artifacts (90-day retention), so there is nothing on a Release page, and the SBOM is the CI runner's `pip list` taken after pytest and, on Linux, three extras were installed: test tooling is listed, there is no dependency graph and no licence data (checked on the real v0.11.0 SBOM: 130 components, pytest and pytest-cov among them). It cannot be reproduced from the tag, because dependencies resolve from PyPI at build time. The pages now give commands that work (`gh run list --workflow publish.yml --branch v<X.Y.Z> --status success`, then `gh run download`; the layout was checked against the v0.11.0 run), say what the file contains, and drop "license extraction in SBOM" from ISO A.5.32, SOC 2 CC9.2 and the statement of applicability. The nightly pip-audit is described as querying PyPI's vulnerability data (its default service; the workflow passes no --vulnerability-service), and "exit 1" in the manual now includes UNKNOWN-severity findings, which the reference page already said. Attaching SBOMs to the release and generating them from a clean runtime-only environment are code changes and stay open. Refs #114 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…le 12 and 14, they do not satisfy them Three pages called audit_log.jsonl "the record-keeping artefact mandated by EU AI Act Article 12" (or implied it satisfies the article) and said the approval gate "implements" Article 14. Article 12 asks the deployed high-risk AI system to technically allow automatic recording of events over its lifetime; it prescribes no format, append-only storage or hash chaining and it does not cover a fine-tuning pipeline. Article 14's oversight (monitor, interpret, override, stop) happens while the system is in use. ForgeLM is a fine-tuning toolkit, so the audit log is development-time evidence that can support the technical documentation (Annex IV) and quality-management record-keeping, and evaluation.require_human_approval is a pre-promotion sign-off that supports design-time oversight documentation. The audit-event catalogue, the audit-log and the human-oversight user-manual pages now say that, in English and Turkish. The exact sentences are in the pull request description for line-by-line approval. Refs #75 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… six languages The marketing site repeated claims the wave-0 triage found false or overstated. In English, Turkish, German, French, Spanish and Chinese, and in the HTML fallbacks of the strings that have one: - a failing benchmark gate "fails (exit 3)": only with evaluation.auto_revert: true; the sample YAML comment in index.html said the same; and a safety evaluation with no verdict keeps the model and exits 0 (#89, #87); - the approval gate drives a "dual-controlled sign-off" and "every model promotion is dual-controlled": it records the approver and does not compare them, and in-use oversight (Article 14) stays with the deployer (#94, #99, #73); - "Slack, Teams, Discord" webhooks: the payload is Slack-format, Teams and Discord need a relay (wizard strings included) (#112); - PII "redacted"/"minimisation enforced upstream"/"satisfying the access-request side of the GDPR": common formats are masked, names and free text are not, and reverse-pii only finds what is literally in the files it scans (#101); - the wizard's retention "Enforce automatic deletion" option, its hint and its three modes, and the operations tutorial's "GDPR-compliant TTLs": the mode is reserved, nothing is deleted (#92, #98); - GGUF "6 quant levels and SHA-256 manifest" and "SHA-256 + size manifest": f16/q8_0 in one step, K-quants through llama-quantize, a SHA-256 sidecar (this wording depends on the sidecar change being merged) (#86); - DeepSpeed "ZeRO-2 / ZeRO-3 presets": bring your own JSON, the preset names need a source checkout (#110). The German, French, Spanish and Chinese strings were written by the assistant, not by a native reviewer. That is an exception to docs/standards/localization.md, which keeps those four tiers deferred and says not to machine-translate them; the project owner chose it for this change. They need review. A new site test keeps the retired phrases out of the English and Turkish tiers and the HTML, and pins the retention hint in every locale. Refs #86, #89, #92, #94, #98, #99, #101, #110, #112 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The quickstart page showed a one-line "✓ Python … · CUDA … · bitsandbytes ok" that the renderer never prints. It now shows the renderer's own format (ASCII header, [+ pass] / [! warn] markers, an abridged row list and the summary line); values are illustrative. Refs #534 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
The Turkish glossary test requires 'Annex IV' unchanged; the Article 12 rewrite had written 'Ek IV'. Refs #75 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Sorry @cemililik, your pull request is larger than the review limit of 150,000 diff characters |
Contributor
|
Warning Review limit reachedEnable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Next included review available in 28 minutes. View limit details
Comment |
This was referenced Oct 18, 2026
Closed
# Conflicts: # CHANGELOG.md
This was referenced Oct 18, 2026
cemililik
added a commit
that referenced
this pull request
Oct 22, 2026
…the code; harden the user-manual schema guard (#1050) ## Summary Finishes #101 (theme D0-6, *Personal data: audit and masking coverage*, 30 findings) and the neighbouring pages that carried the same root cause: documentation, help text, field descriptions, comments and site copy that presented PII masking, the erasure commands and the retention horizons as stronger than the code makes them, plus the guard that should have caught a `chat:` / `output:` block Pydantic rejects. After the first version of this pull request was reviewed, a second pass re-checked every statement against the running commands and the official texts, and a third pass (fact-check, legal and language review, mutation testing) re-checked that delta; this description covers the pull request as it now stands. **This is compliance wording. Please read the before/after table and the legal sign-off list.** ## Related issues Fixes #101. Refs #428, Refs #429, Refs #444, Refs #241, Refs #243, Refs #589, Refs #172, Refs #295, Refs #275, Refs #393, Refs #351, Refs #630, Refs #757, Refs #368, Refs #743, Refs #586, Refs #595, Refs #768, Refs #724, Refs #798, Refs #601, Refs #574, Refs #719, Refs #179, Refs #509, Refs #554, Refs #555, Refs #680, Refs #567, Refs #610, Refs #789. Each of these is a per-page or per-module issue of the October review whose documentation findings this change resolves in whole or in part; a maintainer ticks the findings and closes the ones that are complete (the GPU-cost, encryption-at-rest and notebook issues are, for example). Not worked on, mentioned for context: the `purge --kind artefacts` resolver rewrite stays with the artefacts issue (see the note posted on it) and the chat and `gpu_profiles` blocks the guard lists as known-open belong to other tracked issues; this change also corrects the content of the chat page those issues describe. New issues for what this change found but does not do: #1051 (phone numbers masked only partly), #1052 (wizards omit the retention block at defaults), #1053 (`forgelm ingest` rows have no `id`), #1054 (`--check-policy --config` needs a complete config), #1055 (native review of the DE/FR/ES/ZH strings), #1056 (site localization polish), #1057 (legal sign-off of the wording). ##⚠️ Localization exception `docs/standards/localization.md` keeps German, French, Spanish and Chinese as deferred tiers. As in #1000 (owner decision), **I wrote the DE / FR / ES / ZH site strings myself with no native review**; English and Turkish are authoritative. #1055 tracks the review. Touched keys: `home.cap.gdpr.body`, `home.use.domain.body`, `features.data.pii.body`, `compliance.gdpr15.body`, `compliance.gdpr17.body`, `compliance.art10.body`, `features.gov.{access,purge}.{title,body}`, `privacy.body`, `wizard.retention.*` (title and five hints). The legal pages' "last updated" date moved to 2026-10-22 because the privacy text changed; it is one key shared with the terms page (#1056). ## What this changes for a deployer | Topic | Said | Now says | |---|---|---| | **Article 17(3)(b)** | it "preserves audit / accounting records as a legal-obligation defence"; the audit log is "the legal-obligation record"; AI Act log duties as the example | it disapplies erasure only to the extent a specific Union or Member State obligation (or a public-interest task) requires the processing; keeping entries needs a documented basis; ForgeLM never edits or deletes an entry | | **Article 15 / 17 framing** | "Article 17 erasure with purge and Article 15 access with reverse-pii"; "zero matches is a valid answer" | `reverse-pii` is an input to an Article 15 response, `purge` one step of an Article 17 erasure (trained weights unchanged); a complete response also needs an identity check, one month (12(3)), a copy of the data, the 15(1)(a)-(h) information and the 15(2) safeguards, Article 19 notices, and refusal reasons within one month (12(4)); zero matches does not answer an access request | | **Digest correlation** | query with `--salt-source env_var` to match purge digests | byte-identical only for the same value, `--output-dir` and `FORGELM_AUDIT_SECRET` state, in the default plaintext mode; `--salt-source` is a hash-mask scan for externally masked corpora: not needed for the digests to match, and on a plaintext corpus it reports zero matches even when the person is there (verified end to end; the earlier advice was wrong) | | **Salted digests** | no personal data leaves the log | personal data (recital 26), and pseudonymisation (Art. 4(5)) only if the salt is kept separately; by default the salt file sits next to the log | | **Retention horizons** | "honour" GDPR deadlines, staging TTL is a scheduled cleanup, wizard "matches EU AI Act guidance" | operator-set limits that `--check-policy` reports and never enforces; defaults are ForgeLM choices; no `retention:` block checks nothing, `retention: {}` enables defaults, `0` stops a report; age source is the run's first audit event, else the log's first event, else mtime (so a shared output directory can report a fresh bundle as overdue); ForgeLM creates neither `raw_documents/` nor `ingestion_output/` | | **CI gate / exit codes** | `jq '.violations \| length'`; usage errors exit 1 | `jq -e '.success == true and has("count") and .count == 0'` under `set -o pipefail`; argument-parser usage errors exit 2; the purge JSON envelope section is regenerated from real output | | **`purge --kind artefacts`** | deletes the run's compliance bundle | matches files under `compliance/` whose names contain the run id, so it finds nothing in the standard bundle, emits `data.erasure_failed` (`NoMatchingArtefacts`), exits 1 and deletes nothing; the help text and the error say why | | **PII detectors** | "what GDPR / AI Act Article 10 cares about"; nine categories; German Steuer-ID and French NIR "checksum / key validation"; audit report lists row indices; audit "catches what masking missed" | common structured identifiers, personal data under Art. 4(1); eight detectors, the German ID-card serial and the NIR are shape-only; the report carries counts; `forgelm audit` re-applies the same regexes; `--pii-ml` counts names and locations and reports no `organization` for English | | **GPU-cost page** | a `run_complete` audit event, a model-card cost block, `--gpus`, `nvidia-smi`, a price table that did not match the code | what `_collect_resource_usage` records and where, the real price table and `cost_source` values, the CPU/MPS placeholder rate, a calibration recipe that runs | | **Chat page** | GGUF loading, a YAML `seed:`, `max_length` history, an invented transcript | no GGUF, no seed or greedy mode, last 50 pairs, the real REPL output and every flag | | **Site (six languages)** | masking as automatic; "every row of every split"; "forensically linked" | masking is opt-in (`--pii-mask`); lines containing an identifier you supply, literal and case-sensitive; `0` stops a retention report | | **QMS SOPs / SOC** | "minimum 5 years"; salted SHA-256 "needs no encryption" | a policy-set period (Art. 5(1)(e); Art. 18 is a separate duty); digests are personal data and the log and salt are encrypted alike | ## Changes - Documentation: GDPR guide, purge / reverse-pii / configuration / compliance-summary / event-catalog references, GDPR, GDPR-erasure, human-oversight, PII-masking, PII-ML, audit, ingestion, GPU-cost, chat and LoRA manual pages (EN and TR together), design note, QMS SOPs, deployer guide, localization glossary, `sync-bilingual-docs` skill scope. - Code text only (behaviour, flags and exit codes unchanged): `--help` and epilog strings, field descriptions, docstrings and comments, wizard prompts, the `--kind artefacts` error and the `ingest --pii-mask` note. `config.py`, `_parser.py` and `_purge.py` keep their module-size budgets. - Guard `tools/check_usermanual_schema_drift.py`: top-level blocks are checked everywhere (including full-config summary blocks); two exemption tables with reasons; one block per row per language; unreadable blocks fail; YAML 1.2 booleans; missing PyYAML exits 1; success line counts what was walked. 104 tests; two real snippets it flagged are fixed (`lora.md`, `model-card.md`). - Tests: Article 17(3) invariant per sentence over every public text file; broader retired-phrase lists with line numbers; positive pins for the CI gate, the correlation rule and the exit-2 wording; pages checked against the code (price table, chat flags, detector count); real-exporter `--kind artefacts` tripwire (checked by mutation); site tests per locale and for static fallbacks. ## Finding by finding (#101) | Finding | Resolution | |---|---| | CR-A11-001 (primary) | page rewrite (#1000), runnable example and real output line, `--pii-ml` pointer, the correlation rule and regression tests (this PR) | | CR-A17a-007 | guard and chat / GPU-cost examples (this PR); the gdpr.md block was removed by #1000 | | TECH-T11-002 | Art. 17(3)(b), 5(1)(e), 87 / 32 / 4(1) citations, "compliant out of the box" (this PR) | | DOC-D3B02-001, -007, -008, -009, -010, -024; DOC-BYAML-004 | fixed by the page rewrite in #1000; pinned by retired-phrase tests (EN and TR) | | DOC-D3B02-047 | example passes `--output-dir`, shows the real output line (pinned against the real command) | | DOC-BLEGAL-004 (+ design lines), -005 | Art. 17(3)(b) rewording | | DOC-BLEGAL-041, -047 | site cards in six languages; -050 hedged by #1000, pinned | | DOC-BLEGAL-064, -097, -146, DOC-BFIELD-019 | retention wording and field descriptions | | DOC-BLEGAL-068, -179, -180, -126, DOC-D3B07-044 | PII tier / detector citations | | DOC-D1B05-017, DOC-D2B02-025 | legacy staging marker documented | | DOC-D1B05-005 | **documented, not fixed**: warning in the guide, references and manual, in `--help` and in the error, pinned by a tripwire that fails on purpose when the resolver is fixed. The code half is noted on the artefacts issue | | DOC-D2B02-024, -051 | fixed by #999 / #1000; now pinned | ## Trade-offs worth your review - **`purge --kind artefacts`** is documented rather than fixed (owner decision). - **Known-open guard rows** keep the two fabricated blocks visible; fixing a page forces its row to be deleted. - **Scope.** The second pass widened the diff beyond #101's 30 findings because the same false statements sat on the pages next to them (GPU cost, chat, PII masking, QMS, human oversight). Larger neighbours that need code are filed, not fixed. - **Legal wording.** The statements listed in #1057 were checked against the official texts but not by a lawyer. - **Wording I could not verify here:** the DE / FR / ES / ZH strings (#1055) and the claim about Presidio versions, which I left out of the documentation. ## Review process Each step (erasure / retention, PII citations, GDPR page and site, guard) got two review rounds with independent agents (legal and factual, bilingual and language quality, mutation testing, operator walkthroughs against the running commands). The open pull request then got two independent whole-diff reviews; their findings were checked against the code before applying (one of them showed that the correlation advice added in an earlier round was wrong, and it is corrected on eight surfaces plus an end-to-end test). The tests and the guard were then mutation-tested (237 mutants; the real gaps are closed), the documentation was fact-checked against the running commands, and the legal wording and the Turkish and site strings were reviewed against the official texts. ## Type - [x] Bug fix (documentation, help text and comments that stated more than the code does) - [x] Documentation - [x] Test coverage - [x] CI / infrastructure (the user-manual schema-drift guard) ## Testing - [x] `pytest tests/` passes - [x] `ruff check .` passes - [x] `ruff format --check .` passes - [x] New tests added for changed code - [x] `python -m forgelm --config config_template.yaml --dry-run` works - [x] The full CLAUDE.md gauntlet exits 0 on the final commit (guards, mypy strict, `node --test`, bandit warnings only) ## Checklist - [x] Ruff formatted; module-size budgets unchanged - [x] English docs and their Turkish mirrors updated together (`check_bilingual_parity` / `check_bilingual_code_blocks` pass) - [x] CHANGELOG entry under `[Unreleased]` - [x] The issue I link (#101) has been triaged (no `needs-triage`) and is not blocked; the issues referenced above carry no `blocked` label 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Documentation** * Clarified the scope and limitations of `purge` and `reverse-pii`, including matching requirements, audit digests, retention reporting, and steps needed to complete access or erasure responses. * Updated guidance for PII detection and masking, chat behavior, GPU cost reporting, retention settings, and supported configuration. Clarified that retention checks report overdue items without deleting them and corpus erasure does not change trained weights. * Revised compliance, configuration, and operational guidance in English and Turkish, with related updates to other translated site content. * **Validation** * Expanded strict documentation checks to flag unreadable YAML, undeclared top-level blocks, and stale exceptions.** <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Compliance- and privacy-facing wording corrected to what the code does, plus the six-language site copy that repeated the same claims. Six earlier branches combined into one pull request, replacing #983, #986, #988 and #991 (to be closed as superseded) and two branches that were never opened. This is compliance wording; please read the before/after tables.
docs/standards/localization.mdkeeps German, French, Spanish and Chinese as deferred tiers and says not to machine-translate them. At the owner's decision I wrote the DE / FR / ES / ZH site strings myself, with no native review. English and Turkish are authoritative; the other four need a native reader before they can be trusted. A follow-up issue for that is drafted, not yet filed.Related issues
Refs #73, #99, #90, #94, #101, #87, #114, #75, #86, #89, #92, #98, #110, #112. The code halves stay open: whether the gate should ever compare trainer and approver, failing closed on a missing safety verdict (breaking), attaching SBOMs to the release.
Changes
require_human_approvalis not enforced for strict tiers byConfigError(it only is when safety evaluation is off)international_transferssetting; audit report with source attribution and SHA-256 manifest; retention "auto-deletes originals"evaluation_completed: false) keeps the model and exits 0 (4 with the approval gate) regardless ofauto_revert; code unchangedgh release download …; "re-emit from the git tag"; "licence extraction in SBOM"gh run download; the file is the CI runner'spip list(test tooling included, no dependency graph, no licences)auto_revert: true; records the approver; Slack-format, Teams and Discord need a relay; common formats masked, names not detected; reverse-pii finds only what is literally in the scanned files; retention mode reserved; SHA-256 sidecar; bring your own DeepSpeed JSONThe site GGUF strings describe the
.sha256sidecar, the retention strings the reserved mode, and the DeepSpeed strings the preset caveat; they match the GGUF, code and docs pull requests, so merge those first. A new site test keeps the retired phrases out of English, Turkish and the HTML (mutation-checked).What this changes for a deployer
A DPIA or audit response can no longer cite ForgeLM for: segregation of duties being enforced, personal data being removed or unrecoverable from the model, the SBOM being on the release page, or the audit log being the Article 12 mechanism.
Type
Testing
CLAUDE.mdgauntlet from a clean worktree of this branch: exit 0. (An earlier run failedtests/test_turkish_glossary_terms.pybecause the Article 12 rewrite wrote "Ek IV" instead of "Annex IV", fixed here, then two timing-based regex linearity tests failed while two gauntlets ran at once; they pass alone and in this final run.)node --test tests/site/*.test.cjs: 81 pass; English ↔ Turkish keys in lockstep.Checklist
[Unreleased]🤖 Generated with Claude Code