[Wave 3] Medium — Module sweep: Dataset audit: PII, secrets, dedup, leakage, quality, Croissant (part 1/2) #547

Wave 3 · 23 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A11-025 (code, Medium) — Non-JSONL training inputs (.json array, .csv, .parquet) are audited as 0 rows and pass both gates while the trainer loads them in full
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. The training config points at data.json. The operator gates it with forgelm audit data.json, which exits 0, and the credential is trained on.
  Where. forgelm/data_audit/_splits.py:105-106, forgelm/data_audit/_streaming.py:196-217 (_read_jsonl_split: json.loads per line, parse errors yielded as (None, True, ...)), forgelm/data_audit/_aggregator.py:175-207, forgelm/data.py:99-104 (_SUPPORTED_DATASET_EXTENSIONS), forgelm/data.py:315-322 (load_dataset by extension)
  Recommended fix. Reject or properly parse the non-JSONL formats the trainer accepts, for example by reusing data.py's extension map. Fail with a non-zero exit when a split yields zero parseable object rows, or when parse errors exceed a threshold.

• CR-A11-026 (code, Medium) — Directory audit silently ignores non-canonical *.jsonl (e.g. HF shard names) whenever any canonical/alias file exists
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A sharded HF-exported corpus with a separately named test file passes the secrets gate even though its training shards carry credentials.
  Where. forgelm/data_audit/_splits.py:87-92, forgelm/data_audit/_splits.py:41-63, forgelm/data.py:337, forgelm/data_audit/_splits.py:66-84
  Recommended fix. Note and log every *.jsonl that is not audited, and map HF shard patterns ({split}-NNNNN-of-NNNNN, _train) onto canonical splits, or scan unmatched files as extra pseudo-splits.

• CR-A11-031 (code, Medium) — Forgelm audit secrets/PII gate verdicts and --allow-secrets/--allow-pii overrides are not recorded in the audit log or the report
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A pipeline runs forgelm audit --allow-secrets and trains. The governance bundle shows secrets_summary counts but nothing records that the gate was deliberately bypassed, or by whom.
  Where. forgelm/data_audit/_types.py:121-152, forgelm/cli/subcommands/_audit.py:29-97, forgelm/cli/subcommands/_audit.py:100-154, README.md:87 (confirmed)
  Recommended fix. Add a gate block (status, allow_* flags) to AuditReport, a non-breaking additive key, so the Article 10 artefact records the decision.

• ↳ same root cause: DOC-D4B01-020 (docs, Medium) — README.md: false claim — "Every decision gate appends one line to audit_log.jsonl."
  severity Medium
  Where. README.md:87 · type: false claim
  The document says. "Every decision gate appends one line to audit_log.jsonl."
  What the code does. This holds for the in-training gates, but three gates that set the exit code write no audit event. The forgelm audit secrets and PII gates (exit 3) have no AuditLogger in the audit dispatcher or the data_audit package. --benchmark-only exits 3 without an audit event. Standalone safety-eval only ever logs audit.classifier_load_failed, never safety.evaluation_completed.
  Code references. forgelm/cli/subcommands/_audit.py:1-360, forgelm/cli/_no_train_modes.py:60-125, forgelm/cli/subcommands/_safety_eval.py:221-297, forgelm/safety/_classifier.py:190-207
  Proposed correction. Code: emit gate events from audit, --benchmark-only and safety-eval. Doc, until then: "Every training-run decision gate appends one line to audit_log.jsonl (the standalone audit, safety-eval and --benchmark-only verdicts are reported via exit code and JSON only)."

• CR-A11-037 (code, Medium) — Central exit-code tables misdescribe codes 1, 2 and 3 compared with what the CLI emits
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:782-785 (S15 open)
  Failure scenario. An operator relying on the canonical exit-codes page treats exit 2 from a mistyped flag as retryable, and expects exit 3 to imply a deleted model when forgelm audit exits 3 with no model involved.
  Where. docs/usermanuals/en/reference/exit-codes.md:15-17,19, docs/standards/error-handling.md:24-27,29, forgelm/cli/subcommands/_audit.py:97, forgelm/cli/subcommands/_audit.py:325
  Recommended fix. Bring exit-codes.md and error-handling.md in line with README/cli.md: argparse usage errors are 2, and 3 means 'a gate refused' (training gates under auto_revert, audit data gates, safety-eval, benchmark-only).

• CR-A11-038 (code, Medium) — Documented audit scale claims are contradicted by measurement (streaming, 10M rows in minutes, 50K/500K rows/s, --workers for million-row corpora)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator sizes infrastructure or CI time budgets from the manual: 10M rows in minutes on a laptop, MinHash at 500K rows/s, --workers for big files. The audit step then OOMs or runs for hours, and --workers 8 on a single-file corpus changes nothing.
  Where. docs/usermanuals/en/data/deduplication.md:14-15, docs/usermanuals/en/data/deduplication.md:74, docs/usermanuals/en/data/deduplication.md:85, docs/usermanuals/en/data/audit.md:101
  Recommended fix. Replace the throughput table and the streaming paragraph with measured figures and the real memory model. Drop the --workers advice for single-file corpora. Fix the _audit_split docstring. Re-check after the dedup memory fixes land.

• ↳ same root cause: DOC-D3B07-026 (docs, Medium) — docs/usermanuals/en/data/deduplication.md: false claim — Both algorithms are streaming — they don't load the whole dataset into memory. A 10M-row corpus dedupes in a few minutes on a laptop CPU.
  severity Medium · already tracked: KNOWN-OPEN TRUST-12|TRUST-06|TRUST-09|TRUST-11 docs/roadmap/phase-16-trust-surface-hardening.md:584-599
  Where. docs/usermanuals/en/data/deduplication.md:85 · type: false claim
  The document says. Both algorithms are streaming — they don't load the whole dataset into memory. A 10M-row corpus dedupes in a few minutes on a laptop CPU.
  What the code does. Per-row signatures are kept for the whole split, and simhash candidate-pair bookkeeping grows about n^2. 1M rows took 72.8 s/4.54 GB (simhash) and 396.7 s/6.95 GB (minhash); 10M is not achievable as claimed.
  Code references. forgelm/data_audit/_aggregator.py:334-351, forgelm/data_audit/_simhash.py:294-303
  Proposed correction. Document measured memory/time and the linear signature memory; remove the 10M claim
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• ↳ same root cause: DOC-D2B01-023 (docs, Low) — docs/guides/data_audit.md: misleading — cost drops from O(n^2) to roughly O(n x k)
  severity Low · already tracked: KNOWN-OPEN TRUST-06/TRUST-09/TRUST-11/TRUST-12 (S10) — docs/roadmap/phase-16-trust-surface-hardening.md:586-587,593-601
  Where. docs/guides/data_audit.md:255-256 · type: misleading
  The document says. cost drops from O(n^2) to roughly O(n x k)
  What the code does. k (bucket fan-out) grows linearly with n for 16-bit bands, and identical rows collapse into one bucket, so cost and memory are quadratic in practice (4.54 GB at 1M; 10 GB for 10k identical rows)
  Code references. forgelm/data_audit/_simhash.py:294-303
  Proposed correction. State the n^2/2^band_bits candidate growth and the duplicate-cluster worst case
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• CR-A11-042 (code, Medium) — Secrets user manual documents a non-existent audit_dataset(secrets_mask=...) argument
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A library user calls audit_dataset(path, secrets_mask=True) as the manual says and gets a TypeError. Or they assume the audit masks data, which it never does.
  Where. docs/usermanuals/en/data/secrets.md:123, docs/usermanuals/tr/data/secrets.md:123, forgelm/data_audit/_orchestrator.py:227-240
  Recommended fix. Reword the doc to point at ingest_path(secrets_mask=True) / forgelm ingest --secrets-mask.

• CR-A11-043 (code, Medium) — IBAN/card patterns accept only ASCII space/hyphen separators and uppercase IBANs: NBSP, tab, line-wrapped, dotted and lowercase forms escape both --pii-mask and the PII gate
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator follows docs/guides/ingestion.md:104 (forgelm ingest ./customer_emails/ --output data/anon.jsonl --pii-mask) on hard-wrapped support emails or text pasted from bank PDFs. Ingest exits 0 and reports redactions. The operator then runs the documented forgelm audit, which reports no PII and exits 0, and trains on a JSONL that still carries data subjects' IBANs, card numbers, Steuer-IDs, SSNs and national-format phone numbers.
  Where. forgelm/data_audit/_pii_regex.py:64 (credit_card pattern), forgelm/data_audit/_pii_regex.py:39 (iban pattern is on line 59, comment block 36-58), forgelm/data_audit/_pii_regex.py:64 (confirmed), forgelm/data_audit/_pii_regex.py:77
  Recommended fix. Correct the user manual's category table (Steuer-ID, NIR key, national phone formats) to match the code, or implement them. Make the IBAN and card shapes accept any Unicode space or a single newline between groups. On a checksum failure, retry the validated sub-span (strip leading or trailing short groups) rather than leaving the whole span. Add regression tests for NBSP/U+202F, wrapped, lowercas…

• CR-A11-044 (code, Medium) — Secret detectors for AWS and Slack keys use Unicode-aware \b, so keys next to CJK/non-ASCII letters evade detect_secrets and mask_secrets (secrets gate passes)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. mask_secrets('我的密钥是AKIAIOSFODNN7EXAMPLE请勿泄露', return_counts=True) returns the text unchanged with counts {}. mask_secrets('トークンxoxb-1234567890-abcdefghijです') also returns it unchanged. detect_secrets returns {} for 密钥{AWS key}, {AWS key}密码, é{AWS key}, {AWS key}é and 密钥{slack token}. The control 'token: AKIAIOSFODNN7EXAMPLE' is masked correctly. A corpus with such keys therefore passes the audit secrets gate (the gate in _audit.py fails on critical secrets), ingest --secrets-mask leaves the cre…
  Where. forgelm/data_audit/_secrets.py:71, forgelm/data_audit/_secrets.py:110, forgelm/cli/subcommands/_audit.py:29, forgelm/data_audit/_summary.py:160
  Recommended fix. Add flags=re.ASCII to aws_access_key and slack_token. For credentials that may sit inside underscore-joined identifiers, replace \b with ASCII-only lookarounds such as (?<![A-Za-z0-9]) and (?![A-Za-z0-9]). Add non-ASCII-adjacency fixtures to tests/test_data_audit_phase12.py for every pattern.

• CR-A11-045 (code, Medium) — Single-file audit performs no leakage analysis and says nothing about it, while the user manual tells users to audit a single ingest output for cross-split leakage
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator follows ingestion.md:129 and runs forgelm audit data/output.jsonl. The audit exits 0, shows no leakage line and writes pairs {}. The operator reads the absence of a leakage line as no leakage, but no analysis ran.
  Where. forgelm/data_audit/_splits.py:105-106, forgelm/data_audit/_summary.py:459-463, docs/usermanuals/en/data/ingestion.md:129, docs/usermanuals/tr/data/ingestion.md:128
  Recommended fix. When fewer than two splits are present, add a note such as 'cross-split leakage not evaluated: only one split'. Reword ingestion.md:129 and its TR mirror to point at a split directory.

• CR-A11-046 (code, Medium) — forgelm audit crashes with a raw traceback (exit 1, no JSON envelope, no report) on any dataset row whose key holds a lone-surrogate escape
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C33-CLI-ERROR-ENVELOPE — docs/roadmap/phase-16-trust-surface-hardening.md:497-508 (S7 open)
  Failure scenario. A scraped or third-party JSONL contains one row with a key like "\ud800x". The operator or a CI step runs forgelm audit corpus.jsonl --output-format json. The run dies with a traceback and exit 1, stdout carries no JSON, and the Art. 10 audit report for the whole corpus cannot be produced until someone finds the row. The traceback points to a character offset in the report, not to the row.
  Where. forgelm/data_audit/_orchestrator.py:157-167 (NamedTemporaryFile encoding='utf-8'; fh.write(dumps_strict(payload, indent=2, ensure_ascii=False)) at :167), forgelm/data_audit/_orchestrator.py:491-494 (audit_dataset writes the report when output_dir is set), forgelm/data_audit/_aggregator.py:211-215 (_populate_schema_block copies seen row keys into info['columns']), forgelm/cli/subcommands/_audit.py:199-231 (only OSError and ImportError are caught)
  Recommended fix. Write the report with ensure_ascii=True, or open the tempfile with errors='backslashreplace'/'surrogatepass', or normalise lone surrogates in column names at aggregation. Also catch ValueError/UnicodeError in _audit.py and emit the 2-key envelope with a non-config exit code. Add a regression test with a surrogate key.

• CR-A11-047 (code, Medium) — Forgelm audit prints hostile JSONL keys and split filenames raw to the terminal: escape sequences can clear the screen, print a fake summary and hide the real 'Secrets : CRITICAL' verdict
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator audits a third-party JSONL downloaded from a hub, as the docs suggest, before training on it. It carries a real leaked key in a text value. One row has an extra key that clears the screen, prints a fake clean summary and turns on conceal. The terminal shows 'Secrets : none'; the real CRITICAL line and the gate error are invisible. Exit 3 still fires, so this only deceives an interactive reviewer, and a reviewer who checks $? is not deceived. Other sequences can change the window tit…
  Where. forgelm/data_audit/_aggregator.py:98-102, forgelm/data_audit/_aggregator.py:211-223, forgelm/data_audit/_summary.py:401-402, forgelm/data_audit/_summary.py:447
  Recommended fix. Escape every untrusted string before it reaches text output. Pass column names, split names and paths through a helper that replaces Unicode category Cc/Cf characters with \xNN or \uNNNN, for example repr()-style or str.translate. Apply it in summarize_report and in the logger calls that interpolate split_name. For the JSON envelope, consider ensure_ascii=True on stdout (the on-disk report can st…

• CR-A11-048 (code, Medium) — Private-key detector needs a closing END within an 8192/65536-char bound: truncated or oversized key blocks are neither detected nor masked, and the bound is undocumented
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A scraped Q&A corpus contains a row with a half-pasted OpenSSH private key and no END line. forgelm audit exits 0 with secrets_gate 'passed', and ingest --secrets-mask writes the partial key verbatim into training data.
  Where. forgelm/data_audit/_secrets.py:59, forgelm/data_audit/_secrets.py:96-107 (confirmed), forgelm/data_audit/_aggregator.py:159-160, forgelm/data_audit/_summary.py:140-167
  Recommended fix. Add a fallback private-key-header family that matches a BEGIN ... PRIVATE KEY marker (any label, tolerant of a space after the dashes) plus the following run of base64 and header lines, bounded and linear (e.g. up to the next blank line or a non-base64 line). Mask it, and let it trip the gate. State in secrets.md that the envelope family requires END within 8192/65536 chars.

• CR-A11-049 (code, Medium) — Near-duplicate and cross-split leakage detection is blind to CJK (and other unspaced-script) near-duplicates: one-character edits report 0 pairs and 0 leaks under both simhash and MinHash
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator audits a Chinese SFT corpus whose test split holds lightly edited copies of train rows (augmentation before splitting, or the same FAQ ingested twice with a typo fix). forgelm audit prints 'train__test: leaked=0/0 rate=0.00%/0.00%' and no near-duplicate pairs. The data_governance_report.json then states zero leakage, and benchmark scores are inflated without any signal.
  Where. forgelm/data_audit/_simhash.py:37-38, forgelm/data_audit/_simhash.py:38-39, forgelm/data_audit/_simhash.py:117-153, forgelm/data_audit/_minhash.py:26-55
  Recommended fix. Use character n-gram shingles (for example 3- to 5-grams over NFKC-normalised, case-folded text with whitespace and punctuation stripped) as the simhash/MinHash features, at least for runs of script characters without spaces (Han, Hiragana/Katakana, Thai, Lao, Khmer, Myanmar), or for all text. This is the standard approach in CCNet/RefinedWeb-style dedup and makes the threshold meaning script-ind…

• CR-A11-050 (code, Medium) — Default-on quality filter flags every well-formed Chinese, Japanese and Hindi row (overall_quality_score 0.0 on a 100% clean CJK corpus); Thai and Arabic trip low_punct_endings
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A team audits a clean, curated Chinese or Hindi instruction corpus with the default forgelm audit. Every row is flagged and overall_quality_score = 0.0 is written into the audit report and the Article 10 governance report. Operators either act on the score and drop good data, or learn to ignore the quality block, which also hides real low-quality rows.
  Where. forgelm/data_audit/_quality.py:38, forgelm/data_audit/_quality.py:167-173, forgelm/data_audit/_quality.py:159-164, forgelm/data_audit/_summary.py:291-300
  Recommended fix. Add CJK/Arabic/Devanagari terminators (。！？．｡؟۔।॥ and fullwidth quotes/brackets) to _PUNCT_END_PATTERN. Count combining marks (unicodedata.category M*) as part of words and as alphabetic: tokenise with [\ẁ-ͯ...] or regex's \p{L}\p{M}, and use c.isalpha() or unicodedata.category(c).startswith('M'). Skip or adapt the word-based checks (mean word length, short paragraphs) for rows whose domina…

• CR-A11-052 (code, Medium) — Row-level metadata is claimed to be preserved in audit logs; nothing reads it
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A compliance team relies on per-row provenance in metadata being retained in ForgeLM's audit trail for Art. 10/12 traceability, but it is silently dropped.
  Where. docs/usermanuals/en/concepts/data-formats.md:31, docs/usermanuals/tr/concepts/data-formats.md:31, forgelm/data_audit/_streaming.py:117-163, docs/usermanuals/tr/data/../concepts/data-formats.md:31
  Recommended fix. Remove the claim, or define the destination (for example, a provenance summary in data_audit_report.json) and implement it.

• CR-A11-053 (code, Medium) — PII masker redacts ordinary prose; the code is designed to over-report, contradicting the 'prefer false negatives' promise
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator trusts the 'conservative' claim and runs ingest --pii-mask on a technical or scientific corpus. Citations, IDs and timestamps are silently replaced with [REDACTED], and the corrupted rows are trained on.
  Where. forgelm/data_audit/_pii_regex.py:37-96, forgelm/data_audit/_pii_regex.py:64-96, forgelm/data_audit/_pii_regex.py:265-306, forgelm/data_audit/_pii_regex.py:265-307
  Recommended fix. Make the public docs match the code. State that the masker over-reports on shape-matched families and that only credit_card, iban and tr_id are checksum-validated. Alternatively, mask only the checksum-validated families by default and make the shape-only families opt-in. Correct the site li3 string in all 6 locales, pii-masking.md and compliance/overview.md:91.

• CR-A11-054 (code, Medium) — DPO manual and data-formats page promise a 'chosen == rejected' check that forgelm audit does not have (audit exits 0)
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A preference pipeline emits rows with chosen identical to rejected. Following dpo.md:208, the operator runs forgelm audit, which reports a clean pass (exit 0, no warning). They train DPO on degenerate pairs and the loss collapses to about 0, the very symptom the doc says the audit diagnoses.
  Where. docs/usermanuals/en/training/dpo.md:103, docs/usermanuals/en/training/dpo.md:208, docs/usermanuals/tr/training/dpo.md:94, docs/usermanuals/tr/training/dpo.md:194
  Recommended fix. Remove the claims at dpo.md:103/:208, data-formats.md:158-166 and their TR mirrors, or implement a preference-pair check (chosen == rejected count per split) in the aggregator. Also correct audit.md:122's 'concatenates each row's text fields'.

• CR-A11-055 (code, Medium) — audit_dataset's documented error contract (library_api_reference.md:265) misstates what it raises: plain ImportError, not OptionalDependencyError, plus undocumented RecursionError/SystemExit and row-…
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A CI gate or service embeds the stable-tier audit_dataset and catches the documented (ValueError, OSError, ImportError). One TC-U JSONL row of 100,000 nested '[' raises RecursionError, which the documented handlers do not catch: no report and an unhandled crash. A worker that runs audit_dataset(enable_pii_ml=True, pii_ml_language='de') while the spaCy compatibility server returns an error gets SystemExit(1). It passes through except Exception, and the hosting process exits silently with statu…
  Where. docs/reference/library_api_reference.md:269, docs/reference/library_api_reference-tr.md:269, forgelm/data_audit/_streaming.py:211-215, forgelm/data_audit/_orchestrator.py:167
  Recommended fix. Make the contract true. Convert per-row RecursionError and ValueError to counted parse errors (see M-A11-019). Validate minhash_jaccard and num_perm against datasketch's band limits before scanning. In _require_presidio, catch SystemExit from the spaCy download path, or pre-check spacy.util.is_package(model) and raise ImportError without triggering a download. Then document the full list, includi…

• CR-A11-056 (code, Medium) — Cross-split leakage runs over all s(s-1)/2 split pairs, and pseudo-split mode makes every .jsonl shard a split, so time, memory and report size grow quadratically with shard count
  severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An operator runs forgelm audit on a directory of HF-style shards. This is the documented pseudo-split fallback, used whenever no canonical train/validation/test name exists. With 200 shards of 200 rows, the MinHash audit takes 352 s instead of 14 s for the same rows in one file, and the report grows to 5.2 MB of all-zero pair entries. A hostile or careless dataset publisher who ships 2,000 one-row files (297 KB in total) makes the audit allocate 3.9 GB and write a 513 MB report plus a 100 MB…
  Where. forgelm/data_audit/_orchestrator.py:118-136, forgelm/data_audit/_orchestrator.py:69-75 (confirmed), forgelm/data_audit/_simhash.py:362 (confirmed), forgelm/data_audit/_minhash.py:229-230 (the two _build_minhash_lsh calls; :231-233 are the queries and the return)
  Recommended fix. Ranked by expected gain: (a) Replace the pairwise loop with one global index: a simhash band index or a MinHashLSH holding every non-empty signature keyed by (split, row). Query each row once and attribute each verified hit to its (source split, target split) pair. This reduces the cost from O(sN + s^2 * constructor) to O(N*k). The prototype measured 47x (simhash, s=200) and 180x (minhash, s=50)…

• CR-A11-057 (code, Medium) — Statement of Applicability credits forgelm audit with PII masking and ISO A.8.11 data masking; the audit only counts, and no ForgeLM path masks Presidio NER entities
  severity Medium · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. An ISO 27001 assessor accepts A.8.11 as implemented by forgelm audit. The operator runs forgelm audit d2 on a corpus holding a card number. The run exits 3, and the input file still contains '4111 1111 1111 1111' with no [REDACTED] anywhere. Running forgelm ingest doc.txt --all-mask on 'Hello John Smith from Acme Corp in Berlin.' leaves all three names in the output (exit 0).
  Where. docs/qms/statement_of_applicability.md:63, docs/qms/statement_of_applicability.md:119, docs/qms/statement_of_applicability-tr.md:64, docs/qms/statement_of_applicability-tr.md:120
  Recommended fix. Reword A.5.34 and A.8.11 to say: detection (counts and severity tiers) by forgelm audit, regex-only masking by forgelm ingest --pii-mask/--secrets-mask, Presidio ML-NER detection-only and opt-in via forgelm audit --pii-ml. Name-entity masking stays the deployer's 'Masking policy' gap. Update the TR mirror and fix the ingestion.py comment.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}