Skip to content

[Wave 3] Medium — Module sweep: Dataset audit: PII, secrets, dedup, leakage, quality, Croissant (part 2/2) #548

Open

Description

@cemililik

Wave 3 · 1 finding(s) in 1 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

  • CR-A11-058 (code, Medium) — No test pins the data-integrity note or the gate/exit behaviour for malformed or array-only secret-bearing JSONL; all-malformed and all-array corpora exit 0 with both gates 'passed'
    severity Medium · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
    Where. forgelm/data_audit/_orchestrator.py:446-452, forgelm/data_audit/_splits.py:192-196, forgelm/data_audit/_aggregator.py:179-200, tests/test_data_audit.py:528
    Recommended fix. Add to tests/test_audit_secrets_gate.py a class TestUnscannableCorpusAndGate. Its cases: (1) a corpus of one valid clean line plus one malformed line carrying _AWS_KEY, asserting the 'Data integrity: 1 parse error(s)' note is in report.notes and that the exit code and secrets_gate status match whatever the project decides, fail-closed or documented partial-audit; (2) an all-malformed corpus, asse…

Done when

  • Every finding above is fixed or closed with a reason (tick it here).
  • Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: data-auditCode area: data-auditbugSomething isn't workingseverity: mediumReal defect with limited reach or a workaroundsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 3Roadmap wave 3: medium severity, module sweeps

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions