[Epic] Triage: disputed findings #878

Activity

1. cemililik

   added

   needs-triageNot triaged yet: every new issue starts with it; only a maintainer removes itNot triaged yet: every new issue starts with it; only a maintainer removes it

   source: review-2026-10Found by the October 2026 code and documentation reviewFound by the October 2026 code and documentation review

   epicParent issue grouping one roadmap theme; progress = its sub-issuesParent issue grouping one roadmap theme; progress = its sub-issues

   on Oct 9, 2026
2. cemililik

   added this to ForgeLM Project Planon Oct 9, 2026
3. github-project-automation

   moved this to Todo in ForgeLM Project Planon Oct 9, 2026
4. cemililik

   added sub-issues on Oct 9, 2026

   • [Triage] SFT formatting map spawns up to 8 worker processes for a trivial string-join and gives no measured gain (CR-A03-095) #82
   • [Triage] Code does not match docs/usermanuals/en/training/distributed.md: Launching through accelerate launch makes "DeepSpeed's parameter-partitioning wrapper initialise before the mode… (CR-A04-092) #822
   • [Triage] M-A18-049 as worded is wrong: 'truthfulqa' resolves in lm-eval 0.4.x, as a TAG (neither a task nor a group) (CR-A05-075) #823
   • [Triage] cache-tasks depends on lm_eval.tasks.get_task_dict, deprecated in lm-eval 0.4.12+, with the declared cap at <1.0.0 (CR-A15-062) #824
   • [Triage] Docker build installs ForgeLM with the distro's old pip (jammy python3-pip 22.0.x, never upgraded) under a non-default python3.11, so neither PEP 660 editable install at Dockerfil… (CR-A16-114) #825
   • [Triage] [extends: 'Benchmark gate metadata does not record the lm-eval version'] lm-eval's BOS default flips inside the declared range (0.4.4: never add BOS for causal LMs; 0.4.13: tokeni… (CR-A16-115) #826
   • [Triage] Markdown ignore (PYSEC-2026-89) reasons only about the audited Python env, while the one reachable markdown consumer is the docker-compose TensorBoard image (tensorflow/tensorflow… (CR-A16-116) #827
   • [Triage] Declared floor bandit[toml]>=1.7.0 predates the toml extra that the pinned '-c pyproject.toml' invocation relies on (CR-A16-117) #828
   • [Triage] Help parsing is sensitive to ANSI colour (latent on Python 3.14): coloured {choices} silently disable choice-value drift checking (CR-A17a-053) #829
   • [Triage] Site and CLI help name lm-eval task 'truthfulqa', which does not exist as a task name in lm-eval 0.4.x (CR-A18-099) #830
5. 2 remaining items

   Load more
6. cemililik

   added sub-issues on Oct 9, 2026

   • [Triage] docs/usermanuals/en/evaluation/safety.md: false denial — Guard-verdict scoring is unbatched ... Under classifier_mode: generation (the default) (DOC-D3B08-096) #833
   • [Triage] README.md: false denial — N-10: Integrity manifest and audit log are not keyed without FORGELM_AUDIT_SECRET; editing the last log line leaves the chain self-consistent (DOC-D4B01-032) #834
   • [Triage] site/js/translations.js: false denial — You cannot disable these for high-risk deployments (DOC-D4B09-027) #835
7. cemililik

   removed sub-issues on Oct 20, 2026

   • [Triage] Code does not match docs/usermanuals/en/training/distributed.md: Launching through accelerate launch makes "DeepSpeed's parameter-partitioning wrapper initialise before the mode… (CR-A04-092) #822
   • [Triage] cache-tasks depends on lm_eval.tasks.get_task_dict, deprecated in lm-eval 0.4.12+, with the declared cap at <1.0.0 (CR-A15-062) #824
   • [Triage] Docker build installs ForgeLM with the distro's old pip (jammy python3-pip 22.0.x, never upgraded) under a non-default python3.11, so neither PEP 660 editable install at Dockerfil… (CR-A16-114) #825
   • [Triage] [extends: 'Benchmark gate metadata does not record the lm-eval version'] lm-eval's BOS default flips inside the declared range (0.4.4: never add BOS for causal LMs; 0.4.13: tokeni… (CR-A16-115) #826
   • [Triage] Markdown ignore (PYSEC-2026-89) reasons only about the audited Python env, while the one reachable markdown consumer is the docker-compose TensorBoard image (tensorflow/tensorflow… (CR-A16-116) #827
   • [Triage] Declared floor bandit[toml]>=1.7.0 predates the toml extra that the pinned '-c pyproject.toml' invocation relies on (CR-A16-117) #828
   • [Triage] Help parsing is sensitive to ANSI colour (latent on Python 3.14): coloured {choices} silently disable choice-value drift checking (CR-A17a-053) #829
   • [Triage] docs/reference/configuration.md: false denial — N-21: export/inference/merging unpinned by design; merge-source models cannot be pinned (DOC-D1B01-059) #831
8. cemililik commented on Oct 20, 2026

   

   cemililik

   on Oct 20, 2026

   CollaboratorAuthor

   More actions

   Triage round 1 is done for the 15 sub-issues. Closed: #823 and #830 ('truthfulqa' is a valid lm-eval tag), #82 (measured, not worth changing), #834 (already fixed) and #832, #833, #835 (duplicates of #420, #761 and #599, which now carry their finding IDs). Accepted and moved under the epics of their wave and theme: #822, #824, #825, #826, #827, #828, #829, #831.

   Nothing is left under this epic, so it closes. A finding the reviewers disagree on in the future gets its own needs-triage issue like any other new report.

   Triage round 1, checked against development on 2026-10-20.
9. cemililik

   removed

   needs-triageNot triaged yet: every new issue starts with it; only a maintainer removes itNot triaged yet: every new issue starts with it; only a maintainer removes it

   on Oct 20, 2026
10. cemililik

    closed this as completedon Oct 20, 2026
11. github-project-automation

    moved this from Todo to Done in ForgeLM Project Planon Oct 20, 2026