Skip to content

The tensorboard>=2.15.0 floor installs a TensorBoard whose event reader and tensorboard CLI crash on supported Python 3.13 (NumPy 2, no pkg_resources) (CR-A16-082) #529

Open

Description

@cemililik

Wave 2 · theme D2-7: Next-release technology updates · highest severity Low · 2 finding(s) · effort small

Summary

tensorboard is a core dependency (pyproject.toml:60), and report_to defaults to 'tensorboard' (forgelm/config.py:604-607, trainer.py:519). In the K-3 lower venv (Python 3.13.14, tensorboard 2.15.0, numpy 2.5.3, setuptools 84.0.0), torch's SummaryWriter writes events, so ForgeLM training itself works at the floor. Reading them fails: EventAccumulator raises AttributeError because np.string_ was removed in NumPy 2.0. The tensorboard entry module fails too: import tensorboard.main raises ModuleNotFoundError for pkg_resources. tensorboard 2.21.0 does both. NumPy 2 is the only choice on Python 3.13 and the default elsewhere. setuptools 81.0.0, in the extras venv, still ships pkg_resources (with a removal warning), so the CLI failure depends on the runtime setuptools. The floor's metadata also pins protobuf<4.24 (lower venv: protobuf 4.23.4). This is a viewer defect, not a training or security defect; I found no tensorboard advisory. The T12.17 'current/watch' rating is advisory-only and does not contradict this.

Primary finding

CR-A16-082 — The tensorboard>=2.15.0 floor installs a TensorBoard whose event reader and tensorboard CLI crash on supported Python 3.13 (NumPy 2, no pkg_resources)

severity Low · reachability default · intent intended-but-harmful · confidence likely (static only) · independently confirmed · reproduced end to end

What happens. tensorboard is a core dependency (pyproject.toml:60), and report_to defaults to 'tensorboard' (forgelm/config.py:604-607, trainer.py:519). In the K-3 lower venv (Python 3.13.14, tensorboard 2.15.0, numpy 2.5.3, setuptools 84.0.0), torch's SummaryWriter writes events, so ForgeLM training itself works at the floor. Reading them fails: EventAccumulator raises AttributeError because np.string_ was removed in NumPy 2.0. The tensorboard entry module fails too: import tensorboard.main raises ModuleNotFoundError for pkg_resources. tensorboard 2.21.0 does both. NumPy 2 is the only choice on Python 3.13 and the default elsewhere. setuptools 81.0.0, in the extras venv, still ships pkg_resources (with a removal warning), so the CLI failure depends on the runtime setuptools. The floor's metadata also pins protobuf<4.24 (lower venv: protobuf 4.23.4). This is a viewer defect, not a training or security defect; I found no tensorboard advisory. The T12.17 'current/watch' rating is advisory-only and does not contradict this.

Where.

Evidence.

  • tb_floor.py, lower venv: 'WRITE OK', then 'READ FAIL AttributeError np.string_ was removed in the NumPy 2.0 release', then 'MAIN IMPORT FAIL ModuleNotFoundError No module named 'pkg_resources''. Latest venv (2.21.0): 'WRITE OK', "READ OK ['train/loss']", 'MAIN IMPORT OK'. importlib.metadata: tensorboard 2.15.0 requires 'protobuf <4.24,>=3.19.6' and has no numpy cap; 2.21.0 requires 'protobuf<8.0.0,>=6.31.1'.

Recommended fix. Raise the floor to tensorboard>=2.21.0,<3. This needs protobuf>=6.31.1, so wandb's floor must rise too (0.16.0 caps protobuf<5). Pin the nightly floor lane to match. Also correct experiment-tracking.md:61, which says tensorboard comes 'bundled with PyTorch' and must be installed separately, although it is a core ForgeLM dependency.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

TECH-T09-008 — Experiment tracking, webhooks and monitoring: tensorboard>=2.15.0,<3.0.0 (core dependency, pyproject.toml:60) (range-unsound)

severity Next release

Item. tensorboard>=2.15.0,<3.0.0 (core dependency, pyproject.toml:60) · verdict: range-unsound · urgency: Next release

State in ForgeLM. pyproject.toml:60 declares tensorboard>=2.15.0,<3.0.0 as a core dependency. report_to defaults to tensorboard (config.py:604-606), and the docs tell users to open the logs with tensorboard. The K-3 lower venv resolves 2.15.0; the project and latest venvs resolve 2.21.0.

State of the ecosystem on 2026-10-01. The latest release is tensorboard 2.21.0 (PyPI 2026-06-29). RELEASE.md: NumPy 2.0 fixes in 2.17.1/2.18.0 (#​6871); 2.21.0 'Fixed runtime crashes caused by the missing pkg_resources library' (#​7057) and 'Fix a Projector plugin vulnerability (#​7115)', and requires protobuf>=6.31.1,<8. I reproduced the floor failure in the K-3 lower venv (Python 3.13.14, tensorboard 2.15.0, numpy 2.5.3, setuptools 84.0.0). torch's SummaryWriter writes the event file. EventAccumulator fails with AttributeError: 'np.string_ was removed in the NumPy 2.0 release' (tensorboard/compat/tensorflow_stub/dtypes.py:677). 'import tensorboard.main' fails with ModuleNotFoundError: No module named 'pkg_resources' (tensorboard/default.py:30). In the latest venv, tensorboard 2.21.0 reads the same events and imports main. OSV lists no tensorboard advisories (primary).

Impact. At the declared floor, on the supported Python 3.13 with a current setuptools, an operator using the default backend can write logs but cannot read them or start TensorBoard from the same environment. The floor also predates the 2.21.0 projector fix.

Recommendation. Raise the floor to >=2.21.0 and keep <3. Even >=2.18 still breaks without pkg_resources on new setuptools. This needs the wandb and mlflow floors raised (protobuf>=6.31.1 compatible). Test both bounds in CI.

Sources.

Acceptance criteria

  • A regression test reproduces the failure scenario and asserts the corrected behaviour.
  • Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
  • Every finding in this issue is resolved or explicitly closed with a reason.

Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    dependenciesDependency or ecosystem currency
    severity: lowMinor defect, inaccuracy or rough edge
    source: review-2026-10Found by the October 2026 code and documentation review
    urgency: next-releaseTechnology currency: fix in the next release
    wave: 2Roadmap wave 2: high severity, larger effort; next-release tech
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: packaging-ciCode area: packaging-cibugSomething isn't workingdependenciesDependency or ecosystem currencyseverity: lowMinor defect, inaccuracy or rough edgesource: review-2026-10Found by the October 2026 code and documentation reviewurgency: next-releaseTechnology currency: fix in the next releasewave: 2Roadmap wave 2: high severity, larger effort; next-release tech

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions