Repository navigation
The uncapped wandb>=0.16.0 floor admits wandb 0.16.0-0.17.1, which fails at import wandb under NumPy 2, so report_to: wandb dies at Trainer construction (CR-A16-083) #530
Description
Wave 2 · theme D2-7: Next-release technology updates · highest severity Low · 2 finding(s) · effort trivial
Summary
The [tracking] extra declares wandb>=0.16.0 with no cap (pyproject.toml:92). I read the wandb 0.16.0 wheel (sha256 matches PyPI) and the 0.17.1 macOS arm64 wheel without running them. In both, wandb/sdk/data_types/dtypes.py runs a module-level 'if np:' block (np = get_module('numpy'), lazy) that does NumberType.types.append(np.float) (0.16.0:396; 0.17.1:393) and np.complex_ (0.16.0:399; 0.17.1:396). The block is reached on import through wandb/init.py:63 and wandb/data_types.py:33. NumPy 2.x removed both attributes; in the lower venv (numpy 2.5.3) both raise AttributeError. wandb's CHANGELOG lists 'Fix issues with numpy>=2 support' under 0.17.2 (2024-06-17). transformers 5.3.0's is_wandb_available() runs an unguarded import wandb from WandbCallback.init, so the error surfaces at Trainer construction, after the model has loaded. forgelm doctor probes extras with find_spec only (_doctor.py:84-111) and would report the extra as present. With ForgeLM's NumPy-2 stack this is a concrete broken pair: forced on the declared Python 3.13, and the default resolve elsewhere. A fresh resolve picks 0.30.0, which works per T09. An environment that already holds wandb 0.16.x/0.17.0-…
Primary finding
CR-A16-083 — The uncapped wandb>=0.16.0 floor admits wandb 0.16.0-0.17.1, which fails at import wandb under NumPy 2, so report_to: wandb dies at Trainer construction
severity Low · reachability documented-opt-in · intent intended-but-harmful · confidence likely (static only) · single reviewer · not reproduced (code trace)
What happens. The [tracking] extra declares wandb>=0.16.0 with no cap (pyproject.toml:92). I read the wandb 0.16.0 wheel (sha256 matches PyPI) and the 0.17.1 macOS arm64 wheel without running them. In both, wandb/sdk/data_types/dtypes.py runs a module-level 'if np:' block (np = get_module('numpy'), lazy) that does NumberType.types.append(np.float) (0.16.0:396; 0.17.1:393) and np.complex_ (0.16.0:399; 0.17.1:396). The block is reached on import through wandb/init.py:63 and wandb/data_types.py:33. NumPy 2.x removed both attributes; in the lower venv (numpy 2.5.3) both raise AttributeError. wandb's CHANGELOG lists 'Fix issues with numpy>=2 support' under 0.17.2 (2024-06-17). transformers 5.3.0's is_wandb_available() runs an unguarded import wandb from WandbCallback.init, so the error surfaces at Trainer construction, after the model has loaded. forgelm doctor probes extras with find_spec only (_doctor.py:84-111) and would report the extra as present. With ForgeLM's NumPy-2 stack this is a concrete broken pair: forced on the declared Python 3.13, and the default resolve elsewhere. A fresh resolve picks 0.30.0, which works per T09. An environment that already holds wandb 0.16.x/0.17.0-0.17.1 keeps it. mlflow>=2.9.0 (pyproject.toml:102) is not reportable on the same basis; see reviewed_clean.
Where.
Evidence.
- unzip -p wandb-0.16.0-py3-none-any.whl wandb/sdk/data_types/dtypes.py: line 13 'np = get_module("numpy")', line 365 'if np:', lines 396/399 np.float/np.complex_. The same calls sit at lines 393/396 in the 0.17.1 wheel. Lower venv: 'float_ AttributeError:
np.float_was removed in the NumPy 2.0 release'. CHANGELOG '## [0.17.2] - 2024-06-17' (line 1200), 'Fix issues withnumpy>=2support' (line 1208). PyPI: latest 0.30.0 (requires_python >=3.10).
Recommended fix. Raise the floor to at least wandb>=0.17.2, preferably a release whose protobuf range allows tensorboard 2.21 (protobuf>=6.31.1), and add <1.0. Add an offline (WANDB_MODE=offline) callback smoke test at both bounds.
Other findings with the same root cause
These close with the same fix; update them in the same pull request.
TECH-T09-007 — Experiment tracking, webhooks and monitoring: wandb>=0.16.0, uncapped (extra [tracking], pyproject.toml:91-93) (range-unsound)
severity Next release
Item. wandb>=0.16.0, uncapped (extra [tracking], pyproject.toml:91-93) · verdict: range-unsound · urgency: Next release
State in ForgeLM. pyproject.toml:92 declares wandb>=0.16.0 (released 2023-11-07) with no cap. The K-3 extras venv resolves 0.30.0. ForgeLM declares Python 3.10-3.13 (pyproject.toml:36-40; CI matrix .github/workflows/ci.yml:117) and no longer pins numpy<2.
State of the ecosystem on 2026-10-01. The latest release is wandb 0.30.0 (2026-09-09, Python >=3.10). transformers v5.18.0 WandbCallback (integration_utils.py:695) still uses wandb.init, config.update, define_metric, log and Artifact. Both researchers ran an offline run successfully with transformers 5.13.0 and wandb 0.30.0. I checked the floor itself by downloading the wandb-0.16.0 wheel (sha256 matches PyPI) and reading it. wandb/init.py:63 imports wandb.data_types, which imports sdk/data_types/dtypes.py (data_types.py:33). That module does np = get_module('numpy') and then, at module level, NumberType.types.append(np.float) (line 396) and np.complex_ (line 399). Both attributes were removed in NumPy 2.0; I confirmed np.float_ raises AttributeError with numpy 2.5.3. wandb's changelog lists 'Fix issues with numpy>=2 support' in 0.17.2 (2024-06-17). 0.16.0 also requires protobuf<5, which caps tensorboard at <=2.20 and keeps it from 2.21.0's pkg_resources and projector fixes. OSV lists no wandb advisories (primary).
Impact. With NumPy 2, which is the default resolution and the only option on Python 3.13, wandb 0.16.0 through 0.17.1 should fail at 'import wandb'. A run with report_to: wandb would then die at WandbCallback construction. The declared floor admits versions that cannot work in ForgeLM's own supported environment. This is static evidence plus the upstream changelog; the run was not executed (no pip).
Recommendation. Raise the floor to a version CI exercises, at least >=0.17.2 for NumPy 2 and preferably a 0.2x release whose protobuf range admits tensorboard 2.21. Add <1.0 (pre-1.0 minors remove APIs; 0.30.0 removed InternalApi and the PublicApi aliases) and a CI smoke run at both bounds with WANDB_MODE=offline. Document WANDB_MODE=offline/disabled for air-gapped runs.
Sources.
- wandb 0.16.0 wheel (sdk/data_types/dtypes.py:396,399 np.float/np.complex_ at import) (2023-11-07)
- wandb CHANGELOG (0.30.0 top entry; 0.17.2 numpy>=2 fix; protobuf v4 drop) (2026-09-09)
- PyPI wandb metadata (latest 0.30.0) (2026-09-09)
- wandb 0.16.0 metadata (protobuf<5) (2023-11-07)
- transformers v5.18.0 integration_utils.py (WandbCallback) (2026-09-30)
Acceptance criteria
- A regression test reproduces the failure scenario and asserts the corrected behaviour.
- Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
- Every finding in this issue is resolved or explicitly closed with a reason.
Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity