Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -237,14 +237,17 @@ jobs:
# EXPLICITLY so this job tests the true documented minimum — trl puts
# no upper bound on transformers, so an unpinned resolve would silently
# pull the newest 5.x and stop exercising the floor. transformers
# 5.3.0 is the CVE-2026-4372 fix floor and requires torch>=2.4 /
# huggingface_hub>=1.3 / tokenizers>=0.22; trl 1.0.0 pulls the
# accelerate>=1.4 and datasets>=4.7 floor. Keep 1:1 with the pyproject
# [project].dependencies minimums.
# 5.10.1 is the first installable release past CVE-2026-4372,
# CVE-2026-5241 and CVE-2026-9856 (5.10.0, which carries the last of
# them, was yanked); it requires huggingface_hub>=1.5 /
# tokenizers>=0.22, and ForgeLM itself needs torch>=2.4; trl 1.0.0
# pulls the accelerate>=1.4 and datasets>=4.7 floor. Keep 1:1 with the
# pyproject [project].dependencies minimums (tests/test_packaging.py
# compares them).
pip install \
"torch==2.4.0" \
"transformers==5.3.0" \
"huggingface_hub==1.3.0" \
"transformers==5.10.1" \
"huggingface_hub==1.5.0" \
"peft==0.19.0" \
"accelerate==1.4.0" \
"datasets==4.7.0" \
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ All notable changes to ForgeLM are documented here.

### Changed

- Raise the `transformers` floor from `>=5.3.0` to `>=5.10.1` (and `huggingface_hub` from `>=1.3.0` to `>=1.5.0`, which
transformers 5.10 requires). `5.3.0` was described as the CVE fix floor but still admitted CVE-2026-5241 (a LightGlue
`config.json` overrides `trust_remote_code: false`, fixed in 5.5.0) and CVE-2026-9856 (a `chat_template` name in
`tokenizer_config.json` is used unsanitised as a file name by `save_pretrained`, so `final_model/` saves, GGUF export
and merges could write outside the output directory; fixed in 5.10.0, which PyPI yanked, hence 5.10.1). The nightly
minimum-versions job pins the new floor, and `tests/test_packaging.py` keeps it from drifting from
`pyproject.toml` and checks that a traversal `chat_template` name cannot escape on save. Environments pinned to
transformers 5.3 to 5.9 must upgrade (#109).
- Record monitoring default provenance in `defaulted_fields` and mark literal URL projection with
`endpoint_redacted` (also true when the URL already contains only scheme/host/port). Literal
endpoints retain scheme/host/port in JSON evidence, including IPv6 brackets; userinfo, path,
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/supply_chain_security-tr.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ bir [`tools/pip_audit_ignores.yaml`](../../tools/pip_audit_ignores.yaml)
taşır (şu an dokuz no-fix torch advisory'si ve bir
OSV-yanlış-sınıflandırılmış markdown kaydı; daha önce taşıdığı iki
transformers advisory'si `transformers>=5.3.0` floor'u onları pip-audit
tarafından raporlanamaz hâle getirince kaldırıldı). Bu dosya yalnız projenin kendi workflow'u tarafından
tarafından raporlanamaz hâle getirince kaldırıldı; floor artık `>=5.10.1`). Bu dosya yalnız projenin kendi workflow'u tarafından
`--ignores` ile tüketilir; deployer'lar miras almaz. Her release
cycle'da gözden geçirilir.

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/supply_chain_security.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ ForgeLM's own nightly does carry a checked-in
project-internal triage (currently nine no-fix torch advisories and one
OSV-misclassified markdown record; the two transformers advisories it once
carried were dropped once the `transformers>=5.3.0` floor made them
unreportable by pip-audit). That file is consumed only by the
unreportable by pip-audit; the floor is now `>=5.10.1`). That file is consumed only by the
project's own workflow via `--ignores`; deployers do not inherit it.
Review at every release cycle.

Expand Down
2 changes: 1 addition & 1 deletion docs/usermanuals/en/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ ForgeLM ships as a single PyPI package with optional dependency groups (called *
:::

:::note
**Intel Macs (x86_64) are not supported.** ForgeLM requires `torch>=2.4.0` (pulled in by `transformers>=5.3.0`), and PyPI publishes no `torch>=2.4` wheel for x86_64 macOS. Use Apple Silicon, Linux, or Windows.
**Intel Macs (x86_64) are not supported.** ForgeLM requires `torch>=2.4.0` (the minimum that `transformers` 5 supports), and PyPI publishes no `torch>=2.4` wheel for x86_64 macOS. Use Apple Silicon, Linux, or Windows.
:::

## Base install
Expand Down
2 changes: 1 addition & 1 deletion docs/usermanuals/tr/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ ForgeLM, PyPI'da tek bir paket olarak gelir; ağır özellikler için opsiyonel
:::

:::note
**Intel Mac'ler (x86_64) desteklenmiyor.** ForgeLM `torch>=2.4.0` gerektirir (`transformers>=5.3.0` üzerinden gelir) ve PyPI'da x86_64 macOS için `torch>=2.4` wheel'i yok. Apple Silicon, Linux veya Windows kullanın.
**Intel Mac'ler (x86_64) desteklenmiyor.** ForgeLM `torch>=2.4.0` gerektirir (`transformers` 5'in desteklediği asgari sürüm) ve PyPI'da x86_64 macOS için `torch>=2.4` wheel'i yok. Apple Silicon, Linux veya Windows kullanın.
:::

## Temel kurulum
Expand Down
8 changes: 6 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,19 @@ dependencies = [
# the transformers-5 migration (v0.9.0). Apple Silicon, Linux and Windows
# are unaffected. The previous `numpy<2; darwin x86_64` ABI-guard marker is
# gone with the platform it protected (torch 2.4+ ships against NumPy 2.x).
"transformers>=5.3.0,<6.0.0",
# 5.10.1 is the first installable release past three advisories that reach ForgeLM's own load and save path from
# a Hub repo: CVE-2026-4372 (fixed 5.3.0), CVE-2026-5241 (a LightGlue config.json overrides trust_remote_code;
# fixed 5.5.0) and CVE-2026-9856 (chat_template names used unsanitised as file names by save_pretrained; fixed
# 5.10.0, which PyPI yanked, hence 5.10.1). It also needs huggingface_hub>=1.5.0.
"transformers>=5.10.1,<6.0.0",
"peft>=0.19.0,<1.0.0",
"datasets>=4.7.0,<6.0.0",
"accelerate>=1.4.0,<2.0.0",
"pyyaml>=6.0.1,<7.0.0",
"pydantic>=2.0.0,<3.0.0",
"trl>=1.0.0,<2.0.0",
"tensorboard>=2.15.0,<3.0.0",
"huggingface_hub>=1.3.0,<2.0.0",
"huggingface_hub>=1.5.0,<2.0.0",
"requests>=2.32.2,<3.0.0",
# requests-toolbelt is required by forgelm._http for the HostHeaderSSLAdapter
# used to close the DNS-rebinding TOCTOU window in safe_post / safe_get
Expand Down
41 changes: 41 additions & 0 deletions tests/test_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,47 @@ def test_existing_pad_token_preserved(self):
assert fake_tok.pad_token == "[PAD]"


class TestChatTemplateNameCannotEscapeTheSaveDirectory:
"""CVE-2026-9856: a ``chat_template`` name from a Hub repo was used as a file name by ``save_pretrained``.

ForgeLM loads a tokenizer from a Hub repo and saves it again to ``final_model/`` (trainer, GGUF export, merge), so
a crafted ``tokenizer_config.json`` could write a file outside the output directory. Fixed in transformers
5.10.0 (yanked) and 5.10.1, which is why the declared floor is ``>=5.10.1``. This uses the installed transformers
and a tiny local tokenizer, so it also fails on a floor install that predates the fix.
"""

def _crafted_tokenizer_dir(self, root):
import json

from tokenizers import Tokenizer, models, pre_tokenizers
from transformers import PreTrainedTokenizerFast

tok = Tokenizer(models.WordLevel({"[UNK]": 0, "hello": 1, "</s>": 2}, unk_token="[UNK]"))
tok.pre_tokenizer = pre_tokenizers.Whitespace()
source = root / "hub_repo"
PreTrainedTokenizerFast(tokenizer_object=tok, unk_token="[UNK]", eos_token="</s>").save_pretrained(source)
config_file = source / "tokenizer_config.json"
config = json.loads(config_file.read_text(encoding="utf-8"))
config["chat_template"] = [
{"name": "../../../ESCAPED_BY_CHAT_TEMPLATE", "template": "{{ messages }}"},
{"name": "default", "template": "{{ messages }}"},
]
config_file.write_text(json.dumps(config), encoding="utf-8")
return source

def test_a_traversal_name_is_refused_and_nothing_is_written_outside(self, tmp_path):
source = self._crafted_tokenizer_dir(tmp_path)
out = tmp_path / "run" / "final_model"
cfg = _cfg(model={"name_or_path": str(source)})

with pytest.raises(ValueError, match="Invalid chat template name"):
tokenizer = model_mod._load_tokenizer(cfg, trust_remote_code=False)
tokenizer.save_pretrained(out)

escaped = [p for p in tmp_path.rglob("*") if p.is_file() and "ESCAPED" in p.name]
assert escaped == [], f"a chat_template name wrote outside the save directory: {escaped}"


class TestBuildLoraConfig:
"""``_build_lora_config`` resolves the PEFT method and target modules."""

Expand Down
79 changes: 79 additions & 0 deletions tests/test_packaging.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,85 @@ def test_setuptools_security_floor_meets_pysec_2026_3447() -> None:
)


def _load_runtime_dependencies() -> list:
try:
import tomllib # type: ignore[import-not-found]
except ModuleNotFoundError: # pragma: no cover — Python 3.10 fallback
try:
import tomli as tomllib # type: ignore[import-not-found, no-redef]
except ModuleNotFoundError:
pytest.skip("Neither tomllib (3.11+) nor tomli is available; runtime dependency assertion skipped.")
pyproject_path = Path(__file__).resolve().parent.parent / "pyproject.toml"
with pyproject_path.open("rb") as fh:
pyproject = tomllib.load(fh)
return pyproject.get("project", {}).get("dependencies", [])


# transformers 5.10.1 is the first installable release past three advisories that reach ForgeLM's own load and
# save path from a Hub repo: CVE-2026-4372 (5.3.0), CVE-2026-5241 (a LightGlue config.json overrides
# trust_remote_code; 5.5.0) and CVE-2026-9856 (a chat_template name used as a file name by save_pretrained;
# 5.10.0, which PyPI yanked). A floor of ">=5.3.0" was documented as "the CVE fix floor" and was not.
_TRANSFORMERS_SECURITY_FLOOR = Version("5.10.1")


def test_transformers_floor_is_past_cve_2026_5241_and_cve_2026_9856() -> None:
"""``transformers`` must not admit a release that predates the chat_template / trust_remote_code fixes."""
found = [
Requirement(spec)
for spec in _load_runtime_dependencies()
if canonicalize_name(Requirement(spec).name) == "transformers"
]
assert len(found) == 1, f"expected exactly one transformers requirement in [project].dependencies, found {found!r}"
specifier_set = found[0].specifier
lower_bounds = [
Version(spec.version.rstrip(".*")) for spec in specifier_set if spec.operator in _LOWER_BOUND_OPERATORS
]
assert lower_bounds, f"transformers entry {str(found[0])!r} has no lower bound"
assert max(lower_bounds) >= _TRANSFORMERS_SECURITY_FLOOR, (
f"transformers entry {str(found[0])!r} floors at {max(lower_bounds)}, below {_TRANSFORMERS_SECURITY_FLOOR}, "
"the first installable release past CVE-2026-5241 and CVE-2026-9856. Do not lower it without re-verifying "
"both advisories against the release you lower it to."
)
for vulnerable in ("5.3.0", "5.4.0", "5.9.0", "5.10.0"):
assert not specifier_set.contains(Version(vulnerable), prereleases=True), (
f"transformers entry {str(found[0])!r} still admits {vulnerable}"
)


def test_nightly_floor_job_pins_the_runtime_dependency_minimums() -> None:
"""Every ``name==X`` the nightly minimum-versions job installs is the ``>=X`` floor in pyproject.

The job installs with ``--no-deps`` afterwards, so it can only test the documented minimum if its list is the
same list. It is a second hand-kept copy of every floor, and it kept saying ``transformers==5.3.0`` after the
floor was meant to be a security floor.
"""
import re

nightly = (Path(__file__).resolve().parents[1] / ".github" / "workflows" / "nightly.yml").read_text(
encoding="utf-8"
)
step = nightly.split("Install minimum dependency versions", 1)[1].split("- name:", 1)[0]
pins = {
canonicalize_name(name): version for name, version in re.findall(r'"([A-Za-z0-9_.-]+)==([0-9][^"]*)"', step)
}
assert pins, "the nightly minimum-versions step no longer installs `name==version` pins — update this test"

floors = {}
for spec in _load_runtime_dependencies():
req = Requirement(spec)
lowers = [s.version for s in req.specifier if s.operator == ">="]
if lowers:
floors[canonicalize_name(req.name)] = lowers[0]

for name, version in pins.items():
assert name in floors, f"the nightly floor job pins {name}=={version}, which is not a runtime dependency"
assert version == floors[name], (
f"the nightly floor job pins {name}=={version} but pyproject.toml declares {name}>={floors[name]}; "
"the job would test a minimum nobody documents"
)
assert {"transformers", "huggingface-hub"} <= set(pins), "the floor job must pin transformers and huggingface_hub"


def _lint_job_pins() -> dict[str, Requirement]:
"""Requirements the ci.yml lint job installs directly.

Expand Down
3 changes: 3 additions & 0 deletions tools/pip_audit_ignores.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ ignores:
# so a suppression would be dead weight:
# - CVE-2026-1839 — fixed in transformers 5.0.0rc3.
# - PYSEC-2025-217 / CVE-2025-14929 (X-CLIP RCE) — OSV last_affected 5.0.0rc0.
# The floor has since moved to transformers>=5.10.1, the first installable
# release past CVE-2026-5241 (fixed 5.5.0) and CVE-2026-9856 (fixed 5.10.0,
# yanked), so neither needs a suppression either.

# ----- torch (added 2026-05-21, triage in issue #58) -----
#
Expand Down
Loading