Skip to content

[Triage] Docker build installs ForgeLM with the distro's old pip (jammy python3-pip 22.0.x, never upgraded) under a non-default python3.11, so neither PEP 660 editable install at Dockerfil… (CR-A16-114) #825

Open

Description

@cemililik

Needs triage. Reviewers disagreed on this finding, so it is not counted in the backlog. Decide whether it is a real defect: if yes, set severity and wave; if not, close it with the reason.

CR-A16-114 — Docker build installs ForgeLM with the distro's old pip (jammy python3-pip 22.0.x, never upgraded) under a non-default python3.11, so neither PEP 660 editable install at Dockerfile:52 nor the final i…

severity Medium · confidence disputed · single reviewer · not reproduced (code trace)

What happens. This extends M-A16-035 (the image is never built) with a specific build-time risk. Dockerfile:33-39 installs python3.11, python3.11-venv and the jammy python3-pip, then points /usr/bin/python3 at 3.11. Every later install (Dockerfile:52, 54, 60, 66, 97) runs python3 -m pip, which is jammy's Debian-patched pip 22.0.x from /usr/lib/python3/dist-packages. That pip was packaged for the default python3.10. Nothing upgrades it, and nothing creates a venv, although python3.11-venv is installed. PIP_DISABLE_PIP_VERSION_CHECK=1 (Dockerfile:31) also hides the version warning. On the backend side, the build works. In an offline run, setuptools 84.0.0 (which satisfies the setuptools>=83.0.0 floor at pyproject.toml:14) builds the PEP 660 editable wheel from the 3-file context at Dockerfile:47-48. It needs no import because the version is static at pyproject.toml:19, and LICENSE is not needed. It also builds a full wheel from the .dockerignore-filtered context with all package data. Both emit Metadata-Version 2.4 with only deprecation warnings. On the frontend side, two documented-in-the-wild failure modes of Ubuntu 22.04's pip 22.0.x could not be checked offline: (a) under python3.11, pip's locations code may import distutils.cmd, and jammy may not ship that for 3.11; (b) pip 22.0's build isolation strips only get_purelib()/get_platlib() from sys.path. Modern pip strips site.getsitepackages() instead (pip 26.2.1 build_env/virtual.py:23-26,56-90). Debian's /usr/lib/python3/dist-packages, which holds setuptools 59.6.0, would therefore stay ahead of the isolated setuptools>=83. setuptools 59.6 has no build_editable hook and no PEP 621 support, so Dockerfile:52 would fail ('build backend is missing the build_editable hook'), or Dockerfile:97 would install a distribution c…

Failure scenario. An operator runs docker build -t forgelm . on any host. The RUN at Dockerfile:52 runs jammy pip 22.0.x under python3.11. If that pip imports distutils (hypothesis a) or lets setuptools 59.6 from /usr/lib/python3/dist-packages shadow the isolated setuptools>=83 (hypothesis b), the editable install aborts and the image cannot be built. Alternatively, Dockerfile:97 installs a metadata-less UNKNOWN distribution, and the forgelm console script is missing at Dockerfile:110. If neither holds, the build passes; offline analysis cannot tell which.

Where.

Evidence.

  • Dockerfile:36 installs python3-pip (distro pip). No pip install --upgrade pip, no ensurepip and no venv anywhere in Dockerfile:1-113. Offline setuptools 84.0.0 builds succeed. Log: (local review artefact) (sha256 564ab9a30ce780387e0a09c3a79f55a2c23331fb5776ee08b2fc55760c1477eb). The editable wheel forgelm-0.11.1rc1-0.editable-py3-none-any.whl contains only __editable__.forgelm-0.11.1rc1.pth (adds the project root) plus the dist-info. The full wheel contains py.typed, safety_prompts/defa…

Recommended fix. Do not depend on the distro pip. In the base stage, create a venv (python3.11 -m venv /opt/venv && /opt/venv/bin/python -m pip install --upgrade 'pip>=23') and put /opt/venv/bin on PATH. This also removes the need for update-alternatives. At minimum, run python3.11 -m pip install --upgrade pip before Dockerfile:52. Add a docker build job to CI (M-A16-035) so this is settled empirically.


Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.

Activity

  1. added
    bugSomething isn't working
    needs-triageNot triaged yet: every new issue starts with it; only a maintainer removes it
    source: review-2026-10Found by the October 2026 code and documentation review
    on Oct 9, 2026
  2. cemililik commented on Oct 20, 2026

    @cemililik
    CollaboratorAuthor

    Confirmed, and more serious than the finding's first grading. We reproduced it in a clean ubuntu:22.04 container with the packages the Dockerfile installs (python3.11, python3.11-venv, python3-pip, then update-alternatives to python3.11) and the repository's real pyproject.toml:

    • python3 -m pip install -e . (Dockerfile:52) stops with ERROR: Project file:///app has a 'pyproject.toml' and its build backend is missing the 'build_editable' hook. pip is 22.0.2 from /usr/lib/python3/dist-packages; the isolated build environment evidently does not get the setuptools that build-system.requires asks for.
    • Without -e (Dockerfile:97) the same pip builds a package named UNKNOWN-0.0.0, i.e. it ignores the [project] table.
    • After python3 -m pip install -U pip, both work.

    So docker build -t forgelm . (docs/guides/enterprise_deployment.md, docker-compose.yaml) fails at the first install step of the current Dockerfile. We did not build the full CUDA image; the reproduction covers the pip step, which is where it stops.

    Fix: upgrade pip (or use a venv with a current pip) before the first install. That is the same remedy as #538 and it belongs in the container step #899, but it does not have to wait for it. A CI docker build (as #517 asks) would have caught this.

    Triage result: bug, area: packaging-ci, severity: high, wave: 1.

    Triage round 1, checked against development on 2026-10-20.

  3. added
    severity: highWrong behaviour or a false claim users rely on
    wave: 1Roadmap wave 1: high severity, small effort
    on Oct 20, 2026
  4. removed
    needs-triageNot triaged yet: every new issue starts with it; only a maintainer removes it
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: packaging-ciCode area: packaging-cibugSomething isn't workingseverity: highWrong behaviour or a false claim users rely onsource: review-2026-10Found by the October 2026 code and documentation reviewwave: 1Roadmap wave 1: high severity, small effort

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions