Use -O2 when building wheels for macOS and Linux

[radarhere]

Resolves #10057

[hugovk]

A lot of the other libs are also built unoptimised.

We could add this at the top of wheels-dependencies.sh instead?

export CFLAGS="-O2 $CFLAGS"
export CXXFLAGS="-O2 $CXXFLAGS"

Separately, this is probably worth an upstream multibuild change to default to optimised, this would have an ecosystem-wide improvement.

[radarhere]

Ok, I've updated this PR with those flags, and created multi-build/multibuild#557 to suggest it for multibuild.

[hugovk]

A benchmarking script:

"""Benchmark bundled dependency libraries via Pillow: best-of-N wall time."""

import sys
import time
from io import BytesIO

from PIL import Image, ImageCms, ImageDraw, ImageFont, features

N = 5
im = Image.effect_noise((2000, 1500), 40).convert("RGB")
results = {}


def best(name, fn, n=N):
    t = []
    for _ in range(n):
        s = time.perf_counter()
        fn()
        t.append(time.perf_counter() - s)
    results[name] = min(t) * 1000


def saved(fmt, **kw):
    buf = BytesIO()
    im.save(buf, fmt, **kw)
    return buf.getvalue()


def roundtrip(name, fmt, **kw):
    best(f"{name} save", lambda: saved(fmt, **kw))
    data = saved(fmt, **kw)
    best(f"{name} load", lambda: Image.open(BytesIO(data), formats=[fmt]).load())


roundtrip(
    "JPEG2000 lossy",
    "JPEG2000",
    irreversible=True,
    quality_mode="rates",
    quality_layers=[20],
)
roundtrip("JPEG2000 lossless", "JPEG2000")
roundtrip("PNG", "PNG")
roundtrip("JPEG", "JPEG", quality=90)
roundtrip("TIFF deflate", "TIFF", compression="tiff_adobe_deflate")
roundtrip("TIFF lzw", "TIFF", compression="tiff_lzw")
roundtrip("WebP", "WebP", quality=80)
if features.check("avif"):
    roundtrip("AVIF", "AVIF", quality=60)

# freetype: render text
font = ImageFont.truetype("/System/Library/Fonts/Supplemental/Arial.ttf", 40)
text = "The quick brown fox jumps over the lazy dog " * 4


def draw_text():
    canvas = Image.new("RGB", (2000, 1500))
    d = ImageDraw.Draw(canvas)
    for y in range(0, 1500, 50):
        d.text((0, y), text, font=font)


best("FreeType text", draw_text)

# lcms2: sRGB -> ProPhoto transform
srgb = ImageCms.createProfile("sRGB")
lab = ImageCms.createProfile("LAB")
xform = ImageCms.buildTransform(srgb, lab, "RGB", "LAB")
best("LittleCMS transform", lambda: ImageCms.applyTransform(im, xform))

label = sys.argv[1] if len(sys.argv) > 1 else ""
for k, v in results.items():
    print(f"{label}\t{k}\t{v:.1f}")

Fetch main and PR wheels and install in venvs:

gh run download 36242033748 -n "dist-macOS arm64" -D pr_wheels    # PR
gh run download 35808090359 -n "dist-macOS arm64" -D main_wheels  # main
uv venv -p 3.15 venv_pr   && uv pip install -p venv_pr/bin/python   pr_wheels/pillow-13.0.0.dev0-cp315-cp315-macosx_11_0_arm64.whl
uv venv -p 3.15 venv_main && uv pip install -p venv_main/bin/python main_wheels/pillow-13.0.0.dev0-cp315-cp315-macosx_11_0_arm64.whl

Run and compare:

venv_main/bin/python bench_deps.py main > main.tsv
venv_pr/bin/python   bench_deps.py pr   > pr.tsv
paste main.tsv pr.tsv | awk -F'\t' '{printf "%-24s main %7.1f ms   pr %7.1f ms   %.2fx\n", $2, $3, $6, $3/$6}'

JPEG2000 lossy save      main  2189.4 ms   pr   981.3 ms   2.23x
JPEG2000 lossy load      main   267.5 ms   pr    83.1 ms   3.22x
JPEG2000 lossless save   main  2168.4 ms   pr   970.6 ms   2.23x
JPEG2000 lossless load   main  2280.2 ms   pr   879.9 ms   2.59x
PNG save                 main   140.2 ms   pr   140.5 ms   1.00x
PNG load                 main    34.6 ms   pr    34.9 ms   0.99x
JPEG save                main    10.5 ms   pr    10.2 ms   1.03x
JPEG load                main    16.7 ms   pr    16.6 ms   1.01x
TIFF deflate save        main    93.7 ms   pr    95.0 ms   0.99x
TIFF deflate load        main    26.0 ms   pr    26.0 ms   1.00x
TIFF lzw save            main   104.3 ms   pr    49.1 ms   2.12x
TIFF lzw load            main    30.8 ms   pr    13.3 ms   2.32x
WebP save                main   318.5 ms   pr   323.7 ms   0.98x
WebP load                main    77.9 ms   pr    78.0 ms   1.00x
AVIF save                main   317.4 ms   pr   293.1 ms   1.08x
AVIF load                main    14.4 ms   pr    13.8 ms   1.04x
FreeType text            main   216.6 ms   pr    64.6 ms   3.35x
LittleCMS transform      main   148.0 ms   pr    47.2 ms   3.14x

Speedups of 2x to 3x on JPEG 2000, TIFF LZW, FreeType and LittleCMS. 🚀

Others unchanged as they were already built optimised.

This is worth mentioning in the release notes :)

[hugovk]

Ah, before merge, Linux wheels are x3 bigger. Lucky we have this extra tooling :)

Details

Dist size comparison vs Pillow 12.3.0 on PyPI

File	Size before	Size now	Change
sdist (.tar.gz)	47.0 MB	47.0 MB	🟢 -0.03%
cp310-cp310-macosx_10_10_x86_64.whl	5.4 MB	n/a	n/a
cp310-cp310-macosx_11_0_arm64.whl	4.8 MB	n/a	n/a
cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	n/a	n/a
cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	n/a	n/a
cp310-cp310-musllinux_1_2_aarch64.whl	6.3 MB	n/a	n/a
cp310-cp310-musllinux_1_2_x86_64.whl	7.0 MB	n/a	n/a
cp310-cp310-win32.whl	6.5 MB	n/a	n/a
cp310-cp310-win_amd64.whl	7.2 MB	n/a	n/a
cp310-cp310-win_arm64.whl	2.6 MB	n/a	n/a
cp311-cp311-macosx_10_10_x86_64.whl	5.4 MB	5.6 MB	🟡 +3.35%
cp311-cp311-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.12%
cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.1 MB	🔴 +253.00%
cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	22.7 MB	🔴 +228.01%
cp311-cp311-musllinux_1_2_aarch64.whl	6.3 MB	21.9 MB	🔴 +245.06%
cp311-cp311-musllinux_1_2_x86_64.whl	7.0 MB	22.5 MB	🔴 +219.89%
cp311-cp311-win32.whl	6.5 MB	6.5 MB	🟡 +0.28%
cp311-cp311-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
cp311-cp311-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.72%
cp312-cp312-macosx_10_13_x86_64.whl	5.3 MB	5.6 MB	🟡 +3.84%
cp312-cp312-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.1 MB	🔴 +252.86%
cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	22.7 MB	🔴 +227.76%
cp312-cp312-musllinux_1_2_aarch64.whl	6.3 MB	21.9 MB	🔴 +244.76%
cp312-cp312-musllinux_1_2_x86_64.whl	7.1 MB	22.5 MB	🔴 +219.65%
cp312-cp312-win32.whl	6.5 MB	6.5 MB	🟡 +0.28%
cp312-cp312-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
cp312-cp312-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.72%
cp313-cp313-ios_13_0_arm64_iphoneos.whl	4.2 MB	4.2 MB	🟡 +0.60%
cp313-cp313-ios_13_0_arm64_iphonesimulator.whl	4.3 MB	4.3 MB	🟡 +0.73%
cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl	3.7 MB	3.7 MB	🟡 +0.61%
cp313-cp313-macosx_10_13_x86_64.whl	5.3 MB	5.6 MB	🟡 +3.84%
cp313-cp313-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.1 MB	🔴 +252.99%
cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	22.7 MB	🔴 +227.91%
cp313-cp313-musllinux_1_2_aarch64.whl	6.3 MB	21.9 MB	🔴 +244.97%
cp313-cp313-musllinux_1_2_x86_64.whl	7.0 MB	22.5 MB	🔴 +219.77%
cp313-cp313-win32.whl	6.5 MB	6.5 MB	🟡 +0.28%
cp313-cp313-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
cp313-cp313-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.72%
cp314-cp314-ios_13_0_arm64_iphoneos.whl	4.2 MB	4.2 MB	🟡 +0.60%
cp314-cp314-ios_13_0_arm64_iphonesimulator.whl	4.3 MB	4.3 MB	🟡 +0.73%
cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl	3.7 MB	3.7 MB	🟡 +0.62%
cp314-cp314-macosx_10_15_x86_64.whl	5.4 MB	5.6 MB	🟡 +3.82%
cp314-cp314-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.1 MB	🔴 +252.99%
cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	22.7 MB	🔴 +227.92%
cp314-cp314-musllinux_1_2_aarch64.whl	6.3 MB	21.9 MB	🔴 +245.01%
cp314-cp314-musllinux_1_2_x86_64.whl	7.0 MB	22.5 MB	🔴 +219.82%
cp314-cp314-win32.whl	6.5 MB	6.5 MB	🟡 +0.27%
cp314-cp314-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
cp314-cp314-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.73%
cp314-cp314t-macosx_10_15_x86_64.whl	5.4 MB	5.6 MB	🟡 +3.83%
cp314-cp314t-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.2 MB	🔴 +250.66%
cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	7.0 MB	22.8 MB	🔴 +226.23%
cp314-cp314t-musllinux_1_2_aarch64.whl	6.4 MB	21.9 MB	🔴 +243.05%
cp314-cp314t-musllinux_1_2_x86_64.whl	7.1 MB	22.6 MB	🔴 +218.35%
cp314-cp314t-win32.whl	6.5 MB	6.5 MB	🟡 +0.27%
cp314-cp314t-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.32%
cp314-cp314t-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.77%
cp315-cp315-ios_13_0_arm64_iphoneos.whl	4.2 MB	4.2 MB	🟡 +0.60%
cp315-cp315-ios_13_0_arm64_iphonesimulator.whl	4.3 MB	4.3 MB	🟡 +0.74%
cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl	3.7 MB	3.7 MB	🟡 +0.61%
cp315-cp315-macosx_10_15_x86_64.whl	5.4 MB	5.6 MB	🟡 +3.82%
cp315-cp315-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.1 MB	🔴 +252.59%
cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	6.9 MB	22.8 MB	🔴 +227.55%
cp315-cp315-musllinux_1_2_aarch64.whl	6.4 MB	21.9 MB	🔴 +244.39%
cp315-cp315-musllinux_1_2_x86_64.whl	7.1 MB	22.6 MB	🔴 +219.43%
cp315-cp315-win32.whl	6.5 MB	6.5 MB	🟡 +0.27%
cp315-cp315-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
cp315-cp315-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.73%
cp315-cp315t-macosx_10_15_x86_64.whl	5.4 MB	5.6 MB	🟡 +3.83%
cp315-cp315t-macosx_11_0_arm64.whl	4.8 MB	4.8 MB	🟡 +1.14%
cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	6.3 MB	22.2 MB	🔴 +250.44%
cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	7.0 MB	22.8 MB	🔴 +225.97%
cp315-cp315t-musllinux_1_2_aarch64.whl	6.4 MB	21.9 MB	🔴 +242.67%
cp315-cp315t-musllinux_1_2_x86_64.whl	7.1 MB	22.6 MB	🔴 +218.04%
cp315-cp315t-win32.whl	6.5 MB	6.5 MB	🟡 +0.27%
cp315-cp315t-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.32%
cp315-cp315t-win_arm64.whl	2.6 MB	2.7 MB	🟡 +3.76%
pp311-pypy311_pp73-macosx_10_15_x86_64.whl	5.3 MB	5.5 MB	🟡 +3.81%
pp311-pypy311_pp73-macosx_11_0_arm64.whl	4.7 MB	4.8 MB	🟡 +1.09%
pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl	5.2 MB	21.1 MB	🔴 +302.33%
pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl	5.9 MB	21.7 MB	🔴 +269.62%
pp311-pypy311_pp73-win_amd64.whl	7.2 MB	7.3 MB	🟡 +0.33%
wheel average (77 wheels)	5.7 MB	11.8 MB	🔴 +108.58%
wheel total (77 wheels)	436.2 MB	909.9 MB	🔴 +108.58%
artifacts total (78 artifacts)	483.3 MB	956.9 MB	🔴 +98.01%

https://github.com/python-pillow/Pillow/actions/runs/36242033748

multibuild only sets defaults when CFLAGS is empty. Let's try moving these two after when we source multibuild, so we don't overwrite multibuild's -Wl,-strip-all for Linux:

# Define custom utilities
source wheels/multibuild/common_utils.sh
source wheels/multibuild/library_builders.sh
if [[ -z "$IS_MACOS" ]]; then
    source wheels/multibuild/manylinux_utils.sh
fi

[radarhere]

Ok, I've moved the code, and now the wheels are only 1.39% bigger.

[radarhere]

multi-build/multibuild#557 has been merged, so this PR now simply updates multibuild.

[hugovk]

Thanks!

Release notes suggestion:

Faster JPEG 2000, TIFF, FreeType and LittleCMS in macOS and Linux wheels
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Several of the libraries bundled in the macOS and Linux wheels were built without compiler optimisation. The Windows wheels were not affected.

These libraries are now compiled with ``-O2``. Compared to Pillow 12.3.0, the wheels decode JPEG 2000 images 2 to 3 times faster and encode them about twice as fast, LZW-compressed TIFF images are read and written about twice as fast, text rendering with :py:mod:`~PIL.ImageFont` is up to 3 times faster, and :py:mod:`~PIL.ImageCms` transforms are up to 3 times faster.

PNG, JPEG, WebP and AVIF are unaffected, as those libraries were already built with optimisation.