Skip to content

Commit e78b1ea

Browse files
luhenryluhenry
committed
cvxopt: Add version 1.3.3
Convex optimization package: seven C extensions over BLAS/LAPACK and SuiteSparse, plus the optional fftw one. Upstream publishes no riscv64 wheel. Rocky 10 has no SuiteSparse, GSL or GLPK for riscv64 (and no EPEL to fall back on), so umfpack/cholmod/amd are compiled from the SuiteSparse tarball upstream's own CI pins, and the gsl/glpk/dsdp extensions are left off. Adds gotchas 388 and 389.
1 parent f504ba5 commit e78b1ea

5 files changed

Lines changed: 268 additions & 0 deletions

File tree

‎.github/workflows/build-cvxopt.yml‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# SPDX-FileCopyrightText: 2026 The RISE Project
2+
# SPDX-License-Identifier: MIT
3+
---
4+
# This workflow is based on: https://github.com/cvxopt/cvxopt/blob/1.3.3/.github/workflows/linux_build.yml
5+
name: Build cvxopt wheels (riscv64)
6+
7+
on:
8+
workflow_dispatch:
9+
inputs:
10+
version:
11+
description: 'Version glob to (re)build; empty builds every version of docs/packages/cvxopt.yaml not released yet'
12+
required: false
13+
default: ''
14+
pull_request:
15+
branches: [main]
16+
paths:
17+
- '.github/workflows/build-cvxopt.yml'
18+
- 'docs/packages/cvxopt.yaml'
19+
push:
20+
branches: [main]
21+
paths:
22+
- '.github/workflows/build-cvxopt.yml'
23+
- 'docs/packages/cvxopt.yaml'
24+
25+
concurrency:
26+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
27+
cancel-in-progress: true
28+
29+
permissions:
30+
contents: read # to fetch code (actions/checkout)
31+
32+
env:
33+
MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64
34+
# Rocky 10 has no SuiteSparse for riscv64, so umfpack/cholmod/amd are compiled
35+
# from source; version and hash are upstream's own pin (linux_build.yml).
36+
SUITESPARSE_VERSION: '7.11.0'
37+
SUITESPARSE_SHA256: '93ed4c4e546a49fc75884c3a8b807d5af4a91e39d191fbbc60a07380b12a35d1'
38+
39+
jobs:
40+
setup:
41+
uses: $/.github/workflows/_setup.yml
42+
with:
43+
package: cvxopt
44+
version: ${{ inputs.version }}
45+
46+
build_wheels:
47+
needs: [setup]
48+
if: needs.setup.outputs.versions != '[]'
49+
name: Build cvxopt ${{ matrix.version }} ${{ matrix.python }}-manylinux_riscv64
50+
runs-on: ubuntu-24.04-riscv
51+
timeout-minutes: 180
52+
strategy:
53+
fail-fast: false
54+
matrix:
55+
version: ${{ fromJSON(needs.setup.outputs.versions) }}
56+
python:
57+
- "cp312"
58+
- "cp313"
59+
- "cp314"
60+
- "cp314t"
61+
62+
env:
63+
CVXOPT_VERSION: ${{ matrix.version }}
64+
65+
steps:
66+
- name: Checkout cvxopt ${{ env.CVXOPT_VERSION }}
67+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
68+
with:
69+
repository: cvxopt/cvxopt
70+
ref: ${{ env.CVXOPT_VERSION }}
71+
fetch-depth: 0
72+
persist-credentials: false
73+
74+
- name: Build wheels
75+
uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074 # v4.2.0
76+
with:
77+
output-dir: wheelhouse/
78+
only: ${{ matrix.python }}-manylinux_riscv64
79+
env:
80+
CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}
81+
# Replaces upstream's before_all_linux.sh, which exits on any
82+
# architecture other than x86_64/i686/aarch64.
83+
CIBW_BEFORE_ALL_LINUX: >-
84+
dnf -y install openblas-devel fftw-devel &&
85+
curl -fsSL -o /tmp/suitesparse.tar.gz https://github.com/DrTimothyAldenDavis/SuiteSparse/archive/v${{ env.SUITESPARSE_VERSION }}.tar.gz &&
86+
echo "${{ env.SUITESPARSE_SHA256 }} /tmp/suitesparse.tar.gz" | sha256sum -c - &&
87+
tar xzf /tmp/suitesparse.tar.gz -C {project} &&
88+
cp {project}/SuiteSparse-${{ env.SUITESPARSE_VERSION }}/LICENSE.txt {project}/LICENSE.suitesparse &&
89+
cp /usr/share/licenses/openblas/LICENSE {project}/LICENSE.openblas &&
90+
cp /usr/share/licenses/fftw-libs-double/COPYING {project}/LICENSE.fftw
91+
# Upstream's own linux environment table, minus the gsl, glpk and dsdp
92+
# extensions (no riscv64 packages, and no EPEL for it either). The
93+
# *_LIB_DIR overrides are needed because setup.py only guesses
94+
# /usr/lib64 when it finds SuiteSparse installed there.
95+
CIBW_ENVIRONMENT: >-
96+
CVXOPT_BLAS_LIB=openblas
97+
CVXOPT_LAPACK_LIB=openblas
98+
CVXOPT_BLAS_LIB_DIR=/usr/lib64
99+
CVXOPT_BUILD_FFTW=1
100+
CVXOPT_FFTW_LIB_DIR=/usr/lib64
101+
CVXOPT_SUITESPARSE_SRC_DIR=SuiteSparse-${{ env.SUITESPARSE_VERSION }}
102+
103+
- name: Check the wheel ships the extensions and all licences
104+
run: |
105+
python3 - wheelhouse/*.whl <<'EOF'
106+
import sys, zipfile
107+
for whl in sys.argv[1:]:
108+
names = zipfile.ZipFile(whl).namelist()
109+
for mod in ("base", "blas", "lapack", "umfpack", "cholmod", "amd",
110+
"misc_solvers", "fftw"):
111+
assert any(n.startswith(f"cvxopt/{mod}.") and n.endswith(".so")
112+
for n in names), (whl, mod)
113+
licences = {n.rsplit("/", 1)[1]
114+
for n in names if ".dist-info/licenses/" in n} - {""}
115+
assert licences == {"LICENSE", "LICENSE.openblas", "LICENSE.suitesparse",
116+
"LICENSE.fftw"}, (whl, licences)
117+
print(whl, "ok")
118+
EOF
119+
120+
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
121+
with:
122+
name: cvxopt-${{ env.CVXOPT_VERSION }}-${{ matrix.python }}-manylinux_riscv64
123+
path: wheelhouse/*.whl
124+
if-no-files-found: error
125+
126+
gpl_sources:
127+
needs: [setup]
128+
if: needs.setup.outputs.versions != '[]'
129+
strategy:
130+
fail-fast: false
131+
matrix:
132+
version: ${{ fromJSON(needs.setup.outputs.versions) }}
133+
name: Collect GPL sources
134+
runs-on: ubuntu-24.04-riscv
135+
136+
env:
137+
CVXOPT_VERSION: ${{ matrix.version }}
138+
139+
steps:
140+
- name: Checkout python-wheels
141+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
142+
with:
143+
persist-credentials: false
144+
145+
# OpenBLAS is compiled with gfortran, so auditwheel vendors the image's
146+
# libgfortran alongside libfftw3 (GPL-2.0-or-later).
147+
- uses: ./actions/collect-gpl-sources
148+
with:
149+
image: ${{ env.MANYLINUX_RISCV64_IMAGE }}
150+
packages: gcc fftw
151+
output: rpm-gpl-sources.tar
152+
153+
# UMFPACK and CHOLMOD's supernodal module (GPL-2.0-or-later) are compiled
154+
# straight into the wheel from the pinned tarball, which is not in the
155+
# image's repos, so their source is collected here rather than as an RPM.
156+
- name: Fetch pinned SuiteSparse source
157+
run: |
158+
set -euo pipefail
159+
curl -fsSLO "https://github.com/DrTimothyAldenDavis/SuiteSparse/archive/v${SUITESPARSE_VERSION}.tar.gz"
160+
echo "${SUITESPARSE_SHA256} v${SUITESPARSE_VERSION}.tar.gz" | sha256sum -c -
161+
162+
- name: Combine GPL sources into one archive
163+
run: |
164+
set -euo pipefail
165+
mkdir gpl-sources
166+
tar xf rpm-gpl-sources.tar -C gpl-sources
167+
cp "v${SUITESPARSE_VERSION}.tar.gz" "gpl-sources/SuiteSparse-${SUITESPARSE_VERSION}.tar.gz"
168+
tar cf gpl-sources.tar -C gpl-sources .
169+
170+
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
171+
with:
172+
name: cvxopt-${{ env.CVXOPT_VERSION }}-gpl-sources
173+
path: gpl-sources.tar
174+
if-no-files-found: error
175+
176+
publish:
177+
name: Publish cvxopt ${{ matrix.version }}
178+
needs: [setup, build_wheels, gpl_sources]
179+
if: needs.setup.outputs.versions != '[]'
180+
strategy:
181+
fail-fast: false
182+
matrix:
183+
version: ${{ fromJSON(needs.setup.outputs.versions) }}
184+
permissions:
185+
contents: write
186+
pull-requests: write
187+
uses: $/.github/workflows/_publish-wheel.yml
188+
secrets:
189+
app-private-key: ${{ secrets.RISEPROJECT_APP_PRIVATE_KEY }}
190+
with:
191+
artifact-pattern: cvxopt-${{ matrix.version }}-*-manylinux_riscv64
192+
gpl-sources-artifact: cvxopt-${{ matrix.version }}-gpl-sources
193+
gpl-sources-description: gcc, fftw, suitesparse

‎docs/packages/cvxopt.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
package-name: cvxopt
2+
source-code: https://github.com/cvxopt/cvxopt
3+
license: GPL-3.0-or-later
4+
versions:
5+
- version: 1.3.3

‎skills/python-project-porting/references/gotchas-index.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,8 @@ The porting gotchas (371 of them) live in [`references/gotchas/`](gotchas/), spl
327327
- **378** — A newer libstdc++ on the manylinux image can deprecate calls a project's own
328328
`-DCMAKE_COMPILE_WARNING_AS_ERROR=ON` CI flag then turns into hard errors, purely from
329329
a toolchain-version gap upstream's own (older) runners never see.
330+
- **389** — Rocky 10 riscv64 ships OpenBLAS, LAPACK and FFTW but no SuiteSparse, GSL or
331+
GLPK, and a numeric package's optional-extension set has to be cut along that line.
330332

331333
### Native dependencies & linking — [`gotchas/native-deps-and-linking.md`](gotchas/native-deps-and-linking.md)
332334

@@ -347,6 +349,9 @@ The porting gotchas (371 of them) live in [`references/gotchas/`](gotchas/), spl
347349
- **278** — A vendored, direct-copy (not submodule) header can be missing riscv64 from its
348350
- **363** — A `libraries=[...]` entry can go missing from the link line with *no* error —
349351
- **368** — Linking several codecs against Rocky 10's system libraries instead of
352+
- **388** — A `setup.py` knob that feeds a downloaded dependency's *sources* into
353+
`Extension(sources=...)` needs a path relative to the project root, so the tarball has
354+
to be extracted inside the checkout, not into `/tmp`.
350355

351356
### Compiled-vs-pure detection & the require-extension knob — [`gotchas/compiled-vs-pure-detection.md`](gotchas/compiled-vs-pure-detection.md)
352357

‎skills/python-project-porting/references/gotchas/manylinux-image-and-toolchain.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
5656
- **374** — `find_package(Python3 REQUIRED COMPONENTS Interpreter Development)` fails on
5757
manylinux's static-libpython CPython, on any architecture — only `Development.Module`
5858
is ever needed to build an extension module, not the `Development.Embed` half.
59+
- **389** — Rocky 10 riscv64 ships OpenBLAS, LAPACK and FFTW but no SuiteSparse, GSL or
60+
GLPK, and a numeric package's optional-extension set has to be cut along that line.
5961

6062
---
6163

@@ -1028,3 +1030,37 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
10281030
- **Reproduce locally first**: a plain `python -m build -w` with the flag on vs. off
10291031
on `manylinux_2_39_aarch64` (gotcha 101) settles which of the two is responsible
10301032
in minutes, and shows the exact deprecated symbols by name.
1033+
1034+
389. **Rocky 10 riscv64 ships OpenBLAS, LAPACK and FFTW but no SuiteSparse, GSL or GLPK —
1035+
for a numeric package whose extensions are one-per-library, that split *is* the
1036+
feature set, so settle it before writing any YAML (the cvxopt case).** Same shape as
1037+
gotcha 337's lexbor/re2/uchardet gap, on the numeric side of the catalogue:
1038+
`openblas`/`openblas-devel` (crb, and its `libopenblas.so.0` exports the LAPACK
1039+
entry points too, so `-lopenblas` covers both), `lapack`/`lapack-devel` (crb),
1040+
`flexiblas*` and the whole `fftw*` family (appstream) are all present, while
1041+
`suitesparse`, `gsl` and `glpk` are in none of baseos/appstream/crb — and there is no
1042+
`epel-release` for riscv64 to fall back on, which is where a RHEL-family upstream
1043+
normally gets GLPK (gotcha 51). A package like cvxopt, which compiles a separate
1044+
extension per optional library behind `CVXOPT_BUILD_<LIB>` flags, therefore keeps
1045+
upstream's `fftw` module and drops `glpk`/`gsl`/`dsdp`, while its *mandatory*
1046+
umfpack/cholmod/amd extensions have to come from a from-source SuiteSparse
1047+
(gotcha 388) rather than being droppable at all.
1048+
- **Check it without paying for QEMU `dnf`**: `dnf repoquery` inside the riscv64
1049+
image is minutes per call under emulation (and `dnf provides` re-downloads
1050+
filelists), so pull the three repos' primary metadata over plain HTTPS instead
1051+
(gotcha 369) and grep the name list once.
1052+
- **The musl half of the same image pair is the *opposite* — Alpine 3.22 riscv64 has
1053+
all of them.** `apk search -x` inside `quay.io/pypa/musllinux_1_2_riscv64` finds
1054+
`openblas-dev`, `fftw-dev`, `lapack-dev` **and** `suitesparse-dev` (7.8.2),
1055+
`gsl-dev`, `glpk-dev`, so upstream's own `apk add` line needs no edit at all and the
1056+
musl wheel could carry more extensions than the glibc one. Two things still stand in
1057+
the way of just enabling it: the per-libc feature asymmetry that creates for users of
1058+
one version, and the fact that **Alpine ships no `/usr/share/licenses` at all** — so
1059+
gotcha 137's "copy the licence the package installed" has no source on musl and every
1060+
bundled library's text has to come from somewhere else.
1061+
- **A metapackage's licence text is not under its own name.** `fftw`'s COPYING is
1062+
installed by the subpackage auditwheel actually vendors —
1063+
`/usr/share/licenses/fftw-libs-double/COPYING`, not `/usr/share/licenses/fftw/` —
1064+
so a `cp` written from the `dnf install` name fails the whole `before-all`. `ls
1065+
/usr/share/licenses/` in the image once and copy from what is really there
1066+
(openblas does use the plain `/usr/share/licenses/openblas/LICENSE`).

‎skills/python-project-porting/references/gotchas/native-deps-and-linking.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/native-deps-and-linkin
2222
- **231** — A vendored C library's own CMake can carry a genuine, tested riscv64 branch —
2323
- **363** — A `libraries=[...]` entry can go missing from the link line with *no* error —
2424
- **368** — Linking several codecs against Rocky 10's system libraries instead of
25+
- **388** — A `setup.py` knob that feeds a downloaded dependency's *sources* into
26+
`Extension(sources=...)` needs a path relative to the project root, so the tarball has
27+
to be extracted inside the checkout, not into `/tmp`.
2528

2629
---
2730

@@ -495,3 +498,29 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/native-deps-and-linkin
495498
compiler flag; libpng's own `CHANGES` file (`grep -n cICP CHANGES` against the
496499
tagged release) gives the exact version the symbol was added in, and confirms
497500
Rocky 10's 1.6.40 predates it — a real wall, not a flag away.
501+
502+
388. **A `setup.py` env-var knob that feeds a downloaded dependency's *sources* into
503+
`Extension(sources=...)` needs a path **relative to the project root** — distutils
504+
hard-errors on an absolute one, so the tarball has to be extracted inside the
505+
checkout, not into `/tmp` (the cvxopt/SuiteSparse case).** Gotcha 53's shape is a
506+
`before-all` that curls a dependency tarball, builds it and links the resulting
507+
library; the variant here compiles the dependency's own `.c` files straight into the
508+
extension instead, through a knob like cvxopt's `CVXOPT_SUITESPARSE_SRC_DIR` (its
509+
`setup.py` globs `<dir>/AMD/Source/*.c`, `<dir>/CHOLMOD/Core/c*.c`, … into
510+
`sources=`). Extracting to `/tmp` and pointing the knob there — the obvious choice,
511+
since it keeps the checkout clean — dies at `build_wheel` with `error: Error: setup
512+
script specifies an absolute path: /tmp/<dep>/… setup() arguments must *always* be
513+
/-separated paths relative to the setup.py directory, *never* absolute paths`. That
514+
check is distutils' own and the project cannot opt out of it, so the fix is
515+
`tar xzf /tmp/<dep>.tar.gz -C {project}` plus the bare directory name as the value.
516+
- **The knob's own upstream usage is the tell, and it differs per knob kind**: the
517+
same `setup.py` takes absolute values happily for every `*_LIB_DIR`/`*_INC_DIR`
518+
(they only ever reach `library_dirs`/`include_dirs`), and upstream's own CI writes
519+
the *source* one relative (`CVXOPT_SUITESPARSE_SRC_DIR=SuiteSparse-${VERSION}`
520+
after untarring into the checkout). Sources are the restricted argument; search
521+
paths are not.
522+
- **An untracked dependency tree inside the checkout does not poison a
523+
`setuptools_scm` version.** Gotcha 31's hazard is *modified tracked* files;
524+
`git describe --dirty` ignores untracked paths, so a 31 MB `SuiteSparse-7.11.0/`
525+
plus three staged `LICENSE.<dep>` files at the checkout root still produced a plain
526+
`1.3.3` wheel, with no `SETUPTOOLS_SCM_PRETEND_VERSION` needed.

0 commit comments

Comments
 (0)