Merge origin/main

Author: luhenry

.queue.yml

@@ -1,6 +1,6 @@
 # Gotchas index — router for the themed gotcha files
 
-The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), split by theme so only the relevant slice loads. Every gotcha keeps a **permanent number** cited elsewhere as "gotcha N" (and in workflow comments as "CLAUDE.md gotcha N"). Numbers are stable IDs — **not sequential**, and four are **reused** with different content (two each of 33, 55, 56, 57), disambiguated by theme below.
+The porting gotchas (387 of them) live in [`references/gotchas/`](gotchas/), split by theme so only the relevant slice loads. Every gotcha keeps a **permanent number** cited elsewhere as "gotcha N" (and in workflow comments as "CLAUDE.md gotcha N"). Numbers are stable IDs — **not sequential**, and four are **reused** with different content (two each of 33, 55, 56, 57), disambiguated by theme below.
 
 ## How to find the gotcha you need
 
@@ -158,6 +158,12 @@ The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), spl
   is a dead stub — count the sources that branch globs, diff the built `.so` against the
   published CUDA one, and call an op instead of trusting a "did the extension load" flag
   (the xformers case).
+- **426** — A `-cpu` sibling can be an *x86_64-only label* rather than a portable CPU variant:
+  where the base package's wheel is already CPU-only on every non-x86 arch, the sibling closes
+  no riscv64 gap, is never cheaper than the base, and inherits the base's park — scan the
+  sibling's whole release history for platform tags, compare the base's per-arch wheel sizes,
+  and re-verify any sibling-family blocker at the revision your target actually pins
+  (the tensorflow-cpu case).
 
 ### Sdist source & versioning — [`gotchas/sdist-source-and-versioning.md`](gotchas/sdist-source-and-versioning.md)
 
@@ -355,6 +361,10 @@ The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), spl
 - **424** — Audit a chromium-style DEPS for riscv64-less CIPD packages with
   `cipd describe <pkg>/linux-riscv64` (and `gclient_eval.EvaluateCondition`) before spending a
   build cycle discovering them one abort at a time.
+- **427** — `VPYTHON_BYPASS` also picks the interpreter gsutil runs on, and a chromium-style
+  checkout holds two gsutils: the one its DEPS pins (4.68, vendoring six 1.12) cannot import on
+  python ≥ 3.12, so its `download_from_google_storage` hooks fail — reproducible on x86 in
+  seconds, fixed by conditioning those test-data hooks off.
 
 ### The manylinux image & toolchain — [`gotchas/manylinux-image-and-toolchain.md`](gotchas/manylinux-image-and-toolchain.md)
 
@@ -413,6 +423,11 @@ The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), spl
   library (XNNPACK's `zvfh` kernels), where the fix is that dependency's own feature
   `--define` rather than an `-march` probe — and `--keep_going` hides a single-cause failure
   behind five hours of unrelated progress, making it look like a timeout.
+- **428** — A project still on the deprecated `find_package(PythonLibs REQUIRED)` has no
+  `Development.Module` way out of gotcha 374's static-libpython wall: satisfying it with
+  manylinux's non-PIC `libpython3.XX.a` only moves the failure to the final link after hours,
+  the project may already strip libpython from its own non-Windows link lines (making the
+  `REQUIRED` vestigial), and the header-only fix rehearses locally on x86_64 in a minute.
 
 ### Native dependencies & linking — [`gotchas/native-deps-and-linking.md`](gotchas/native-deps-and-linking.md)
 
@@ -572,6 +587,9 @@ The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), spl
 - **350** — A suite's own `try: import X except ImportError: X = None` plus
 - **355** — Gotcha 339 generalizes past `pytest` to any unpinned runtime dependency whose
   own heuristic changed across a major version — pin it for the test venv only.
+- **429** — A media project's suite is written against upstream's *full* FFmpeg; an FFmpeg
+  you configure yourself has no H.264/HEVC/VP9/AV1/MP3 encoder at all, and the failures
+  blame the wrong codec.
 
 ### Test failures, flakes & arch-specific bugs — [`gotchas/test-failures-and-flakes.md`](gotchas/test-failures-and-flakes.md)
 
@@ -672,6 +690,9 @@ The porting gotchas (385 of them) live in [`references/gotchas/`](gotchas/), spl
   arch macros in a scratch copy to exercise its generic architecture path natively — a
   restricted-egress host can still prove compilability without a container (the
   bitsandbytes case).
+- **430** — A `-k`/`--ignore` change is verifiable offline with no wheel at all: rebuild the
+  failed run's node ids into a synthetic test tree, then run the YAML-folded
+  `CIBW_TEST_COMMAND` through `sh -c`.
 
 ### PR, CI, triggers, publishing & maintainer signals — [`gotchas/pr-ci-and-maintainer.md`](gotchas/pr-ci-and-maintainer.md)
 

skills/python-project-porting/references/gotchas-index.md

@@ -94,6 +94,9 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/feasibility-and-triage
 - **419** — Gotcha 411's "is the CPU backend the default?" test can pass and still not yield a
   port: the non-CUDA branch of a torch extension can compile operator *schemas* with no
   implementations, so the build succeeds and the wheel is a dead stub (the xformers case).
+- **426** — A `-cpu` sibling can be an *x86_64-only label* rather than a portable CPU variant:
+  where the base package's wheel is already CPU-only on every non-x86 arch, the sibling name
+  closes no gap and inherits the base's park (the tensorflow-cpu case).
 
 ---
 
@@ -2372,3 +2375,54 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/feasibility-and-triage
       note reading "1 Linux wheel (abi: py39)" on a torch-extension package is that
       convention, not a pure-Python tell — and it means the port, had it been feasible,
       would have been one wheel rather than a per-interpreter matrix.
+
+426. **A `-cpu` sibling can be an *x86_64-only label*, not a portable CPU variant — if the
+    base package already ships a CPU-only wheel on every non-x86 arch, the sibling name
+    closes no riscv64 gap (the tensorflow-cpu case).** Gotcha 79's `-headless`/`-gpu`/`-lite`
+    sibling is usually a legitimate second port, and gotcha 50's `-binary` sibling is where
+    the wheels actually live. This is the third shape: a sibling that exists **only because
+    one architecture's default wheel is the GPU one**. `tensorflow-cpu` and `tensorflow`
+    2.21.0 are the same tree — `tensorflow/tools/pip_package/utils/tf_wheel.bzl` reads
+    `WHEEL_NAME` out of `@python_version_repo` and its own docstring says "Should be set via
+    `--repo_env=WHEEL_NAME=tensorflow_cpu`" — and their PyPI metadata is identical down to
+    the same 12 `nvidia-*; extra == "and-cuda"` requirements. What differs is only which
+    arch each name is *built* for, and that is the whole triage:
+    - **Scan the sibling's entire release history for platform tags, not just the version in
+      the queue entry.** One pass over `https://pypi.org/pypi/<sibling>/json`'s `releases`
+      counting the trailing tag of every file: `tensorflow-cpu` has published `win_amd64`
+      and `manylinux*_x86_64` **only** — across every release ever, zero aarch64, zero
+      ppc64le, zero other Linux arch, and no sdist. A sibling that has never left x86_64 is
+      a label for "x86_64 without the GPU bits", not a CPU variant.
+    - **Then compare the base package's per-arch wheel *sizes* to find which arches are
+      already CPU-only under the base name.** `tensorflow` 2.21.0 is 545 MB on
+      `manylinux_2_27_x86_64` but 268 MB on `manylinux_2_27_aarch64` — and `tensorflow_cpu`
+      x86_64 is 261 MB. The aarch64 wheel matching the *cpu* wheel's size rather than its own
+      arch's GPU wheel is the proof: on aarch64 upstream ships the CPU-only build under the
+      plain name, because there is no CUDA there to ship. riscv64 is in exactly that
+      position, so the riscv64 deliverable for "CPU-only TensorFlow" is `tensorflow`, and a
+      `manylinux_riscv64` wheel named `tensorflow-cpu` would invent a name upstream uses on
+      exactly one Linux architecture — gotcha 50's divergence-with-no-gap-closed, reached
+      from the sibling side.
+    - **So the `-cpu` sibling is never *cheaper* than the base, and inherits its verdict.**
+      The tempting inference is "the CPU-only variant sidesteps whatever made the full
+      package impractical (no CUDA build to worry about)". It is backwards: on an arch with
+      no CUDA the base package's build *is already* the CPU build, so the sibling saves
+      nothing and is the identical compile under a worse name. If the base entry is
+      `parked`, park the sibling for the base's reason plus the naming one, and say so in
+      both notes — an under-documented base park is what makes an agent re-derive this.
+    - **Don't inherit a sibling-family blocker citation across versions — re-verify it at
+      the revision your target actually pins.** jaxlib (PR #526, parked) died in XLA's
+      `xla/codegen/intrinsic/cpp` `embed_bitcode`, which links every LLVM backend except
+      RISC-V, and TensorFlow vendors the whole XLA tree at `third_party/xla/`, so that reads
+      like a ready-made blocker for TF too. At TF 2.21.0 it is not one: that tag's XLA
+      predates the restructure (the rule is `cc_ir_header` in `cc_to_llvm_ir.bzl`, whose
+      `ir_to_string` tool deps are just `llvm:Object`+`llvm:Support`, no per-arch CodeGen),
+      and `xla/backends/cpu/codegen/BUILD` *does* wire `if_llvm_riscv_available(["@llvm-project//llvm:RISCVCodeGen"])`
+      for the CPU JIT, with `linux_riscv64` and `riscv64_or_cross` defined in
+      `xla/tsl/BUILD`. Citing the sibling's hunk anyway would put a false hard blocker in the
+      queue; the honest note says "scope and naming, *not* Bazel-blocked like jaxlib".
+    - **Before costing a big Bazel build, check whether its wheel is per-interpreter.**
+      `_get_full_wheel_name` formats `cp{v}-cp{v}` from `HERMETIC_PYTHON_VERSION`, so TF is
+      one full build **per** interpreter (cp310–cp313 = 4), with none of the abi3/`py3-none`
+      collapse that let mediapipe serve every interpreter from a single ctypes-loaded `.so`.
+      That multiplier belongs in the estimate before anything else.

skills/python-project-porting/references/gotchas/feasibility-and-triage.md

@@ -30,6 +30,9 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/local-validation-and-r
 - **410** — Gotcha 188's "lower the optimisation level for the local rehearsal only" can
   silently produce a broken wheel when the project has a C99 `inline` helper with no
   `static` — and the suite still passes, because the pure-Python fallback catches it.
+- **430** — A `-k`/`--ignore` change is verifiable offline with no wheel at all: rebuild the
+  failed run's node ids into a synthetic test tree, then run the YAML-folded
+  `CIBW_TEST_COMMAND` through `sh -c`.
 
 ---
 
@@ -444,3 +447,25 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/local-validation-and-r
       FFmpeg build into a few minutes and changed nothing about the bug under
       investigation — but say so in the PR, because it does mean the *decode* tests were
       left to CI.
+
+430. **A `-k`/`--ignore` change is verifiable offline with no wheel at all: rebuild the
+    failed run's node ids into a synthetic test tree, then run the YAML-folded
+    `CIBW_TEST_COMMAND` through `sh -c` (the torchcodec case).** Dropping a couple of
+    hundred failing tests by name risks two silent mistakes, each costing a full CI cycle:
+    a clause that misses some failures (job still red) and a substring that also matches a
+    test that *passed* (coverage lost quietly, job green). Both are decidable on the host.
+    Scrape `FAILED <nodeid>` out of the failed job's log, generate one throwaway module per
+    test file — a class per class, and for a parametrised test
+    `@pytest.mark.parametrize("p", [pytest.param(0, id="<the exact param string>")])` so
+    the ids match character for character — add a handful of ids you know passed, and run
+    `pytest --collect-only -q -k "<expr>"` in a plain `python:3.x-slim` container: the
+    deselected count must equal the failures in scope, and every known-passing id must
+    still be selected. Then close the loop on the workflow file itself rather than on your
+    draft of the expression: `yaml.safe_load()` it, pull `CIBW_TEST_COMMAND` out of the
+    `cibuildwheel` step's `env`, assert `cmd.count("\n") == 0` (gotcha 93's folding trap)
+    and run `subprocess.run(["sh", "-c", cmd], cwd=<synthetic tree>)` — which is exactly
+    how cibuildwheel invokes it, so this also catches a shell-quoting bug in the `-k`
+    string. One artefact to expect: the log truncates a long parametrised id in its
+    `FAILED` line, so the generated tree grows both a truncated and a full variant of the
+    same test and the "passing test dropped" list fills with truncated twins — compare
+    names, not counts, before believing you have collateral damage.

skills/python-project-porting/references/gotchas/local-validation-and-rehearsal.md

@@ -64,6 +64,9 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
   `/usr/include/ev.h`.
 - **401** — Rocky 10 riscv64 ships OpenBLAS, LAPACK and FFTW but no SuiteSparse, GSL or
   GLPK, and a numeric package's optional-extension set has to be cut along that line.
+- **428** — A project on the *deprecated* `find_package(PythonLibs REQUIRED)` has no
+  `Development.Module` way out of gotcha 374's static-libpython wall — and satisfying it
+  with manylinux's non-PIC `libpython3.XX.a` only moves the failure to the final link.
 
 ---
 
@@ -1129,3 +1132,46 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
       `grep -c '^ERROR'` plus `grep -oE 'from target @@[^)]*' | sort -u`: one distinct
       target and one distinct extension name is what tells you a single `--define` fixes
       the whole run, rather than guessing from the first error you happen to see.
+
+428. **A project still on the *deprecated* `find_package(PythonLibs REQUIRED)` has no
+    `Development.Module` escape hatch from gotcha 374's static-libpython wall, and the
+    obvious way to satisfy it is a trap that only fails at the very end of the build (the
+    paddlepaddle case).** manylinux configures every interpreter it ships with
+    `--disable-shared` — `pypa/manylinux`'s `build_scripts/build-cpython.sh`, with no
+    architecture condition — so `/opt/python/cp3XX-cp3XX` carries `Python.h` and a
+    `libpython3.XX.a` but no `libpython3.XX.so`, and the old `FindPythonLibs` module, which
+    never consults `PYTHON_EXECUTABLE` at all, stops configure with `Could NOT find
+    PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)`. Two things to establish
+    before touching it:
+    - **Never point `PYTHON_LIBRARY` at the static archive to make the error go away.** It
+      configures, compiles for hours and *then* fails at the final link. CPython's
+      `configure` adds `CFLAGSFORSHARED` (i.e. `-fPIC`) only
+      `if test ! "$LIBRARY" = "$LDLIBRARY"`, which a `--disable-shared` build never
+      satisfies, so `libpython3.XX.a` holds no position-independent code and cannot be
+      linked into a shared object on any architecture. The cycle this wastes is the whole
+      build, not the configure step.
+    - **Check whether the project already refuses to link libpython, which makes the
+      `REQUIRED` vestigial.** Paddle's `cmake/generic.cmake` strips `python` out of every
+      non-Windows target's `target_link_libraries()`, keeps it only as an
+      `add_dependencies()` ordering edge and links `-Wl,-undefined,dynamic_lookup`
+      instead — citing pybind11's own "Building manually" notes — so `PYTHON_LIBRARIES` is
+      read only by the `cc_test()` executables that embed an interpreter (off under
+      `WITH_TESTING=OFF`) and by two dead variables. A `grep -rn '${PYTHON_LIBRARIES}'`
+      across the cmake tree is the whole audit, and it decides whether dropping the library
+      changes any link line at all.
+    The fix is to require only the headers, and to take them from the interpreter being
+    built against rather than from whatever the module finds on the host: pre-seed the
+    `PYTHON_INCLUDE_DIR` cache entry from `sysconfig.get_config_var('INCLUDEPY')`, which
+    keeps pointing at the real installation from inside a virtualenv. Pre-seeding also makes
+    a now-optional `find_package(PythonLibs)` skip its own `find_path()` and still report the
+    right `PYTHONLIBS_VERSION_STRING` out of `patchlevel.h`, so a distro build that does have
+    a shared libpython keeps behaving exactly as before. Guard the imported target too: a
+    `SHARED IMPORTED` target with an empty `IMPORTED_LOCATION` is invalid, so create
+    `add_library(<name> INTERFACE IMPORTED GLOBAL)` when no library was found.
+    **All of this rehearses locally on x86_64 in a minute, with no image pull and no QEMU**,
+    which matters when the real build is a multi-hour riscv64 job: `include()` the patched
+    `.cmake` from a throwaway CMake project, stub the project's own helper modules, and force
+    the manylinux branch with `-DCMAKE_DISABLE_FIND_PACKAGE_PythonLibs=TRUE`; then build a
+    real `.so` that calls a `Py_*` function the way the project builds its extension module,
+    and confirm `nm -D --undefined-only` reports the symbols as `U` and `readelf -d` shows no
+    `libpython` in `DT_NEEDED`.