skills: add gotcha 446; queue: comfy-angle round 4 built everything

Round 4 got through gclient sync, gn gen with the prefixed GCC toolchain and all
493 ninja targets, built the wheel and passed the smoke test on cp312/cp313/cp314
- then exited 127 on the last line of the script because the free-threaded
interpreter directory is /opt/python/cp314-cp314t, not cp314t-cp314t.

Author: luhenry

.queue.yml

@@ -2317,7 +2317,7 @@ packages:
     a gclient sync plus a full ANGLE build does not fit. Validation was static - patch applies with
     patch --dry-run --fuzz=0, py_compile of both patched scripts, a unit test of the new
     stage_licenses() against a synthetic checkout (including non-UTF8 licence bytes), bash -n of the
-    container script and compile() of both heredocs, and pending_versions.py resolving 0.1.1. PR #2123 open; three CI rounds so far, every one inside gclient sync before anything compiled.
+    container script and compile() of both heredocs, and pending_versions.py resolving 0.1.1. PR #2123 open; four CI rounds so far - the first three never got out of gclient sync, the fourth built everything.
     Round 1 (~2 min): vpython3 could not build its venv, since infra/python/wheels publishes no
     linux-riscv64 crcmod/lxml (gotcha 423) - fixed by setting VPYTHON_BYPASS for riscv64 hosts in
     build_linux.py. Round 2 (~4 min): the bypass worked and moved the wall one layer down - with
@@ -2352,12 +2352,26 @@ packages:
     third_party/depot_tools at the same revision the driver clones, so gotcha 427''s gsutil
     4.68/six split does not apply (both are 5.35, fine on 3.12), and the Linux hooks that follow
     (clang, llvm_objdump, rust, lastchange, configure_siso) import nothing outside the stdlib and
-    depot_tools. Still not rehearsed end to end locally (no docker daemon, 4.5 GB free). Left to
-    watch: those hooks (all 23 use plain python3, so the vpython class of failure is behind us,
-    and update.py maps every Linux host to Linux_x64 so the x86-only prebuilts still download),
-    gn gen with the prefixed GCC toolchain, and whether the statically linked
-    Vulkan-Loader falls back to its C unknown-ext-chain path on riscv64 instead of wanting
-    per-arch assembly.'
+    depot_tools. Still not rehearsed end to end locally (no docker daemon, 4.5 GB free). Round 4
+    (~64 min) closed every open question at once: gclient sync completed, all 23 hooks ran, gn gen
+    with the prefixed GCC toolchain worked, and ninja built all 493 targets - including
+    third_party/vulkan-loader/src/libvulkan.a, so the statically linked Vulkan-Loader does take its
+    portable C unknown-ext-chain path on riscv64 and needs no per-arch assembly. Both of gotcha
+    438''s watch items are therefore closed. libEGL.so and libGLESv2.so linked, scripts/build.py
+    produced the 3.4 MB comfy_angle-0.1.1-py3-none-manylinux_2_39_riscv64.whl with all four
+    expected members, and the smoke test loaded both libraries and printed the real EGL client
+    extension string (EGL_ANGLE_platform_angle_vulkan among them) on cp312, cp313 and cp314; the
+    gpl_sources job passed in parallel. The job then failed on the very last line of the container
+    script with exit 127, purely because the smoke-test loop spelled the free-threaded interpreter
+    /opt/python/cp314t-cp314t while manylinux names that directory cp314-cp314t, so pip was not
+    there and set -e took the step down after the whole build had already succeeded (new gotcha
+    446; the only compile diagnostics in the log are warnings, none of them riscv64-specific).
+    Round 5 changes that one path and nothing else. The fix needed no CI to validate:
+    build-mujoco.yml drives the same manylinux_2_39_riscv64 image with /opt/python/cp314-cp314t and
+    docs/packages/mujoco.yaml lists published cp314-cp314t riscv64 wheels, while
+    build-onnxruntime.yml, build-labmaze.yml and build-vtk.yml all derive the directory as
+    ${TAG%t}-${TAG}. Barring a free-threading-specific surprise from loading a py3-none ctypes
+    wrapper, round 5 should be green and publishable.'
 - pkg: solders
   version: 0.29.0
   home: https://github.com/kevinheavey/solders

skills/python-project-porting/references/gotchas-index.md

@@ -488,6 +488,10 @@ The porting gotchas (431 of them) live in [`references/gotchas/`](gotchas/), spl
   only through `TARGET=`, so pass `TARGET=RISCV64_GENERIC` (the rv64gc baseline) as the twin of
   the `TARGET=ARMV8` the project already has — and first ask whether gotcha 401's Rocky
   `openblas` package would do.
+- **446** — The image's free-threaded interpreter directory is `/opt/python/cp3XX-cp3XXt`
+  (`<implementation tag>-<ABI tag>`), not `cp3XXt-cp3XXt`, so a hand-written per-interpreter
+  loop that doubles the `t` exits 127 — derive it as `${TAG%t}-${TAG}`, and confirm any
+  `/opt/python` path with a `grep` over the green workflows rather than a CI round.
 
 ### Native dependencies & linking — [`gotchas/native-deps-and-linking.md`](gotchas/native-deps-and-linking.md)
 

skills/python-project-porting/references/gotchas/manylinux-image-and-toolchain.md

@@ -71,6 +71,9 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
 - **428** — A project on the *deprecated* `find_package(PythonLibs REQUIRED)` has no
   `Development.Module` way out of gotcha 374's static-libpython wall — and satisfying it
   with manylinux's non-PIC `libpython3.XX.a` only moves the failure to the final link.
+- **446** — The image's free-threaded interpreter directory is `/opt/python/cp3XX-cp3XXt`, not
+  `cp3XXt-cp3XXt`; a hand-written loop that doubles the `t` dies with exit 127, possibly on the
+  last line of an hour-long build.
 
 ---
 
@@ -1229,3 +1232,29 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/manylinux-image-and-to
       whole transaction on one unmatched argument, so a speculative package name is a build
       failure, not insurance. Install what the build needs and let the dependency solver
       pull the runtimes.
+
+446. **The manylinux image's free-threaded interpreter directory is
+     `/opt/python/cp3XX-cp3XXt`, not `/opt/python/cp3XXt-cp3XXt` — the directory is
+     `<implementation tag>-<ABI tag>`, and only the ABI tag carries the `t`.** A hand-written
+     per-interpreter loop that appends the free-threaded tag to itself (`cp314t-cp314t`) names
+     a path that does not exist, `"$pybin/bin/pip"` is "No such file or directory", and `set -e`
+     ends the step with **exit 127**. This is a one-line typo with an expensive failure mode:
+     in comfy-angle's round 4 (run 35496806279) it landed on the *last* line of the container
+     script, after a 64-minute ANGLE compile had already produced the wheel and the smoke test
+     had passed on cp312/cp313/cp314 — the log's final error says nothing about the build that
+     worked, so read *upwards* from an exit 127 before concluding the port regressed.
+     - **Derive the directory, don't write it out.** The rule is `${TAG%t}-${TAG}`, which
+       `build-onnxruntime.yml`, `build-labmaze.yml` and `build-vtk.yml` all encode as
+       `case "$PYTHON_TAG" in *t) python_dir="/opt/python/${PYTHON_TAG%t}-${PYTHON_TAG}/bin" ;;
+       *) python_dir="/opt/python/${PYTHON_TAG}-${PYTHON_TAG}/bin" ;; esac`. A literal list is
+       fine only if every free-threaded entry is spelled `cp3XX-cp3XXt` (as
+       `build-mujoco.yml`'s matrix and `build-cryptography.yml`'s do).
+     - **Verify a `/opt/python` path off-target instead of in CI.** No container is needed:
+       `grep -rn 'opt/python' .github/workflows/` shows how every green workflow on the same
+       `MANYLINUX_RISCV64_IMAGE` spells it, and `docs/packages/<pkg>.yaml` listing a published
+       `…-cp314-cp314t-manylinux_2_39_riscv64.whl` (mujoco has two) is proof that that exact
+       directory exists on that exact image. That pair of greps is seconds against an hour.
+     - **A `py3-none` wheel still wants the free-threaded leg in the smoke test.** The
+       interpreter list here is not a build matrix (gotcha 145): one platform wheel is loaded
+       on every interpreter we ship for, so a wrong path in the list fails a job that has
+       nothing else left to do.