queue: comfy-angle round 3 reached cipd ensure; skills: add gotcha 445

The httplib2 pin got the sync through the whole DEPS graph; it now fails in
the "cipd ensure" that ends it, on four packages with no linux-riscv64
build. The custom_deps null-out the patch carried for two of them had never
had any effect, which is gotcha 445: gclient applies custom_deps while
resolving a git dep's URL, and a cipd entry goes straight from DEPS into a
CipdDependency that never consults it.

Also records the second half of that fix - ANGLE's .gn turns siso on in
default_args, and autoninja reads use_siso back out of args.gn, so a
dropped siso package needs use_siso=false in the gn args to go with it.

Author: luhenry

.queue.yml

@@ -2288,21 +2288,21 @@ packages:
     Wayland / GBM, tests / SwiftShader / dawn / GL / wgpu / validation-layers all off), so riscv64
     needed only the architecture added to that build rather than a Chromium-scale port. Every layer
     below already supports riscv64, which is what makes this cheap: depot_tools detect_host_arch.py
-    maps riscv* to riscv64; CIPD publishes linux-riscv64 for the cipd client, cpython3, ninja, gn
-    and luci-go; Chromium build/toolchain/linux/BUILD.gn has gcc_toolchain("riscv64") with toolprefix
+    maps riscv* to riscv64; CIPD publishes linux-riscv64 for the cipd client, cpython3, ninja and gn;
+    Chromium build/toolchain/linux/BUILD.gn has gcc_toolchain("riscv64") with toolprefix
     riscv64-linux-gnu, which BUILDCONFIG selects once is_clang=false, and config/compiler/BUILD.gn
     carries riscv64 cflags; ANGLE''s own gni/angle.gni lists riscv64 among the 64-bit CPUs, so gn
-    analysis does not hit its "Unknown current CPU" assert. Only two real gaps: buildtools/reclient
-    and third_party/siso/cipd publish no linux-riscv64 package and would abort gclient sync before
-    anything compiles. Neither is used here (use_remoteexec is false, and use_siso_default in
-    build/toolchain/siso.gni is false unless build_with_chromium, which a standalone ANGLE checkout
-    is not), so the patch nulls both through the same custom_deps mechanism build_linux.py already
-    uses for SwiftShader / VK-GL-CTS / catapult; reclient has no linux-arm64 build either, which is
-    why upstream''s DEPS already carves arm64 out of that dep. The patch otherwise reuses upstream''s
+    analysis does not hit its "Unknown current CPU" assert. The real gap is CIPD: three DEPS entries
+    pull packages with no linux-riscv64 build and abort gclient sync before anything compiles -
+    buildtools/reclient (infra/rbe/client), third_party/siso/cipd (build/siso) and tools/luci-go
+    (isolate and swarming; only its cas tool is published for riscv64). None is reachable from a
+    local build, so the patch deletes the three entries from the checkout''s DEPS before the sync;
+    reclient has no linux-arm64 build either, which is why upstream''s DEPS already carves arm64 out
+    of that dep. The patch otherwise reuses upstream''s
     existing arm64 recipe (alias the container''s gcc/g++/ar/readelf/nm under the prefixed names
     ANGLE expects, is_clang=false, use_custom_libcxx=false, treat_warnings_as_errors=false),
-    generalised to a toolprefix table, and both new tables are keyed by arch so x86_64/arm64 render
-    byte-identical .gclient and gn args. The x86-only prebuilt host tools the DEPS hooks fetch
+    generalised to a toolprefix table, and the new tables are all keyed by arch so x86_64/arm64
+    render a byte-identical .gclient and byte-identical gn args against an untouched DEPS. The x86-only prebuilt host tools the DEPS hooks fetch
     (clang, llvm_objdump, rust, glslang_validator, flex_bison) are not a blocker: update.py maps
     every Linux host to Linux_x64 with no arch check, and upstream''s green aarch64 job proves those
     binaries are never executed by the libEGL/libGLESv2 targets. Wheel is a single
@@ -2325,7 +2325,22 @@ packages:
     httplib2.socks at module scope (ModuleNotFoundError: No module named ''httplib2'', ~80 s in) -
     fixed in the workflow with pip install httplib2==0.13.1, the version depot_tools'' own
     .vpython3 pins; unpinned resolves to 0.32.0, which no longer ships the httplib2.socks that
-    import needs (new gotcha 441). Round 3 in flight. That round-2 diagnosis was settled off-
+    import needs (new gotcha 441). Round 3 (~7 min): the httplib2 pin worked - gclient fetched
+    depot_tools and ANGLE, evaluated the whole DEPS graph and reached the "cipd ensure" that ends
+    the sync, which then failed to resolve four packages with no linux-riscv64 build:
+    infra/rbe/client, build/siso and the luci isolate and swarming tools. So the custom_deps
+    null-out the patch carried for reclient and siso had never done anything: gclient applies
+    custom_deps while resolving a "git" dep''s URL, whereas a "cipd" entry is copied out of DEPS
+    into a CipdDependency that never consults custom_deps, so a None there is accepted and
+    silently ignored (new gotcha 445, verified by reading gclient.py at the revision the run
+    pinned). Round 4 deletes the three DEPS entries outright before the sync and raises if one is
+    missing, and adds use_siso=false to the gn args - ANGLE''s .gn sets use_siso=true in
+    default_args for a standalone checkout (the earlier siso.gni reading was wrong), and autoninja
+    reads use_siso back out of args.gn, so dropping the siso package without that arg would have
+    swapped one failure for another one layer later. Validated off-target: drop_deps() run against
+    the real pinned DEPS removes exactly those four packages (421 of 426 CIPD packages kept, the
+    fifth being luci cas from the same entry), the rewritten DEPS still parses to 495 deps and 23
+    hooks, and driving main() for all three arches confirms x86_64/arm64 are byte-identical. That round-2 diagnosis was settled off-
     target on x86 in seconds rather than in CI: the failure reproduces exactly under a bare cp312
     venv against the pinned depot_tools tree (fetched as a gitiles archive, no clone), and with
     httplib2 installed every module the sync path touches imports cleanly and a real
@@ -2336,7 +2351,9 @@ packages:
     4.68/six split does not apply (both are 5.35, fine on 3.12), and the Linux hooks that follow
     (clang, llvm_objdump, rust, lastchange, configure_siso) import nothing outside the stdlib and
     depot_tools. Still not rehearsed end to end locally (no docker daemon, 4.5 GB free). Left to
-    watch: those hooks, gn gen with the prefixed GCC toolchain, and whether the statically linked
+    watch: those hooks (all 23 use plain python3, so the vpython class of failure is behind us,
+    and update.py maps every Linux host to Linux_x64 so the x86-only prebuilts still download),
+    gn gen with the prefixed GCC toolchain, and whether the statically linked
     Vulkan-Loader falls back to its C unknown-ext-chain path on riscv64 instead of wanting
     per-arch assembly.'
 - pkg: solders

skills/python-project-porting/references/gotchas-index.md

@@ -412,6 +412,10 @@ The porting gotchas (431 of them) live in [`references/gotchas/`](gotchas/), spl
   `include(third_party)` already ran the matching `add_definitions()`, so the feature is
   compiled in and not linked — check the include line numbers and use the project's own
   early default switch instead.
+- **445** — A `.gclient` `custom_deps: None` drops a *git* dep and is silently ignored for a
+  `cipd` one, so a riscv64-less CIPD package survives into the `cipd ensure` that ends
+  `gclient sync` — delete the entry from the checkout's `DEPS` instead, and add
+  `use_siso=false` to the gn args when siso is one of them.
 
 ### The manylinux image & toolchain — [`gotchas/manylinux-image-and-toolchain.md`](gotchas/manylinux-image-and-toolchain.md)
 

skills/python-project-porting/references/gotchas/native-build-bazel-and-drivers.md

@@ -42,6 +42,8 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/native-build-bazel-and
   `httplib2==0.13.1` on the ambient interpreter or the sync dies on `import httplib2`.
 - **443** — An upstream CMake arch block that `set(... CACHE ... FORCE)`s features OFF can
   run *after* `include(third_party)`, making the FORCE dead for that configure.
+- **445** — gclient ignores a `custom_deps` `None` for a `cipd` dep: delete the DEPS entry
+  instead, and add `use_siso=false` when the dropped package is siso.
 
 ---
 
@@ -759,3 +761,45 @@ To pull up one entry: `grep -n '^N\. ' references/gotchas/native-build-bazel-and
       emitting *unsuffixed* scalar declarations. The generated riscv64 `sleef.h` therefore
       declares `Sleef_sinf1_u35purec` and never `Sleef_sinf1_u35`. Worth reporting upstream
       to SLEEF, but not something a wheel port should carry a patch for.
+
+445. **A `.gclient` `custom_deps` entry set to `None` removes a *git* dep and is silently
+    ignored for a `cipd` one, so the standard "null it out" trick does not keep a
+    riscv64-less CIPD package out of a depot_tools checkout — you have to delete the entry
+    from the checkout's `DEPS`.** In `gclient.py`, `_postprocess_deps` copies the DEPS dict
+    verbatim and only ever *adds* custom_deps whose value is truthy; the removal path for a
+    git dep is `Dependency._OverrideUrl`, which asks `get_custom_deps` for the URL and skips
+    the dep when it comes back `None`. `_deps_to_objects` builds a `CipdDependency`
+    straight from the DEPS entry and passes it only `custom_vars` — nothing in that path
+    consults `custom_deps`, and nothing warns. The null looks right in the `.gclient`, the
+    sync proceeds normally for several minutes, and the package still turns up in the
+    `cipd ensure` that ends the sync: `failed to resolve <pkg>/linux-riscv64 (line N): no
+    such package`, one line per unresolvable package, then a bare `CalledProcessError`.
+    - **Delete the DEPS entries instead, before the sync, and raise if one is not found.**
+      The driver script already has the checkout on disk before it writes `.gclient`, so a
+      `re.subn(rf"\n  '{re.escape(path)}': \{{\n.*?\n  \}},\n", "\n", text, flags=re.DOTALL)`
+      per path is enough for Chromium-family DEPS formatting. Assert the count is exactly 1:
+      a future revision that reformats DEPS must fail loudly rather than quietly restore the
+      dep you thought you had dropped.
+    - **`cipd ensure` reports *every* unresolvable package at once**, because gclient batches
+      all cipd deps in the tree into a single ensure file for the root. That list is the
+      complete set — fix them in one round instead of expecting another wall per package.
+    - **Read the ensure-file failure by dep entry, not by package.** ANGLE's `tools/luci-go`
+      names three packages and only `isolate` and `swarming` lack a riscv64 build, but they
+      share one entry, so dropping the entry also drops `cas`. Fine when the whole entry is
+      test-distribution tooling; check before assuming.
+    - **Dropping siso means saying so in the GN args, or you just move the failure later.**
+      `//build/toolchain/siso.gni` defaults `use_siso` to false for a non-Chromium checkout,
+      but a project's own `.gn` can override that in `default_args` — ANGLE's sets
+      `use_siso = true` — and `autoninja` chooses siso or ninja by reading `use_siso` back out
+      of `args.gn`. Add `use_siso=false` to the args wherever you remove the siso package.
+      It does reach autoninja despite the args being passed as one space-separated
+      `--args=` string: gn runs `gn format` over that string in `Setup::SaveArgsToFile`, so
+      `args.gn` lands one `key = value` per line, which is the only shape depot_tools'
+      `gn_helper.args` regex matches. Leave `use_remoteexec` unset and `use_reclient` stays
+      false on its own (`use_reclient = use_remoteexec && !use_siso`), so nothing wants the
+      reclient package either.
+    - **All of this is checkable off-target in seconds.** Fetch the pinned `DEPS` and
+      `gclient.py` from gitiles (`?format=TEXT`, base64 — no clone), run the rewrite against
+      the real DEPS, `exec` the result to confirm it still parses and count the deps and
+      hooks, and diff the rendered `.gclient` and gn args for the untouched architectures to
+      prove the port changed nothing for them.