[velonica0]

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Reset the build number to 0 (if the version changed)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

• ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
• ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/41171243632. Examine the logs at this URL for more detail.}

[velonica0]

I built llvmdev locally on a DP-1000 rather than through CI. Same recipe and commit as this PR:

runner	cores	duration
ubuntu-24.04-arm	4	99 min
linux_ppc64le_ (cross on x86)	4	132 min
linux_64_ (x86)	4	174 min
DP-1000, native riscv64, local build	8	**226 min **
RISE EM-RV1, native riscv64	4	cancelled at 360 min, unfinished

The RISE job had reached ninja target 1546/4081 when it was cancelled after 6 hours.

On the memory side: the complete llvmdev build peaked at 9.8 GiB with zero swap used, so it fits comfortably in what an EM-RV1 has.

Since this was a local build, the figure will get somewhat worse once the same machines run as CI runners, with the container, network and job overhead that adds. We'll measure again after the machines are integrated, and if the degradation turns out to be serious we'll take it up with RISE.

[velonica0]

Hi conda team,
Could you please re-approve the CI for RISC-V? The hardware should be ready for use now.
Thanks!

[velonica0]

I have increased the disk space for the DP-1000 (30GB Root Disk + 60GB Ephemeral Disk); this should be sufficient for compiling llvmdev.

cc @scorbunny-cinderace

[velonica0]

I noticed the CI task was cancelled. Just checking if that was intentional.

[h-vetinari]

It's unlikely that this was a human IMO; could be that it got OOM-killed? 🤔

I'll restart it again

[velonica0]

It might be a network issue; continuing to check.
ci-cancellation-analysis-2026-10-28.en.md

[luhenry]

In case of networking issue, there is usually a message along the lines of "network connectivity lost to the runner", and it shows up after many more hours.

Also, let me know and I can set some environment variables just for the pods that are in conda-forge and based on labels, so we can have some more logging in the pod.

[velonica0]

Thanks @luhenry.

1. Yes, please enable the extra logging for the conda-forge pods. That would be very helpful.
2. Could you also check the RISE scheduler logs for runner rise-riscv-runner-tkzx9lwzy around 2026-10-27 10:57 UTC? The build was progressing normally (step 1481/4081, no errors) and then ended with just The operation was canceled. I wonder if the health check removed the runner because it briefly looked offline.

Hi @h-vetinari,
We’ve made some adjustments to the network and would like to test them by running another build. Could you please approve the CI? Thanks!

[scorbunny-cinderace]

We recently maintained a collector to get infos from runner, such as diag and pods logs.
And from the latest record, we‘ve now determine why the runner cancelled the task. But we need more info to determine why the request to github's broker was timeout.
We have now strengthened the collector’s sampling. Please re-run build.

[velonica0]

Our network to github using egress path corresponding with fake-ip mapping.
We are implimented a reversable network setting to make connection to github broker directly wtih fully tested.
And more data collection points have been added. Please re-run the build.

[h-vetinari]

Our network to github using egress path corresponding with fake-ip mapping.
We are implimented a reversable network setting to make connection to github broker directly wtih fully tested.
And more data collection points have been added.

This is hard for me to parse

Please re-run the build.

Restarted

[velonica0]

Hi, @h-vetinari

Good news—the network issue has been resolved.

Currently, I think the timeout might be caused by the ⁠LLVM_PARALLEL_LINK_JOBS⁠ parameter.

I have removed it and would like to give it another try.

If it still fails this time, it's possible that ⁠llvmdev⁠ is too massive to be built locally on RISC-V at the moment.

[h-vetinari]

With reduced testing, the job manages to build all outputs (until llvmdev), but then ran out of time to run the (relatively short) test phase for each output. We could disable tests completely of course, but then we don't actually win anything compared to cross-compilation, and the runtime would be much higher, even on the fastest available runners.

[velonica0]

I looked into where the time went: the job wasn't stuck in tests, it ran out of time while compressing llvmdev at zstd_compression_level: 22 (the log goes silent after Time taken to mark (prefix) for llvmdev and is cut off 34m45s later). That step is single-threaded.

To check, we recompressed the contents of llvmdev-23.1.4 (linux-riscv64, 774 MB) locally, the same way conda-build does:

Machine	level 22, 1 thread	level 22, 8 threads	level 19, 1 thread
x86_64 (i7-12700)	7m36s	4m49s	5m00s
riscv64 (DP-1000, idle)	38m32s	24m28s	24m10s

So the job was cut off ~4 min before llvmdev would have been packaged.

Some options:

1. Raise github_actions: timeout_minutes (e.g. to 600) — self-hosted runners aren't limited to 6h, so we could see a full run including the tests. We'll run the full build locally on a DP-1000 (build-locally.py) first to get an estimate of the total time needed, so we can suggest a concrete value instead of guessing.
2. Longer term, Add zstd_compression_threads setting for multi-threaded .conda compression conda/conda-build#6242 adds zstd_compression_threads, which would bring that step down to ~24 min on the DP-1000.
3. Alternatively, we could first validate native riscv64 builds on a smaller feedstock, and come back to llvmdev once (1) or (2) is in place.

Would you be OK with trying (1) on this PR, or would you prefer (3)?

[h-vetinari]

Native riscv builds have been validated from my POV, e.g.

• TST: fully native linux-riscv64 & win-arm64 builds numpy-feedstock#401
• TST: fully native linux-riscv64 & win-arm64 builds scipy-feedstock#331

(even though I've used these PRs for other testing afterwards by now).

I don't mind doing (1) & (2) here for testing. Right now I really don't see the benefit of merging native riscv builds for big feedstocks like llvmdev, because CI as a whole will just take ~6h instead of currently ~2.5h, for basically no benefit.

IMO, as long as the native riscv jobs are such an outlier compared to all other targets w.r.t. runtime, it's better to stay in cross-compilation, except in specific cases where the testing is critical (or cross-compilation is somehow infeasible).