Skip to content

Commit 6734861

Browse files
authored
Try different apt-get mirrors in CI (#13082)
* Try different apt-get mirrors in CI Every other run is currently taking 30+ minutes due to apt-get talking to azure's default servers being heavily throttled. Try out different mirrors with configuration I've cribbed from a few other locations. prtest:full * Update ocaml install too
1 parent c994bc4 commit 6734861

2 files changed

Lines changed: 58 additions & 12 deletions

File tree

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: "Run apt-get to install packages"
2+
description: "Helper to work around some common github actions issues"
3+
4+
inputs:
5+
packages:
6+
description: "The packages to install with apt-get"
7+
required: true
8+
default: ""
9+
10+
runs:
11+
using: composite
12+
steps:
13+
# See https://github.com/actions/runner-images/issues/12949 and
14+
# https://github.com/actions/runner-images/issues/7048 for some background,
15+
# but the default mirrors can sometimes take up to 30+ minutes to install a
16+
# package. Seems like the azure mirrors are heavily throttled occasionally.
17+
# Ideally something like
18+
# https://github.com/marketplace/actions/configure-fast-apt-mirror would be
19+
# used to figure out the fastest mirror but in the interest of trying out
20+
# something simple we instead just tweak some priorities.
21+
#
22+
# This is lifted from
23+
# https://github.com/servo/servo/blob/main/.github/actions/apt-mirrors/action.yml
24+
# which is in turn lifted from
25+
# https://github.com/CrowdStrike/glide-core/pull/1113 apparently.
26+
- name: Change Mirror Priorities
27+
shell: bash
28+
run: |
29+
sudo sed -i '/archive.ubuntu.com\/ubuntu\/\tpriority/ s/priority:2/priority:0/' /etc/apt/apt-mirrors.txt
30+
sudo sed -i '/azure.archive.ubuntu.com\/ubuntu\/\tpriority/ s/priority:0/priority:1/' /etc/apt/apt-mirrors.txt
31+
sudo sed -i '/security.ubuntu.com\/ubuntu\/\tpriority/ s/priority:3/priority:2/' /etc/apt/apt-mirrors.txt
32+
sudo cat /etc/apt/apt-mirrors.txt
33+
34+
- name: Update apt-get
35+
shell: bash
36+
run: sudo apt-get update
37+
38+
- name: Install packages
39+
shell: bash
40+
run: sudo apt-get install -y ${{ inputs.packages }}

.github/workflows/main.yml

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -616,9 +616,10 @@ jobs:
616616
curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash
617617
cargo binstall --no-confirm cross
618618
if: ${{ matrix.cross }}
619-
- name: Install apt packages
619+
- uses: ./.github/actions/apt-get-install
620620
if: ${{ matrix.apt_packages }}
621-
run: sudo apt-get update && sudo apt-get install -y ${{ matrix.apt_packages }}
621+
with:
622+
packages: ${{ matrix.apt_packages }}
622623
- run: ${{ matrix.test }}
623624
env:
624625
CARGO_BUILD_TARGET: ${{ matrix.target }}
@@ -660,7 +661,9 @@ jobs:
660661
# Install the OCaml packages necessary for fuzz targets that use the
661662
# `wasm-spec-interpreter`.
662663
- run: cargo install cargo-fuzz --vers "^0.11" --locked
663-
- run: sudo apt-get update && sudo apt install -y ocaml-nox ocamlbuild ocaml-findlib libzarith-ocaml-dev
664+
- uses: ./.github/actions/apt-get-install
665+
with:
666+
packages: ocaml-nox ocamlbuild ocaml-findlib libzarith-ocaml-dev
664667
- run: cargo fetch
665668
working-directory: ./fuzz
666669
- run: cargo fuzz check --dev
@@ -796,15 +799,16 @@ jobs:
796799

797800
- run: cargo fetch --locked
798801

799-
- name: Install cross-compilation tools
800-
run: |
801-
set -ex
802-
803-
sudo apt-get update
804-
sudo apt-get install -y ${{ matrix.gcc_package }}
802+
- uses: ./.github/actions/apt-get-install
803+
name: Install cross-compilation tools
804+
if: matrix.gcc_package != ''
805+
with:
806+
packages: ${{ matrix.gcc_package }}
805807

806-
# Configure Cargo for cross compilation and tell it how it can run
807-
# cross executables
808+
# Configure Cargo for cross compilation and tell it how it can run
809+
# cross executables
810+
- name: Configure cross-compilation tools
811+
run: |
808812
upcase=$(echo ${{ matrix.target }} | awk '{ print toupper($0) }' | sed 's/-/_/g')
809813
echo CARGO_TARGET_${upcase}_LINKER=${{ matrix.gcc }} >> $GITHUB_ENV
810814
if: matrix.gcc != ''
@@ -1053,8 +1057,10 @@ jobs:
10531057
curl --retry 5 --retry-all-errors -OL https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-30/wasi-sdk-30.0-x86_64-linux.tar.gz
10541058
tar -xzf wasi-sdk-30.0-x86_64-linux.tar.gz
10551059
mv wasi-sdk-30.0-x86_64-linux wasi-sdk
1060+
- uses: ./.github/actions/apt-get-install
1061+
with:
1062+
packages: gdb lldb-18 llvm
10561063
- run: |
1057-
sudo apt-get update && sudo apt-get install -y gdb lldb-18 llvm
10581064
# workaround for https://bugs.launchpad.net/ubuntu/+source/llvm-defaults/+bug/1972855
10591065
sudo mkdir -p /usr/lib/local/lib/python3.10/dist-packages/lldb
10601066
sudo ln -s /usr/lib/llvm-15/lib/python3.10/dist-packages/lldb/* /usr/lib/python3/dist-packages/lldb/

0 commit comments

Comments
 (0)