Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions .agents/skills/change-maple-ci/references/changing-ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Paths are relative to the repository root. `scripts/ci/lanes.py` and
| `submodules` | Recursive submodules with full blobless history (Nix needs `revCount`) |
| `hook` | Pre-commit component that runs the same checks locally |
| `restore` | Read-only restore of a cache another workflow writes |
| `requires` | Lanes whose jobs this lane's special job needs. The planner adds them to every plan that selects the lane, dispatches included; a required lane has no paths, labels or gate of its own, so nothing else selects it (lint). `metal-toolchain` is the example: GitHub's xcode-27 image exports Xcode's Metal Toolchain for `agent-package-macos` |

Event sets:

Expand Down Expand Up @@ -99,6 +100,8 @@ Its id is the lane's `job`. Checked by `plan.py lint` and the policy tests:
- `runs-on` exactly `${{ vars.<VARIABLE> || '<label>' }}` when the runner has
an override variable, otherwise the plain label; `timeout-minutes` equals
the lane's `timeout` (lint).
- A lane with `requires` has its own job, whose `needs:` lists each required
lane's job (lint).
- `ci-ok` needs it (test 8); a new runner label is declared in
`.github/actionlint.yaml` (test 9 and lint); checkouts use
`persist-credentials: false` (test 3).
Expand Down Expand Up @@ -132,7 +135,7 @@ The `Runner` constants at the top of `lanes.py` are authoritative.
| `MACOS` (Blacksmith macOS) | `CI_MACOS_RUNNER` |
| `WINDOWS` (Blacksmith Windows) | `CI_WINDOWS_RUNNER` |
| `GITHUB_WINDOWS` (`maple-windows-2025-16core`) | `CI_WINDOWS_GITHUB_RUNNER`, read only by special jobs |
| `GITHUB`, `GITHUB_WINDOWS_FREE`, `EIF_BUILDER`, inline `ubuntu-24.04-arm` | none |
| `GITHUB`, `GITHUB_WINDOWS_FREE`, `GITHUB_XCODE` (`xcode-27`), `EIF_BUILDER`, inline `ubuntu-24.04-arm` | none |

- Matrix lanes resolve their runner when `plan` runs. A new matrix variable
must be passed in the `plan` step's `env`; `lint` checks this.
Expand All @@ -159,7 +162,10 @@ The `Runner` constants at the top of `lanes.py` are authoritative.
- The Agent release key's hash is a step output from
`scripts/ci/cargo_graph_key.py`, not `hashFiles`. C4 runs that step, with
the script working and failing, so an empty or misnamed output cannot turn
the exact key into a family prefix.
the exact key into a family prefix. The macOS build also passes
`scripts/ci/apple-toolchain.json`, as the macOS lanes list it in
`cache_files`: an Xcode upgrade must not restore objects built against the
previous SDK.
- Turn off implicit saves on PR paths: setup-bun `no-cache: true` and no
Magic Nix Cache (both checked), and Trivy `cache: "false"` (convention).
- After a lockfile change, master's first save deletes the lane's target
Expand Down Expand Up @@ -252,6 +258,9 @@ selected or nightly; run it yourself after doc edits.
`just lane <id>` runs exactly what CI runs. Limits:

- macOS lanes need macOS and the pinned Xcode (`scripts/ci/select-xcode.py`).
The Agent's Clippy and native suite build gpui with `runtime_shaders`, so
they need no Metal Toolchain; the packaging rehearsal compiles the shaders
and needs it (`xcodebuild -downloadComponent MetalToolchain`).
- Windows lanes need Windows with the toolchain from
`.github/actions/setup-agent-windows`.
- `os-sdk-e2e` needs PostgreSQL on `127.0.0.1:5432` with the credentials in
Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/change-maple-ci/references/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ the cancellation to finish, so rapid re-pushes each add some delay.
variable `CI_LINUX_HEAVY_RUNNER=blacksmith-16vcpu-ubuntu-2404` (same
vendor, so caches still work) and unset it afterwards. For the 8-vCPU
lanes, `CI_LINUX_RUNNER=blacksmith-16vcpu-ubuntu-2404` keeps Blacksmith's
store. In a Blacksmith-wide outage use `ubuntu-latest` (or `macos-26`,
store. In a Blacksmith-wide outage use `ubuntu-latest` (or `xcode-27`,
`windows-2025`): cold, saves nothing, slower.
- Matrix lanes take their runner from `plan`, so re-plan: cancel a PR run,
then **Re-run all jobs**. Never cancel a master run; the variable applies
Expand Down
4 changes: 3 additions & 1 deletion .github/actionlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ self-hosted-runner:
labels:
- blacksmith-16vcpu-ubuntu-2404-arm
- blacksmith-8vcpu-windows-2025
- blacksmith-6vcpu-macos-26
- blacksmith-6vcpu-macos-27
- blacksmith-8vcpu-ubuntu-2404
- blacksmith-32vcpu-ubuntu-2404
# GitHub-hosted larger runner (org runner group "Maple"); actionlint only knows standard labels.
- maple-windows-2025-16core
# GitHub-hosted Xcode 27 image; newer than the pinned actionlint label list.
- xcode-27
28 changes: 28 additions & 0 deletions .github/actions/export-metal-toolchain/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Export the Metal Toolchain
description: >-
Download the selected Xcode's Metal Toolchain and upload it as this run's
one-day `metal-toolchain` artifact. Blacksmith's macOS image has no Metal
Toolchain and cannot download it (its MobileAsset catalog fetch fails), so a
GitHub xcode-27 job exports it and the Blacksmith jobs import it. Run it after
selecting the pinned Xcode.
runs:
using: composite
steps:
- name: Download and export the Metal Toolchain
shell: bash
run: |
set -euo pipefail
export_dir="$RUNNER_TEMP/metal-toolchain"
mkdir -p "$export_dir"
xcodebuild -downloadComponent metalToolchain -exportPath "$export_dir"
# Keep symlinks and modes intact; artifacts flatten them.
tar -C "$export_dir" -cf "$RUNNER_TEMP/metal-toolchain.tar" .
- name: Upload the Metal Toolchain
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: metal-toolchain
path: ${{ runner.temp }}/metal-toolchain.tar
if-no-files-found: error
# Apple's toolchain, re-hosted only for this run's jobs.
retention-days: 1
compression-level: 0
30 changes: 30 additions & 0 deletions .github/actions/import-metal-toolchain/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Import the Metal Toolchain
description: >-
Install the Metal Toolchain that this run's metal-toolchain job exported, so
gpui can compile its Metal shaders on Blacksmith's macOS image. A runner whose
Xcode already has it (GitHub's xcode-27 image, during a failover) skips the
import. Run it after selecting the pinned Xcode.
runs:
using: composite
steps:
- name: Download the exported Metal Toolchain
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: metal-toolchain
path: ${{ runner.temp }}/metal-toolchain-artifact
- name: Import the Metal Toolchain
shell: bash
run: |
set -euo pipefail
if xcodebuild -showComponent metalToolchain | grep -q '^Status: installed'; then
echo "The selected Xcode already has its Metal Toolchain."
else
import_dir="$RUNNER_TEMP/metal-toolchain"
mkdir -p "$import_dir"
tar -C "$import_dir" -xf "$RUNNER_TEMP/metal-toolchain-artifact/metal-toolchain.tar"
bundle="$(find "$import_dir" -maxdepth 1 \( -name '*.exportedBundle' -o -name '*.dmg' \) -print -quit)"
test -n "$bundle"
xcodebuild -importComponent metalToolchain -importPath "$bundle" \
|| sudo xcodebuild -importComponent metalToolchain -importPath "$bundle"
fi
xcrun -sdk macosx metal --version
31 changes: 26 additions & 5 deletions .github/workflows/agent-desktop-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,31 @@ jobs:
# Dispatch, the nightly run, an unusable push base and selector errors all build.
run: python3 scripts/ci/plan.py only agent-desktop >> "$GITHUB_OUTPUT"

# Blacksmith's macOS image cannot download Xcode's Metal Toolchain, so GitHub's
# xcode-27 image exports it for the macOS build, which compiles gpui's shaders.
metal-toolchain:
name: Export the Metal Toolchain
needs: changes
if: ${{ always() && !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.agent != 'false') && github.repository == 'MaplePrivacyLabs/Maple' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule') }}
runs-on: xcode-27
permissions:
contents: read
timeout-minutes: 15
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Select pinned Xcode
working-directory: .
run: python3 scripts/ci/select-xcode.py
- uses: ./.github/actions/export-metal-toolchain

macos-build:
name: Build macOS binary (${{ matrix.variant }})
needs: changes
needs: [changes, metal-toolchain]
if: ${{ always() && !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.agent != 'false') && github.repository == 'MaplePrivacyLabs/Maple' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule') }}
runs-on: blacksmith-6vcpu-macos-26
runs-on: blacksmith-6vcpu-macos-27
permissions:
contents: read
timeout-minutes: 120
Expand All @@ -82,9 +102,10 @@ jobs:
- name: Select pinned Xcode
working-directory: .
run: python3 scripts/ci/select-xcode.py
- uses: ./.github/actions/import-metal-toolchain
- name: Hash the Agent's third-party dependency graph and toolchain
id: graph
run: python3 ../../scripts/ci/cargo_graph_key.py Cargo.lock flake.lock >> "$GITHUB_OUTPUT"
run: python3 ../../scripts/ci/cargo_graph_key.py Cargo.lock flake.lock ../../scripts/ci/apple-toolchain.json >> "$GITHUB_OUTPUT"
- name: Restore the exact-key Agent release cache
id: cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
Expand Down Expand Up @@ -138,7 +159,7 @@ jobs:
contents: read
id-token: write
attestations: write
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 120
strategy:
fail-fast: false
Expand Down Expand Up @@ -513,7 +534,7 @@ jobs:
permissions:
contents: read
attestations: read
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 30
strategy:
fail-fast: false
Expand Down
34 changes: 26 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -229,11 +229,27 @@ jobs:
if-no-files-found: error
retention-days: 7

# Blacksmith's macOS image cannot download Xcode's Metal Toolchain, so GitHub's
# xcode-27 image exports it for the lanes that compile gpui's shaders.
metal-toolchain:
name: metal-toolchain
needs: plan
if: fromJSON(needs.plan.outputs.lanes)['metal-toolchain']
runs-on: xcode-27
timeout-minutes: 15
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: Select pinned Xcode
run: python3 scripts/ci/select-xcode.py
- uses: ./.github/actions/export-metal-toolchain

agent-package-macos:
name: agent-package-macos
needs: plan
needs: [plan, metal-toolchain]
if: fromJSON(needs.plan.outputs.lanes)['agent-package-macos']
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-26' }}
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-27' }}
timeout-minutes: 60
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
Expand All @@ -242,13 +258,15 @@ jobs:
- uses: ./.github/actions/setup-nix
- name: Select pinned Xcode
run: python3 scripts/ci/select-xcode.py
- uses: ./.github/actions/import-metal-toolchain
- name: Restore the packaging rehearsal cache
id: package-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
# Thin-LTO release artifacts; master's fat-LTO cache cannot serve them.
key: ci-v1-agent-package-${{ runner.os }}-${{ hashFiles('apps/maple-agent/flake.lock', 'apps/maple-agent/Cargo.lock') }}
restore-keys: ci-v1-agent-package-${{ runner.os }}-
# The Xcode pin leads, so a prefix restore never brings back another SDK's objects.
key: ci-v1-agent-package-${{ runner.os }}-${{ hashFiles('scripts/ci/apple-toolchain.json') }}-${{ hashFiles('apps/maple-agent/flake.lock', 'apps/maple-agent/Cargo.lock') }}
restore-keys: ci-v1-agent-package-${{ runner.os }}-${{ hashFiles('scripts/ci/apple-toolchain.json') }}-
path: |
~/.cargo/registry/index
~/.cargo/registry/cache
Expand Down Expand Up @@ -355,7 +373,7 @@ jobs:
name: research-ios
needs: plan
if: fromJSON(needs.plan.outputs.lanes)['research-ios']
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-26' }}
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-27' }}
timeout-minutes: 120 # a missed ONNX Runtime cache builds it from source
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
Expand Down Expand Up @@ -388,7 +406,7 @@ jobs:
name: research-desktop-macos
needs: plan
if: fromJSON(needs.plan.outputs.lanes)['research-desktop-macos']
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-26' }}
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-27' }}
timeout-minutes: 45
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
Expand Down Expand Up @@ -498,7 +516,7 @@ jobs:
name: research-dev-macos
needs: plan
if: fromJSON(needs.plan.outputs.lanes)['research-dev-macos']
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-26' }}
runs-on: ${{ vars.CI_MACOS_RUNNER || 'blacksmith-6vcpu-macos-27' }}
timeout-minutes: 45
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
Expand Down Expand Up @@ -655,7 +673,7 @@ jobs:
ci-ok:
name: ci-ok
if: always()
needs: [plan, nix, windows, agent-package-linux, agent-package-macos, agent-package-windows, research-rust,
needs: [plan, nix, windows, agent-package-linux, metal-toolchain, agent-package-macos, agent-package-windows, research-rust,
research-ios, research-desktop-macos, research-desktop-linux, research-desktop-windows,
research-dev-macos, os-sdk-e2e, os-helpers, os-eif]
runs-on: ubuntu-latest
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/desktop-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
environment: desktop-signing
needs: changes
if: ${{ always() && !cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.frontend != 'false' || needs.changes.outputs.macos != 'false') }}
runs-on: blacksmith-6vcpu-macos-26
runs-on: blacksmith-6vcpu-macos-27
timeout-minutes: 120
permissions:
contents: read
Expand Down Expand Up @@ -214,7 +214,7 @@ jobs:
verify-macos-desktop-artifacts:
needs:
- build-macos
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 30
permissions:
contents: read
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ios-dev-testflight.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
(github.event_name == 'workflow_dispatch' || (github.event_name == 'push' &&
(needs.changes.result != 'success' || needs.changes.outputs.frontend != 'false' || needs.changes.outputs.ios != 'false')))
environment: apple-signing
runs-on: blacksmith-6vcpu-macos-26
runs-on: blacksmith-6vcpu-macos-27
timeout-minutes: 240
permissions:
contents: read
Expand Down Expand Up @@ -144,7 +144,7 @@ jobs:
(github.event_name == 'push' || github.event_name == 'workflow_dispatch')
needs: build-ios-dev
environment: apple-signing
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 45
permissions:
contents: read
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/mobile-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
github.ref == 'refs/heads/master' &&
(github.event_name == 'push' || github.event_name == 'workflow_dispatch') &&
(needs.changes.result != 'success' || needs.changes.outputs.frontend != 'false' || needs.changes.outputs.ios != 'false')
runs-on: blacksmith-6vcpu-macos-26
runs-on: blacksmith-6vcpu-macos-27
# The ONNX source build and the app build each allow 90 minutes.
timeout-minutes: 240
permissions:
Expand Down Expand Up @@ -120,7 +120,7 @@ jobs:

verify-ios-artifacts:
needs: build-ios
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 30
permissions:
contents: read
Expand Down Expand Up @@ -156,7 +156,7 @@ jobs:
github.ref == 'refs/heads/master' &&
(github.event_name == 'push' || github.event_name == 'workflow_dispatch')
needs: verify-ios-artifacts
runs-on: macos-26
runs-on: xcode-27
timeout-minutes: 45
permissions:
contents: read
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-26, windows-latest]
os: [ubuntu-latest, xcode-27, windows-latest]
defaults:
run:
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/proxy-rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
archive: maple-proxy-linux-x86_64.tar.gz
- runner: ubuntu-24.04-arm
archive: maple-proxy-linux-aarch64.tar.gz
- runner: blacksmith-6vcpu-macos-26
- runner: blacksmith-6vcpu-macos-27
archive: maple-proxy-macos-aarch64.tar.gz
- runner: windows-2025
archive: maple-proxy-windows-x86_64.zip
Expand Down
Loading
Loading