Skip to content

vinca-gha: a job budget spent where it shortens the workflow most, under GitHub's size limit - #180

Merged
Tobias-Fischer merged 1 commit into
RoboStack:masterfrom
Tobias-Fischer:feat/job-budget
Oct 23, 2026
Merged

Tobias-Fischer merged 1 commit into
RoboStack:masterfrom
Tobias-Fischer:feat/job-budget

Conversation

@Tobias-Fischer

Copy link
Copy Markdown
Contributor

#179 added packing by measured times, but kept the old count batching next to it, and neither respected GitHub's 500 KiB workflow limit by itself. On Windows, where each job carried about 5.7 KiB of inlined scripts, a workflow packed by time came out at up to 1.8 MiB. RoboStack had to retry with fewer runners and larger batches, and finally fall back to count batching.

One batching logic, with or without times (batch_stages):

  1. Fewest jobs first: every stage gets as few jobs as --batch_size (packages per job) allows. Consecutive short stages, of at most 2 minutes each, share a job up to --batch_size packages. They run one after another anyway, so sharing costs no time.
  2. Spend the budget where it helps most: while the job budget lasts, the stage whose duration more jobs cut the most, per added job, gets them. Its packages are packed longest first, with at most one job per runner. The search looks past plateaus: 8 equal packages get faster again only at 8 jobs once they have 4.
  3. Without --timings, every package counts the same (BuildTimings' default), which splits the largest stages first. Count mode as such is gone.

The budget comes from the size limit: when the generated workflow is too large (WorkflowTooLarge), vinca-gha batches again with a budget scaled down by how much it is over. The slowest stages stay split, and callers no longer need their own retries.

Smaller jobs, so more of them fit:

  • The Windows jobs call .scripts\build_win.bat from their checkout instead of carrying its text, which was 2–3 KiB per job. That's the same file vinca inlined, which vinca-gha still requires.
  • The Build steps no longer repeat the job's package list in their name.

Simulated on RoboStack's jazzy full rebuild (the generated workflows, with the measured per-recipe times):

jobs workflow size wall time (20 runners / 8 runners)
linux-64, today (count) 117 292 KiB 6.41 h
linux-64, #179 timed 219 374 KiB 2.65 h / 3.04 h
linux-64, this PR, timed 214 302 KiB 2.60 h / 3.00 h
linux-64, this PR, no times 363 470 KiB 2.96 h / 3.16 h
win-64, today (count) 90 514 KiB (over the limit) 25.0 h
win-64, #179 timed 401 1859 KiB (over the limit) 6.0 h
win-64, this PR, timed 181 493 KiB 8.87 h / 9.14 h
win-64, this PR, no times 178 486 KiB 11.6 h

The size loop needed at most one retry.

The remaining Windows overhead is the inlined conda-forge build setup script, about 1.3 KiB per job. Moving it into the repository as well would fit about twice as many Windows jobs again, which could be a follow-up.

Tests:

  • updated for the merged short stages and longest-first packing;
  • new tests for the budget going to the slowest stages, splitting without times, and the Windows build script being called;
  • all 308 pass, and ruff is clean.

🤖 Claude (AI assistant), on behalf of @Tobias-Fischer

🤖 Generated with Claude Code

…ep it under GitHub's size limit

batch_stages now has one way of batching, with or without measured times:
each stage first gets as few jobs as --batch_size allows (consecutive short
stages, one job of at most 2 minutes each, share a job up to --batch_size
packages: they run one after another anyway), then, as long as the job budget
lasts, the stage whose duration more jobs cut the most (per added job) gets
them, its packages packed longest first, at most one job per runner. Without
--timings every package counts the same (BuildTimings' default), which splits
the largest stages first.

The budget comes from GitHub's 500 KiB workflow limit: when the generated
workflow is too large (WorkflowTooLarge), vinca-gha batches again with a
budget scaled down by how much it's over, so callers no longer retry with
larger batches or fewer runners themselves.

Smaller jobs, so that more fit: the Windows jobs call the repository's
.scripts/build_win.bat instead of carrying its text (2-3 KiB per job), and
the Build steps no longer repeat the job's package list in their name.

Simulated on RoboStack's jazzy full rebuild with its measured times (20 runners):
linux-64 6.41 -> 2.60 h (214 jobs, 302 KiB); win-64 25.0 h (90 jobs, over the
limit at batch size 25) -> 8.87 h (181 jobs, 493 KiB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Tobias-Fischer
Tobias-Fischer merged commit 7eef4ea into RoboStack:master Oct 23, 2026
3 checks passed
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Oct 23, 2026
…size retries

vinca-gha now keeps the workflow under GitHub's size limit itself (a job budget
spent where it shortens the workflow most, with or without build times), and
its Windows jobs call .scripts/build_win.bat instead of inlining it. rs gha
drops the retries with fewer runners and the fallback to batching by count;
larger batches remain the last resort when even the fewest jobs the batch
size allows don't fit. The check that the Windows workflow uses the
repository's build script looks for the call.

lyrical's full rebuild: win-64 149 jobs (483 KiB), linux-64 197 jobs (341 KiB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tobias-Fischer added a commit to RoboStack/robostack that referenced this pull request Oct 23, 2026
* lyrical: on the shared conda-forge pinning, Qt6 mapping shared (from #82)

Drops the distribution's own conda-forge-pinning version and migrations: it uses
shared/pinning/conda_forge.yaml (2026.10.09.13.11.31), with the package fixes the new
pins needed. Other distributions keep their own pinning version until their PR; their
conda_build_config.yaml only gets the comment of the new shared openexr override
(the value, 3.4, is what they pin already).

shared/robostack.yaml maps the Qt5 rosdep keys to Qt6 (qt6-main, pyqt6), so lyrical's
per-package Qt workarounds go. humble and jazzy keep Qt5 through overrides that their
own sync PRs add: merge those first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lyrical: rosdistro sync 2026-10-05

`rs lyrical update-snapshot`, with the fixes from the full-rebuild runs:
- lyrical: rtabmap without its tests (MSVC can't build test_optimizer_gtsam.cpp), as jazzy
- lyrical: python_qt_binding uses PyQt6 on macOS too (RoboStack/ros-lyrical#86)
- rcutils test: Python import only on humble and jazzy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lyrical: foxglove_bridge on Windows with /permissive

With C++20 (pulled in by lyrical's dependencies) MSVC enables /permissive-, which
rejects the Foxglove SDK 0.28's explicit specializations of ParameterView::get after
their first use (C2908/C3856 in parameter.hpp); humble/jazzy build it as C++17.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lyrical: Rust message crates match the rosidl_buffer sequence layout [skip ci]

From Lyrical on, rosidl_runtime_c's primitive and string sequences carry two
trailing rosidl_buffer flags (ros2/rosidl#942), 32 instead of 24 bytes.
rosidl_runtime_rs 0.7 has the matching layout (ros2-rust/rosidl_runtime_rs#22),
and rclrs 0.8 uses it, but rosidl_generator_rs 0.5.0 still pins the generated
crates to 0.6, so every primitive-sequence field is misread: a Marker or
JointState round trip segfaults (RoboStack/ros-lyrical#17,
ros2-rust/ros2_rust#659). Pin 0.7. The generated code needs no other change.

Checked locally: the generated std_msgs crate pins 0.7, and with 0.7 the
visualization_msgs/Marker and sensor_msgs/JointState round trips are correct
(rmw Marker 480 bytes, as in C), while 0.6 segfaults.

Also point gz_ros2_control's plugin path hook at bin/ on Windows, where the
plugin DLL is installed (#104).

Evict the generator, every package that generates interfaces (each ships its
generated Rust crate) and gz_ros2_control from the PR build cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update mutex_package configuration in vinca.yaml

Removed build number and upper bound for mutex_package.

* Clear evict_cache in ci.yaml

Reset evict_cache to an empty list.

* humble and rolling keep Qt5 for the Qt5 rosdep keys

This PR maps the Qt5 rosdep keys to Qt6 for every distribution. humble's own
override comes with its sync (#62), rolling moves to Qt6 with #99: until then
both get the override jazzy has (distros/<d>/robostack.yaml, layered over the
shared mapping with conda_index). Their generated recipes (humble linux-64,
win-64, osx-arm64; rolling linux-64, win-64) are identical to main's, except
for the new rcutils package test, which doesn't change any build hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* rosbag2_performance_benchmarking_msgs: no own short-paths flag

vinca's Windows build scripts pass -DCMAKE_INTERMEDIATE_DIR_STRATEGY=SHORT for
every package since RoboStack/vinca#178 (pinned with #139, which has to be
merged before this PR).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build branches: a newer run cancels the older one

A new push to a build branch (after a merge to main) brings the latest
recipes and workflow; the older run's plan is out of date. The new run skips
what the old one uploaded already, so only the packages its cancelled jobs
had built but not uploaded are built again. Re-run failed jobs only in the
newest run: re-running an older one would cancel the newer run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build times from the jazzy full rebuild (linux-64, linux-aarch64, osx-arm64)

Per-recipe build times of the jazzy build-branch runs 39770179894,
39770118028 and 39770103102, from their job logs, so that this sync's
lyrical rebuild is already packed by time (osx-64 and win-64 weren't
finished yet). The weekly update-build-timings PR adds the times the build
jobs record from now on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build times for all five platforms, also for lyrical's compatibility recipes

osx-64 and win-64 from the jazzy build-branch runs' job logs (39770120710 and
39907983469; 39770136120 and the finished jobs of 39907979075). Recipes those
runs didn't reach are estimated from their linux-64 time with a fit per
platform (osx-64: 37 s + 1.61 x; win-64: 77 s + 2.13 x; median error 33% and
20%), compatibility recipes at the platform's median (0.3-0.4 s).

Every ros-jazzy-<name> time is copied to ros-lyrical-<name> (under
"lyrical"), so that lyrical's compatibility recipes have a time until its own
build jobs record theirs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test_generate_gha: independent of the repository's build times

With shared/build_timings/win-64.json present, rs gha packs by time and first
retries an oversized workflow with fewer runners, so the batch-size retry test
now runs without timings; a new test covers the retry with timings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* rs gha: fall back to batching by count when a timed workflow stays too large

Packing by time keeps short stages apart (they share a job only up to two
minutes), so lyrical's win-64 workflow, whose jobs carry the long Windows
scripts, stayed at 621 KiB even with --runners 2. After the runner retries,
generate it without --timings; batching by count then merges the short
stages and the larger-batch retries shrink it as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* vinca with job budgets (RoboStack/vinca#180); rs gha without its own size retries

vinca-gha now keeps the workflow under GitHub's size limit itself (a job budget
spent where it shortens the workflow most, with or without build times), and
its Windows jobs call .scripts/build_win.bat instead of inlining it. rs gha
drops the retries with fewer runners and the fallback to batching by count;
larger batches remain the last resort when even the fewest jobs the batch
size allows don't fit. The check that the Windows workflow uses the
repository's build script looks for the call.

lyrical's full rebuild: win-64 149 jobs (483 KiB), linux-64 197 jobs (341 KiB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Tobias Fischer <info@tobiasfischer.info>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant