Skip to content

bench(jit): measure mixed gpui-shell workload - #17

Closed
huacnlee wants to merge 2 commits into
mainfrom
gpui-mixed-benchmark
Closed

bench(jit): measure mixed gpui-shell workload#17
huacnlee wants to merge 2 commits into
mainfrom
gpui-mixed-benchmark

Conversation

@huacnlee

@huacnlee huacnlee commented Sep 4, 2026

Copy link
Copy Markdown
Member

Issue # (if available)

Follow-up to longbridge/gpui-kit#2903.

Description of changes

  • add a real gpui-shell market workload that computes and aggregates 96 quote scores, sorts them, and builds a 12-row visible list
  • extend the paired fresh-process runner and report format with native-required observational workloads
  • support exact crates.io versions as well as Git revisions in the integration runner
  • record clean x86_64 Linux evidence: mixed steady-state 2.32x..2.34x and P99 2.20x..2.27x, with snapshot parity and zero fallback/deopt
  • document the next target: diagnose four invalid artifacts per mixed automatic process, then isolate array/object sorting and host-list stages

x86_64 Linux gpui-shell performance evidence

Measured from clean commits 034c9f04f80ef0464ee2589dbb202608ac090e4d (quickjs-jit) and c0b989691707d5c18d973cf9cf805bbcadde37b5 (gpui-kit).

Sampling: 5 discarded warmup processes, 30 interleaved paired fresh processes, 200 observations per process, and 10,000 paired bootstrap resamples. Every process was pinned to CPU 0.

Workload Steady-state speed CI P99 speed CI Native entries Fallback / deopt
Realistic 443-node host-heavy panel 0.99x..0.99x 0.99x..1.00x 0 0 / 0
Render-driven numeric layout checksum 15.34x..18.33x 10.73x..15.24x 7,028 0 / 0
Market compute, sort, aggregate, and visible list 2.32x..2.34x 2.20x..2.27x 732,467 0 / 0

Lifecycle speed CIs: first window 0.99x..1.00x; hot reload 0.99x..1.00x. Interpreter and automatic mode produced identical checksums, snapshot SHA-256 values, and script-render counts in all 30 pairs. The overall acceptance report passed.

The mixed workload computes 96 quote scores, aggregates and sorts them, then builds a 12-row visible list through the real GPUI builder API. This demonstrates a stable product-shaped gain above 2x while retaining the host-heavy panel regression guard.

Known follow-up: mixed automatic samples produced 120 optimizing-tier invalid-artifact compilation failures (four per process). They did not affect semantics or the installed native path, but tracing and eliminating or correctly classifying them is the next optimization target.

Verification:

  • cargo test --workspace passed.
  • gpui-shell release suite: 697 passed, 1 ignored.
  • gpui-shell integration tests: 4 passed.
  • gpui-shell doctests: 4 passed, 2 ignored.

Tracked evidence: benchmarks/results/gpui-shell-mixed-v1.json and .md.

Verification

  • cargo test --workspace
  • scripts/bench-gpui-shell.sh /home/jason/work/gpui-kit /tmp/gpui-shell-mixed-clean.json
  • gpui-shell release suite: 697 passed, 1 ignored; integration tests: 4 passed; doctests: 4 passed, 2 ignored

Checklist

  • Added change to the changelog (benchmark/documentation-only change)
  • Created unit tests for my feature if needed

性能进展

场景 95% CI 结论
混合场景 steady-state 2.32x–2.34x 明确超过解释器 2 倍
混合场景 P99 2.20x–2.27x 尾延迟也稳定提升
纯计算 steady-state 15.34x–18.33x 显著加速
纯计算 P99 10.73x–15.24x 显著加速
Host-heavy panel 0.99x 慢约 1%,通过 5% guard
首窗口 0.99x–1.00x 基本持平
热重载 0.99x–1.00x 基本持平

@huacnlee

huacnlee commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

x86_64 Linux gpui-shell performance evidence

Measured from clean commits 034c9f04f80ef0464ee2589dbb202608ac090e4d (quickjs-jit) and c0b989691707d5c18d973cf9cf805bbcadde37b5 (gpui-kit).

Sampling: 5 discarded warmup processes, 30 interleaved paired fresh processes, 200 observations per process, and 10,000 paired bootstrap resamples. Every process was pinned to CPU 0.

Workload Steady-state speed CI P99 speed CI Native entries Fallback / deopt
Realistic 443-node host-heavy panel 0.99x..0.99x 0.99x..1.00x 0 0 / 0
Render-driven numeric layout checksum 15.34x..18.33x 10.73x..15.24x 7,028 0 / 0
Market compute, sort, aggregate, and visible list 2.32x..2.34x 2.20x..2.27x 732,467 0 / 0

Lifecycle speed CIs: first window 0.99x..1.00x; hot reload 0.99x..1.00x. Interpreter and automatic mode produced identical checksums, snapshot SHA-256 values, and script-render counts in all 30 pairs. The overall acceptance report passed.

The mixed workload computes 96 quote scores, aggregates and sorts them, then builds a 12-row visible list through the real GPUI builder API. This demonstrates a stable product-shaped gain above 2x while retaining the host-heavy panel regression guard.

Known follow-up: mixed automatic samples produced 120 optimizing-tier invalid-artifact compilation failures (four per process). They did not affect semantics or the installed native path, but tracing and eliminating or correctly classifying them is the next optimization target.

Verification:

  • cargo test --workspace passed.
  • gpui-shell release suite: 697 passed, 1 ignored.
  • gpui-shell integration tests: 4 passed.
  • gpui-shell doctests: 4 passed, 2 ignored.

Tracked evidence: benchmarks/results/gpui-shell-mixed-v1.json and .md.

@huacnlee huacnlee closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant