Skip to content

Add static-tls profile: 20-file static serving over HTTP/1.1 + TLS - #1087

Merged
MDA2AV merged 3 commits into
mainfrom
feat/static-tls-profile
Aug 7, 2026
Merged

Add static-tls profile: 20-file static serving over HTTP/1.1 + TLS#1087
MDA2AV merged 3 commits into
mainfrom
feat/static-tls-profile

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Exactly the static workload transported over TLS on :8081 (the json-tls port), driven by wrk with the same static-rotate.lua. Scored at 1024/4096/6800 connections like static.

  • profiles.sh: spec + order + wrk dispatch for the static-tls endpoint
  • wrk adapter: https://localhost:8081 with the existing rotation script
  • framework.sh: TLS readiness probe on /static/reset.css
  • validate.sh: full static check suite over TLS (ALPN http/1.1 negotiation, Content-Types, 20 file sizes, compression, 404), and static-tls now triggers the 8081 port mapping + static mount
  • docs: implementation/validation/index pages + isolated section card
  • catalog: scored, not engine-scored (mirrors static)

Also subscribes genhttp-11-ioxide to the new test. Validation passes except vendor.js (300 KB): ioxide.tls 0.0.17 hangs on TLS responses larger than exactly 256 KB (262,144 B verified by bisection; plaintext unaffected, no configurable buffer in TlsOptions). Needs an engine fix before this framework's static-tls results can land.

Description


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@MDA2AV

MDA2AV commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-11 -t static-tls --save

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

⚠️ /benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

👋 Heads up! This PR modifies the following frameworks:

MDA2AV added 2 commits August 7, 2026 20:44
Exactly the static workload transported over TLS on :8081 (the json-tls
port), driven by wrk with the same static-rotate.lua. Scored at
1024/4096/6800 connections like static.

- profiles.sh: spec + order + wrk dispatch for the static-tls endpoint
- wrk adapter: https://localhost:8081 with the existing rotation script
- framework.sh: TLS readiness probe on /static/reset.css
- validate.sh: full static check suite over TLS (ALPN http/1.1
  negotiation, Content-Types, 20 file sizes, compression, 404), and
  static-tls now triggers the 8081 port mapping + static mount
- docs: implementation/validation/index pages + isolated section card
- catalog: scored, not engine-scored (mirrors static)

Also subscribes genhttp-11-ioxide to the new test. Validation passes
except vendor.js (300 KB): ioxide.tls 0.0.17 hangs on TLS responses
larger than exactly 256 KB (262,144 B verified by bisection; plaintext
unaffected, no configurable buffer in TlsOptions). Needs an engine fix
before this framework's static-tls results can land.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
genhttp-11's standard TLS stack (SslStream via X509 PEM binding on
:8081, shared handler tree with Assets.From) passes the full static-tls
validation suite — 56/56 checks including the 300 KB vendor.js that
trips the ioxide.tls 256 KB write-path hang. genhttp-11-ioxide's
subscription is withdrawn until the engine fix lands.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@MDA2AV
MDA2AV force-pushed the feat/static-tls-profile branch from 52ad005 to 402f83c Compare August 7, 2026 19:45
@MDA2AV

MDA2AV commented Aug 7, 2026

Copy link
Copy Markdown
Owner Author

/benchmark -f genhttp-11 -t static-tls --save

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: genhttp-11 | Test: static-tls

Test Conn RPS CPU Mem Δ RPS Δ Mem
static-tls 1024 524,716 5734.1% 514MiB NEW NEW
static-tls 4096 491,305 6126.1% 996MiB NEW NEW
static-tls 6800 462,125 6280.7% 1.4GiB NEW NEW
Full log

#4 [internal] load .dockerignore
#4 transferring context:
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [stage-1 1/3] FROM mcr.microsoft.com/dotnet/runtime:10.0@sha256:8fb7ff015fcf0ebc6e90105bd6db06875954e6dc3d374b9dbb34c732867d13e4
#5 resolve mcr.microsoft.com/dotnet/runtime:10.0@sha256:8fb7ff015fcf0ebc6e90105bd6db06875954e6dc3d374b9dbb34c732867d13e4 0.1s done
#5 DONE 0.1s

#6 [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0@sha256:8a90a473da5205a16979de99d2fc20975e922c68304f5c79d564e666dc3982fc
#6 resolve mcr.microsoft.com/dotnet/sdk:10.0@sha256:8a90a473da5205a16979de99d2fc20975e922c68304f5c79d564e666dc3982fc 0.1s done
#6 DONE 0.1s

#7 [internal] load build context
#7 transferring context: 537B done
#7 DONE 0.0s

#8 [stage-1 2/3] WORKDIR /app
#8 CACHED

#9 [build 3/6] COPY genhttp.csproj ./
#9 CACHED

#10 [build 6/6] RUN dotnet publish -c Release --no-self-contained -o /app
#10 CACHED

#11 [build 2/6] WORKDIR /source
#11 CACHED

#12 [build 4/6] RUN dotnet restore
#12 CACHED

#13 [build 5/6] COPY . .
#13 CACHED

#14 [stage-1 3/3] COPY --from=build /app .
#14 CACHED

#15 exporting to image
#15 exporting layers done
#15 exporting manifest sha256:7f48f265d00df1e7571744016c59294a40078ca0a999c4a5ac09f3c5e959490c done
#15 exporting config sha256:fc2b2b326eee7babc9822bb8316be0dc3877b81e6118e68b2fadc913c35ec4b9 done
#15 exporting attestation manifest sha256:c558b82a388f39c95e48246be6e1451ee478e80b29aad99176aff352f2a4b8f1 0.0s done
#15 exporting manifest list sha256:0d38e06c9f31c15bfcfb36c4913f254880e5f1fcb25b51b525ecebcfc0cd2ca4
#15 exporting manifest list sha256:0d38e06c9f31c15bfcfb36c4913f254880e5f1fcb25b51b525ecebcfc0cd2ca4 0.0s done
#15 naming to docker.io/library/httparena-genhttp-11:latest done
#15 unpacking to docker.io/library/httparena-genhttp-11:latest done
#15 DONE 0.1s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] starting postgres sidecar
[info] postgres ready (seeded)
[info] starting redis sidecar (cpuset=0,64)
[info] redis ready

==============================================
=== genhttp-11 / static-tls / 1024c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ https://localhost:8081
  64 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.99ms    3.22ms  47.91ms   91.01%
    Req/Sec     6.35k     1.82k   16.58k    75.11%
  2022086 requests in 5.10s, 30.92GB read
Requests/sec: 396468.95
Transfer/sec:      6.06GB
[info] CPU 5866.6% | Mem 395MiB

[run 2/3]
Running 5s test @ https://localhost:8081
  64 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.50ms    2.54ms  45.05ms   88.07%
    Req/Sec     8.07k     1.04k   15.01k    82.53%
  2611522 requests in 5.10s, 39.94GB read
Requests/sec: 512125.50
Transfer/sec:      7.83GB
[info] CPU 5595.4% | Mem 426MiB

[run 3/3]
Running 5s test @ https://localhost:8081
  64 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.36ms    2.24ms  29.34ms   88.39%
    Req/Sec     8.24k     0.97k   20.00k    83.79%
  2676131 requests in 5.10s, 40.92GB read
Requests/sec: 524716.34
Transfer/sec:      8.02GB
[info] CPU 5734.1% | Mem 514MiB

=== Best: 524716 req/s (CPU: 5734.1%, Mem: 514MiB) ===
[info] saved results/static-tls/1024/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / static-tls / 4096c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ https://localhost:8081
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.90ms    4.59ms  71.06ms   78.21%
    Req/Sec     6.88k     2.90k   22.09k    80.72%
  1749893 requests in 5.10s, 26.76GB read
Requests/sec: 343146.55
Transfer/sec:      5.25GB
[info] CPU 5909.5% | Mem 742MiB

[run 2/3]
Running 5s test @ https://localhost:8081
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.53ms    3.63ms  88.43ms   90.73%
    Req/Sec     7.62k     1.26k   17.21k    91.81%
  2471848 requests in 5.10s, 37.80GB read
Requests/sec: 484623.72
Transfer/sec:      7.41GB
[info] CPU 6172.6% | Mem 920MiB

[run 3/3]
Running 5s test @ https://localhost:8081
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.42ms    3.51ms  68.36ms   90.58%
    Req/Sec     7.70k     1.23k   17.33k    91.59%
  2505948 requests in 5.10s, 38.32GB read
Requests/sec: 491305.71
Transfer/sec:      7.51GB
[info] CPU 6126.1% | Mem 996MiB

=== Best: 491305 req/s (CPU: 6126.1%, Mem: 996MiB) ===
[info] saved results/static-tls/4096/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11

==============================================
=== genhttp-11 / static-tls / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ https://localhost:8081
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.94ms    6.92ms 131.57ms   78.92%
    Req/Sec     5.79k     2.47k   16.14k    82.72%
  1634538 requests in 5.10s, 25.00GB read
Requests/sec: 320770.30
Transfer/sec:      4.91GB
[info] CPU 5814.2% | Mem 1.1GiB

[run 2/3]
Running 5s test @ https://localhost:8081
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.10ms    4.19ms 124.94ms   90.93%
    Req/Sec     7.30k     1.60k   16.09k    88.51%
  2357523 requests in 5.10s, 36.05GB read
Requests/sec: 462125.85
Transfer/sec:      7.07GB
[info] CPU 6280.7% | Mem 1.4GiB

[run 3/3]
Running 5s test @ https://localhost:8081
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.53ms    4.93ms 117.11ms   92.13%
    Req/Sec     7.23k     1.52k   16.11k    90.75%
  2351244 requests in 5.10s, 35.96GB read
Requests/sec: 461296.78
Transfer/sec:      7.05GB
[info] CPU 6146.4% | Mem 1.6GiB

=== Best: 462125 req/s (CPU: 6280.7%, Mem: 1.4GiB) ===
[info] saved results/static-tls/6800/genhttp-11.json
httparena-bench-genhttp-11
httparena-bench-genhttp-11
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/genhttp-11.json - 3 new, 31 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@MDA2AV
MDA2AV merged commit c697c20 into main Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant