Skip to content

test(e2e): run Grafana 13.2.3, 12.4.12 and Logs Drilldown 2.6.0 - #768

Merged
szibis merged 2 commits into
mainfrom
build/grafana-13.2.3
Oct 18, 2026
Merged

szibis merged 2 commits into
mainfrom
build/grafana-13.2.3

Conversation

@szibis

@szibis szibis commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Moves the compatibility stack to the newest Grafana and Logs Drilldown releases that are at least two weeks old.

component was now released
Grafana (pinned runtime, e2e-compat, root compose, review override, compat workflows) 13.2.1 13.2.3 2026-09-29
Grafana previous-family smoke 12.4.10 12.4.12 2026-09-29
Logs Drilldown (grafana-lokiexplore-app) 2.5.2 2.6.0 (commit e894c0eb) 2026-09-22
VictoriaLogs datasource 0.32.0 0.32.0 newer 0.33.x is from 2026-10-15

Grafana 13.2.4 / 12.4.13 (2026-10-13), Drilldown 3.0.0 (2026-10-08) and the datasource 0.33.x are newer than two weeks and are left for a later bump.

What 2.6.0 changes for the proxy

  • grafanaDependency is unchanged from 2.5.2 (>=12.2.5-0 among others), so it installs on 12.4.12 and 13.2.3.
  • The new "volume aggregated by other log fields" view is behind the drilldown.logs.logsVolumeByField feature flag, off by default. When enabled it sends sum(count_over_time({…} <parser> <filters> [$__auto])) by (<field>), the field-breakdown shape the proxy already serves.
  • ServiceScene.subscribeToLokiConfig() keeps the $patternsData === undefined guard, so the Patterns tab initialisation issue documented for 2.0.4 to 2.5.2 also applies to 2.6.0 and the patterns-autodetect variant stays the default datasource.

Matrix

compatibility-matrix.json: Grafana pinned_version and runtime profiles (13.2.3 full and current smoke, 12.4.12 previous smoke); the datasource contract matrix adds 13.2.3 and 12.4.12 and keeps 13.2.1 and 12.4.10; the Drilldown contract pins 2.6.0 and keeps 2.5.2 in matrix_versions. The generated matrix report and matrix-link.json are regenerated. Docs (testing, e2e guide, Drilldown and datasource compatibility, compatibility matrix, patterns, manual acceptance) name the new pins.

Checks

  • TestPinnedCompatibilityMatrixMatchesCompose passes (pins in the manifest match the compose files).
  • Conformance gate, matrix_link.py --check, changelog gate, CI script tests and the bench A/B and visual unit tests pass.

Grafana 13.2.1 -> 13.2.3 (pinned runtime and the CI workflows), the previous-family smoke 12.4.10 -> 12.4.12, Logs Drilldown 2.5.2 -> 2.6.0 (commit e894c0eb). The VictoriaLogs datasource stays on 0.32.0. The datasource and Drilldown contract matrices keep the previous pins. Drilldown 2.6.0 keeps the Patterns tab initialisation guard, so the patterns-autodetect default datasource is still needed; its volume by other fields is behind a feature flag that is off by default.
@github-actions github-actions Bot added size/M Medium change scope/ci CI/CD scope/docs Documentation labels Oct 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⚪ Performance A/B: no change beyond noise

8 shape×range — 8 ⚪ same · results vs Loki: 7 same, 0 new difference(s), 1 pre-existing

Every shape is within noise of the base build (±30% and 100 ms).

8 unchanged shape×range (within noise)
shape range base p50 PR p50 change Loki p50 PR ÷ Loki result vs Loki
⚪ drilldown service landing 1h 25 ms 25 ms ⚪ +0% 20 ms 1.25× ✅ same
⚪ instant ungrouped rate instant 5 ms 5 ms ⚪ +0% 54 ms 0.09× ✅ same
⚪ json rate by level (no filter, no drop) 1h 400 400 ⚪ +0% 400 — ✅ same
⚪ json volume no filters 1h 159 ms 157 ms ⚪ -1% 37 ms 4.24× ✅ same
⚪ logs plain 1h 49 ms 53 ms ⚪ +8% 451 ms 0.12× ✅ same
⚪ logs | json 1h 69 ms 72 ms ⚪ +4% 50 ms 1.44× ✅ same
⚪ stream-label count by app 1h 18 ms 18 ms ⚪ +0% 31 ms 0.58× ✅ same
⚪ topk rate by app 1h 31 ms 31 ms ⚪ +0% 68 ms 0.46× ➖ differs (as base)
Cold (first-run) timings — what a user opening a panel sees
shape range base cold PR cold Loki cold PR ÷ Loki cold
drilldown service landing 1h 28 ms 24 ms 464 ms 0.05×
instant ungrouped rate instant 6 ms 7 ms 55 ms 0.13×
json rate by level (no filter, no drop) 1h 400 400 400 —
json volume no filters 1h 148 ms 147 ms 520 ms 0.28×
logs plain 1h 68 ms 50 ms 633 ms 0.08×
logs | json 1h 116 ms 118 ms 47 ms 2.51×
stream-label count by app 1h 20 ms 18 ms 404 ms 0.04×
topk rate by app 1h 31 ms 34 ms 507 ms 0.07×
Why these shapes
  • control smoke → 8: control/drilldown service landing, control/instant ungrouped rate, control/json rate by level (no filter, no drop), control/json volume no filters, control/logs plain, control/logs | json, control/stream-label count by app, control/topk rate by app

base 53038c1208 → PR 560d1289df · 4 runs (first is cold), warm p50 · ranges 1h · noise ±30% and 100 ms · VictoriaLogs restarts 0→0 · 1.9 min. Same runner, one fresh stack, targets interleaved per request. How to read and reproduce: bench/ab/README.md.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Visual smoke: passed with warnings

Base 53038c1208 vs PR 742a976f41, Grafana through the proxy builds and Loki on one fresh stack; run.
Core set: 5 capture(s) at 1h. Detailed set: 0 entries (nothing visual touched). 5 capture(s), 15m/1h/6h, 24h and 7d stay out of PR runs.

page range set base = PR (data) vs Loki (PR) pixel diff result
dd-fields 1h core ✅ identical identical (4 explained, 1 history-dependent) 0.12% ⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
dd-landing 1h core ✅ identical identical (3 explained) 0.14% ✅
dd-service-logs 1h core ✅ identical identical (3 explained, 1 history-dependent) 0.12% ⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
explore-logs-plain 1h core ✅ identical identical (1 explained) 0.07% ✅
explore-metric-plain 1h core ✅ identical identical (1 explained) 0.04% ✅

Gate: a base-vs-PR data difference passes when the PR is closer to Loki, and fails when the PR diverges from Loki or Loki cannot decide it (at a range Loki does not hold, a difference passes when the same page proved it closer to Loki at a shorter range; otherwise label the pull request visual-change-expected to accept an intended change). Also failing: a panel empty on the PR but not on the base, an error banner, panel error, error answer or new "No data" panel on the PR, a PR side that never settled, a difference that did not reproduce on the recapture. Pixel differences only warn.

dd-fields-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-18T07%3A34%3
  • history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-18T07%3A34%3A00.000Z&query=%7Bservic: body: content differs (rows 4 vs 2)
  • vs Loki, explained: query ('sum by (service_name_extracted) (count_over_time({service_name="otel-collector"} | servi: request only on one side (right) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision)
  • vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-18T07%3A34%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
  • vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-18T07%3A34%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
  • vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-fields-1h

dd-service-logs-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-18T07%3A34%3
  • history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-18T07%3A34%3A00.000Z&query=%7Bservic: body: content differs (rows 4 vs 2)
  • vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-18T07%3A34%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
  • vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-18T07%3A34%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
  • vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-service-logs-1h

Core set montages (3): base | PR | Loki

dd-landing-1h

dd-landing-1h

explore-logs-plain-1h

explore-logs-plain-1h

explore-metric-plain-1h

explore-metric-plain-1h

All montages of this run: pr-visuals/pr-768.

@github-actions

Copy link
Copy Markdown
Contributor

PR Quality Report

Compared against base branch main.

Coverage and tests

Signal Base PR Delta
Test count 11242 11242 0
Coverage 88.3% 88.3% 0.0% (stable)

Compatibility

Track Base PR Delta
Loki API 100.0% 11/11 (100.0%) 0.0% (stable)
Logs Drilldown 100.0% 17/17 (100.0%) 0.0% (stable)
VictoriaLogs 100.0% 11/11 (100.0%) 0.0% (stable)

Performance smoke

Lower CPU cost (ns/op) is better. Lower benchmark memory cost (B/op, allocs/op) is better. Higher throughput is better. Lower load-test memory growth is better. Benchmark rows are medians from repeated samples.

Signal Base PR Delta
QueryRange cache-hit CPU cost 3389.0 ns/op 3363.0 ns/op -0.8% (stable)
QueryRange cache-hit memory 1240.0 B/op 1240.0 B/op 0.0% (stable)
QueryRange cache-hit allocations 27.0 allocs/op 27.0 allocs/op 0.0% (stable)
QueryRange cache-bypass CPU cost 4073.0 ns/op 4137.0 ns/op +1.6% (stable)
QueryRange cache-bypass memory 1303.0 B/op 1303.0 B/op 0.0% (stable)
QueryRange cache-bypass allocations 27.0 allocs/op 27.0 allocs/op 0.0% (stable)
Labels cache-hit CPU cost 885.6 ns/op 894.9 ns/op +1.1% (stable)
Labels cache-hit memory 48.0 B/op 48.0 B/op 0.0% (stable)
Labels cache-hit allocations 3.0 allocs/op 3.0 allocs/op 0.0% (stable)
Labels cache-bypass CPU cost 1479.0 ns/op 1484.0 ns/op +0.3% (stable)
Labels cache-bypass memory 399.0 B/op 400.0 B/op +0.3% (stable)
Labels cache-bypass allocations 14.0 allocs/op 14.0 allocs/op 0.0% (stable)

State

  • Coverage, compatibility, and sampled performance are reported here from the same PR workflow.
  • This is a delta report, not a release gate by itself. Required checks still decide merge safety.
  • Performance is a smoke comparison, not a full benchmark lab run.
  • Delta states use the same noise guards as the quality gate (percent + absolute + low-baseline checks), so report labels match merge-gate behavior.

@szibis
szibis marked this pull request as ready for review October 18, 2026 08:12
@szibis
szibis merged commit 52670e2 into main Oct 18, 2026
64 checks passed
@szibis
szibis deleted the build/grafana-13.2.3 branch October 18, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/ci CI/CD scope/docs Documentation size/M Medium change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant