fix: a scalar answer carries its time in milliseconds, like Loki

[szibis]

Summary

Follow-up to #735. On live Loki 3.7.7, a scalar answer such as query=1&time=1791811168123456789 returns its time as integer milliseconds: {"resultType":"scalar","result":[1791811168123,"1"]}. A vector(1) answer returns seconds: [1791811168.123,"1"]. #735 returned the scalar time in seconds.

Why Loki returns milliseconds. The value comes from Loki 3.7.7's query-frontend encoding:

• engine.go:700-702 evalLiteral keeps the time in milliseconds.
• queryrange/marshal.go:97-112 turns the scalar into a LokiPromResponse with TimestampMs: v.T.
• queryrange/prometheus.go:163-176 marshalScalar then builds the time with model.TimeFromUnix(TimestampMs). TimeFromUnix takes seconds and Time.String divides by 1000, so the millisecond count is printed as if it were seconds.
• marshalVector (prometheus.go:97-100) uses model.Time(TimestampMs) directly, which is why vectors stay in seconds.
• The JSON and protobuf codec paths feed the same function, so the result does not depend on configuration.

Fix. Scalar answers now carry integer milliseconds, as Loki does. That covers the constant instant answer and the scalar-op-scalar binary answer. vector() answers stay in seconds.

Tests

• Unit tests with exact JSON for scalar and vector times. The previous expectations, which asserted seconds, are corrected.
• e2e vs Loki: a new "constant sample time" subtest in TestCompat_InstantResultTypeAlwaysSet compares the time and value with Loki for 1, -1, 1 + 1, 2 * (3 + 4), 1 > bool 2, vector(1) and vector(1) + 2.
  • It was run locally against Loki 3.7.7 with VictoriaLogs v1.52.0.
  • It passes on this branch. On main it fails: loki=[1.791814361544e+12 1] proxy=[1.791814361544e+09 1].

Notes

• 1/0 and other literal divisions by zero answer HTTP 500 on Loki 3.7.7 after about 40 s. The query frontend cannot marshal NaN: grpc: error while marshaling: unsupported value: NaN. The proxy answers the scalar NaN that Loki's own MergeBinOp fold computes, and does not reproduce that internal error. 1/0 is left out of the e2e.
• The query_range scalar-op-scalar path still answers +Inf for 1/0. fix: a division or modulo by zero is NaN, like Loki #745 makes it NaN.

[github-actions]

⚪ Performance A/B: no change beyond noise

26 shape×range — 26 ⚪ same · results vs Loki: 25 same, 0 new difference(s), 1 pre-existing

Every shape is within noise of the base build (±30% and 100 ms).

26 unchanged shape×range (within noise)

	shape	range	base p50	PR p50	change	Loki p50	PR ÷ Loki	result vs Loki
⚪	bytes_over_time by app	1h	72 ms	70 ms	⚪ -3%	89 ms	0.79×	✅ same
⚪	drilldown service landing	1h	32 ms	33 ms	⚪ +3%	25 ms	1.32×	✅ same
⚪	instant json volume	instant	440 ms	429 ms	⚪ -3%	94 ms	4.56×	✅ same
⚪	instant ungrouped rate	instant	7 ms	7 ms	⚪ +0%	72 ms	0.10×	✅ same
⚪	json rate by level (no filter, no drop)	1h	400	400	⚪ -33%	400	—	✅ same
⚪	json volume no filters	1h	1 ms	1 ms	⚪ +0%	3 ms	0.33×	✅ same
⚪	logs plain	1h	57 ms	61 ms	⚪ +7%	36 ms	1.69×	✅ same
⚪	logs | json	1h	88 ms	84 ms	⚪ -5%	59 ms	1.42×	✅ same
⚪	stream-label count by app	1h	25 ms	29 ms	⚪ +16%	38 ms	0.76×	✅ same
⚪	topk rate by app	1h	45 ms	44 ms	⚪ -2%	90 ms	0.49×	➖ differs (as base)
⚪	ungrouped rate	1h	14 ms	11 ms	⚪ -21%	86 ms	0.13×	✅ same
⚪	DB1 label breakdown pod	1h	104 ms	88 ms	⚪ -15%	116 ms	0.76×	✅ same
⚪	DB2 label breakdown namespace	1h	15 ms	16 ms	⚪ +7%	30 ms	0.53×	✅ same
⚪	DB3 json field breakdown user_id	1h	47 ms	48 ms	⚪ +2%	24 ms	2.00×	✅ same
⚪	DB4 json field breakdown batch_id	1h	101 ms	99 ms	⚪ -2%	74 ms	1.34×	✅ same
⚪	A explore volume, 2 filters, drop	1h	159 ms	151 ms	⚪ -5%	113 ms	1.34×	✅ same
⚪	D volume, no filters	1h	211 ms	214 ms	⚪ +1%	123 ms	1.74×	✅ same
⚪	E field breakdown pipeline (explore)	1h	101 ms	101 ms	⚪ +0%	35 ms	2.89×	✅ same
⚪	F field breakdown service_version (explore)	1h	160 ms	162 ms	⚪ +1%	44 ms	3.68×	✅ same
⚪	G field breakdown pipeline (drilldown)	1h	44 ms	44 ms	⚪ +0%	22 ms	2.00×	✅ same
⚪	H field breakdown service_version (drilldown)	1h	95 ms	97 ms	⚪ +2%	25 ms	3.88×	✅ same
⚪	I grafana volume of B	1h	100 ms	103 ms	⚪ +3%	35 ms	2.94×	✅ same
⚪	O2 explore log volume (Grafana form), underscore structured-metadata filter	1h	71 ms	67 ms	⚪ -6%	34 ms	1.97×	✅ same
⚪	DL1 drilldown field breakdown json-expr, level filter	1h	485 ms	485 ms	⚪ +0%	22 ms	22×	✅ same
⚪	DL2 drilldown field breakdown, level + stream-label filter before json	1h	81 ms	82 ms	⚪ +1%	7 ms	12×	✅ same
⚪	DL6 explore volume, level filter before json	1h	508 ms	506 ms	⚪ -0%	39 ms	13×	✅ sum ±0.31%

Cold (first-run) timings — what a user opening a panel sees

shape	range	base cold	PR cold	Loki cold	PR ÷ Loki cold
bytes_over_time by app	1h	72 ms	71 ms	828 ms	0.09×
drilldown service landing	1h	35 ms	45 ms	703 ms	0.06×
instant json volume	instant	473 ms	466 ms	98 ms	4.76×
instant ungrouped rate	instant	9 ms	13 ms	74 ms	0.18×
json rate by level (no filter, no drop)	1h	400	400	400	—
json volume no filters	1h	216 ms	188 ms	2 ms	94.00×
logs plain	1h	87 ms	67 ms	38 ms	1.76×
logs | json	1h	94 ms	94 ms	58 ms	1.62×
stream-label count by app	1h	31 ms	25 ms	817 ms	0.03×
topk rate by app	1h	42 ms	47 ms	856 ms	0.05×
ungrouped rate	1h	11 ms	12 ms	777 ms	0.02×
DB1 label breakdown pod	1h	106 ms	96 ms	948 ms	0.10×
DB2 label breakdown namespace	1h	22 ms	20 ms	661 ms	0.03×
DB3 json field breakdown user_id	1h	53 ms	50 ms	737 ms	0.07×
DB4 json field breakdown batch_id	1h	97 ms	91 ms	673 ms	0.14×
A explore volume, 2 filters, drop	1h	2.90 s	1.78 s	1.26 s	1.41×
D volume, no filters	1h	200 ms	204 ms	1.94 s	0.11×
E field breakdown pipeline (explore)	1h	108 ms	113 ms	683 ms	0.17×
F field breakdown service_version (explore)	1h	1.35 s	1.33 s	827 ms	1.61×
G field breakdown pipeline (drilldown)	1h	46 ms	58 ms	692 ms	0.08×
H field breakdown service_version (drilldown)	1h	145 ms	151 ms	725 ms	0.21×
I grafana volume of B	1h	112 ms	115 ms	657 ms	0.18×
O2 explore log volume (Grafana form), underscore structured-metadata filter	1h	169 ms	151 ms	636 ms	0.24×
DL1 drilldown field breakdown json-expr, level filter	1h	488 ms	488 ms	570 ms	0.86×
DL2 drilldown field breakdown, level + stream-label filter before json	1h	86 ms	86 ms	30 ms	2.87×
DL6 explore volume, level filter before json	1h	1.03 s	1.03 s	850 ms	1.22×

Why these shapes

• internal/proxy/metric_agg.go → 22: control/bytes_over_time by app, control/instant json volume, control/instant ungrouped rate, control/json volume no filters, control/stream-label count by app, control/topk rate by app, control/ungrouped rate, drilldown-breakdown-exact/DB1 label breakdown pod …
• control smoke → 8: control/drilldown service landing, control/instant ungrouped rate, control/json rate by level (no filter, no drop), control/json volume no filters, control/logs plain, control/logs | json, control/stream-label count by app, control/topk rate by app

_{base 08f27eafe5 → PR 5a37059ed3 · 4 runs (first is cold), warm p50 · ranges 1h · noise ±30% and 100 ms · VictoriaLogs restarts 0→0 · 3.6 min. Same runner, one fresh stack, targets interleaved per request. How to read and reproduce: bench/ab/README.md.}

[github-actions]

✅ Visual smoke: passed with warnings

Base 08f27eafe5 vs PR f92077ebd9, Grafana through the proxy builds and Loki on one fresh stack; run.
Core set: 5 capture(s) at 1h. Detailed set: 7 entries (explore-metric-plain, explore-A, explore-E, explore-F, dd-labels-namespace, dd-field-service\_version, dd-field-pipeline). 25 capture(s), 15m/1h/6h, 24h and 7d stay out of PR runs.

Fix proof

Captures of the exact cases this pull request adds or fixes (registry case, base | PR | Loki). The verdict compares the PR with Loki where the base did not match. The gate fails a case unless one capture is fixed or improved, or, beyond the Loki data window, the base failed where Loki and the PR both answered with data (the label does not excuse it; only visual: none on the case does).

• semantics/scalar-sample-time-units: no visual proof, exempt: instant scalar literals only; no Explore range panel renders them, the fix is proven by the sample-time comparison against Loki in the e2e

Regression check

page	range	set	base = PR (data)	vs Loki (PR)	pixel diff	result
dd-fields	1h	core	✅ identical	identical (4 explained, 1 history-dependent)	0.11%	⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
dd-landing	1h	core	✅ identical	identical (3 explained)	0.13%	✅
dd-service-logs	1h	core	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
explore-logs-plain	1h	core	✅ identical	identical (1 explained)	0.07%	✅
explore-metric-plain	1h	core	✅ identical	identical (1 explained)	0.04%	✅
dd-field-pipeline	15m	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	✅
dd-field-pipeline	1h	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
dd-field-pipeline	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.11%	✅
dd-field-service_version	15m	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	✅
dd-field-service_version	1h	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
dd-field-service_version	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.11%	✅
dd-labels-namespace	15m	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	✅
dd-labels-namespace	1h	detailed	✅ identical	identical (3 explained, 1 history-dependent)	0.11%	⚠️ history-dependent difference, not gated (1): resource /api/datasources/uid/*/res
dd-labels-namespace	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.11%	✅
explore-A	15m	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-A	1h	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-A	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.04%	✅
explore-E	15m	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-E	1h	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-E	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.04%	✅
explore-F	15m	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-F	1h	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-F	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.04%	✅
explore-metric-plain	15m	detailed	✅ identical	identical (1 explained)	0.04%	✅
explore-metric-plain	6h	detailed	✅ identical	n/a (beyond the Loki window)	0.04%	✅

Gate: a base-vs-PR data difference passes when the PR is closer to Loki, and fails when the PR diverges from Loki or Loki cannot decide it (at a range Loki does not hold, a difference passes when the same page proved it closer to Loki at a shorter range; otherwise label the pull request visual-change-expected to accept an intended change). Also failing: a panel empty on the PR but not on the base, an error banner, panel error, error answer or new "No data" panel on the PR, a PR side that never settled, a difference that did not reproduce on the recapture. Pixel differences only warn.

dd-fields-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3

• history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3A00.000Z&query=%7Bservic: body: content differs (rows 6 vs 6)
• vs Loki, explained: query ('sum by (service_name_extracted) (count_over_time({service_name="otel-collector"} | servi: request only on one side (right) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision)
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-service-logs-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3

• history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3A00.000Z&query=%7Bservic: body: content differs (rows 6 vs 6)
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-field-pipeline-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3

• history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3A00.000Z&query=%7Bservic: body: content differs (rows 6 vs 6)
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-field-service_version-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3

• history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3A00.000Z&query=%7Bservic: body: content differs (rows 6 vs 6)
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

dd-labels-namespace-1h: history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3

• history-dependent difference, not gated (1): resource /api/datasources/uid/*/resources/patterns?end=2026-10-12T17%3A08%3A00.000Z&query=%7Bservic: body: content differs (rows 6 vs 6)
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_fields?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 13 vs 15) -- explained: Loki's _extracted suffix for a key named like a stream label, where the build does not return it (detected_fields: open, semantics/detected-fields-extracted-suffix; service/service.name: owner decision); the proxy also returns data, status; Loki lists fields in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/detected_labels?end=2026-10-12T17%3A08%3A00.000Z&query=%7: body: content differs (rows 9 vs 9) -- explained: sampled cardinality: Loki counts every stream its ingesters hold, not the requested window (pkg/ingester/instance.go LabelsWithValues), the proxy the window's last 5 minutes (metadataMaxFieldNamesWindow); the label names match; the proxy also returns data, limit, status; Loki lists labels in no fixed order
• vs Loki, explained: resource /api/datasources/uid/*/resources/drilldown-limits: body: content differs (rows 0 vs 0) -- explained: configuration probe: each backend publishes its own limits, version and settings (the proxy's pattern persistence)

Core set montages (3): base | PR | Loki

dd-landing-1h

explore-logs-plain-1h

explore-metric-plain-1h

All montages of this run: pr-visuals/pr-752.

[github-actions]

PR Quality Report

Compared against base branch main.

Coverage and tests

Signal	Base	PR	Delta
Test count	9724	9724	0
Coverage	87.8%	87.8%	0.0% (stable)

Compatibility

Track	Base	PR	Delta
Loki API	100.0%	11/11 (100.0%)	0.0% (stable)
Logs Drilldown	100.0%	17/17 (100.0%)	0.0% (stable)
VictoriaLogs	100.0%	11/11 (100.0%)	0.0% (stable)

Performance smoke

Lower CPU cost (ns/op) is better. Lower benchmark memory cost (B/op, allocs/op) is better. Higher throughput is better. Lower load-test memory growth is better. Benchmark rows are medians from repeated samples.

Signal	Base	PR	Delta
QueryRange cache-hit CPU cost	3307.0 ns/op	3294.0 ns/op	-0.4% (stable)
QueryRange cache-hit memory	1240.0 B/op	1240.0 B/op	0.0% (stable)
QueryRange cache-hit allocations	27.0 allocs/op	27.0 allocs/op	0.0% (stable)
QueryRange cache-bypass CPU cost	3785.0 ns/op	3808.0 ns/op	+0.6% (stable)
QueryRange cache-bypass memory	1301.0 B/op	1299.0 B/op	-0.2% (stable)
QueryRange cache-bypass allocations	27.0 allocs/op	27.0 allocs/op	0.0% (stable)
Labels cache-hit CPU cost	910.4 ns/op	908.6 ns/op	-0.2% (stable)
Labels cache-hit memory	48.0 B/op	48.0 B/op	0.0% (stable)
Labels cache-hit allocations	3.0 allocs/op	3.0 allocs/op	0.0% (stable)
Labels cache-bypass CPU cost	1521.0 ns/op	1513.0 ns/op	-0.5% (stable)
Labels cache-bypass memory	399.0 B/op	399.0 B/op	0.0% (stable)
Labels cache-bypass allocations	14.0 allocs/op	14.0 allocs/op	0.0% (stable)

State

• Coverage, compatibility, and sampled performance are reported here from the same PR workflow.
• This is a delta report, not a release gate by itself. Required checks still decide merge safety.
• Performance is a smoke comparison, not a full benchmark lab run.
• Delta states use the same noise guards as the quality gate (percent + absolute + low-baseline checks), so report labels match merge-gate behavior.