Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed

- **A scalar instant answer carries its sample time in integer milliseconds, as
Loki does, not in seconds (#752).** `GET /loki/api/v1/query?query=1` (and
`1 + 1`, `2 * (3 + 4)`, `1 > bool 2`) was answered `[1791811168.123,"1"]`;
Loki 3.7.7 answers `[1791811168123,"1"]`, while `vector(1)` is in seconds on
both. The engine keeps the scalar time in milliseconds (`pkg/logql/engine.go`
`evalLiteral`), the encoder of `pkg/util/marshal` would write seconds, but the
HTTP answer is written by the query frontend: `pkg/querier/queryrange/prometheus.go`
`marshalScalar` builds the time with `model.TimeFromUnix(TimestampMs)`, which
reads seconds, so the millisecond count is printed as is, whereas
`marshalVector` uses `model.Time(TimestampMs)`. The earlier fix (#735) followed
`pkg/util/marshal` and stamped seconds. Every scalar shape the proxy answers
without a backend (a literal, a binary operation of literals, a comparison
with `bool`) now carries the evaluation time truncated to the millisecond;
vector samples are unchanged. `1 / 0` is left as it is: Loki 3.7.7 answers a
500 `failed to enqueue request` after about 40 s (its frontend cannot ship
the folded NaN to the querier) where the proxy answers a 200 scalar `NaN`; that
difference awaits an owner decision. `TestCompat_InstantResultTypeAlwaysSet` now compares the
sample time and value of every constant against Loki
(`semantics/scalar-sample-time-units`).

## [2.14.1] - 2026-10-12

### CI
Expand Down
69 changes: 69 additions & 0 deletions conformance/registry/cases/semantics/scalar-sample-time-units.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
id: semantics/scalar-sample-time-units
title: A scalar instant answer is stamped in integer milliseconds, like Loki (fixed)
proves:
- loki_api_v1_query
consumer: grafana-datasource
why: |
`/loki/api/v1/query?query=1&time=1791811168123456789` is answered by Loki
3.7.7 (measured on an isolated instance, the compose config) with
`{"resultType":"scalar","result":[1791811168123,"1"]}`: the sample time is the
count of milliseconds, not seconds. `1 + 1`, `2 * (3 + 4)` and
`1 > bool 2` are stamped the same way. `vector(1)` and `1 + vector(1)` are
stamped `1791811168.123`, in seconds.

The cause is the query frontend's own encoder, not the engine's. The engine
keeps the time as milliseconds (pkg/logql/engine.go evalLiteral:
`T: q.params.Start().UnixNano() / int64(time.Millisecond)`), and the encoder
of pkg/util/marshal (encodeScalar, `model.Time(v.T).String()`) would write
seconds, but the HTTP answer does not go through it: ResultToResponse
(pkg/querier/queryrange/marshal.go) turns the promql.Scalar into a
LokiPromResponse, and LokiPromResponse.marshalScalar
(pkg/querier/queryrange/prometheus.go) builds the time with
`model.TimeFromUnix(r.Samples[0].TimestampMs)`. TimeFromUnix takes seconds,
so the millisecond count is multiplied by 1000 and model.Time.String divides
it by 1000 again: the millisecond count is printed as if it were seconds.
marshalVector uses `model.Time(TimestampMs)` and is correct, which is why a
vector is in seconds.

The proxy stamped a scalar with seconds (`[1791811168.123,"1"]`), written
from the reading of pkg/util/marshal that the frontend does not follow (#735).
A client that reads the scalar time as seconds reads a date in the year
58000; one that reads it as Loki's wire format (the Grafana Loki datasource
decodes through loghttp.Scalar, which parses a number) now gets the same
number. The value and the resultType were already Loki's.

Not covered here: `1 / 0`, `0 / 0` and `-1 / 0`. Loki folds a division of
literals through MergeBinOp, which makes a division by zero NaN
(pkg/logql/syntax/ast.go mustNewBinOpExpr -> reduceBinOp -> MergeBinOp,
OpTypeDiv). Loki 3.7.7 with the compose config answers HTTP 500
`failed to enqueue request` after about 40 s for these queries (its frontend
retries the grpc send, which fails with `grpc: error while marshaling:
unsupported value: NaN`), while the proxy answers 200 with a scalar `NaN`
(stamped in milliseconds like the other scalars). Whether to follow that
failure is an owner decision, not made here.
request:
method: GET
path: /loki/api/v1/query
params:
query: '1'
time: '1791811168123456789'
headers:
X-Scope-OrgID: "0"
X-Loki-Response-Encoding-Flags: categorize-labels
compare:
mode: observed
fields: [data.result]
expect: |
[1791811168123,"1"] for a scalar; [1791811168.123,"1"] for vector(1).
gap:
status: fixed
fixed_by: "#752"
impact: grafana-datasource
visual: none
visual_reason: 'instant scalar literals only; no Explore range panel renders them, the fix is proven by the sample-time comparison against Loki in the e2e'
effort: S
area: 'internal/proxy evaluateConstantInstantQuery (metric_agg.go), binaryConstantResponse (binary_scalar.go)'
loki: 'pkg/logql/engine.go:700-702 evalLiteral; pkg/querier/queryrange/marshal.go:97-112 ResultToResponse; pkg/querier/queryrange/codec.go:1221-1227,1761-1772 DecodeResponse/toProtoScalar; pkg/querier/queryrange/prometheus.go:163-176 marshalScalar (model.TimeFromUnix) vs :90-102 marshalVector'
proxy: 'a scalar answer carries the evaluation time as integer milliseconds; vector() keeps seconds'
planned_test: 'TestCompat_InstantResultTypeAlwaysSet (test/e2e-compat, subtest constant sample time); TestInstantConstantsCarrySecondsTimestamps and TestBinaryScalarSubtreesAndIEEEValues (internal/proxy)'
note: 'The e2e ran against an isolated Loki 3.7.7 and VictoriaLogs v1.52.0 with a proxy built from the branch.'
22 changes: 17 additions & 5 deletions conformance/registry/generated/wiring.json
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@
"test/e2e-compat/metadata_scan_admission_test.go"
],
"loki_api_v1_query": [
"internal/proxy/binary_matching_test.go",
"internal/proxy/instant_result_type_test.go",
"internal/proxy/parenthesized_range_test.go",
"internal/proxy/series_limit_routes_test.go",
Expand Down Expand Up @@ -477,6 +478,11 @@
"internal/proxy/plain_stream_labels_test.go",
"test/e2e-compat/plain_name_after_parser_test.go"
],
"semantics/scalar-sample-time-units": [
"internal/proxy/binary_matching_test.go",
"internal/proxy/instant_result_type_test.go",
"test/e2e-compat/instant_result_type_test.go"
],
"semantics/sliding-window-recurrence-exact": [
"internal/proxy/sliding_window_stats_test.go",
"test/e2e-compat/heavy_query_protection_test.go"
Expand Down Expand Up @@ -569,7 +575,7 @@
"test/e2e-compat/unwrap_ungrouped_sum_test.go"
]
},
"registry_items": 300,
"registry_items": 301,
"routes_without_registry_item": [],
"test_to_items": {
"cmd/proxy/main_test.go": [
Expand Down Expand Up @@ -620,6 +626,10 @@
"operator-configurable-limits",
"semantics/backend-aborted-response"
],
"internal/proxy/binary_matching_test.go": [
"loki_api_v1_query",
"semantics/scalar-sample-time-units"
],
"internal/proxy/compat_cache_freshness_test.go": [
"semantics/metadata-answers-include-last-24h-like-loki"
],
Expand Down Expand Up @@ -662,7 +672,8 @@
],
"internal/proxy/instant_result_type_test.go": [
"loki_api_v1_query",
"semantics/instant-result-type-missing"
"semantics/instant-result-type-missing",
"semantics/scalar-sample-time-units"
],
"internal/proxy/instant_unwrap_sum_test.go": [
"profiles/sanitized-json-key-grouping"
Expand Down Expand Up @@ -999,7 +1010,8 @@
],
"test/e2e-compat/instant_result_type_test.go": [
"loki_api_v1_query",
"semantics/instant-result-type-missing"
"semantics/instant-result-type-missing",
"semantics/scalar-sample-time-units"
],
"test/e2e-compat/json_filter_pushdown_test.go": [
"parser-error-and-label-collision",
Expand Down Expand Up @@ -1132,7 +1144,7 @@
"profiles/structured-metadata-keys-per-profile"
]
},
"tests_with_markers": 105,
"tests_with_markers": 106,
"unknown_ids": {},
"unwired_items": [
"api_prom_label",
Expand Down Expand Up @@ -1238,5 +1250,5 @@
"versions/capabilities-floor-vs-matrix",
"versions/rejected-stats-query-wording-vl-1.47"
],
"wired_items": 109
"wired_items": 110
}
1 change: 1 addition & 0 deletions conformance/reports/parity-gaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,4 +115,5 @@ Last run: 2026-10-05T20:16:15Z on `fcaccfcc`, window [1791220800, 1791224400].
| `severity/otlp-severity-before-vl-1.50` — OTLP rows stored by VictoriaLogs before v1.50 (one `severity` field, no severity_number) get detected_level "Unspecified" / "Info2" (fixed) | #666 | 0 | 0 |
| `versions/rejected-stats-query-wording-vl-1.47` — The proxy did not recognise VictoriaLogs v1.47 parse errors, so a rejected query was not Loki's 400 (fixed) | #659 | 0 | 0 |
| `semantics/instant-result-type-missing` — An instant query answers its failure status and a resultType, like Loki (fixed) | #735 | 1 | 2 |
| `semantics/scalar-sample-time-units` — A scalar instant answer is stamped in integer milliseconds, like Loki (fixed) | #752 | 0 | 0 |
| `versions/capabilities-floor-vs-matrix` — logsql.Capabilities treated VictoriaLogs below v1.40 as unsupported while the matrix claimed v1.30 (fixed) | #659 | 0 | 0 |
3 changes: 2 additions & 1 deletion internal/proxy/binary_matching_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,7 @@ func TestBinaryInstantUsesSingleDefaultTimestamp(t *testing.T) {
}
}

// conformance: semantics/scalar-sample-time-units, loki_api_v1_query
func TestBinaryScalarSubtreesAndIEEEValues(t *testing.T) {
p := newTestProxy(t, "http://127.0.0.1:1")
for _, tc := range []struct{ query, want string }{
Expand Down Expand Up @@ -253,7 +254,7 @@ func TestBinaryScalarSubtreesAndIEEEValues(t *testing.T) {
}
if resultType == "vector" {
var point []any
if response.Data.ResultType != "scalar" || json.Unmarshal(response.Data.Result, &point) != nil || !reflect.DeepEqual(point, []any{float64(1700000000), tc.want}) {
if response.Data.ResultType != "scalar" || json.Unmarshal(response.Data.Result, &point) != nil || !reflect.DeepEqual(point, []any{float64(1700000000000), tc.want}) {
t.Fatalf("scalar response %s", w.Body)
}
} else {
Expand Down
8 changes: 5 additions & 3 deletions internal/proxy/binary_scalar.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"net/http"
"strconv"
"strings"
"time"

logqlpkg "github.com/ReliablyObserve/Loki-VL-proxy/internal/logql"
)
Expand Down Expand Up @@ -85,9 +86,10 @@ func binaryConstantResponse(r *http.Request, value float64, resultType string) (
if !ok {
return nil, fmt.Errorf("invalid scalar evaluation time")
}
// Loki encodes a scalar with model.Time(ms).String(): seconds with
// millisecond precision, the same unit as vector and matrix samples.
return json.Marshal(map[string]any{"status": "success", "data": map[string]any{"resultType": "scalar", "result": []any{float64(ts/1e6) / 1e3, encoded}}})
// Loki's frontend marshalScalar passes the millisecond timestamp to
// model.TimeFromUnix (seconds), so a scalar is stamped in integer
// milliseconds, unlike the vector and matrix samples.
return json.Marshal(map[string]any{"status": "success", "data": map[string]any{"resultType": "scalar", "result": []any{ts / int64(time.Millisecond), encoded}}})
}
start, startOK := parseLokiTimeToUnixNano(r.FormValue("start"))
end, endOK := parseLokiTimeToUnixNano(r.FormValue("end"))
Expand Down
65 changes: 53 additions & 12 deletions internal/proxy/instant_result_type_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,11 @@ import (
// vector, whatever the expression (queryrange/codec.go NewEmptyResponse,
// LokiInstantRequest).
//
// The sample time of a vector or scalar is seconds (model.Time, millisecond
// precision), not nanoseconds (pkg/util/marshal/query.go NewVector, NewScalar).
// The sample time of a vector is seconds with millisecond precision, not
// nanoseconds (queryrange/prometheus.go marshalVector, model.Time(ms)); the
// time of a scalar is integer milliseconds, because marshalScalar hands the
// millisecond timestamp to model.TimeFromUnix, which reads seconds
// (queryrange/prometheus.go marshalScalar).

type instantBackendMode string

Expand Down Expand Up @@ -277,18 +280,18 @@ func requireInstantAnswer(t *testing.T, label string, a instantAnswer, want stri
}

// requireInstantResultShape checks the result against loghttp's types: a vector
// sample is {metric, value:[seconds, "v"]}, a scalar is [seconds, "v"].
// sample is {metric, value:[seconds, "v"]}, a scalar is [milliseconds, "v"].
func requireInstantResultShape(t *testing.T, label string, a instantAnswer, want string) {
t.Helper()
checkPoint := func(raw json.RawMessage) {
checkPoint := func(raw json.RawMessage, lo, hi float64, unit string) {
var point []any
if err := json.Unmarshal(raw, &point); err != nil || len(point) != 2 {
t.Errorf("%s: sample point %s is not [time, value]\n%s", label, raw, a.body)
return
}
ts, ok := point[0].(float64)
if !ok || ts < 1e9 || ts > 1e11 {
t.Errorf("%s: sample time %v is not seconds since the epoch\n%s", label, point[0], a.body)
if !ok || ts < lo || ts > hi {
t.Errorf("%s: sample time %v is not %s since the epoch\n%s", label, point[0], unit, a.body)
}
if _, ok := point[1].(string); !ok {
t.Errorf("%s: sample value %v is not a string\n%s", label, point[1], a.body)
Expand All @@ -300,7 +303,7 @@ func requireInstantResultShape(t *testing.T, label string, a instantAnswer, want
Result json.RawMessage `json:"result"`
}
_ = json.Unmarshal(a.raw["data"], &data)
checkPoint(data.Result)
checkPoint(data.Result, 1e12, 1e14, "milliseconds")
case "vector":
for _, item := range a.resp.Data.Result {
var sample struct {
Expand All @@ -312,7 +315,7 @@ func requireInstantResultShape(t *testing.T, label string, a instantAnswer, want
t.Errorf("%s: %s is not a vector sample {metric, value}\n%s", label, item, a.body)
continue
}
checkPoint(sample.Value)
checkPoint(sample.Value, 1e9, 1e11, "seconds")
}
}
}
Expand Down Expand Up @@ -370,9 +373,11 @@ func TestInstantQueryPostProcessedBackendFailureKeepsStatus(t *testing.T) {
}
}

// Sample times of vector() are seconds, like every other sample, and the
// scalar of a literal follows pkg/util/marshal NewScalar.
// conformance: semantics/instant-result-type-missing, loki_api_v1_query
// Sample times of vector() are seconds, like every other vector sample. The
// time of a scalar is integer milliseconds, as Loki's query frontend writes it
// (queryrange/prometheus.go marshalScalar passes the millisecond timestamp to
// model.TimeFromUnix), which a live Loki 3.7.7 answers as [1791811168123,"1"].
// conformance: semantics/instant-result-type-missing, semantics/scalar-sample-time-units, loki_api_v1_query
func TestInstantConstantsCarrySecondsTimestamps(t *testing.T) {
p := newTestProxy(t, instantBackend(t, instantBackendData).URL)
for _, tc := range []struct{ query, time string }{
Expand All @@ -390,6 +395,42 @@ func TestInstantConstantsCarrySecondsTimestamps(t *testing.T) {
}
requireInstantAnswer(t, tc.query+" at "+tc.time, a, want, false)
}
// the exact sample time, as the JSON text a client reads
for _, tc := range []struct{ query, time, want string }{
{`1`, "1704067200", `[1704067200000,"1"]`},
{`1`, "1704067200.5", `[1704067200500,"1"]`},
{`1`, "1704067200123456789", `[1704067200123,"1"]`},
{`1`, "2024-01-01T00:00:00.789Z", `[1704067200789,"1"]`},
{`-1`, "1704067200", `[1704067200000,"-1"]`},
{`1 + 1`, "1704067200123456789", `[1704067200123,"2"]`},
{`2 * (3 + 4)`, "1704067200", `[1704067200000,"14"]`},
{`1 > bool 2`, "1704067200", `[1704067200000,"0"]`},
} {
a := askInstant(t, p, tc.query, "0", url.Values{"time": {tc.time}})
var data struct {
ResultType string `json:"resultType"`
Result json.RawMessage `json:"result"`
}
_ = json.Unmarshal(a.raw["data"], &data)
if data.ResultType != "scalar" || string(data.Result) != tc.want {
t.Errorf("%s at %s: scalar %s %s, want scalar %s", tc.query, tc.time, data.ResultType, data.Result, tc.want)
}
}
for _, tc := range []struct{ query, time, want string }{
{`vector(1)`, "1704067200123456789", `[1704067200.123,"1"]`},
{`vector(1) + 1`, "1704067200", `[1704067200,"2"]`},
} {
a := askInstant(t, p, tc.query, "0", url.Values{"time": {tc.time}})
var data struct {
Result []struct {
Value json.RawMessage `json:"value"`
} `json:"result"`
}
_ = json.Unmarshal(a.raw["data"], &data)
if len(data.Result) != 1 || string(data.Result[0].Value) != tc.want {
t.Errorf("%s at %s: vector %s, want value %s", tc.query, tc.time, a.body, tc.want)
}
}
// without a time, the evaluation time is now
req := httptest.NewRequest("GET", "/loki/api/v1/query?query="+url.QueryEscape(`1`), nil)
req.Header.Set("X-Scope-OrgID", "0")
Expand All @@ -404,7 +445,7 @@ func TestInstantConstantsCarrySecondsTimestamps(t *testing.T) {
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil || got.Data.ResultType != "scalar" || len(got.Data.Result) != 2 {
t.Fatalf("scalar literal without time: HTTP %d %s", rec.Code, rec.Body.String())
}
if ts, _ := got.Data.Result[0].(float64); time.Since(time.Unix(int64(ts), 0)) > time.Minute {
if ts, _ := got.Data.Result[0].(float64); time.Since(time.UnixMilli(int64(ts))) > time.Minute {
t.Fatalf("scalar literal without time is stamped %v, want now", got.Data.Result[0])
}
}
Expand Down
22 changes: 16 additions & 6 deletions internal/proxy/metric_agg.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ func unwrapEmptySumWithout(query string) (logqlpkg.Expr, int) {
// evaluateConstantInstantQuery answers an instant query of constants, which has
// no data behind it. vector(n), and a binary operation of two of them, is a
// vector of one sample with no labels (Loki pkg/logql/engine.go evalVector); a
// literal, or a binary operation of literals, is a scalar (evalLiteral).
// literal, or a binary operation of literals, is a scalar (evalLiteral), whose
// sample time is integer milliseconds (see constantInstantMillis).
func evaluateConstantInstantQuery(expr, timeParam string) ([]byte, bool) {
parsed, err := logqlpkg.Parse(strings.TrimSpace(expr))
if err != nil {
Expand All @@ -63,17 +64,26 @@ func evaluateConstantInstantQuery(expr, timeParam string) ([]byte, bool) {
"status": "success",
"data": map[string]interface{}{
"resultType": "scalar",
"result": []interface{}{constantInstantSeconds(timeParam), strconv.FormatFloat(value, 'f', -1, 64)},
"result": []interface{}{constantInstantMillis(timeParam), strconv.FormatFloat(value, 'f', -1, 64)},
},
})
return result, true
}

// constantInstantSeconds is the evaluation time as a sample time: seconds with
// millisecond precision (Loki pkg/util/marshal NewVector and NewScalar), not
// the nanoseconds the request carries.
// constantInstantSeconds is the evaluation time as a vector sample time: seconds
// with millisecond precision (Loki pkg/querier/queryrange/prometheus.go
// marshalVector, model.Time(ms)), not the nanoseconds the request carries.
func constantInstantSeconds(timeParam string) float64 {
return float64(parseInstantVectorTime(timeParam)/int64(time.Millisecond)) / 1e3
return float64(constantInstantMillis(timeParam)) / 1e3
}

// constantInstantMillis is the evaluation time as a scalar sample time. Loki
// answers it as integer milliseconds, not seconds: marshalScalar
// (pkg/querier/queryrange/prometheus.go) hands the millisecond timestamp of
// promql.Scalar.T (pkg/logql/engine.go evalLiteral) to model.TimeFromUnix,
// which reads seconds, so model.Time.String prints the millisecond count.
func constantInstantMillis(timeParam string) int64 {
return parseInstantVectorTime(timeParam) / int64(time.Millisecond)
}

func buildConstantVectorResponse(timeParam string, value float64) []byte {
Expand Down
Loading
Loading