fix: reject group(), label_join() and other names Loki does not have, and accept keywords in any case, like Loki

CHANGELOG.md

@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- **`group()`, `label_join()` and every other name Loki does not have are a 400
+  parse error, and keywords are accepted in any case (#748).** Loki 3.7.7 has
+  no `group` aggregation and no `label_join`: its lexer makes a function token
+  only of the names in `functionTokens` (the range operations, `sum` `avg`
+  `max` `min` `count` `stddev` `stdvar` `bottomk` `topk` `sort` `sort_desc`
+  `approx_topk`, `vector`, `label_replace` and the unwrap conversions), so any
+  other name is an `IDENTIFIER` the grammar cannot place and the query fails
+  with `parse error at line 1, col N: syntax error: unexpected IDENTIFIER`, N
+  being the column of the name. The proxy kept any unknown `name(...)` as an
+  opaque call: `group(count_over_time({...}[5m]))` answered a series valued 1
+  per stream on `query_range` and a line count on `query`, `label_join(...)`
+  joined labels in the response, `abs` and `clamp_min` were answered as well,
+  and `time()` and `count_values()` were a 400 with the proxy's own text
+  (`unexpected )`, `unexpected STRING`). Both endpoints now answer Loki's
+  status and text, byte for byte, with the column of the name wherever it
+  stands (`sum(group(...))` is col 5, and inside a `label_replace` argument it
+  is relative to the whole query); only `label_replace`, `vector` and
+  `approx_topk` stay opaque calls. Loki's lexer also lower-cases a word before
+  it looks it up, so `TOPK(3, ...)`, `SUM BY (a) (...)`, `RATE(...)`, `OFFSET`,
+  `UNWRAP`, `BOOL`, `ON` and `GROUP_LEFT` are the keywords they spell, while a
+  label called `Rate` or `Count` stays a label. The proxy matched the
+  lower-case spelling only: `TOPK(...)` fell through the opaque path and
+  answered a `streams` result, and `SUM BY (app) (...)` was a parse error. The
+  scanner now reads keywords in any case, and the `query` parameter is written
+  with lower-case keywords before a handler reads it. The group-normalisation
+  and `label_join` post-processing of the response, now unreachable, is
+  removed.
+
 ## [2.11.2] - 2026-10-12
 
 ### CI

conformance/registry/cases/semantics/proxy-accepts-operators-not-in-loki.yaml

@@ -0,0 +1,75 @@
+id: semantics/proxy-accepts-operators-not-in-loki
+title: group(), label_join() and other names Loki does not have are answered; uppercase keywords are not (fixed)
+proves:
+  - loki_api_v1_query_range
+  - loki_api_v1_query
+consumer: api
+why: |
+  Loki 3.7.7 has no `group` aggregation and no `label_join`: its lexer makes a
+  function token only of the names in pkg/logql/syntax/lex.go functionTokens
+  (the range operations, sum avg max min count stddev stdvar bottomk topk sort
+  sort_desc approx_topk, vector, label_replace and the unwrap conversions), so
+  any other name is an IDENTIFIER the grammar cannot place (syntax.y vectorOp,
+  rangeOp, labelReplaceExpr, vectorExpr). `group(count_over_time({...}[5m]))`
+  and `label_join(...)` are HTTP 400 `parse error at line 1, col 1: syntax
+  error: unexpected IDENTIFIER`, and so is every PromQL name Loki lacks (abs,
+  clamp, round, ceil, sqrt, ln, scalar, time, quantile, count_values, ...),
+  with the column of the name wherever it stands (`sum(group(...))` is col 5).
+  The proxy's parser kept any unknown `name(...)` as an opaque metric call
+  (OpaqueMetricExpr) for the translator, which implemented group (every value
+  1) and label_join as proxy-only extensions: a client got an answer where
+  Loki answers 400, and a client written against the proxy broke on Loki.
+
+  The other direction: Loki's lexer lower-cases a word before it looks it up
+  (lex.go Lex: tokenTextLower, tokenNext), so `TOPK(3, X)`, `SUM BY (a) (X)`,
+  `RATE(...)`, `OFFSET`, `UNWRAP`, `AND`, `ON`, `GROUP_LEFT`, `BOOL` and the
+  parser keywords are the keywords they spell. A function name counts only when
+  a call follows (lex.go isFunction: "(" or by/without and "("), so a label
+  called Rate or Count stays a label. The proxy matched the lower-case
+  spelling only: `TOPK(...)` fell through the opaque path and answered a
+  `streams` result, `SUM BY (app) (...)` was a parse error.
+
+  Fixed in internal/logql: the scanner reads a keyword in any case (the
+  keyword and function-name sets the dotted-name check already uses), the parser
+  keeps opaque only label_replace, vector and approx_topk and reports any other
+  name as Loki does (the column of the name, relative to the whole query also
+  inside a label_replace argument), and LowercaseKeywords writes the keywords
+  in lower case in the `query` parameter before a handler reads it, so the
+  translator and every handler see the spelling they match. The group
+  post-processing and label_join code (translator markers, proxy response
+  rewriting) became unreachable and was removed.
+
+  Not changed (separate causes, still open): a name Loki rejects with a
+  different grammar error (`sum({app="a"})` col 14 unexpected ")", `avg_over_time`
+  without unwrap "invalid aggregation", `approx_topk by (...)` "grouping not
+  allowed"); keyword-named labels in a selector (`{by="x"}`), whose proxy text
+  is its own; and leading white space, which the proxy trims before it
+  reports a column.
+request:
+  method: GET
+  path: /loki/api/v1/query_range
+  params:
+    query: 'group(count_over_time({app="api-gateway"}[5m]))'
+    start: 'now-1h'
+    end: 'now'
+    step: '60'
+  headers:
+    X-Scope-OrgID: "0"
+compare:
+  mode: observed
+  fields: [status, error_text]
+  expect: |
+    Loki's 400 `parse error at line 1, col 1: syntax error: unexpected
+    IDENTIFIER`, byte for byte; `TOPK(3, RATE({...}[5m]))` answers like the
+    lower-case form.
+gap:
+  status: fixed
+  fixed_by: "#748"
+  impact: api-only
+  effort: S
+  area: 'internal/logql scanner (keyword case) + parser (opaque calls) + semantic (label_replace argument position); internal/proxy lowercaseQueryKeywords'
+  loki: 'pkg/logql/syntax/lex.go:95-137 functionTokens, :226-252 Lex lower-casing, :398 isFunction; syntax.y:477-490 vectorOp, :492-508 rangeOp'
+  proxy: '400 with Loki''s text; keywords accepted in any case'
+  planned_test: 'TestCompat_OperatorsNotInLokiRejected and TestCompat_UppercaseKeywordsMatchLoki (test/e2e-compat, group semantics); TestValidateLogQL_NamesLokiDoesNotKnowAreRejected, TestValidateLogQL_KeywordsAreCaseInsensitive and TestLowercaseKeywords (internal/logql); TestUnknownLogQLNamesRejectedLikeLoki and TestUppercaseKeywordsSendTheLowercaseBackendRequests (internal/proxy)'
+  note: 'Messages from Loki v3.7.7 syntax.ParseExpr, run on the same queries. The e2e vs Loki is written, not yet run.'
+  clusters: []

conformance/registry/generated/wiring.json

@@ -456,6 +456,11 @@
       "internal/proxy/plain_stream_labels_test.go",
       "test/e2e-compat/plain_name_after_parser_test.go"
     ],
+    "semantics/proxy-accepts-operators-not-in-loki": [
+      "internal/logql/loki_unknown_names_test.go",
+      "internal/proxy/unknown_logql_names_test.go",
+      "test/e2e-compat/operators_not_in_loki_test.go"
+    ],
     "semantics/sliding-window-recurrence-exact": [
       "internal/proxy/sliding_window_stats_test.go",
       "test/e2e-compat/heavy_query_protection_test.go"
@@ -541,7 +546,7 @@
       "test/e2e-compat/unwrap_ungrouped_sum_test.go"
     ]
   },
-  "registry_items": 299,
+  "registry_items": 300,
   "routes_without_registry_item": [],
   "test_to_items": {
     "cmd/proxy/main_test.go": [
@@ -559,6 +564,9 @@
       "semantics/line-filter-or-alternatives",
       "semantics/parenthesized-log-range"
     ],
+    "internal/logql/loki_unknown_names_test.go": [
+      "semantics/proxy-accepts-operators-not-in-loki"
+    ],
     "internal/logql/parser_captures_test.go": [
       "parsed-label-series-identity"
     ],
@@ -827,6 +835,9 @@
       "series-limits-and-partial-results",
       "window-bounds-and-step-alignment"
     ],
+    "internal/proxy/unknown_logql_names_test.go": [
+      "semantics/proxy-accepts-operators-not-in-loki"
+    ],
     "internal/proxy/unwrap_bytes_gate_property_test.go": [
       "semantics/unwrap-gate-parsefloat-divergences",
       "semantics/unwrap-sample-validity"
@@ -982,6 +993,9 @@
       "loki_api_v1_label_name_values",
       "loki_api_v1_labels"
     ],
+    "test/e2e-compat/operators_not_in_loki_test.go": [
+      "semantics/proxy-accepts-operators-not-in-loki"
+    ],
     "test/e2e-compat/parenthesized_range_or_filters_test.go": [
       "loki_api_v1_query",
       "loki_api_v1_query_range",
@@ -1064,7 +1078,7 @@
       "profiles/structured-metadata-keys-per-profile"
     ]
   },
-  "tests_with_markers": 96,
+  "tests_with_markers": 99,
   "unknown_ids": {},
   "unwired_items": [
     "api_prom_label",
@@ -1172,5 +1186,5 @@
     "versions/capabilities-floor-vs-matrix",
     "versions/rejected-stats-query-wording-vl-1.47"
   ],
-  "wired_items": 106
+  "wired_items": 107
 }

conformance/reports/parity-gaps.md

@@ -114,4 +114,5 @@ Last run: 2026-10-05T20:16:15Z on `fcaccfcc`, window [1791220800, 1791224400].
 | `semantics/unwrap-rate-ungrouped-sum-per-stream` — An ungrouped sum over rate with unwrap is one series on query_range, like Loki (fixed) | #716 | 0 | 0 |
 | `severity/otlp-severity-before-vl-1.50` — OTLP rows stored by VictoriaLogs before v1.50 (one `severity` field, no severity_number) get detected_level "Unspecified" / "Info2" (fixed) | #666 | 0 | 0 |
 | `versions/rejected-stats-query-wording-vl-1.47` — The proxy did not recognise VictoriaLogs v1.47 parse errors, so a rejected query was not Loki's 400 (fixed) | #659 | 0 | 0 |
+| `semantics/proxy-accepts-operators-not-in-loki` — group(), label_join() and other names Loki does not have are answered; uppercase keywords are not (fixed) | #748 | 0 | 0 |
 | `versions/capabilities-floor-vs-matrix` — logsql.Capabilities treated VictoriaLogs below v1.40 as unsupported while the matrix claimed v1.30 (fixed) | #659 | 0 | 0 |

docs/KNOWN_ISSUES.md

@@ -102,8 +102,7 @@ These are not current open issues in this codebase:
 - route-aware proxy, cache, and upstream request telemetry
 - prefixed app metrics under `loki_vl_proxy_*` with CI guard coverage
 - `label_replace()` — fully implemented in translator with proxy-side post-processing (v1.21.0)
-- `label_join()` — fully implemented in translator with proxy-side post-processing (v1.21.0)
-- `group()` — implemented: inner metric translated normally, proxy normalises all matrix values to `1` (v1.21.0)
+- `label_join()` and `group()` — not in Loki 3.7.7's LogQL; both are a 400 parse error like Loki's (they were proxy-only extensions from v1.21.0)
 - bare label matcher malformed VL output — queries like `app="value"` (missing braces) now return a descriptive HTTP 400 instead of silently producing double-quoted VL syntax (v1.20.0)
 - `detected_level` on log responses — derived per row from stored fields and the line with Loki's rules, including `unknown`, normalised values and structured metadata placement under `categorize-labels` (Unreleased; replaces the v1.20.0 JSON/logfmt inference)
 - circuit breaker sliding window — failure counting uses a 30-second sliding window; sporadic slow-query resets no longer open the breaker (v1.18.0)

docs/logql-parser.md

@@ -46,7 +46,7 @@ flowchart TD
         LQ["*LogQuery\n(stream + pipeline)"]
         RA["*RangeAggregation\n(rate, max_over_time…)"]
         BE["*BinOpExpr\n(left op right)"]
-        OM["*OpaqueMetricExpr\n(label_replace, label_join…)"]
+        OM["*OpaqueMetricExpr\n(label_replace, vector…)"]
         VA["*VectorAggregation\n(sum by, topk…)"]
     end
 
@@ -128,7 +128,7 @@ Expr (interface)
 ├── *VectorAggregation       sum by (label) (rate(...)) / topk(5, ...)
 ├── *BinOpExpr               left op right, optional VectorMatching
 ├── *LiteralExpr             scalar 3.14
-└── *OpaqueMetricExpr        label_replace(...), label_join(...) — raw pass-through
+└── *OpaqueMetricExpr        label_replace(...), vector(...) — raw pass-through
 ```
 
 ### Opaque pass-through nodes
@@ -137,7 +137,7 @@ Two node types capture content without parsing it:
 
 - **`LabelFilterStage`** / **`LabelFormatStage`** — the inner expression grammar for `| level > 1` or `| label_format dst=src` is complex and context-dependent. These stages capture the raw text after the keyword so the translator sees exactly what Loki would see.
 
-- **`OpaqueMetricExpr`** — metric-level functions that aren't yet expressible in the AST (e.g. `label_replace`, `label_join`). The parser captures the full raw text including balanced parentheses and returns it verbatim so VictoriaLogs receives the original expression unchanged.
+- **`OpaqueMetricExpr`** — metric-level functions that aren't yet expressible in the AST (`label_replace`, `vector`, `approx_topk`; any other name is a Loki parse error). The parser captures the full raw text including balanced parentheses and returns it verbatim so VictoriaLogs receives the original expression unchanged.
 
 ## Scanner
 
@@ -262,7 +262,7 @@ The AST-to-AST translator (`logql.Translate`) maps LogQL pipeline stages to `log
 | `*RangeAggregation` | any | `errFallthrough` | Metric path via string translator |
 | `*VectorAggregation` | any | `errFallthrough` | Metric path via string translator |
 | `*BinOpExpr` | any | `errFallthrough` | Binary metric path |
-| `*OpaqueMetricExpr` | any | raw pass-through | `label_replace`, `label_join` etc. |
+| `*OpaqueMetricExpr` | any | raw pass-through | `label_replace`, `vector`, `approx_topk` |
 
 `errFallthrough` is a package-private sentinel — callers route to `TranslateLogQLWithCapabilities` unchanged.
 

docs/roadmap.md

@@ -51,9 +51,9 @@ description: Shipped milestones and committed, already-decided work for loki-vl-
 - [x] `IsScalar` supports negative and scientific notation
 - [x] Circuit breaker half-open metrics fix
 - [x] Tenant map reload race condition fix
-- [x] `group()` outer aggregation — inner metric translated normally, proxy normalises all values to `1` (v1.21.x)
+- [x] ~~`group()` outer aggregation~~ — removed: Loki has no `group()`, the proxy answers its 400 parse error
 - [x] `label_replace()` — proxy post-processing via marker suffix; Prometheus no-match semantics (v1.21.x)
-- [x] `label_join()` — proxy post-processing via marker suffix; missing src labels skipped (v1.21.x)
+- [x] ~~`label_join()`~~ — removed: Loki has no `label_join()`, the proxy answers its 400 parse error
 - [x] `count_values()` — returns descriptive error (not translatable to VL; VL has no group-by-value primitive) (v1.21.x)
 - [x] Circuit breaker sliding-window failure counting — 30s window replaces consecutive-failure model; sporadic slow-query resets no longer open the breaker (v1.18.0)
 - [x] Bare label matcher error — `app="value"` (missing braces) returns HTTP 400 with descriptive Loki-style parse error instead of silently emitting malformed VL syntax (v1.20.0)

docs/translation-reference.md

@@ -148,11 +148,11 @@ The rejected rows form groups of their own, so every other group aggregates exac
 | `avg(rate({...}[5m])) by (x)` | same normalized-rate path grouped by `(x)` |
 | `topk(10, rate({...}[5m]))` | normalized-rate path with stream grouping; on range queries the proxy ranks series independently at each timestamp, so a range can return more than `k` distinct series, each with only its winning samples |
 
-Supported: `sum`, `avg`, `max`, `min`, `count`, `topk`, `bottomk`, `stddev`, `stdvar`, `sort`, `sort_desc`, `group`, `label_replace`, `label_join`.
+Supported: `sum`, `avg`, `max`, `min`, `count`, `topk`, `bottomk`, `stddev`, `stdvar`, `sort`, `sort_desc`, `label_replace`.
 
-`group()` returns `1` for every series that has data. The inner metric is translated normally for grouping/presence detection; the proxy normalises all result values to `1` in post-processing.
+Loki has no `group()` and no `label_join()` (nor the PromQL math and time functions): any name outside Loki's function tokens is a 400 `parse error at line 1, col N: syntax error: unexpected IDENTIFIER` on both query endpoints. Keywords are read in any case, as Loki's lexer does (`TOPK(3, ...)`, `SUM BY (a) (...)`).
 
-`label_replace(v, dst, repl, src, regex)` and `label_join(v, dst, sep, src1, ...)` are handled via a marker-suffix pattern: the inner expression is translated to VL, the spec is base64-encoded and embedded as a query suffix, the proxy strips the marker before sending to VL and applies the label operation to the matrix response.
+`label_replace(v, dst, repl, src, regex)` is handled via a marker-suffix pattern: the inner expression is translated to VL, the spec is base64-encoded and embedded as a query suffix, the proxy strips the marker before sending to VL and applies the label operation to the matrix response.
 
 `count_values(label, v)` is not translatable — VictoriaLogs has no primitive that groups by metric values. Queries using `count_values` return a descriptive error.
 
@@ -186,7 +186,6 @@ The following Loki semantics are implemented in the proxy to bridge gaps where V
 | Subquery `max_over_time(rate(...)[1h:5m])` | Not LogQL: Loki's grammar has no `[range:step]` form. Rejected with Loki's HTTP 400 parse error (`syntax error: unexpected RATE, expecting NUMBER or { or (`) before any backend call |
 | Range-vector metric windows (`*_over_time`, `rate`, `count_over_time`, `bytes_*`, `rate_counter`) | Proxy applies Loki-compatible sliding-window evaluation over step-aligned timestamps and emits matrix/vector responses |
 | `label_replace(expr, dst, repl, src, regex)` | Proxy post-processing: inner expr translated to VL, spec embedded as marker, applied to matrix response (Prometheus semantics: no-match leaves dst unchanged) |
-| `label_join(v, dst, sep, src1, ...)` | Proxy post-processing: same marker pattern as `label_replace`; missing src labels are skipped |
 
 ### Parser-Stage Metric Compatibility Path
 

internal/logql/ast.go

@@ -516,7 +516,7 @@ func (l *LiteralExpr) String() string {
 func (l *LiteralExpr) expr() {}
 
 // OpaqueMetricExpr holds the raw text of a metric-level function call that the
-// parser does not understand (e.g. label_replace, label_join). It passes
+// parser does not understand (label_replace, vector, approx_topk). It passes
 // through to VictoriaLogs without modification.
 type OpaqueMetricExpr struct {
 	Raw string

internal/logql/canonical.go

@@ -6,8 +6,10 @@ import "strings"
 // expression into the plain `{selector} pipeline [range]` form the translator
 // handles: parentheses around a log query or a log range, and a pipeline or
 // unwrap written after the range (`({a} | json)[5m]`, `rate({a}[5m] | json)`).
-// Every other query is returned unchanged, without parsing it.
+// Every other query is returned unchanged, without parsing it. It also writes
+// the keywords in lower case (LowercaseKeywords).
 func CanonicalizeLogRanges(query string) string {
+	query = LowercaseKeywords(query)
 	if !hasAlternativeLogRange(query) {
 		return query
 	}
@@ -26,6 +28,34 @@ func CanonicalizeLogRanges(query string) string {
 	return string(append(out, query[last:]...))
 }
 
+// LowercaseKeywords writes in lower case the words Loki's lexer reads without
+// regard to case (pkg/logql/syntax/lex.go lower-cases a word before it looks it
+// up): SUM, TOPK(3, ...), RATE(...), BY, OFFSET, UNWRAP and the like. Names,
+// strings and label names that are not keywords keep their spelling. The
+// translator and the handlers match the lower-case spelling.
+func LowercaseKeywords(query string) string {
+	upper := false
+	for i := 0; i < len(query) && !upper; i++ {
+		upper = query[i] >= 'A' && query[i] <= 'Z'
+	}
+	if !upper {
+		return query
+	}
+	var out []byte
+	last := 0
+	s := newScanner(query)
+	for tok := s.next(); tok.Typ != TokEOF; tok = s.next() {
+		if span := query[s.start:s.pos]; span != tok.Val && strings.EqualFold(span, tok.Val) {
+			out = append(append(out, query[last:s.start]...), tok.Val...)
+			last = s.pos
+		}
+	}
+	if out == nil {
+		return query
+	}
+	return string(append(out, query[last:]...))
+}
+
 // hasAlternativeLogRange is the cheap pre-check for CanonicalizeLogRanges. It
 // looks, outside string literals, for a `(` that opens a log query (it is
 // followed by `{` or `(` and does not follow an identifier or a number, which