Skip to content

chore: sync upstream GreptimeTeam/promql-parser v0.11.1 - #10

Merged
milindsrivastava1997 merged 12 commits into
mainfrom
chore/sync-upstream-v0.11.1
Nov 2, 2026
Merged

milindsrivastava1997 merged 12 commits into
mainfrom
chore/sync-upstream-v0.11.1

Conversation

@milindsrivastava1997

Copy link
Copy Markdown
Collaborator

Merges the 11 upstream commits since our v0.10.0 sync.

  • Adds integral and start_timestamp and makes first_over_time non-experimental, so the weekly Function Comparison run (red since at least Sep 13) passes: 0 missing, 0 differences. The 3 MetricsQL extras remain as warnings.
  • Includes the non-recursive lexer fix (stack overflow).
  • Our MetricsQL extensions (distinct_over_time, entropy_over_time, l2_over_time) and their tests are kept.

Conflicts: Cargo.toml, scripts/compare_functions.py and function-comparison.yml were older copies of upstream, so upstream wins. Note that the workflow now also runs on PRs and pushes to main, and a failed scheduled run opens an issue.

Merge with a merge commit (not squash) so upstream history stays shared for the next sync.

Tested: cargo fmt --check, cargo clippy --all-targets -D warnings, cargo test, scripts/compare_functions.py. ASAPPlanner's asap-frontend-promql and asap-frontend-metricsql tests also pass against this branch via a local patch. Follow-up: bump the promql-parser rev in ASAPPlanner crates/frontend-promql/Cargo.toml after merge.

🤖 Generated with Claude Code

sunng87 and others added 12 commits May 25, 2026 15:11
* feat: add function sync script with upstream

* fix: correct variadic -1 argument count in error message

* chore: lint fix

* chore: header

* feat: sync functions, remove holt_winters, add start/end/range/step

* chore: format

* feat: add vardict test s

* refactor: make arg count check easier to read

* fix: when variadict is -1 it means the last argument is 0 or more
Add Prometheus min_of and max_of functions
* feat(function): Add custom function registration API

Add register_extra_functions() and clear_extra_functions() to allow users
to extend the parser with custom functions not in upstream Prometheus.

This is inspired by Thanos's XFunctions pattern
(thanos-io/promql-engine/execution/parse/functions.go) where custom
functions like xrate/xdelta/xincrease are registered alongside built-in
Prometheus functions.

The implementation uses a RwLock-protected HashMap that get_function()
falls back to when a function name is not found in the built-in
FUNCTIONS map. This allows:
- Registering functions once at startup (zero read-path contention)
- Calling register_extra_functions() multiple times (additive)
- Full argument type validation at parse time (same as built-ins)
- Rejecting registrations that conflict with built-in function names

Closes GreptimeTeam#144

* Apply suggestion from @sunng87

* style: Apply rustfmt formatting

---------

Co-authored-by: Ning Sun <classicning@gmail.com>
…parse (#151)

lrpar 0.13 generates a `parse()` that calls `_reconstitute` on every
invocation, deserializing the grammar and LR state table (~47 KiB of
tables, ~57 KiB of allocation per call). lrpar 0.14.2 caches them in a
`OnceLock` (softdevteam/grmtools#631).

On a 14-core M-series, `parse("sum(up)")` drops from ~25 us to ~7 us and
steady-state allocation from ~61 KB to ~4.6 KB per parse; 16 threads
parsing concurrently go from 438k to 1.48M parses/s.

Also:
- switch the deprecated `ct_token_map` to `CTTokenMapBuilder`
- enable serde's `derive` feature explicitly: the `ser` feature relied on
  grmtools 0.13 enabling it transitively, which 0.14 no longer does


Claude-Session: https://claude.ai/code/session_01VrH837ZJiH2gpReQPza32S

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* ci: create issue when ci job finds function mismatches

* ci: fix failure check

* ci: only create issue on confirmed function mismatch

Distinguish the mismatch exit status (1) from operational errors (2)
in compare_functions.py, and gate the issue-creation step on the
mismatch exit code instead of a generic failure(). This prevents a
false "out of sync" issue (which would then suppress later genuine
reports via dedup) when the comparison fails for operational reasons
such as a network error fetching Prometheus functions.go.

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
`<Lexer as Iterator>::next` tail-calls itself for every state transition
that does not yield a lexeme, and rustc does not guarantee tail-call
elimination. Two of those transitions form cycles that can repeat without
producing a lexeme, so the recursion depth grew with the length of the
input instead of staying constant:

- `String` -> `Escape` -> `String`, one frame pair per escape sequence
  within a single quoted string;
- `Start` -> `LineComment`/`Space` -> `Start`, one frame pair per comment
  line or whitespace run.

A stack overflow aborts the process and cannot be caught, so a single long
query takes down the whole program rather than returning a parse error.
This was reached in production by a 0.8 MB PromQL query whose label matcher
held a large regex alternation with 29,416 escape sequences: it exhausted
an 8 MB stack inside one `next` call.

Rewrite the function as a loop. The state machine, the order of the match
arms and the values returned are unchanged, so this only affects how much
stack the iteration uses. With the loop, the same 0.8 MB query parses on a
1 MB stack.

Add a regression test that lexes both cycles on a deliberately small
(256 KB) stack, so a return to per-item recursion is caught rather than
hidden by a large default stack.
Signed-off-by: evenyag <realevenyag@gmail.com>
Brings integral and start_timestamp, makes first_over_time
non-experimental (fixes the weekly Function Comparison run), and the
non-recursive lexer fix. Keeps our MetricsQL extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milindsrivastava1997
milindsrivastava1997 merged commit 1d1562e into main Nov 2, 2026
3 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the chore/sync-upstream-v0.11.1 branch November 2, 2026 17:28
milindsrivastava1997 added a commit to ProjectASAP/ASAPPlanner that referenced this pull request Nov 2, 2026
…nc (#1184)

ProjectASAP/promql-parser#10 merged upstream GreptimeTeam v0.11.1:
integral and start_timestamp parse, first_over_time is no longer
experimental, and the lexer no longer recurses (stack overflow fix).
Newly parsed functions stay unsupported until the lowerer maps them.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants