Repository navigation
Freeze current-time functions into literals in DuckDB SQL - #413
Conversation
CURRENT_DATE, now() and CURRENT_TIMESTAMP are replaced with literals from a single instant per query, so every reference in one query sees the same time and engines that cannot fold these functions before pruning still skip files. On by default for DuckDB only, since other dialects use a server timezone sidemantic cannot see. The cached SQL stays unfrozen and is frozen per call.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 58fab67efe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return sql | ||
| from sidemantic.sql.frozen_time import freeze_current_time | ||
|
|
||
| return freeze_current_time(sql, dialect, now=self.clock()) |
There was a problem hiding this comment.
When a DuckDB connection sets TimeZone through the supported init_sql option or a prior SET TimeZone, this call omits the session timezone, so freeze_current_time derives CURRENT_DATE from the Python process timezone instead. If those zones have different calendar dates—such as a UTC process querying an America/Los_Angeles session near midnight—the generated literal is a day off and silently changes filter results; obtain the connection's effective timezone or avoid enabling date freezing unless it is known to match.
Useful? React with 👍 / 👎.
| def _is_single_query(tokens: list[Token]) -> bool: | ||
| if not tokens or tokens[0].token_type not in _QUERY_START_TYPES: | ||
| return False | ||
| semicolons = [i for i, token in enumerate(tokens) if token.token_type == TokenType.SEMICOLON] | ||
| return not semicolons or semicolons == [len(tokens) - 1] |
There was a problem hiding this comment.
For a CTE-prefixed DML statement such as WITH x AS (...) INSERT INTO history SELECT CURRENT_DATE, the first token is WITH and there is only one statement, so this check classifies it as a query and freezes the value despite the function's contract that DDL/DML remain unchanged. Non-strict SQL transports can pass such statements through, and rewritten SQL retained for later execution then persists a stale compile-time literal; inspect the parsed root expression rather than inferring statement kind from the first token.
Useful? React with 👍 / 👎.
…xed DML CURRENT_DATE depends on the connection's TimeZone setting, which init_sql or SET can change, so read it from the layer's own DuckDB session; the setting is session-scoped, so a cursor would see the default. When the timezone is unknown, leave CURRENT_DATE unchanged unless freezing was explicitly enabled. Classify statements by their parsed root so WITH ... INSERT and other CTE-prefixed DML keep their current-time calls.
The parity harness replaced the dates module's layer fixture with a fresh Rust layer, so its pinned clock was dropped and the frozen dates followed the wall clock, failing whenever the UTC date differed from the pinned one. Build the layer from the module fixture, which already uses the patched Rust SemanticLayer.
The Rust engine renders pushed-down filters as (timestamp) >= ...; the frozen literals were correct but the assertions matched only the Python engine's rendering.
1 participant
Compiled DuckDB SQL now replaces
CURRENT_DATE,today(),now(),CURRENT_TIMESTAMP,get_current_timestamp()andtransaction_timestamp()with literals taken from one instant per query. Every reference in a query sees the same time, and scans that cannot fold these functions before partition pruning (Iceberg on DuckDB 1.5) still skip files.CURRENT_DATEdepends on the session timezone, which sidemantic can match for in-process DuckDB but not for remote servers.SemanticLayer(freeze_current_time=...)overrides it.CURRENT_DATEarithmetic with day/week/hour/minute/second intervals folds into a DATE or TIMESTAMP literal.now()becomes an exact UTC TIMESTAMPTZ literal; day intervals on it stay as intervals so DST behavior is unchanged. Month/quarter/year arithmetic is never folded.t.current_date, and non-SELECT statements are left alone.compile()andrewrite_transport_sql(). Generator andsql()caches keep the unfrozen SQL and freeze on each call, so cached queries never reuse a stale time.layer.clock).