Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 32 additions & 4 deletions docs/docs/usage/dbt-pr-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,13 +209,33 @@ jobs:
| `target-base/compiled/…` | Compile the base ref into `target-base/` (below) | Equivalence proofs and column-breakage for modified models. |
| `target/catalog.json` | `dbt docs generate` | Complete warehouse columns for column-lineage breakage. |

Compiling the base (optional, for equivalence and column-breakage):
The action can build these for you with `dbt_prepare` (off by default):

```yaml
- uses: AltimateAI/altimate-code/github/review@main

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This snippet pins the action to @main, but the rest of the page pins @v0.8.5. Pin to the same release tag so users get a reviewed, reproducible action version.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 215:

<comment>This snippet pins the action to `@main`, but the rest of the page pins `@v0.8.5`. Pin to the same release tag so users get a reviewed, reproducible action version.</comment>

<file context>
@@ -209,12 +209,30 @@ jobs:
+The action can build these for you with `dbt_prepare` (off by default):
+
+```yaml
+      - uses: AltimateAI/altimate-code/github/review@main
+        with:
+          dbt_prepare: compile        # none | compile | full
</file context>
Suggested change
- uses: AltimateAI/altimate-code/github/review@main
- uses: AltimateAI/altimate-code/github/review@v0.8.5

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing yet: @main is intentional until a release that includes dbt_prepare ships, and we'll re-pin it then.

with:
dbt_prepare: compile # none | compile | full
dbt_command: dbt # or "uv run dbt", "poetry run dbt"
dbt_profiles_dir: . # relative to project_dir
```

- **`compile`:** runs `dbt compile` on the head, then compiles the PR's merge-base
into `target-base/` in a temporary worktree. This enables equivalence and
column-breakage proofs. dbt needs a profile it can load, and many adapters
connect to the warehouse while compiling. The head is compiled into the
directory of `manifest_path`. When `head` is set to a commit other than the
checkout, preparation is skipped.
- **`full`:** also runs `dbt docs generate` for `catalog.json` (complete column
types). That needs warehouse credentials in the profile.
- A head or base compile that fails is a warning: the review still runs, and the
affected models are listed as not verified.

To build them yourself, compile the **merge-base**, not the base branch's tip:

```yaml
- name: Compile the base into target-base/
run: |
# The review diffs against the merge-base, not the target branch tip.
git worktree add ../base "$(git merge-base "origin/${{ github.base_ref }}" HEAD)"
git worktree add ../base "$(git merge-base origin/${{ github.base_ref }} HEAD)"
(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
```

Expand Down Expand Up @@ -246,7 +266,7 @@ ways, or neither (it self-disables, leaving a deterministic-only review):
Always pass keys as repo **secrets**. Warehouse credentials are consumed by the
`dbt compile` step (via your `profiles.yml`), **not** by the review step. A
complete, copy-paste workflow lives at
[`github/review/examples/altimate-ingestion.yml`](https://github.com/AltimateAI/altimate-code/blob/main/github/review/examples/altimate-ingestion.yml).
[`github/review/examples/dbt-pr-review.yml`](https://github.com/AltimateAI/altimate-code/blob/main/github/review/examples/dbt-pr-review.yml).

Re-pushing commits updates the same summary comment in place; fixed findings are
dropped on the next run. `--post` targets **GitHub** PRs (it reads `GITHUB_TOKEN` /
Expand Down Expand Up @@ -509,6 +529,14 @@ The deterministic rules group into four families:
- **dbt unit tests.** A unit test whose `given` inputs are all `rows: []` is
reported: it either never tests the model's logic, or expects rows the model has nothing to
build from.
- **Data tests that can't do their job.** These are checked on tests added or
changed in the PR:
- a test that compares a model with itself (`dbt_utils.equality` or
`equal_rowcount` against its own `ref`, or a `relationships` test pointing
at its own column) passes by construction;
- new tests that inherit `severity: warn` or an `error_if` threshold from
`dbt_project.yml` report failures without failing the build;
- a `dbt_project.yml` change that loosens those thresholds for a whole folder.
- **Incremental and snapshot semantics.** Config is read from the manifest
(project, yml and inline, as dbt merges them) with the head's inline
`{{ config() }}` on top. Formatting-only edits never count as a change.
Expand Down
83 changes: 83 additions & 0 deletions github/review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,17 @@ inputs:
description: "Post the verdict to the PR (summary comment + inline review)."
required: false
default: "true"
dbt_prepare:
description: "Build the dbt artifacts the full review uses before reviewing. `none` (default): you build them in earlier steps. `compile`: `dbt compile` on the head, and on the PR's merge-base into `target-base/` (equivalence and lineage proofs). `full`: also `dbt docs generate` for `catalog.json` (complete column types). `compile` needs a profile dbt can load, and many adapters connect while compiling; `full` needs warehouse credentials. A failed compile is a warning and the review runs without that side's artifacts."
required: false
default: "none"
dbt_command:
description: "How to invoke dbt for `dbt_prepare`, e.g. `dbt`, `uv run dbt`, `poetry run dbt`. Split on whitespace (no shell quoting)."
required: false
default: "dbt"
dbt_profiles_dir:
description: "`--profiles-dir` for `dbt_prepare`, relative to `project_dir`. Empty = dbt's default lookup."
required: false
signing_key:
description: "Optional HMAC key to sign the verdict envelope (ALTIMATE_REVIEW_SIGNING_KEY)."
required: false
Expand Down Expand Up @@ -167,6 +178,78 @@ runs:
echo "Advisory lane: no model/credentials provided — running deterministic-only."
fi

# Optional: build the head and base artifacts the full-tier review needs.
# Inputs are passed as ENV, never interpolated into the script.
- name: Prepare dbt artifacts
if: inputs.dbt_prepare != 'none'
shell: bash
env:
IN_PREPARE: ${{ inputs.dbt_prepare }}
IN_DBT: ${{ inputs.dbt_command }}
IN_PROFILES: ${{ inputs.dbt_profiles_dir }}
IN_PROJECT_DIR: ${{ inputs.project_dir }}
IN_MANIFEST: ${{ inputs.manifest_path }}
IN_BASE: ${{ inputs.base }}
IN_HEAD: ${{ inputs.head }}
EVENT_BASE: ${{ github.event.pull_request.base.ref }}
run: |
set -euo pipefail
case "$IN_PREPARE" in compile|full) ;; *) echo "::error::dbt_prepare must be none, compile or full (got '$IN_PREPARE')"; exit 1 ;; esac
read -ra DBT <<< "$IN_DBT"
if [[ ${#DBT[@]} -eq 0 ]]; then echo "::error::dbt_command is empty; set it to how dbt is invoked (e.g. dbt)."; exit 1; fi
# Artifacts are built from the checkout: a different `head` would be reviewed
# against another commit's compiled SQL, so skip preparing instead.
if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then
echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts."
exit 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A failed head compile leaves existing or partial artifacts in place before exit 0, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 204:

<comment>A failed head compile leaves existing or partial artifacts in place before `exit 0`, so the next review can use stale SQL and manifest data instead of treating the head as unverified. Clear the head artifact set safely on this failure path before continuing.</comment>

<file context>
@@ -188,12 +188,21 @@ runs:
+        # against another commit's compiled SQL, so skip preparing instead.
+        if [[ -n "${IN_HEAD:-}" && "$(git rev-parse --verify -q "$IN_HEAD^{commit}" || true)" != "$(git rev-parse HEAD)" ]]; then
+          echo "::warning::head ($IN_HEAD) is not the checked-out commit; skipping dbt_prepare. Check out the head to build its artifacts."
+          exit 0
+        fi
         PROJECT="$(cd "${IN_PROJECT_DIR:-.}" && pwd)"
</file context>

fi
PROJECT="$(cd "${IN_PROJECT_DIR:-.}" && pwd)"
REL="$(git -C "$PROJECT" rev-parse --show-prefix)"
PROFILES=()
[[ -n "${IN_PROFILES:-}" ]] && PROFILES=(--profiles-dir "$(cd "$PROJECT" && cd "$IN_PROFILES" && pwd)")
build() { # $1 = project dir, $2 = target path
( cd "$1"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING]: A failed cd inside build() is swallowed, so the base compile can run in the wrong directory

build is only invoked from if ! build ... (line 220) and build ... || echo ... (line 251), and bash disables set -e for the whole function body in both contexts. When $WT/$REL does not exist at the merge-base (this PR adds or moves project_dir), this cd fails and the subshell keeps the step's cwd, then runs dbt deps/dbt compile there. If that cwd is itself a dbt project (monorepo with a root project plus the added/moved one), the wrong project is compiled into $PROJECT/target-base and reported as success, so the review treats head-vs-root SQL as the base side instead of warning. Make the cd failure explicit:

Suggested change
( cd "$1"
( cd "$1" || exit 1

[[ -f packages.yml || -f dependencies.yml ]] && "${DBT[@]}" deps ${PROFILES[@]+"${PROFILES[@]}"}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: "${DBT[@]}" expands an array that can be empty

read -ra DBT <<< "$IN_DBT" produces an empty array when dbt_command is empty/whitespace. Under set -u on bash < 4.4 (e.g. macOS 3.2) "${DBT[@]}" is an unbound-variable error; on newer bash the command word vanishes and deps/compile fails obscurely. Validate it ([[ ${#DBT[@]} -gt 0 ]] || { echo "::error::dbt_command is empty"; exit 1; }) and use the file's own ${DBT[@]+"${DBT[@]}"} idiom for consistency.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing: dbt_command defaults to dbt, and an empty value is a configuration error that fails loudly.

"${DBT[@]}" compile --target-path "$2" ${PROFILES[@]+"${PROFILES[@]}"} )
}
# The head goes where the review reads the manifest (`manifest_path`).
MANIFEST="${IN_MANIFEST:-target/manifest.json}"
[[ "$MANIFEST" == /* ]] || MANIFEST="$PROJECT/$MANIFEST"
TARGET="$(dirname "$MANIFEST")"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: manifest_path now sets the head compile target, but the review still reads compiled SQL from target/compiled

TARGET moves dbt compile's manifest and compiled/ to dirname(manifest_path), but makeCompiledResolver defaults headDir to "target/compiled" (packages/opencode/src/altimate/review/compiled.ts:96) and run.ts:407 never passes headDir. With manifest_path: build/manifest.json the manifest and catalog are found, yet every compiled-SQL lookup misses - the review falls back to raw Jinja, or reads a target/compiled left by an earlier run, instead of the SQL this step just built. Either derive the resolver's headDir from dirname(manifestPath) in run.ts, or document that dbt_prepare requires the default target/manifest.json.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Custom manifest_path: head compiled SQL goes where the review does not read it

TARGET="$(dirname "$MANIFEST")"

The head is now compiled into dirname(manifest_path), but the compiled-SQL resolver defaults to target/compiled (compiled.ts:96; run.ts passes no headDir). With manifest_path: build/manifest.json the manifest is found while every model reports no compiled SQL (before this change it was the reverse). A stale target/compiled could also be read instead.

Fix: pass headDir: <manifest dir>/compiled to makeCompiledResolver, or keep compiling into target/ and document that manifest_path only affects where the manifest is read.

echo "Compiling the head…"
if ! build "$PROJECT" "$TARGET"; then
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
echo "::warning::The head compile failed (dbt may need a warehouse connection to compile); the review runs without compiled artifacts."
exit 0
fi
if [[ "$IN_PREPARE" == "full" ]]; then
echo "Generating catalog.json…"
( cd "$PROJECT" && "${DBT[@]}" docs generate --target-path "$TARGET" ${PROFILES[@]+"${PROFILES[@]}"} ) ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: A failed docs generate leaves an earlier catalog.json in place, and the review still reads it

run.ts:376 reads dirname(manifest)/catalog.json unconditionally with no freshness check. On a workspace that is not cleaned between runs (clean: false, self-hosted, or a persisted target/), a full run whose docs generate fails here keeps the previous catalog, so column types for equivalence and column-lineage breakage come from an older warehouse snapshot while the step only warns. Remove the stale file before generating (e.g. rm -f "$TARGET/catalog.json") or say so in the warning.

echo "::warning::dbt docs generate failed; continuing without catalog.json."
fi
# The base side: the PR's merge-base, compiled into target-base/.
BASE="${IN_BASE:-$EVENT_BASE}"
BASE="${BASE#origin/}"
if [[ -z "$BASE" ]]; then echo "::warning::No base branch known; skipping the base compile."; exit 0; fi
git fetch --no-tags origin "+refs/heads/$BASE:refs/remotes/origin/$BASE" >/dev/null 2>&1 || true
MB="$(git merge-base "origin/$BASE" HEAD 2>/dev/null || git merge-base "$BASE" HEAD 2>/dev/null || true)"
if [[ -z "$MB" && "$(git rev-parse --is-shallow-repository)" == "true" ]]; then
git fetch --no-tags --deepen=1000 origin "+refs/heads/$BASE:refs/remotes/origin/$BASE" "$(git rev-parse HEAD)" >/dev/null 2>&1 || true
MB="$(git merge-base "origin/$BASE" HEAD 2>/dev/null || true)"
fi
if [[ -z "$MB" ]]; then echo "::warning::Could not find the merge-base with $BASE; skipping the base compile."; exit 0; fi
WT="$RUNNER_TEMP/altimate-review-base"
trap 'git worktree remove --force "$WT" >/dev/null 2>&1 || true' EXIT
git worktree add --detach "$WT" "$MB" >/dev/null || { echo "::warning::Could not check out the base; skipping the base compile."; exit 0; }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION]: A cancelled run can make every later base compile skip

The fixed $WT is cleaned only by the EXIT trap. If a run is cancelled or killed, the directory disappears but .git/worktrees/altimate-review-base stays registered on a self-hosted/cached workspace, so the next git worktree add fails with git's "missing but already registered worktree" error, the trap's worktree remove cannot fix a missing path, and this line warns and skips the base compile on every subsequent job. Prune the registration first, e.g. git worktree remove --force "$WT" >/dev/null 2>&1 || true (or git worktree prune) before adding.

# A stale target-base/ from an earlier run must not stand in for a failed compile.
rm -rf "$PROJECT/target-base"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The stale target-base/ cleanup runs only after the guards that can skip the base compile

When BASE is empty, the merge-base can't be found, or git worktree add fails, the script exits 0 before this line, so a target-base/ left by an earlier run survives. makeCompiledResolver reads target-base/compiled whenever it exists (compiled.ts:97), so on a cached or self-hosted workspace the head is compared against the previous run's base, and because the old side still resolves, those models are not even listed as unverified. Move the cleanup above the BASE/MB/worktree guards, and make it non-fatal (rm -rf ... || { echo "::warning::..."; exit 0; }) so an rm failure cannot abort the step and skip the review.

# A profiles.yml generated into the project dir at runtime is untracked, so
# the base worktree lacks it.
if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This unguarded cp can fail when project_dir does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 248:

<comment>This unguarded `cp` can fail when `project_dir` does not exist at the merge-base, causing the whole preparation step to fail before the base compile’s warning fallback. Guard the copy or turn its failure into a warning so the review still runs.</comment>

<file context>
@@ -226,6 +242,11 @@ runs:
+        # A profiles.yml generated into the project dir at runtime is untracked, so
+        # the base worktree lacks it.
+        if [[ ${#PROFILES[@]} -eq 0 && -f "$PROJECT/profiles.yml" && ! -f "$WT/$REL/profiles.yml" ]]; then
+          cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
+        fi
         echo "Compiling the base ($BASE @ ${MB:0:12})…"
</file context>
Suggested change
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml to the base worktree; base compile may fail."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unguarded cp under set -e can fail the whole prepare step and skip the review

Every other failure on this path degrades to a warning, but this cp does not. If the dbt project directory does not exist at the merge-base (a PR that adds or renames project_dir, e.g. a new analytics/ folder, or a rename), or the copy fails for any reason, set -e exits the step non-zero and the following "Run dbt PR review" step never runs - contradicting the input description and docs/docs/usage/dbt-pr-review.md ("A head or base compile that fails is a warning: the review still runs"). Add a -d "$WT/$REL" guard to the condition and make the copy non-fatal:

Suggested change
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"
cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml" || echo "::warning::Could not copy profiles.yml into the base worktree."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] profiles.yml copy aborts the step when the base has no project dir

cp "$PROJECT/profiles.yml" "$WT/$REL/profiles.yml"

This runs under set -e outside any || warning guard. If the project directory does not exist at the merge-base (the PR adds the dbt project or moves project_dir) and a runtime profiles.yml sits in the project, cp fails with No such file or directory and the whole step exits 1 (reproduced), where previously this path degraded to the base-compile warning.

Fix: guard with [[ -d "$WT/$REL" ]] && or append || true.

fi
echo "Compiling the base ($BASE @ ${MB:0:12})…"
build "$WT/$REL" "$PROJECT/target-base" || echo "::warning::The base compile failed; equivalence will be marked unverified."
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved

- name: Run dbt PR review
shell: bash
# Inputs are passed as ENV (data), never interpolated into the command —
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Example: drop-in dbt PR review for a dbt repo (e.g. altimate-ingestion).
# Example: drop-in dbt PR review for a dbt repo.
# Copy to the target repo as `.github/workflows/dbt-pr-review.yml` and adjust the
# dbt adapter + profiles/secrets to match the warehouse.
#
Expand Down
78 changes: 68 additions & 10 deletions packages/opencode/src/altimate/review/dbt-patterns.ts
Original file line number Diff line number Diff line change
Expand Up @@ -225,15 +225,76 @@ const detectNotIn: Detector = (c) => {
})
}

/** Head-side line numbers of the added lines; undefined when the diff has no hunk headers. */
function addedLineNumbers(diff: string | undefined): Set<number> | undefined {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This hunk walker duplicates the exported commentableLines

commentableLines (orchestrate.ts:268) already walks @@ hunks with the same regex and the same right counter to compute head-side line numbers; this copy already differs (it does not drop the diff's trailing-newline row) and the two will drift. dbt-patterns cannot import orchestrate without a cycle (the file already keeps a local modelNameFromPath for exactly that reason), so extract the walker into a shared module (e.g. diff-filter.ts) and take the added-line set from it, or pass the numbers in through Ctx.

if (!diff || !/^@@ /m.test(diff)) return undefined
const out = new Set<number>()
let right = 0
let inHunk = false
for (const line of diff.split("\n")) {
const h = /^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@/.exec(line)
if (h) {
right = Number(h[1])
inHunk = true
continue
}
if (!inHunk || line.startsWith("\\")) continue
if (line.startsWith("+")) out.add(right++)
else if (line.startsWith(" ") || line === "") right++
else if (!line.startsWith("-")) inHunk = false
}
return out
}

/**
* `row_number() over (…)` windows in the model, read from the whole file so a
* window split across lines is seen whole. A window counts as part of the
* change when one of the PR's added lines falls inside its line span (by text
* only when the diff carries no hunk headers, and then only for added lines
* that themselves hold window syntax).
*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
// Comments blanked with newlines kept, so offsets map to head line numbers.
const sql = c.newSql.replace(/\/\*[\s\S]*?\*\/|--[^\n]*|\{#[\s\S]*?#\}/g, (m) => m.replace(/[^\n]/g, " "))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This comment blanking is not quote-aware; blankComments() already does this

A -- inside a string literal blanks the rest of the line: select '--' as sep, row_number() over (partition by id) as rn loses the window entirely (a missed dedup finding), and a /* in a literal can blank a multi-line span. blankComments() in select-list.ts is the existing length-preserving, quote-aware helper (it skips '...', "..." and backticked identifiers), so reuse it here and keep only the {# ... #} alternative.

const lineAt = (offset: number) => sql.slice(0, offset).split("\n").length
const addedLines = addedLineNumbers(c.file.diff)
// Text fallback (diff without hunk headers): only added lines that are part
// of a window spec can touch one; a bare identifier elsewhere never does.
const squash = (t: string) => t.replace(/\s+/g, " ").trim()
const added = c.added
.map((l) => squash(stripComments(l)))
.filter((l) => l.length >= 3 && /\b(row_number|over|partition\s+by|order\s+by)\b/i.test(l))
const out: Array<{ spec: string; text: string }> = []
for (const m of sql.matchAll(/row_number\s*\(\s*\)\s*over\s*\(/gi)) {
const open = m.index! + m[0].length - 1
let depth = 0
let close = -1
for (let i = open; i < sql.length; i++) {
if (sql[i] === "(") depth++
else if (sql[i] === ")" && --depth === 0) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The parenthesis scan counts parentheses inside SQL string literals. A value like status = ')' closes the window early, truncating spec before a real ORDER BY; mask quoted literals or use a quote-aware scanner while finding the matching window close.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 243:

<comment>The parenthesis scan counts parentheses inside SQL string literals. A value like `status = ')'` closes the window early, truncating `spec` before a real `ORDER BY`; mask quoted literals or use a quote-aware scanner while finding the matching window close.</comment>

<file context>
@@ -225,15 +225,42 @@ const detectNotIn: Detector = (c) => {
+    let close = -1
+    for (let i = open; i < sql.length; i++) {
+      if (sql[i] === "(") depth++
+      else if (sql[i] === ")" && --depth === 0) {
+        close = i
+        break
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. We'll run stripLiterals before the paren scan.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Paren scan runs on comment-stripped but not literal-stripped SQL

sql comes from stripComments, but string literals are not blanked (the file already has stripLiterals). A ( or ) inside a literal in the OVER spec — e.g. partition by coalesce(customer_id, ')') or regexp_replace(code, '\\(', '') — miscounts depth, truncating the window before its ORDER BY (false "no ORDER BY") or over-extending it into the rest of the file (dropping/dirtying the window). Apply stripLiterals before the depth scan.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Same as r4626962367.

close = i
break
}
}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
const touched = addedLines
? [...addedLines].some((n) => n >= lineAt(m.index!) && n <= lineAt(close))
: added.some((l) => squash(text).includes(l) || l.includes(squash(text)))
if (!touched) continue
out.push({ spec: sql.slice(open + 1, close).replace(/\s+/g, " ").trim(), text: text.replace(/\s+/g, " ") })
}
return out
}

// 9. Dedup via ROW_NUMBER()/QUALIFY whose ORDER BY has no unique tiebreaker.
const detectDedupTie: Detector = (c) => {
const hit = c.added.find((l) => /row_number\s*\(\s*\)\s*over\s*\(/i.test(stripComments(l)))
if (!hit) return null
// Only flag dedup with NO ORDER BY — that is unambiguously non-deterministic.
// A present ORDER BY (even single-column) is the developer's deterministic
// choice and the dbt-recommended pattern; warning "it might not be unique"
// on every dedup is noise (false positives on the common, correct case).
if (/order\s+by/i.test(stripComments(hit))) return null
const hit = dedupWindows(c).find((w) => !/order\s+by/i.test(w.spec))
if (!hit) return null
return makeFinding({
severity: "warning",
category: "dedup",
Expand All @@ -244,7 +305,7 @@ const detectDedupTie: Detector = (c) => {
file: c.file.path,
model: c.model,
confidence: "high",
evidence: { tool: "dbt-patterns", result: { rule: "dedup_no_tiebreaker", line: hit.trim() } },
evidence: { tool: "dbt-patterns", result: { rule: "dedup_no_tiebreaker", line: hit.text } },
ruleKey: "dedup:no-tiebreaker",
})
}
Expand All @@ -257,11 +318,8 @@ const detectDedupTie: Detector = (c) => {
const LOAD_TS_RE =
/^(_?(ingest(ion|ed)?|load(ed)?|etl|extract(ed)?|batch)(_at|_ts|_time|_timestamp|_date|_dt)|_?(ingestion|load|etl|batch)_timestamp|_loaded_at)$/i
const detectDedupLoadTimestamp: Detector = (c) => {
for (const line of c.added) {
const s = stripComments(line)
const m = /row_number\s*\(\s*\)\s*over\s*\(([^)]*)\)/i.exec(s)
if (!m) continue
const order = /order\s+by\s+(.+)$/i.exec(m[1])?.[1]
for (const w of dedupWindows(c)) {
const order = /order\s+by\s+(.+)$/i.exec(w.spec)?.[1]
if (!order) continue
const keys = order.split(",").map((k) => k.trim().replace(/\s+(asc|desc)\b.*$/i, "").replace(/\s+nulls\s+(first|last)$/i, ""))
if (keys.length !== 1) continue
Expand All @@ -277,7 +335,7 @@ const detectDedupLoadTimestamp: Detector = (c) => {
file: c.file.path,
model: c.model,
confidence: "high",
evidence: { tool: "dbt-patterns", result: { rule: "dedup_load_timestamp_only", line: line.trim() } },
evidence: { tool: "dbt-patterns", result: { rule: "dedup_load_timestamp_only", line: w.text } },
ruleKey: "dedup:load-timestamp-only",
})
}
Expand Down
Loading
Loading