Skip to content
Open
257 changes: 247 additions & 10 deletions packages/opencode/src/altimate/review/ai-review.ts

Large diffs are not rendered by default.

41 changes: 35 additions & 6 deletions packages/opencode/src/altimate/review/ai-verify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,15 @@ import { readStreamStep, type StepToolCall } from "../llm/stream-text"

const log = Log.create({ service: "ai-verify" })

// Connection profiles, env files, key material and Terraform variables hold credentials: never sent to a model.
const CREDENTIAL_FILE =
/(^|\/)(profiles\.ya?ml|\.env(\.[^/]*)?|\.netrc|\.pgpass|id_(rsa|dsa|ecdsa|ed25519)|[^/]+\.(pem|key|p12|pfx|tfvars))$/i

/** A path whose content may hold credentials; the AI lane and the verifier never read or show it. */
export function isCredentialPath(path: string): boolean {
return CREDENTIAL_FILE.test(path)
}

/** What the verifier may read: the PR's files, the engine, and the dbt project. */
export interface AiVerifyEvidence {
dialect?: string
Expand All @@ -41,6 +50,8 @@ export interface AiVerifyEvidence {
prBody?: string
/** Head-side line numbers each changed file adds (see `addedLinesByFile`). */
addedLines: Map<string, Set<number>>
/** Every file the PR changes, repo-relative, including tests and CI outside the dbt project. */
prFiles?: string[]
}

export type Decision = "keep" | "discard" | "summary"
Expand Down Expand Up @@ -382,12 +393,27 @@ export function hunkAround(diff: string | undefined, line?: number): string {
: text || "(the line is outside the diff)"
}

/** Most PR file paths listed in a verifier message. */
const PR_FILES_MAX = 40

/** The user message for one batch: the PR title and each candidate with its hunk. Not the whole file, not the grounding. */
export function buildVerifyMessage(
prTitle: string | undefined,
candidates: Array<{ id: string; finding: Finding; diff?: string }>,
prFiles: string[] = [],
): string {
const parts = [`PR: ${(prTitle ?? "(untitled)").slice(0, 200)}`, "", "Candidates:"]
const parts = [`PR: ${(prTitle ?? "(untitled)").slice(0, 200)}`]
// A test-coverage claim is checked against the PR's own test files, which may sit outside the dbt project.
if (prFiles.length && candidates.some((c) => c.finding.category === "test_coverage")) {
const shown = prFiles.slice(0, PR_FILES_MAX)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The verifier cannot discover changed files after the first 40 paths, so a test added at path 41+ can be missed by the test-gap check; add a bounded file-list tool or otherwise expose all paths to the verifier.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-verify.ts, line 395:

<comment>The verifier cannot discover changed files after the first 40 paths, so a test added at path 41+ can be missed by the test-gap check; add a bounded file-list tool or otherwise expose all paths to the verifier.</comment>

<file context>
@@ -378,12 +380,27 @@ export function hunkAround(diff: string | undefined, line?: number): string {
+  const parts = [`PR: ${(prTitle ?? "(untitled)").slice(0, 200)}`]
+  // A test-coverage claim is checked against the PR's own test files, which may sit outside the dbt project.
+  if (prFiles.length && candidates.some((c) => c.finding.category === "test_coverage")) {
+    const shown = prFiles.slice(0, PR_FILES_MAX)
+    parts.push(
+      "",
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as r4627044412; tracked.

parts.push(
"",
"Files this PR changes (read_file can open any of them):",
...shown.map((p) => `- ${p}`),
...(prFiles.length > shown.length ? [`- … and ${prFiles.length - shown.length} more`] : []),
)
}
parts.push("", "Candidates:")
for (const { id, finding: f, diff } of candidates) {
parts.push(
"",
Expand Down Expand Up @@ -784,8 +810,9 @@ export function verifierTools(
...ev.addedLines.keys(),
...graph.parents.map((p) => p.path),
...graph.downstream.map((d) => d.path),
...(ev.prFiles ?? []),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The verifier's read allowlist now includes every PR file, including paths the user excluded with exclude or rubric excludeGlobs and out-of-project files. prFiles comes from the unfiltered collected list in run.ts, so those contents can reach the model provider. Build prFiles from the reviewable set after exclusions, or filter the allowlist with the same exclusion rules.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-verify.ts, line 783:

<comment>The verifier's read allowlist now includes every PR file, including paths the user excluded with `exclude` or rubric `excludeGlobs` and out-of-project files. `prFiles` comes from the unfiltered `collected` list in run.ts, so those contents can reach the model provider. Build `prFiles` from the reviewable set after exclusions, or filter the allowlist with the same exclusion rules.</comment>

<file context>
@@ -763,6 +780,7 @@ export function verifierTools(
       ...ev.addedLines.keys(),
       ...graph.parents.map((p) => p.path),
       ...graph.downstream.map((d) => d.path),
+      ...(ev.prFiles ?? []),
     ]),
   ]
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 31cb5c4: prFiles (the verifier's read allowlist and the test-gap list) now drops paths that match the user's exclude globs, plus build output and vendored paths (isExcludedPath). Changed files outside the dbt project stay readable on purpose (a Python test elsewhere in the repo is evidence for a test-gap claim), as OrchestrateInput.prFiles documents.

]),
]
].filter((p) => !isCredentialPath(p))
/** A requested path inside the review scope, as the repo-relative path to read; a project-relative path resolves too. */
const resolve = (p: unknown): string | undefined => {
const want = String(p ?? "").replace(/^\.?\/+/, "")
Expand All @@ -799,11 +826,11 @@ export function verifierTools(
graph.parents.find((p) => p.name === name && (p.kind === "model" || p.kind === "snapshot"))?.path ??
graph.downstream.find((d) => d.name === name)?.path
const side = z.enum(["old", "new"]).optional().describe("old=base, new=head (default)")
const scope = NA("outside review scope (changed files, their dbt parents and direct downstream models only)")
const scope = NA("outside review scope (files the PR changes, their dbt parents and direct downstream models only)")

return {
read_file: {
description: "Up to 200 numbered lines of a changed file, its dbt parents or direct children.",
description: "Up to 200 numbered lines of any file the PR changes (tests and CI included), a dbt parent or direct child.",
input: z.object({
path: z.string(),
side,
Expand Down Expand Up @@ -920,8 +947,9 @@ export function verifierTools(
async run({ pattern, glob }) {
if (!ev.grepProject) return NA("project search unavailable")
if (String(pattern).length < 2) return NA("pattern needs at least 2 characters")
const hits = await ev.grepProject(String(pattern), glob as string | undefined)
if (!hits) return NA("the search failed (git error, timeout, or a glob outside the project); this is not evidence of absence")
const found = await ev.grepProject(String(pattern), glob as string | undefined)
if (!found) return NA("the search failed (git error, timeout, or a glob outside the project); this is not evidence of absence")
const hits = found.filter((h) => !isCredentialPath(h.path))
return hits.length ? hits.map((h) => `${h.path}:${h.line}: ${h.text}`).join("\n") : "no matches"
},
},
Expand Down Expand Up @@ -1176,6 +1204,7 @@ export async function runAiVerify(
finding: input.findings[i],
diff: fileByPath.get(input.findings[i].file)?.diff,
})),
ev.prFiles,
)
return { idx, ids, message }
})
Expand Down
4 changes: 3 additions & 1 deletion packages/opencode/src/altimate/review/dbt-patterns.ts
Original file line number Diff line number Diff line change
Expand Up @@ -986,7 +986,9 @@ export function redundantJoinPredicates(sql: string): string[] {
} else if (rel) i += rel[0].length
const tail = /^\s+(?:as\s+)?(\w+)\s+on\b/i.exec(s.slice(i)) ?? /^\s+on\b/i.exec(s.slice(i))
if (!tail) {
joins.push({ preds: [], start: m.index!, end: i, inner, outer })
// A join with no ON (`cross join params as p`): step over its alias so the chain continues.
const bare = /^\s+(?:as\s+)?(?!(?:natural|inner|left|right|full|cross|outer|join|lateral|on|using|where|group|order|qualify|having|union|limit|window|select|from)\b)\w+\b/i.exec(s.slice(i))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A join ... using (...) has no ON clause, but this change does not step over it, so the join chain still breaks there and a redundant predicate on the next join is missed. Let the bare-join match also consume a USING (...) list.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/dbt-patterns.ts, line 933:

<comment>A `join ... using (...)` has no ON clause, but this change does not step over it, so the join chain still breaks there and a redundant predicate on the next join is missed. Let the bare-join match also consume a `USING (...)` list.</comment>

<file context>
@@ -929,7 +929,9 @@ export function redundantJoinPredicates(sql: string): string[] {
     if (!tail) {
-      joins.push({ preds: [], start: m.index!, end: i, inner })
+      // A join with no ON (`cross join params as p`): step over its alias so the chain continues.
+      const bare = /^\s+(?:as\s+)?(?!(?:natural|inner|left|right|full|cross|outer|join|lateral|on|using|where|group|order|qualify|having|union|limit|window|select|from)\b)\w+\b/i.exec(s.slice(i))
+      joins.push({ preds: [], start: m.index!, end: i + (bare?.[0].length ?? 0), inner })
       continue
</file context>
Suggested change
const bare = /^\s+(?:as\s+)?(?!(?:natural|inner|left|right|full|cross|outer|join|lateral|on|using|where|group|order|qualify|having|union|limit|window|select|from)\b)\w+\b/i.exec(s.slice(i))
const bare = /^\s+(?:as\s+)?(?!(?:natural|inner|left|right|full|cross|outer|join|lateral|on|using|where|group|order|qualify|having|union|limit|window|select|from)\b)\w+\b|^\s+using\s*\([^)]*\)/i.exec(s.slice(i))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. The result is a missed redundant predicate, never a false one.

joins.push({ preds: [], start: m.index!, end: i + (bare?.[0].length ?? 0), inner, outer })
continue
}
const fallback = rel?.[1].split(".").pop()?.toLowerCase()
Expand Down
8 changes: 8 additions & 0 deletions packages/opencode/src/altimate/review/diff-filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,14 @@ export function partitionChangedFiles(
return { reviewable, dropped }
}

/** True when a path is build output / vendored or matches one of the user's exclude globs:
* files the review must not read, even outside the reviewable set. */
export function isExcludedPath(path: string, extraExcludeGlobs: string[] = []): boolean {
const p = norm(path)
if (SKIP_DIR_PATTERNS.some((re) => re.test(p)) || SKIP_FILE_PATTERNS.some((re) => re.test(p))) return true
return extraExcludeGlobs.some((g) => globToRegExp(g).test(p))
}

/** Filter changed files down to the reviewable set, with their dbt kind. */
export function filterChangedFiles(
files: ChangedFile[],
Expand Down
32 changes: 26 additions & 6 deletions packages/opencode/src/altimate/review/orchestrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
type NotReviewed,
partitionChangedFiles,
hasOldSide,
isExcludedPath,
walkDiffLines,
walkDiffRows,
} from "./diff-filter"
Expand Down Expand Up @@ -61,12 +62,10 @@ import {
import { type AiContextFile, type AiLaneStatus, type AiReviewFile, type AiReviewInput, type AiReviewResult, aiInputBudget, safeReason, selectAiFiles, selectAiFilesByQuota } from "./ai-review"
import { type MacroCard, type ModelCard, attachContextCards } from "./ai-context-card"
import { ruleTier } from "./rule-tiers"
import { type AiVerifyEvidence, numberedSlice } from "./ai-verify"
import { type AiVerifyEvidence, isCredentialPath, numberedSlice } from "./ai-verify"
import type { ColumnReach } from "../native/dbt/column-reach"

const AI_SUPPORTING_KINDS = new Set(["macro", "test", "snapshot", "schema_yml", "source_yml", "seed", "project_config"])
// Connection profiles hold warehouse credentials: never send their diff or content to the AI lane.
const AI_CREDENTIAL_FILE = /(^|\/)profiles\.ya?ml$/i

/**
* The deterministic review recipe.
Expand Down Expand Up @@ -289,6 +288,8 @@ export interface OrchestrateInput {
aiVerify?: boolean
/** Fixed-string search over the dbt project at head, for the verifier. */
grepProject?: AiVerifyEvidence["grepProject"]
/** Every file the PR changes (repo-relative), including files outside the dbt project; the verifier may read them. */
prFiles?: string[]
/** PR metadata passed to the AI reviewer for intent checking. */
prTitle?: string
prBody?: string
Expand All @@ -302,6 +303,8 @@ export interface OrchestrateInput {
layout?: DbtProjectLayout
/** Names `ref()` can resolve to at head (repo resources + installed packages); undefined = unknown. */
resourceNames?: () => Promise<Set<string> | undefined>
/** `{% docs %}` names installed packages define; undefined when that can't be known. */
packageDocNames?: () => Promise<Set<string> | undefined>
/** What the review covered: base resolution and files left out of scope. */
scope?: Omit<ReviewScope, "unverified" | "notReviewed"> & { notReviewed?: NotReviewed[] }
/** Drop the regex catalog's cross-engine portability findings (single known engine). */
Expand Down Expand Up @@ -2378,20 +2381,31 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// dozens of yml files; serial `git show` per file adds up).
// A `{% docs name %}` block in any of the project's .md files at head (cached per name).
const docLookups = new Map<string, Promise<boolean>>()
// Docs blocks of installed packages (cached); undefined when they can't be known.
let pkgDocs: Promise<Set<string> | undefined> | undefined
const packageDocs = () => (pkgDocs ??= input.packageDocNames?.().catch(() => undefined) ?? Promise.resolve(undefined))
const docExists = (name: string) => {
if (!docLookups.has(name)) {
const block = new RegExp(`\\{%-?\\s*docs\\s+${name}\\s*-?%\\}`)
docLookups.set(
name,
(input.grepProject?.(`docs ${name}`, "*.md") ?? Promise.resolve([]))
// An unavailable search (undefined) is no evidence the block is missing.
.then((hits) => hits === undefined || hits.some((h) => block.test(h.text)))
.then(async (hits) => hits === undefined || hits.some((h) => block.test(h.text)) || !!(await packageDocs())?.has(name))
// A failed search is no evidence the block is missing.
.catch(() => true),
)
}
return docLookups.get(name)!
}
// The project defines docs blocks of its own (any `{% docs x %}` in its .md files), and its
// installed packages' blocks are known: a package may define the name otherwise.
let docsAny: Promise<boolean> | undefined
const projectHasDocs = () =>
(docsAny ??= (input.grepProject?.("docs", "*.md") ?? Promise.resolve([]))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: the generic docs pattern can exhaust the search cap before reaching a docs block

grepFixedAt stops after 50 total hits (git.ts:194-210; -m 50 per file), and docs shows up in ordinary markdown prose, so those hits can be used up before the file holding {% docs ... %} is scanned. projectHasDocs then returns false and the new doc-block-missing finding stays silent — the recall gap this change targets. A block-specific fixed string (e.g. separate searches for {% docs and {%- docs) would keep unrelated prose from diluting the result.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: low priority, tracked. Hitting the cap makes the rule go silent (a missed finding, not a wrong one); grepping for {% docs / {%- docs fixes it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The generic docs search can exhaust grepFixedAt’s 50-hit limit on Markdown prose before reaching a docs block, causing projectHasDocs() to suppress missing-doc() findings. Use a targeted, complete docs-block search instead.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 2287:

<comment>The generic `docs` search can exhaust `grepFixedAt`’s 50-hit limit on Markdown prose before reaching a docs block, causing `projectHasDocs()` to suppress missing-`doc()` findings. Use a targeted, complete docs-block search instead.</comment>

<file context>
@@ -2279,6 +2281,13 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
+  // The project defines docs blocks of its own (any `{% docs x %}` in its .md files).
+  let docsAny: Promise<boolean> | undefined
+  const projectHasDocs = () =>
+    (docsAny ??= (input.grepProject?.("docs", "*.md") ?? Promise.resolve([]))
+      .then((hits) => hits.some((h) => /\{%-?\s*docs\s+\w+\s*-?%\}/.test(h.text)))
+      // A failed search is no evidence: stay silent.
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as r4627012892; tracked.

.then(async (hits) => !!hits?.some((h) => /\{%-?\s*docs\s+\w+\s*-?%\}/.test(h.text)) && (await packageDocs()) !== undefined)
// A failed or unavailable search is no evidence: stay silent.
.catch(() => false))
const schemaFiles = reviewable.filter((f) => f.kind === "schema_yml")
touch(schemaFiles)
const sqlByModel = new Map([...ctxByPath.values()].filter((c) => c.newSql).map((c) => [modelNameFromPath(c.file.path).toLowerCase(), c.newSql!]))
Expand All @@ -2410,7 +2424,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
...detectUnitTestFixtures(file, oldContent, newContent),
...snapshotYmlFindings(file, oldContent, newContent),
...detectTestAdequacy({ file, oldContent, newContent, projectYml, layout: input.layout }),
...(lanes.has("dbt_patterns") && input.grepProject ? await docRefFindings(file, oldContent, newContent, docExists) : []),
...(lanes.has("dbt_patterns") && input.grepProject ? await docRefFindings(file, oldContent, newContent, docExists, projectHasDocs) : []),
]
} catch (err) {
// One malformed schema file degrades that file only.
Expand Down Expand Up @@ -2586,7 +2600,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// Macros, data tests, snapshots, yml and seeds carry defects no model diff shows.
// Seeds go as a diff only: the full CSV is data, not logic.
const supporting = reviewable.filter(
(f) => AI_SUPPORTING_KINDS.has(f.kind) && f.status !== "deleted" && !AI_CREDENTIAL_FILE.test(f.path),
(f) => AI_SUPPORTING_KINDS.has(f.kind) && f.status !== "deleted" && !isCredentialPath(f.path),
)
const contents = await Promise.all(
supporting.map((f) => (f.kind === "seed" ? undefined : getContent?.(f.path, "new")?.catch(() => undefined))),
Expand All @@ -2600,6 +2614,10 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
touch(aiBudget.coverage ? selectAiFilesByQuota(aiFiles, aiBudget.coverage.quotas).sent : selectAiFiles(aiFiles, aiBudget.maxFiles))
if (input.aiContextCard && anyManifest) await attachContextCards(aiFiles, input.runner)
const context = input.aiGraphContext ? await aiGraphContext(aiFiles.map((f) => f.path), input.runner, getContent) : undefined
// Files the PR changes that the AI may list and the verifier may read; never the user's exclusions or credentials.
const prFiles = (input.prFiles ?? input.changedFiles.map((f) => f.path)).filter(
(p) => !isExcludedPath(p, input.rubric.exclusions.excludeGlobs) && !isCredentialPath(p),
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] Verifier allowlist bypasses the credential-file exclusion

const prFiles = (input.prFiles ?? input.changedFiles.map((f) => f.path)).filter(
  (p) => !isExcludedPath(p, input.rubric.exclusions.excludeGlobs),
)

The AI file list above (line ~2599) deliberately drops profiles.yml via AI_CREDENTIAL_FILE, but prFiles only drops generated paths and user globs. Because prFiles is added to the verifierTools allowlist, read_file can open it. Scenario: a monorepo PR changes dbt/models/m.sql and config/profiles.yml; with ALTIMATE_REVIEW_AI_VERIFY=1 the verifier calls read_file("config/profiles.yml") and the contents are sent to the model provider. Reproduced with the changed verifierTools in isolation (synthetic file): returns the content with prFiles, returns "outside review scope" without it. .env, *.tfvars and *.pem changed by the PR are reachable the same way.

Fix: also filter AI_CREDENTIAL_FILE (and ideally a small secret-like deny list) when building prFiles, and enforce it in resolve() in ai-verify.ts so every allowlist source is covered. Add a regression test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed in 7b5bb4b (the "verifier credential scope" part). One isCredentialPath in ai-verify.ts covers profiles.yml/.yaml, .env*, .netrc, .pgpass, SSH keys, *.pem/*.key/*.p12/*.pfx and *.tfvars. It is applied in three places:

  • when prFiles is built in orchestrate.ts, so the AI lane's file list and the test-gap list never name these files;
  • inside verifierTools, on the merged allowlist (files, added lines, graph parents/children and prFiles), so no allowlist source can expose them;
  • on grep_project hits.

The AI lane's supporting-file filter now uses the same predicate. Tests: read_file refuses each credential path, even when it comes in through prFiles, addedLines or a graph parent, and grep drops a profiles.yml hit. A runReview-level test checks that the evidence's prFiles excludes them.

// The verifier reads engine SQL first (the same SQL the engine lanes saw), then compiled output.
const evidence: AiVerifyEvidence | undefined = input.aiVerify
? {
Expand All @@ -2611,6 +2629,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
prTitle: input.prTitle,
prBody: input.prBody,
grepProject: input.grepProject,
prFiles,
readFile: async (p, side, start, end) => numberedSlice(await getContent?.(p, side), start, end),
compiledSql: async (p, side) => {
const ctx = ctxByPath.get(p)
Expand All @@ -2627,6 +2646,7 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
prTitle: input.prTitle,
prBody: input.prBody,
model: input.aiModel,
prFiles,
...(evidence ? { evidence } : {}),
})
const aiFindings = Array.isArray(res) ? res : res.findings
Expand Down
28 changes: 22 additions & 6 deletions packages/opencode/src/altimate/review/project-rules.ts
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,19 @@ export function yamlParseFindings(file: ChangedFile, oldContent: string | undefi
if (!/\.ya?ml$/i.test(file.path) || file.status === "deleted" || !newContent) return []
const err = yamlError(newContent)
if (!err || (oldContent !== undefined && yamlError(oldContent))) return []
const text = err.line ? newContent.split("\n")[err.line - 1] : undefined
// Some errors land one past the edit (an unterminated quote fails where the scan ends): anchor
// on the added line at or before the error line, else on the error line when a hunk shows it
// (a deletion-only edit), else on the file. An error with no line goes on the first added line.
const rows = [...walkDiffRows(file.diff)]
const added = rows.filter((r) => r.kind === "+")
const errLine = err.line
const at =
errLine === undefined
? added[0]
: (added.find((r) => r.line === errLine) ??
[...added].reverse().find((r) => r.line < errLine) ??
rows.find((r) => r.kind === " " && r.line === errLine))
const text = at?.text ?? (errLine ? newContent.split("\n")[errLine - 1] : undefined)
const f = finding(
file,
"yaml-parse-error",
Expand All @@ -414,7 +426,7 @@ export function yamlParseFindings(file: ChangedFile, oldContent: string | undefi
: `${err.message}${err.line ? ` (line ${err.line})` : ""}. The file parsed before this change; tools that read it now fail.`,
text,
)
return [err.line ? { ...f, startLine: err.line, endLine: err.line } : f]
return [at ? { ...f, startLine: at.line, endLine: at.line } : f]
}

/** New models named like the public models a `.quickstart/quickstart.yml` lists, but missing from
Expand Down Expand Up @@ -489,14 +501,17 @@ export function packageLockFindings(file: ChangedFile, isPackage: boolean): Find

const DOC_REF = /\bdoc\s*\(\s*['"](\w+)['"]\s*\)/g

/** `doc('x')` names this change adds to a yml file that no docs block in the project defines:
* dbt fails to parse the project. Only when another `doc()` in the file resolves in the project,
* so docs kept in an installed package aren't mistaken for missing ones. */
/** `doc('x')` names this change adds to a yml file that no docs block in the project or its
* installed packages defines: dbt fails to parse the project. Only when another `doc()` in the
* file resolves, or, when the file has no other `doc()`, the project keeps its own docs blocks
* (`projectHasDocs` also requires the packages' blocks to be known); so docs kept in an
* uninstalled package aren't mistaken for missing ones. */
export async function docRefFindings(
file: ChangedFile,
oldContent: string | undefined,
newContent: string | undefined,
docExists: (name: string) => Promise<boolean>,
projectHasDocs?: () => Promise<boolean>,
): Promise<Finding[]> {
if (!newContent || file.status === "deleted") return []
const before = new Set([...(oldContent ?? "").matchAll(DOC_REF)].map((m) => m[1]))
Expand All @@ -509,6 +524,7 @@ export async function docRefFindings(
const others = [...new Set([...newContent.matchAll(DOC_REF)].map((m) => m[1]))].filter((n) => !missing.includes(n)).slice(0, 5)
let resolves = false
for (const n of others) if ((resolves = await docExists(n))) break
if (!others.length && projectHasDocs) resolves = await projectHasDocs()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: projectHasDocs() cannot tell a project docs block from one that resolves in an installed package

docExists only greps the dbt project's own paths (grepProject pathspecs come from the dbt_project.yml model/macro/seed/snapshot/test dirs, run.ts:431-438), so a doc('name') defined by an installed package already counts as "missing". This new fallback then treats any docs block anywhere in the project as proof that the name resolves, so for a file whose only doc() is the new one (typically a newly added schema yml) a package-provided doc is reported as "dbt fails to parse" — exactly the package case the guard above was added to avoid. The finding body also says "the other doc() references in this file resolve there", which is false on this branch (there are none). Consider also searching installed-package .md files before concluding the name is missing, or keep the previous "another doc() in this file resolves" requirement for names that could come from a package.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 31cb5c4: doc() names defined in the .md files of installed packages (dbt_packages/) now count as defined. The no-other-doc() fallback also requires the packages' blocks to be known (no packages declared, or packages installed), so a package-provided doc() stays silent when packages aren't installed. The body now says why the finding is trusted ("the other doc() references resolve" or "the project defines its own docs blocks") and no longer claims the first on the fallback path.

Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
if (!resolves) return []
return missing.map((name) =>
finding(
Expand All @@ -517,7 +533,7 @@ export async function docRefFindings(
"warning",
"sql_correctness",
`\`doc('${name}')\` has no docs block`,
`No \`{% docs ${name} %}\` block exists in the project's \`.md\` files, while the other \`doc()\` references in this file resolve there. dbt fails to parse the project on a \`doc()\` it can't resolve. Add the docs block, or fix the name.`,
`No \`{% docs ${name} %}\` block exists in the project's \`.md\` files or its installed packages, while ${others.length ? "the other \`doc()\` references in this file resolve" : "the project defines its own docs blocks"}. dbt fails to parse the project on a \`doc()\` it can't resolve. Add the docs block, or fix the name.`,
lines.get(name),
),
)
Expand Down
6 changes: 5 additions & 1 deletion packages/opencode/src/altimate/review/rubric.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,17 +70,21 @@ export function blockingCategories(rubric: Rubric): Set<string> {
return new Set(rubric.blockOn)
}

/** Findings that break `dbt parse` for the whole project, so a dev/sandbox/scratch folder doesn't excuse them. */
const PROJECT_BREAKING_RULES = new Set(["dbt-project-rules:yaml-parse-error", "dbt-project-rules:doc-block-missing"])

/**
* Apply the rubric's exclusion predicates. Returns the reason a finding should
* be dropped, or null to keep it. Centralizing this here (vs prompt text) is
* what keeps the false-positive rate down deterministically.
*/

export function exclusionReason(finding: Finding, rubric: Rubric): string | null {
const ex = rubric.exclusions
const isStaging = /(^|\/)stg_|(^|\/)staging\//.test(finding.file) || finding.model?.startsWith("stg_")
const isDev = /(^|\/)(dev|sandbox|scratch)\//.test(finding.file)

if (ex.skipNonProdModels && isDev) return "non-prod model (dev/sandbox/scratch)"
if (ex.skipNonProdModels && isDev && !PROJECT_BREAKING_RULES.has(finding.rule ?? "")) return "non-prod model (dev/sandbox/scratch)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: ref-target-missing is still dropped in dev/sandbox/scratch folders even though it breaks dbt parse

This exemption only keeps yaml-parse-error and doc-block-missing, but missingRefFindings emits dbt-project-rules:ref-target-missing, whose own body says "so dbt fails to parse the project" (project-rules.ts:828). skipNonProdModels defaults to true, so a ref('typo') added under models/dev/, sandbox/ or scratch/ is still filtered out here — the exact case the new constant is meant to stop. Add dbt-project-rules:ref-target-missing to the set (and jinja-unbalanced, which breaks compilation of every dependent model, if that is in scope for this fix).


if (
ex.allowSelectStarInStaging &&
Expand Down
3 changes: 2 additions & 1 deletion packages/opencode/src/altimate/review/rule-catalog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -793,9 +793,10 @@ const CORRECTNESS: Rule[] = [
{
// A literal holding SQL (it has whitespace) that ends, or starts, with a word character right at the `~`,
// inside an open `{{`/`{%` on the line (a bare `col ~ 'a b'` is a Postgres regex match, not Jinja).
// Single- or double-quoted; an escape (`\n`) at the seam is not a word.
added:
// The whitespace check is a lookahead so the literal is scanned once (no split backtracking on long lines).
/\{[{%](?:(?![}%]\}).)*?(?:'(?=[^'\n]*\s)[^'\n]*[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9](?=[^'\n]*\s)[^'\n]*')/,
/\{[{%](?:(?![}%]\}).)*?(?:'(?=[^'\n]*\s)[^'\n]*(?<!\\)[A-Za-z0-9]'\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*'[A-Za-z0-9](?=[^'\n]*\s)[^'\n]*'|"(?=[^"\n]*\s)[^"\n]*(?<!\\)[A-Za-z0-9]"\s*~\s*[A-Za-z_][\w.]*|[\w)]\s*~\s*"[A-Za-z0-9](?=[^"\n]*\s)[^"\n]*")/,
// Messages are not SQL.
unless: /\b(?:log|print|raise_compiler_error|warn)\s*\(/,
macros: true,
Expand Down
Loading
Loading