Skip to content

fix: prevent false dbt validator findings and preserve build artifacts - #1817

Draft
anandgupta42 wants to merge 19 commits into
mainfrom
fix/dbt-validator-false-alarms-v2
Draft

anandgupta42 wants to merge 19 commits into
mainfrom
fix/dbt-validator-false-alarms-v2

Conversation

@anandgupta42

@anandgupta42 anandgupta42 commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

  • Partial YAML column lists triggered schema fixes → verification reports enforced-contract mismatches and missing declared columns with tests, including when a legacy manifest leaves test attachment uncertain.
  • A valid model with no selected tests looked untestable → the CLI says “Model has no tests” and the validator records nothing to check. Failed tests and dbt errors still surface.
  • Concurrent commands reinstalled packages and raced manifest writes → package installs are conditional and locked, incomplete manifest reads retry, and eligible validator errors get one serial retry.
  • Automatic verification replaced build results with compile results → it uses a temporary target and reports that in the build output. Real build failures still block.
  • Package resets and missing SQL connections were opaque → builds report when dependencies were reinstalled, and an empty connection registry points dbt users to altimate-dbt execute or their project profile.

The changes are model-agnostic and remove false findings while preserving real failure detection. Validators are off by default. ALTIMATE_VALIDATORS_ENABLED=1 enables enforcement. ALTIMATE_VALIDATORS_SHADOW=1 runs checks without enforcement.

Issue for this PR

Supersedes #1420
Closes #1419

Introduced by #849 (2026-05-29) and #1175 (2026-09-03).

The advisory PR #1815 overlaps the schema and test validator result handling and validator/dbt-tools documentation. This draft targets main directly and has no dependency on those advisory changes.

Type of change

  • Bug fix

How did you verify your code works?

  • Real dbt/DuckDB on detached main reproduced the artifact overwrite. Single-model and project builds now retain build provenance. A real failed build exits with an error and blocks the completion gate. The dbt-tools E2E job runs these cases and the package-concurrency regressions.
  • Ported regression tests cover partial documentation, enforced contracts, tested missing columns, empty selections, real test failures, transient contention, package repair and manifest retry.
  • Local checks passed: 942 validator and SQL-tool tests, 191 dbt-tools tests, six real-dbt integration cases, typecheck, and the strict marker check. The validator suite had 132 skips.
  • Independent review covered 2ee636b861fcad3944696eda73a535eb3ec55296. Follow-up commits update the stale-manifest assertion, bound serial retries, and improve package and tool diagnostics.
  • Exact-head local review ran after each push, including 711cb81a7ac57cef84cdc4965ae40ba6cf0b2857. GitHub CI passed on this head, including the real-dbt integration cases.
  • Round 2 at d49d77dd3e08bab3c410528ce69d13e1456418a2 passed 204 dbt-tools tests, 858 validator tests, 21 real-dbt cases with isolated HOME, typecheck, marker and tracker checks. Exact-head Codex review found zero P0–P3 issues.
  • The coordinator runs a paired cross-model regression check before readiness.

Known limit

The lock stays held while a recorded dbt installer is alive, even if its CLI caller exits. A crash during installer startup may require clearing the lock after confirming the installer has stopped. When the CLI cannot inspect the package destination, dbt resolves it and the lock covers installation through command completion. Direct edits to the generated package lock during an install are also outside that lock’s coordination. When packages were installed externally and have no Altimate stamp, local and git package directories are checked by count. Legacy schema responses without evidence stay unclassified, as in the superseded PR’s settled review.

Cost side effects

No new model calls or cloud resources. Verification uses a separate local dbt parse and the existing column checks. Eligible subprocess errors get one serial retry per affected model, capped at four per validator run.

Screenshots / recordings

CLI and validator behavior only.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects

Require evidence for schema drift, handle empty test selections, and
coordinate package installs and manifest reads. Preserve build results
when automatic schema checks compile, and clarify dbt tool diagnostics.
@coderabbitai

coderabbitai Bot commented Nov 21, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@anandgupta42

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 711cb81a7a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T21:21:07.196922Z 711cb81 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Let dbt resolve uninspectable package paths under the install lock.
Keep the lock while an orphaned installer is alive, and use legacy
relationship test metadata to identify the model under test.
Keep commands using unresolved package paths serialized through parsing
and execution. Track explicit installer subprocesses and distinguish
reused process IDs when recovering an abandoned lock.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt validators raise false alarms and concurrent altimate-dbt corrupts dbt_packages

1 participant