Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
2ee636b
fix: prevent false dbt validator findings
anandgupta42 Nov 21, 2026
c31f75e
test: verify dbt diagnostics against current fixtures
anandgupta42 Nov 21, 2026
98d63a1
fix: scope text-mode notices to build output
anandgupta42 Nov 21, 2026
945fc78
fix: bound serial retries for dbt validator errors
anandgupta42 Nov 21, 2026
00cf459
fix: handle stale and templated dbt package installs
anandgupta42 Nov 21, 2026
7f9d3dd
fix: preserve missing dbt relation and dependency failures
anandgupta42 Nov 21, 2026
c0b7727
fix: use dbt command status for package repair and builds
anandgupta42 Nov 21, 2026
4339694
fix: invalidate dbt package stamps when inputs change
anandgupta42 Nov 21, 2026
b38667f
fix: keep stamps for successful explicit dbt installs
anandgupta42 Nov 21, 2026
7a01e4b
fix: keep successful dbt progress output out of errors
anandgupta42 Nov 21, 2026
0dc5e3d
fix: keep confirmed dbt test failures blocking
anandgupta42 Nov 21, 2026
48b34f3
fix: verify dbt install inputs stayed stable before stamping
anandgupta42 Nov 21, 2026
711cb81
fix: stamp dbt packages from the verified input snapshot
anandgupta42 Nov 21, 2026
a2df794
test: isolate dbt integration test configuration
anandgupta42 Nov 21, 2026
514e5b2
fix: recover dbt installs and legacy test attribution
anandgupta42 Nov 21, 2026
b478f6e
fix: resolve wrapped dbt legacy test targets
anandgupta42 Nov 21, 2026
e04cac4
fix: protect dbt package readers and explicit installs
anandgupta42 Nov 21, 2026
cd9becc
fix: recognize quoted dbt test columns
anandgupta42 Nov 21, 2026
d49d77d
fix: track the Python Core package installer
anandgupta42 Nov 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .opencode/skills/dbt-develop/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,9 @@ altimate-dbt info # confirm every req

**Diff column shape against the spec — use the `dbt-schema-verify` skill.**
For each model the task touched, run `altimate-dbt schema-verify --model
<name>` and treat any `mismatch` verdict as "not done." Full procedure,
<name>` and treat a `mismatch` verdict (an enforced contract the table does
not match, or a declared column with tests that the table lacks) as "not
done." Columns the YAML does not list are not a mismatch. Full procedure,
output interpretation, and fallback (when `altimate-dbt` is missing) live
in the dedicated **dbt-schema-verify** skill, which auto-loads alongside
this one.
Expand Down Expand Up @@ -206,7 +208,7 @@ Use `altimate-dbt children` and `altimate-dbt parents` to verify the DAG is inta
5. **Fix ALL errors, not just yours.** After creating/modifying models, run a full `dbt build`. If ANY model fails — even pre-existing ones you didn't touch — fix them. Your job is to leave the project in a fully working state.
6. **Verify transformation correctness, not just mechanics.** For non-trivial models, generate and run dbt unit tests as part of the validate step (use the `dbt-unit-tests` skill). Passing `dbt build` only proves the SQL is syntactically valid — it doesn't prove the *values* are right.
7. **Enumerate deliverables, then check them off.** The task is not done until every model, column, test, and config change explicitly requested exists on disk and in the manifest. Re-read the prompt at the end and verify each requested item — don't trust your own intermediate "done" feeling.
8. **Match the column spec exactly — and verify it mechanically, not by inspection.** Use the dedicated **dbt-schema-verify** skill. Before declaring any model task done, run `altimate-dbt schema-verify --model <name>` and treat any `mismatch` verdict as "not done." Adding "helpful" extras (rank breakdowns, name-resolved fields, lineage metadata), reordering columns "more logically", or substituting synonyms (`supplier_id` for `supplier_company`, `transaction_type_name` for `transaction_type`) all break equality tests. The contract is what the spec says, not what you think would be useful.
8. **Produce the columns the task asks for — and check declared columns mechanically, not by inspection.** Use the dedicated **dbt-schema-verify** skill. Before declaring any model task done, run `altimate-dbt schema-verify --model <name>` and treat a `mismatch` verdict as "not done." Adding "helpful" extras (rank breakdowns, name-resolved fields, lineage metadata) or substituting synonyms (`supplier_id` for `supplier_company`) can break the task's own expectations: follow the columns the task and the existing consumers of the model call for. YAML that lists only some columns does not mean the others should be removed, and a YAML column nothing tests or enforces may be stale rather than required.

## Common Pitfalls in Transformation Logic

Expand Down
111 changes: 63 additions & 48 deletions .opencode/skills/dbt-schema-verify/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,17 @@ applyPaths:
- "dbt_project.yml"
- "**/dbt_project.yml"
description: |
REQUIRED after building or modifying ANY dbt model that has columns declared
in `schema.yml` / `_models.yml`. Run `altimate-dbt schema-verify --model
<name>` to diff actual columns against the spec, and treat any `mismatch`
verdict as "not done."

The most common reason "the build is green but the tests still fail" is
that the model produces the right *data values* in the wrong *column
shape* — extra columns, missing columns, wrong order, wrong types. Many
dbt equality tests grade the column tuple `(name, type, position)`
exactly, and the agent's prior bias is to add "helpful" extras
(`p1`/`p2`/`p3` rank breakdowns, name-resolved variants, lineage
metadata) or reorder columns "more logically." Both break the contract.

This skill enforces the mechanical check that catches those bugs before
declaring done. Use it before declaring any model task complete.
Use after building or modifying a dbt model whose columns are declared in
YAML (`schema.yml` / `_models.yml`). Run `altimate-dbt schema-verify --model
<name>` to compare the built table with the declared columns, and treat a
`mismatch` verdict (a non-empty `findings` list) as "not done" until you have
decided whether the model or the YAML is wrong.

YAML `columns:` is documentation plus the anchor for column tests. It is not
an exhaustive list of what a model may produce, so columns the YAML does not
mention are not an error unless a contract is enforced. What does count: an
enforced contract the table does not match, and a declared column that has
tests attached but is not produced.
---

# dbt schema-verify
Expand Down Expand Up @@ -59,8 +55,17 @@ Returns a structured JSON result:

```json
{
"model": "int_asana__project_user_agg",
"model": "int_work_items__project_user_agg",
"verdict": "mismatch",
"spec": { "declared_in": "models/schema.yml", "package": "work_items", "contract_enforced": false },
"findings": [
{
"kind": "tested-column-missing",
"columns": ["number_of_users_involved"],
"evidence": "models/schema.yml declares number_of_users_involved with tests (not_null_...), but the model does not produce it"
}
],
"notes": [],
"expected_columns": ["project_id", "users", "number_of_users_involved"],
"actual_columns": ["project_id", "users"],
"columns_extra": [],
Expand All @@ -74,42 +79,53 @@ Returns a structured JSON result:

| verdict | meaning | what to do |
|---|---|---|
| `match` | actual columns match the spec exactly (case-insensitive on names) | DONE — proceed |
| `mismatch` | one or more of `columns_extra`, `columns_missing`, `columns_reordered`, `type_mismatches` is non-empty | NOT DONE — read the diff, fix the model SQL, rebuild, re-run schema-verify |
| `no-spec` | the model has no columns declared in `schema.yml` | DONE for shape-fidelity purposes — no contract to verify against |

## How to act on a `mismatch`

For each non-empty list, the fix is mechanical:

| Field | What it means | What to change in the model SQL |
|---|---|---|
| `columns_extra` | columns in your model NOT in the spec | REMOVE them from the `SELECT` |
| `columns_missing` | columns in the spec NOT in your model | ADD them to the `SELECT` (compute them, or rename an existing column if you used a synonym) |
| `columns_reordered` | columns present in both but at different positions | REORDER the columns in your `SELECT` to match the spec's order |
| `type_mismatches` | declared `data_type` in spec disagrees with the warehouse's reported type | CAST in the `SELECT` or change the upstream source |

Then run `altimate-dbt build --model <name>` again, then re-run
`altimate-dbt schema-verify --model <name>` until verdict is `match`.
| `match` | nothing dbt treats as an error was found (`notes` may still hold true observations) | proceed |
| `mismatch` | `findings` is non-empty: an enforced contract the table does not match, or a missing declared column with an attached test or a legacy test whose attachment could not be verified | NOT DONE until you decide which side is wrong |
| `no-spec` | the model has no columns declared in YAML | nothing to compare |

`spec` says which YAML file declares the columns, in which package, and
whether a contract is enforced. Each entry in `findings` carries the
evidence. `columns_extra` / `columns_missing` / `columns_reordered` /
`type_mismatches` are the raw diff against the YAML, not instructions:
`columns_extra` only means "not listed in the YAML".

## How to act on a finding

A finding says two things disagree. It does not say which one is wrong.

- **Contract enforced.** dbt will refuse to build a model whose columns differ
from its contract. Either the model should produce exactly the contract's
columns, or the contract is out of date. Decide from the task.
- **Declared column with tests, not produced.** The tests read a column that
is not there and will fail or error. Either the model should produce the
column, or the YAML entry (and its tests) is stale. Decide from the task.
- **Test attachment unverified.** The table lacks the declared column, and
the legacy manifest lists the model as a test dependency without identifying
which relation the test reads. Confirm the attachment in the test's YAML
before deciding whether to change the model or its declaration.
- **Never** delete or invent a column just because the YAML does or does not
list it. Undocumented columns are normal, and a declared column with no
tests and no contract (a `note`) may simply be stale. If the task states
which columns the output must have, follow the task.

Then run `altimate-dbt build --model <name>` again and re-run
`altimate-dbt schema-verify --model <name>`.

## Iron Rules

1. **The verdict is the source of truth, not your inspection.** Reading the
columns yourself and concluding "looks right to me" does not count.
Run the command and read its output.
2. **A `mismatch` is "not done", even if the build is green.** dbt build
only proves the SQL compiled and ran without errors. It does not prove
the column shape is correct. Equality tests grade shape AND values.
3. **Do not reinterpret the spec to make the model right.** The spec is
the contract. If the spec lists `supplier_company` and your model has
`supplier_id`, the answer is to fix your model, not to argue that
`supplier_id` is more useful.
does not validate the full declared column list unless a contract is enforced;
attached tests still run and check the columns they reference.
3. **Do not change the model to satisfy YAML that only documents some
columns.** The YAML is evidence about intent, not the spec of the whole
table. The task and the existing consumers of the model decide which
columns are wanted.
4. **Run schema-verify on every model touched, not just the last one.**
The most common "almost-pass" is N-1 models passing and the Nth one
silently failing on column shape. Walk the list.
5. **Skip only on `no-spec`.** Do not skip on the grounds that the model
is small, or trivial, or "obvious." The spec is small only because
the dbt project author already curated it.
5. **`no-spec` and `match` with notes need no action.** Read the notes; act
on them only if they match what the task asked for.

## Fallback when altimate-dbt is unavailable

Expand All @@ -125,10 +141,9 @@ cat models/**/*.yml | grep -A 50 "name: <name>" # or: yq eval '...' models/**/
dbt show --select <name> --limit 0
```

Compare the two ordered lists. Produce the same four-bucket diff
(`columns_extra`, `columns_missing`, `columns_reordered`,
`type_mismatches`) in your head, and apply the same fix logic. The
mechanics don't change; only the tool name does.
Compare the two lists and apply the same rules: only an enforced
contract, or a declared column with tests that the table lacks, is a
problem. Columns the YAML does not list are not.

## What this skill does NOT cover

Expand Down
1 change: 1 addition & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions docs/docs/data-engineering/tools/dbt-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,14 @@ All commands provide friendly error diagnostics with actionable fix suggestions

> **Tip:** In builder mode, the agent prefers `altimate-dbt` over the raw `dbt_run` tool for better error handling and auto-detection.

**Packages.** If the project declares packages (`packages.yml`, `dependencies.yml`), `altimate-dbt` installs them when an adapter-backed command starts (not `doctor`), but only when they are missing or out of date (a package directory is absent or has no `dbt_project.yml`, or the declarations changed since the last install). Installation is serialised across processes with a lock under `target/.altimate-dbt/`, so several `altimate-dbt` commands started together install at most once and cannot corrupt `dbt_packages/`. A lock left by a crashed process is broken automatically (owner process gone, or no heartbeat for 30 seconds) and the interrupted install is repeated. `altimate-dbt deps` always runs `dbt deps`, under the same lock. Because every command re-parses the project and dbt rewrites `target/manifest.json` in place, a command that reads the manifest while a sibling rewrites it re-reads it (up to six times, with backoff) before reporting it missing.

For package install paths with Jinja beyond a literal `env_var(...)`, use `altimate-dbt deps` to let dbt resolve the path and install the packages.

Automatic schema verification after `altimate-dbt build` uses a temporary target directory. Its compile artifacts leave the project's build results available for inspection. The build reports when it ran verification or reinstalled dependencies. Reinstalling dependencies resets the package directory, so changes made directly there may be lost.

When `altimate-dbt test --model <name>` selects a model with no tests, it reports “Model has no tests.” Invalid selectors and failed tests retain dbt's diagnostics.

---

## dbt Skills
Expand Down
80 changes: 68 additions & 12 deletions docs/docs/data-engineering/validators.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,24 +53,73 @@ deterministic-validators work and are currently soak-tested in shadow mode

After the agent declares done, runs `altimate-dbt test --model <name>`
against every dbt model the agent modified during this session. Refuses
to terminate if any model's tests fail or error.
to terminate if a model's tests fail, or if tests could not be run
because dbt itself errored.

**Catches**: row-data correctness errors (`relationships`, `unique`,
`not_null`, `accepted_values`, `AUTO_*_equality` tests) — the kind of
bug that column-shape verification cannot detect because the schema can
be green while the SELECT logic produces wrong values or wrong row
counts.

**What counts as what**

| dbt result for the model | Validator |
|---|---|
| Tests ran and all passed | passes |
| Tests ran and some failed or errored | **fails**, naming captured failing tests when available |
| dbt selected no tests (it prints `Nothing to do` and no summary) | passes, and sends nothing. A model with no tests has nothing to check. Recorded as `no_tests` in telemetry. |
| The selector matched no enabled node | passes (`unmatched_selector_models` in telemetry) |
| dbt aborted (connection, lock, compile or runtime error) before or while running tests | **fails**, with dbt's error text |

A run error can be retried once in isolation to recover from warehouse contention,
including contention with another invocation. Each schema or test validator run retries
up to four models (`retried_serially` in telemetry). Compilation and parsing errors,
missing profiles, timeouts, and spawn failures keep their original diagnostics.

### `dbt-schema-verify`

After the agent declares done, runs `altimate-dbt schema-verify --model
<name>` on every modified model. Reports a mismatch if the produced
column shape diverges from the `schema.yml` spec (extra, missing,
reordered, or type-mismatched columns).

**Catches**: column-level drift that wouldn't be caught by `dbt build`
alone — equality tests against the spec would fail later but the
agent has already declared done.
<name>` on every modified model and compares the built table with the
columns its YAML declares. It reports only what dbt's own semantics
establish as a problem, and says which file declares what:

- **Enforced contract.** The model has `contract: {enforced: true}` and
the built table has columns the contract does not list, or lacks columns
it lists. dbt itself rejects such a model. Only column names are compared
here; `dbt build` is what checks a contract's data types.
- **Declared column with tests, not produced.** The YAML declares a column
that has at least one test attached, but the built table does not have it.
The test reads a column that does not exist.
- **Test attachment unverified.** The table lacks a declared column and a
legacy manifest lists the model as a dependency of a test on that column.
Check the test's YAML to establish which relation it reads.

It does **not** report, and never tells the agent to remove, add or
reorder anything because of:

- columns the model produces that the YAML does not list. YAML commonly
documents only some columns (those with tests or descriptions); that is
not an exhaustive list unless a contract is enforced;
- a declared column with no tests and no contract that the model lacks.
This may be a stale or aspirational entry, so it is only a note in the
`altimate-dbt schema-verify` output;
- column order, or a declared `data_type` that differs from the warehouse
type outside a contract (dbt does not enforce either, and adapters spell
types differently);
- a model with no YAML entry or no declared columns (`no-spec`).

The message states the evidence and leaves the decision with the agent
(change the model if the YAML is right, or the YAML if its entry is
stale). The full classification appears in `altimate-dbt schema-verify`
and `altimate-dbt build --model` as `findings` (mismatches and evidence),
`notes` (true observations) and `spec` (declaring file, package, contract
flag). The project-wide `altimate-dbt build` returns these detailed fields
only for models with a mismatch; matching models are summarised as a count.
`columns_extra`, `columns_missing`, `columns_reordered` and
`type_mismatches` remain as the raw diff against the YAML and are not
instructions. A tool error is still a
failure (drift cannot be ruled out) and now includes the first error text.

## Completion gates (shadow mode)

Expand Down Expand Up @@ -234,6 +283,10 @@ When a validator runs, it returns:
// validator-specific extras:
// dbt-tests-pass:
passed?: number
no_tests?: number
no_tests_models?: string[]
unmatched_selector_models?: string[]
retried_serially?: number
failed?: number
errored?: number
spawn_failures?: number
Expand All @@ -244,13 +297,14 @@ When a validator runs, it returns:
match?: number
mismatch?: number
no_spec?: number
diff_not_established?: number
mismatch_models?: string[]
}
}
```

`reason` names the failing models inline (e.g. `"2 of 3 models you
edited have a column-shape mismatch against schema.yml: foo, bar"`).
edited contradict the YAML that declares their columns: foo, bar"`).

## Phased rollout plan

Expand Down Expand Up @@ -284,9 +338,11 @@ default-on. Track progress in
deeper (workspace layouts) are not detected.
- Multiple `dbt_project.yml` candidates pick the alphabetically-first
match deterministically.
- The validator surfaces "schema mismatch" even when the real cause
is "model never materialized" — distinguishing these requires
changes inside `altimate-dbt`.
- A model that was never materialized is reported as a tool error
("has no warehouse table. Build it first"), not as a column finding.
- Retries consume the shared validator budget (three by default) whether
or not the failure text changed. A failure that is re-sent verbatim on
every retry spends the budget without giving the agent new information.

## Writing custom validators

Expand Down
Loading
Loading