Add data-quality checks, contracts, and CI for core datasets

[bdevierno1]

What

Adds automated data-quality enforcement for the two core datasets
(operations, billing_sessions) across four dimensions from our
Data Quality Dimensions policy, with YAML data contracts and a GitHub
Actions CI job that gates every push.

New files:

Path	Purpose
backend/data_quality/checks.py	Sync SA primitives: completeness, validity, uniqueness, freshness
backend/data_quality/runner.py	CLI runner — loads contracts, exits 1 on failure
backend/data_quality/contracts/operations.yaml	Data contract for operations table
backend/data_quality/contracts/billing_sessions.yaml	Data contract for billing_sessions table
backend/tests/test_data_quality.py	14 unit tests (all dimensions, pass + fail paths)
.github/workflows/data-quality.yml	CI: contract lint → DQ unit tests → full suite

Why

No automated enforcement of the six quality dimensions means regressions
(null floods, bad status values, stale rows) only surface in production.
This wires checks into CI so failures block merge, and the data contracts
codify the producer's promises for consumers.

Check dimensions

Dimension	Rule	Threshold
Completeness	null rate on NOT-NULL cols	< 0.5 %
Validity	status ∈ allowed set; cost ≥ 0	zero violations
Uniqueness	duplicate PKs	zero
Freshness	MAX(updated_at) age	≤ 3 600 s (Tier-1 SLA)

How to verify

cd backend
# Run the data-quality unit tests
pytest tests/test_data_quality.py -v          # expect 14 passed

# Smoke-test the contract linter locally
python -c "
import pathlib, yaml, sys
REQUIRED = {'dataset','ownership','schema','quality_rules','slas'}
for p in pathlib.Path('data_quality/contracts').glob('*.yaml'):
    doc = yaml.safe_load(p.read_text())
    assert REQUIRED <= doc.keys(), f'{p.name} missing {REQUIRED - doc.keys()}'
print('contracts OK')
"

# Run the runner against an empty local DB (should print 0/0 or all-pass empty tables)
DATABASE_URL=sqlite:///./test_run.db python -m data_quality.runner

The .github/workflows/data-quality.yml job runs all three steps above
on every push/PR. Check the Actions tab after this PR merges.

Rollback

Delete or disable .github/workflows/data-quality.yml to remove the CI
gate. The data_quality/ module has no runtime coupling to the application
(it is not imported by FastAPI routes) — removing it is a pure file delete.

Out of scope

• Accuracy dimension (requires an authoritative reference dataset; deferred)
• Consistency / referential-integrity checks (FK enforcement already handled by SQLite constraints in the app layer)
• Wiring the runner into a scheduled cron job (post-merge follow-up)

---

🤖 Generated with Vecna

Co-Authored-By: Vecna noreply@vecna.ai