v0.8 W65 — Bounded data-quality policies and monitoring

[ammarheidari]

Status

ACTIVE — Slice 1/2, Slice 3 evaluator runtime/corrective, Slice 4A lifecycle contracts, Slice 4B durable SQLite/PostgreSQL store, and Slice 4C bounded monitoring worker are protected-main admitted. API/UI closeout remains.

Scope

• declarative bounded data-quality policy definitions;
• authorized topic/schema/record monitoring;
• safe predicates/checks for completeness/shape/schema/range/null/rate patterns;
• aggregate violation counters/evidence/trends;
• policy lifecycle and operator UI/API.

Safety

• record monitoring is explicit opt-in and separately authorized;
• bounded sampling/window/rate/byte/concurrency with no hidden unlimited consumer;
• masking/data policy applies before exposed evidence;
• no arbitrary JavaScript/Python/user code execution;
• raw payloads are not durably stored by default.

Authority: scope #208; tracker #209; W61/W62 complete.

Activation boundary

Implementation remains bounded by the closed rule grammar and durable aggregate/progress-only evidence model. No implementation may introduce arbitrary-code predicates, unbounded historical scans, consumer-group offset commits, raw-payload persistence or masking bypass.

Admitted slices

Slice 1 / 2

• policy/evidence contracts: PR feat(data-quality): define bounded W65 policy contracts #239 / protected-main 6ea6dfc6ea441b1917cce4a4beb54320a297aabf;
• evaluator/progress contracts: PR feat(data-quality): define bounded W65 evaluator contracts #243 / protected-main e1abda30d3d8ae57b4ddbf983f4bb12e145eed26.

Slice 3

• runtime evaluator: PR feat(w65): add bounded data-quality evaluator runtime #250;
• post-merge correctness corrective: PR fix(w65): harden bounded evaluator policy semantics #251 exact head d44165496f90445952d541f2cddddbf2c85091f6;
• exact-head CI, fresh Codex and CODEOWNER gates completed.

Slice 4A — COMPLETE / PROTECTED-MAIN

• lifecycle/persistence contracts: PR feat(data-quality): add W65 lifecycle persistence contracts #252;
• final exact head: ce6e7060f44d82e9523c1193b2aaf8fb5ca44312;
• protected-main merge baseline: 9572726f9880e82aaf1b836b0a8c1e330c5ca568;
• all exact-head CI gates: SUCCESS;
• final Codex review: CLEAN;
• fresh CODEOWNER approval: APPROVED;
• lifecycle create vs revision-guarded replace semantics admitted;
• policy list cursor/truncation bounded;
• durable evidence requires validated evidence/progress pairing and a closed server-derived source identity.

Slice 4B — COMPLETE / PROTECTED-MAIN

• durable lifecycle/evidence store: PR feat(data-quality): add W65 durable lifecycle store #253;
• final exact head: db3feabe59f8b4139b5dcf4900afe9094ceae952;
• protected-main merge baseline: 8d066a2dacdbf0c647d5be81dc61eea8c8680e34;
• SQLite standalone + PostgreSQL HA-compatible persistence admitted;
• optimistic revision CAS, stable pagination, idempotent aggregate/progress evidence persistence admitted;
• explicit persistence DTO used instead of loosening domain constructors;
• shared kafdeck_schema_info migration lock protocol aligned across persistence initializers;
• all exact-head CI, v05 persistence, CodeQL, Codex and CODEOWNER gates clean.

Slice 4C — COMPLETE / PROTECTED-MAIN

• bounded monitoring worker: PR feat(data-quality): add W65 bounded monitoring worker #254;
• final exact head: 40e447c4a503bacde36d1167f14271f5c33ac9f9;
• protected-main merge baseline: a2685bf54778d113878f1a819bfa65bb4a0cf476;
• exact-head Quality / v03 / v04 / Dependency / Release / CodeQL: SUCCESS;
• final Codex review: CLEAN;
• fresh CODEOWNER approval: APPROVED;
• authorization-before-read, bounded rolling policy rate ceilings, fair cluster/policy/partition rotation, truthful partial/deadline/cancellation semantics and durable continuation behavior are admitted.

API/UI activation is now the remaining W65 closeout scope.

[ammarheidari]

W65 Slice 1 canonical head is now 8ad70afe95679bf2b422f207540250f87b884e79 on protected-main 4beeab13ab9d251024d95aa28bcf4c6b8f3e9f8a. Current contract includes pre-normalization input caps and hard evidence throughput ceilings; fresh exact-head gates are active.

[ammarheidari]

W65 Slice 1 final exact head: fd7c8f43ae3d9c14dc0ecd9a10c3181b10d588f3. Raw string/cardinality/throughput bounds are closed; fresh gates active.

[ammarheidari]

W65 Slice 1 closeout

• PR feat(data-quality): define bounded W65 policy contracts #239 final head fd7c8f43ae3d9c14dc0ecd9a10c3181b10d588f3;
• protected-main merge 6ea6dfc6ea441b1917cce4a4beb54320a297aabf;
• exact-head CI/CodeQL/CODEOWNER/Codex gates: clean;
• post-merge quality #1654: SUCCESS;
• post-merge CodeQL #1496: SUCCESS.

W65 Slice 1 bounded policy/evidence contract foundation is COMPLETE / protected-main verified. Runtime evaluator/reader work remains a later bounded slice.

[ammarheidari]

W65 Slice 1 closeout

• PR feat(data-quality): define bounded W65 policy contracts #239 final exact head: fd7c8f43ae3d9c14dc0ecd9a10c3181b10d588f3;
• protected-main merge: 6ea6dfc6ea441b1917cce4a4beb54320a297aabf;
• closed typed rule grammar, explicit topic/partition scope and aggregate-only evidence contract;
• raw strings/cardinality are bounded before normalization/materialization;
• exact-duration record/byte ceilings enforced;
• no arbitrary code execution, hidden consumer-group join, offset commit or raw-payload durable evidence;
• exact-head quality/dependency/CodeQL/supply-chain/benchmarks: SUCCESS;
• CODEOWNER approval and final Codex review: CLEAN;
• post-merge quality #1654: SUCCESS;
• post-merge CodeQL #1496: SUCCESS.

W65 remains ACTIVE for later bounded evaluator/reader/runtime slices; Slice 1 contract foundation is COMPLETE and protected-main verified.

[ammarheidari]

W65 Slice 2 admitted next scope

Slice 1 contract foundation is complete on protected main 6ea6dfc6ea441b1917cce4a4beb54320a297aabf.

Next dependency-safe slice is now defined as bounded evaluator + progress-state contracts, with these hard boundaries:

• evaluator consumes only an explicitly authorized bounded record window;
• no Kafka consumer-group offset commit and no hidden long-lived consumer ownership;
• per-cycle records/bytes/duration/concurrency are server-owned and cannot exceed the Slice 1 hard budget;
• closed rule grammar only; no script/CEL/JQ/regex execution extension in this slice;
• evaluation output is aggregate counters + rule evidence state only;
• no raw key/value/header payload is durable state;
• cancellation/timeout produces Partial/Unavailable evidence rather than fabricated zero;
• durable progress may store policy/version/topic/partition/window/offset-range metadata, never payload content;
• masking remains mandatory before any future operator-visible exemplar path.

Implementation of the evaluator can begin after current W63/W66 corrective gates stabilize protected main, avoiding unnecessary branch rebases.