Skip to content

WIP [draft]: Admin Layer PR A — the rewrite (create) - #526

Merged
MrParamecium merged 18 commits into
mainfrom
feat/admin-layer-prA
Sep 23, 2026
Merged

MrParamecium merged 18 commits into
mainfrom
feat/admin-layer-prA

Conversation

@MrParamecium

@MrParamecium MrParamecium commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

⛔ Draft until CI's Postgres tier is green on this branch — merge only after that and the checklist below.

Implements docs/superpowers/plans/2026-09-15-admin-layer-redesign-prA-rewrite-create.md (14 tasks), the data model for design §6: user_activity, user_daily_facts, lifecycle_transitions, agent_runs.owner_user_id, the read-time lifecycle calculator, the two-field day claim, population-wide operational signals, and the daily job on its own worker thread. No response shape changes; no table dropped; no frontend file touched; no cohort added.

Acceptance checklist (plan §Acceptance)

  • user_activity, user_daily_facts (user_group DEFAULT 'unknown', nullable operational_reason_code, no cohort) and lifecycle_transitions (UNIQUE per user/day, data_quality) in both DDL constants, twin-parity column check green (Task 1)
  • agent_runs.owner_user_id on both run-history twins, threaded from the authenticated caller through the four hops, NULL for callerless runs (Task 2)
  • Ingestion maintains user_activity with one MIN/MAX upsert per accepted lifecycle event, zero history reads; live singleton maintain_activity=True, project_snapshots=False; seed idempotent, never regresses (Task 3)
  • build_lifecycle_inputs clamps evidence to as_of, prefers the day's own activity, unrouted (Task 4)
  • Ledger/run aggregates on both credits and run-history twins; list_commercial_values/list_credit_activity byte-identical through the owning stores (Task 5)
  • backfill.py has no _get_connection/database_url; its _DIALECT_BRANCH_ALLOWLIST entry removed in the same commit; the four source reads exist on both twins (Task 6)
  • list_operational_signals agrees with get_operational_facts on every seeded state, same call count at 10 and 40 users, no scalar user id; consecutive_failed_terminal_runs single-owner (Task 7)
  • Two-field day lease: double claim refused, 2h-old running reclaimable, completed cursor never re-run, released day retries at once (Task 8)
  • run_daily_facts writes every §6.7 column, excludes admins/excluded/pre-D accounts, survives post-midnight activity, counts D in trailing window, transitions DO UPDATE with data_quality, partial-day retry, late-arrival recompute, retention coordinator per tick (Task 9)
  • Eight weeks of history copied into user_daily_facts as partial rows, idempotent, never overwrites complete rows (Task 10)
  • Own analytics-daily-facts thread, ANALYTICS_DAILY_JOB_INTERVAL_SECONDS (default 300, junk-tolerant), app.py starts it beside the reaper; retention moves off the reaper; maintenance keeps the two repairs (Task 11)
  • Retention sweep past the horizon leaves user_activity untouched; retention.py never names the table (guard test + probe-verified teeth) (Task 12)
  • _raise_service_error logs category-only and raises 503 from exc; admin_analytics in BLOCKING_IO_ROUTER_MODULES; dead users-list stack deleted; route set unchanged (Task 13)
  • Read budget: 23 store calls per claimed tick at 200 and 400 users, zero scalar user-id calls, wall-clock budget + 2.5× ratio floor 0.25s, idle tick = 1 call; rules 1–7 pinned in test_architecture_boundaries.py (Task 14)

Verification

  • Full suite with live Postgres 18 (TEST_POSTGRES_URL): 5496 passed, 48 skipped, 0 failed (same skip count as the pre-branch baseline — all non-PG skips are pre-existing).
  • Twin parity: every new method on both twins; _DUPLICATED_BODIES / _DIALECT_BRANCH_ALLOWLIST respected (allowlist entry for value_repository.py rewritten with the new reason; backfill's entry removed in the same commit as the fix).
  • No git add -A used anywhere; seed backtest.db verified clean after every task.

Deviations from the plan text (all cosmetic, none semantic)

  1. Task 4 clamp fixture off-by-one: the plan's last_active_date=2026-09-04 is 7 days before the computed day — lifecycle.py (≥8 since 3c05049) classifies onboarding, not the documented at_risk. Fixture uses the 3rd.
  2. Task 9 insertion amputated two methods: the plan's "insert after list_existing_source_event_ids" matches up to the first return existing (the early exit), truncating the batch loop on both twins. Restored in a follow-up commit.
  3. Task 11 conftest/app edits: the plan's literal snippets are identical to feat: draw the universal top nav on /admin #490's app.py header text, so the retention-block deletion and worker insertion were applied against the post-feat: draw the universal top nav on /admin #490 header (same code, different surrounding comments).
  4. test_admin_analytics_value_frontend.py no longer exists (deleted by the feat: consolidate admin navigation on /admin #488 nav consolidation); Task 13's verification ran against the surviving conformance oracle (test_admin_analytics_frontend.py, test_app_composition.py).
  5. test_analytics_maintenance.py and test_run_lifecycle_unification.py carried pre-existing source pins contradicting Task 11 (retention-on-reaper); rewritten per the plan's own replacement snippets.

PR B cannot land until this merges and prod writes 8 days of facts — merged-to-main auto-deploys, so the clock starts at merge.

@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agentic-trading-lab Ready Ready Preview Sep 23, 2026 5:23pm UTC

Comment thread dashboard/backend/tests/domain/analytics/test_user_activity.py Fixed
The plan's clamp fixture carried an off-by-one: last_active_date 2026-09-04
is seven UTC days before the day under test, which lifecycle.py (>=8, since
3c05049) classifies onboarding, not at_risk. The fixture now uses the 3rd,
matching the at_risk the test documents.
Comment on lines +31 to +38
from dashboard.backend.domain.credits.repository import (
_assemble_billing_states,
_assemble_commercial_ledger,
_assemble_ledger_day,
_day_bounds,
_unique_user_ids,
_utc_text,
)
The plan's absent-user fixture needed the user row to exist: the per-user
reader lazily creates the credit_accounts row and its FK points at users,
so comparing the two paths on a truly nonexistent id failed before any
signal was computed. The agent-source ordering fixture now pins
created_at — created_at has second resolution and the SQL tie-breaks on
the random agent_id.
The list_daily_subjects insertion matched lazily up to the first
'return existing' — the early-exit inside the if, not the method end —
which amputated the 500-batch loop from list_existing_source_event_ids
on both twins.
The worker owns rollup_day and the retention coordinator (the reaper keeps
only the two throttled snapshot repairs), runs the idempotent user_activity
seed and the eight-week history copy once before its first tick, and its
interval comes from ANALYTICS_DAILY_JOB_INTERVAL_SECONDS (default 300,
junk-tolerant), stripped in conftest and documented in CLAUDE.md.
Comment thread dashboard/backend/domain/analytics/repository.py Fixed
Comment thread dashboard/backend/domain/analytics/repository_postgres.py Fixed
Comment thread dashboard/backend/tests/test_analytics_maintenance.py Fixed
with _worker_lock:
thread, stop = _worker_thread, _worker_stop
_worker_thread = None
_worker_stop = None
Comment thread dashboard/backend/domain/analytics/value_repository.py Fixed
Comment thread dashboard/backend/tests/test_analytics_maintenance.py Fixed
"""Design SS6.9 step 1 / D23: the daily job owns rollup_day now, so the
reaper tick must not be able to write the same rollup rows."""
parameters = inspect.signature(maintenance.run_analytics_maintenance).parameters
source = inspect.getsource(maintenance)
@MrParamecium
MrParamecium marked this pull request as ready for review September 23, 2026 15:05
@MrParamecium

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b08ed2474

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +389 to +392
_compute_day(
stale_day, now=current, store=store, run_history_store=runs, rollup=rollup_fn
)
recomputed.append(stale_day)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Surface failed stale-day recomputations

When a late-arrival recomputation fails, _compute_day returns an outcome with failed_steps, but that result is discarded and the date is still appended to recomputed_dates. This makes the report claim the stale fact was refreshed even though the old rows remain, and callers have no indication that the retry was unsuccessful; capture the outcome and report the failure (or leave the date out) so recomputation errors remain observable.

Useful? React with 👍 / 👎.

…redits import cycle, remove unreachable code and dead names

- daily_facts: a failed stale-day recompute now releases the day lease and
  stays out of recomputed_dates (Codex P2)
- credits: the six shared ledger helpers move to repository_common.py,
  killing the repository_postgres → repository import cycle CodeQL flagged
- analytics twins: the dangling pre-repair loop bodies after
  list_daily_subjects (unreachable since the Task 9 fix) are removed
- dead names: unused pytest import, unused module constant, duplicate
  test definition
@MrParamecium

Copy link
Copy Markdown
Collaborator Author

Review findings addressed in 78d5fc4:

Codex P2 — surface failed stale-day recomputations: _compute_day's outcome is now captured; a failed recompute releases the day lease (so the next tick retries), logs WARNING: analytics.daily_facts.recompute_failed day=… steps=…, and stays out of recomputed_dates — the report no longer claims stale facts were refreshed when they weren't.

CodeQL findings:

  • Cyclic import repository_postgres → repository: the six shared ledger helpers moved to repository_common.py; both twins import them from there. No cycle.
  • Unreachable code after list_daily_subjects in both analytics twins: the dangling pre-repair loop bodies (left over from the Task 9 fix) removed.
  • Unused import (pytest), unused module constant (PLATFORM_CREDENTIAL_ENVIRONMENT_SECRET), duplicate test definition, unused locals in test_analytics_maintenance.py: removed.

All findings fixed; 1951 tests across the touched slices green.

from collections.abc import Iterator, Sequence
from contextlib import contextmanager
from datetime import datetime, timedelta
from datetime import date, datetime, timedelta, timezone
def test_maintenance_no_longer_owns_rollups_or_the_lifecycle_backfill():
"""Design SS6.9 step 1 / D23: the daily job owns rollup_day now, so the
reaper tick must not be able to write the same rollup rows."""
parameters = inspect.signature(maintenance.run_analytics_maintenance).parameters
The discipline guard forbids store calls inside loops; the Codex-P2 fix
put release_projection_day inside the stale-day loop. The lease belongs
to the job row, so one release after the loop carries the same retry
semantics — failed days stay in list_days_needing_recompute and the next
tick retries them.
@MrParamecium
MrParamecium merged commit 9e4c6d7 into main Sep 23, 2026
8 checks passed

This branch was successfully deployed

1 active deployment
Preview — 3897d77a Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants