diff --git a/CLAUDE.md b/CLAUDE.md index c28ea8a5..ba21ea52 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -77,6 +77,7 @@ python dashboard/scripts/backtest_hourly_agent.py # main hourly agent backte - `ATL_BACKTEST_WORKER` (**never set by an operator**): `run_backtest_background` sets it to `1` in a dashboard backtest child's environment and nowhere else. Every Postgres store twin that owns a schema — **eleven of the twelve pairs in `tests/test_store_twin_parity.py::_TWINS`**; the twelfth, `PostgresValueAnalyticsStore`, composes the other stores and has no `_init_schema`, which that file's `_NO_OWN_DDL_TWINS` already records — routes its constructor's DDL through `db_url.init_schema_unless_worker`, which skips it on the literal `1` and prints ` backend: schema init skipped (backtest worker)`, and otherwise runs it, times it into `db_url.schema_init_seconds()` and prints ` backend: schema init s`. The parent ran that DDL at import, and the child was paying a pooled Neon checkout plus a batch of round trips per store before reading a bar: on today's import graph six twins construct in a child — `PostgresBacktestDatabase` (55 statements), `PostgresModelProviderStore` (18), `PostgresAnalyticsStore` (16), `PostgresAgentStore` (14), `PostgresCreditsStore` (3), `BrokerConnectionStorePostgres` (1), 107 statements over six checkouts. A warm schema does not shrink that: 72 of the 107 are `ADD COLUMN IF NOT EXISTS` and the other five `ALTER`s are constraint drops and re-adds, so they round-trip whether or not the column is there. ⚠ **`_init_schema` is not purely DDL, so the guard skips more than its name says.** Three of the eleven twins run data statements inside it — `PostgresCreditsStore` seeds the `default` grant pool and backfills settled reservation amounts plus ledger bucket/operation columns, `PostgresModelProviderStore` scrubs `api_key_enc` on revoked credentials and seeds `SEEDED_PROVIDERS`, `PostgresUserStore` repairs `users.user_group` values outside the allowed set. A child skips those too, and they are safe to skip **only because the parent applies them at boot before it spawns anything** — a property of how dashboard backtests are launched rather than of this code, so a worker that ever ran without a parent ahead of it (a bare CLI run with the flag exported, a future worker-only service) would silently not have them. `test_backtest_worker_schema_skip.py` holds that set as a declared registry and fails in both directions, so a fourth cannot arrive unnoticed and a stale entry cannot linger; read it before adding a migration to any guarded `_init_schema`. **Both log directions are deliberate:** the timed line is the parent's own boot cost, which is the only pre-change baseline the deploy that removes the child's copy can still produce, and a silent skip would look exactly like a build with no guard. ⚠ **All eleven are guarded, not just those six, on purpose.** Which stores a child imports is an accident of the import graph, and the accident arrives by **two independent routes**. The script imports `db` deliberately (`backtest_hourly_agent.py:51`) and then reaches `credits_store` and `model_provider_store` through the two service imports at `:188-189` (`credits/service.py:27`, `model_providers/service.py:38`), with `model_providers/repository.py:13-14` pulling in `agents/repository.py` and `brokers/repository.py` — five of the six twins, with `domain/analytics/service.py` never imported. Only the sixth, `analytics_store`, needs the analytics chain those same service modules also pull in (`domain/analytics/instrumentation.py:15` → `analytics/service.py:17`, whose module-level `analytics_service = _build_analytics_service()` at `:253` then constructs a value store that resolves `credits_store`, `model_provider_store`, `agent_store` and `run_store` as constructor-time defaults — the first three already built by route one, `run_store` (like `analytics_store`) built here for the first time). `run_store` does not move the count: it is a seventh module-level singleton (`domain/runs/repository.py:469`), not a seventh twin — `domain/runs/` has no `repository_postgres.py`, so protocol runs are SQLite either way and nothing here guards them. Neither route is legible from the child's own import block, which names `db` and two service modules and nothing else, so one new module-level import in the engine's reach would otherwise re-add an unguarded payer with nothing going red. `test_backtest_worker_schema_skip.py` pins the invariant that survives that, in the AST rather than as a substring: no non-test `*_postgres.py` calls **any** `self._…schema…()` method straight from `__init__`, any twin defining a schema method names `init_schema_unless_worker`, and guarded ∪ exempt equals `_TWINS`. It matched the literal `"self._init_schema()"` until 2026-09-21, which is a check on one spelling — a twin calling `self._ensure_schema()` passed it while doing the forbidden thing. One shared helper rather than eleven copies, for the reason `db_url.py`'s header gives about its scrubber — an inverted copy fails in the *parent*, as an `UndefinedTable` on prod from a store nobody edited, and a dropped log line makes "skipped" and "never constructed" the same empty stdout. Named for the process role, not the DDL, so nobody exports it globally and later child-only behaviour has a home. The child also publishes pre-loop **phases** to its progress file (`engine.py:PROGRESS_PHASES`: `starting`, `loading_bars`, `indicators`, `first_decision`, `running`, `saving`), which `/backtest/status` turns into the card's sentence and which double as the start-up measurement — read `phases[]` off a run's progress file for the per-phase timings, or the child's `⏱ phase …` stdout lines out of the service log, which is the copy that survives (the parent unlinks the progress file when the run ends). Every finished entry carries `duration_seconds`, measured on `time.monotonic()` — do **not** re-derive a duration by differencing `started_at`/`ended_at`, which are deliberately wall-clock stamps kept for correlating against a log line: `time.time()` is not monotonic, so an NTP step between two reads distorts the interval and a backward step yields a negative one. ⚠ `starting` is the one phase whose duration *is* that wall-clock difference, and it cannot be otherwise — it spans the parent→child boundary (it opens at the parent's `--launched-at` and closes in the child) and `monotonic()` has a **per-process epoch**, so the parent's reading and the child's are not comparable at all. `_init_progress_phases` leaves the steady mark `None` to say exactly that, and `_set_progress_phase` reads `None` as "fall back to the wall clock"; every later phase is entirely in-child and uses the steady clock. The `starting` entry additionally carries `child_entered_at`, `imports_done_at` and `schema_init_seconds`, splitting that one interval into spawn, imports-including-stores, of-which-DDL and preflight: one phase name, four numbers, because a single figure covering all four cannot say which of them an optimisation moved. ⚠ **That exemption covers the phase's total and exactly one of those four splits — do not read it as covering the record.** Only `spawn+interpreter` (`child_entered_at - started_at`) reaches back into the parent; `imports+stores` and `preflight` both begin and end inside the child, so both are measured on the steady clock and published as `imports_seconds` and `preflight_seconds` beside the stamps. The launch script takes a second, steady pair for this (`CHILD_ENTERED_STEADY`/`IMPORTS_DONE_STEADY`, handed over in `startup_clock`) and the engine closes `preflight` against its own `steady_clock()` read, which is the same clock in the same process. Those two marks are deliberately **not** published — a raw `monotonic()` reading has a per-process epoch and is meaningless in a file — so what `phases[]` gains is the two durations, not the marks. The wall-clock fallback for a caller that hands over only the three stamps is kept and is *silent*, so it is pinned in the source rather than at runtime: `test_backtest_launch_phases.py` AST-checks that the script still hands over the steady pair, that both are `time.monotonic()` reads, and that they bracket the imports. Shipping the #509 fix with these two differences still in the breakdown line is the mistake that guard exists to prevent repeating. `loading_bars` carries `fetch_seconds` for the same reason and prints `fetch s | aggregate+verify s` on the transition out: the phase is one name, but with the bar cache warm the residual **is** the aggregation cost, measured rather than inferred — which is what decides whether caching the aggregated output is worth a second change. `record_phase_metric` attaches a number to whichever phase is open — and drops it when none is — and extras are cleared on every transition, so a figure can never be reported against the wrong phase. The startup clock is seeded only when `--launched-at` actually opens `starting`; a bare CLI run passes the clock without a launch time, and those keys must not surface on `loading_bars`. `tests/conftest.py` strips the flag. - `MAX_AI_HEDGE_FUND_TRADING_DAYS` (optional, default **10**, range 0–60, **0 disables the hosted runtime**): the pre-flight bound `POST /backtest/run` applies to an AI Hedge Fund window (`_enforce_ai_hedge_fund_window` in `api/routers/backtests.py`). The hosted runtime spends one upstream **subprocess per trading day**, each loading its own lookback window beside a parent already holding uvicorn, FastAPI and the Postgres pools, and the kernel's victim is the whole web process — one backtest denies service to everyone (issue #308). An over-long window answers **422** naming the bound and the requested size (the caller can shorten it); `0` answers **503** (the deployment's own configuration, nothing the caller can change). Measured in trading days via `_estimated_decision_days`, the same upper bound `_backtest_subprocess_timeout` sizes the parent budget from, so the two cannot disagree about how big a window is. Junk/negative/over-ceiling values log and fall back rather than raising at import — a bare `int()` at module scope in this very module once killed app boot. **The default stays 10 even though prod is now 2GB** (2026-09-11): nothing has ever measured one child's resident set, so 10 was a guess against the old ceiling and a larger guess against a larger ceiling is the same mistake with more RAM behind it. Profile a run, then raise it from the Render dashboard — no deploy needed. - `MAX_LEGACY_ACTIVE_PER_SESSION` / `MAX_LEGACY_ACTIVE_GLOBAL` (optional, defaults **5** / **50**, 0 disables): concurrency budgets for the legacy `/api/v1/backtest/*` surface. That surface authenticates nothing — `_require_session` accepts any non-empty `X-Session-Id` — and writes **no `protocol_runs` row**, so the per-agent, per-account and global protocol caps are all blind to its runs; before these it was the one unbounded path into the same engine. The per-session budget bounds a looping client (its key is caller-chosen, so it is not a bound against someone who rotates it); the global one is the memory bound that holds regardless, since every live session pins a loaded bar window. Enforced via `start_backtest(enforce_session_cap=True)`, which **only** the legacy route passes — the protocol surfaces reach the same function through `run_service.create_run`, which has already applied its three caps. +- `ANALYTICS_DAILY_JOB_INTERVAL_SECONDS` (optional, default **300**, range 5–3600): how often the analytics daily-facts worker (`domain/analytics/daily_job.py`, started from `app.py` beside the run reaper) wakes to ask whether yesterday's `user_daily_facts` row set is due. It runs on **its own thread**, not the 60-second reaper tick — the reaper exists to keep run heartbeats fresh against `RUN_HEARTBEAT_STALE_SECONDS`, and a whole-population batch across three databases on that thread would let a slow analytics night mark live runs as orphaned (design D23). An idle wake costs exactly one store call (the refused day claim on `analytics_projection_jobs`), so the interval buys freshness after midnight, not cost. Junk or out-of-range values log and fall back to 300 rather than raising at import — a bare `int()` at module scope has killed app boot in this repo before. `tests/conftest.py` strips it. The worker also runs the idempotent `user_activity` seed and the eight-week history copy (`domain/analytics/facts_migration.py`) once before its first tick, and owns `rollup_day` and the analytics retention coordinator; the reaper keeps only the two throttled snapshot repairs in `maintenance.py` until PR B deletes them. Design: `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md` §6.9, §6.12. - `LEADERBOARD_DAILY_AUTO_DEPLOY` (optional, **strict opt-in — this one spends money**): when truthy, serving the *public, unauthenticated* `GET /api/v1/leaderboard?period=daily` may start a background thread that runs `deploy_model_run` for every competition LLM entry — real billable API calls initiated by an anonymous request. It is therefore off unless explicitly set (`1`/`true`/`yes`/`on`); do **not** restore the old "on whenever `RENDER` is unset" default, which armed it on the Docker image, every self-host and fork, and the test suite (`tests/conftest.py` deliberately strips `RENDER`, and strips this var too). Prod previously relied on the nightly refresh job instead — **but that schedule is paused as of PR #352** (see `LEADERBOARD_DAILY_REFRESH_SECRET` above), so prod currently runs *neither* this flag nor the cron. Nothing refreshes the daily board automatically today, by design: the Live Trading Leaderboard that replaced its tab is a Season 0 preview with no advance engine yet. Do not "restore" automation here to close the gap — arming this flag is the anonymous-billable-request path the strict opt-in exists to prevent. Note the in-progress guard and the per-window state file (`storage/data/leaderboard_daily_refresh.json`) are **per-process**: fine for the current single-instance Render deploy, but a multi-replica deploy would duplicate model deploys. - Alpaca paper-trading credentials also live in `credentials/alpaca.json` (gitignored; see `credentials/alpaca.json.example`). @@ -329,4 +330,4 @@ The FinSearch news adapter (`dashboard/backend/integrations/news_sentiment.py`) **Merging to Open-Finance-Lab `main` auto-deploys prod**: a CI job hits the Render Deploy Hook once backend tests pass on `main` (PR #95, live since 2026-07-11) — no manual trigger or fork-sync needed. (Render's own `autoDeploy` is `"no"`; the CI hook drives every deploy.) - **Phantom `test_deleted_shim_is_not_importable` failures = stale bytecode, not a regression.** If those cases fail locally with `DID NOT RAISE ModuleNotFoundError`, it's leftover `dashboard/backend/{engines,services}/__pycache__/*.pyc` from the pre-refactor layout, which Python resolves as a PEP-420 namespace package. The dirs are untracked so CI is green; `rm -rf dashboard/backend/engines dashboard/backend/services` clears it. - **User accounts were silently lost on every prod redeploy until 2026-07 (see `docs/superpowers/plans/2026-07-08-user-account-persistence-fix.md`).** `users.py` originally shared `DB_PATH` with backtest data; on the live Render service (then free tier, `disk: null` — still true today, `DATABASE_PATH` unset) that file resets to the git-committed seed DB on every deploy, deleting the `users`/`auth_sessions` tables with no error surfaced anywhere. The fix is an optional Postgres backend selected via `USERS_DATABASE_URL` — set it in prod (see the **Prod deploy reality** bullet above); leave it unset for local dev/tests, which keep using SQLite exactly as before. The same fix was extended to agents, agent versions, and strategies in 2026-07 via `CONTENT_DATABASE_URL` (see `docs/superpowers/specs/2026-07-15-agent-strategy-persistence-design.md`) — before that, every registered agent and issued API key died on each deploy, breaking all SDK/Discord integrations, with `resolve_api_key()` as the sole auth path for `/api/v1` and `/api/v2`. -- **Anything registered with `register_reaper_sweep` runs on the same 60-second tick** (`RUN_REAPER_INTERVAL_SECONDS`, `domain/runs/service.py`) — there is no per-sweep interval, so a sweep that does per-user work multiplies its cost by every tick, not by however slow that work actually is. The analytics snapshot-repair sweep registered there (`domain/analytics/maintenance.py::run_analytics_maintenance` → `states.py::repair_stale_snapshots`) was one of two September 2026 outage burners for exactly this reason; it is throttled to a 24-hour staleness window by PR 0 and deleted outright by PR A once `user_daily_facts` replaces `user_analytics_snapshots`. The admin layer's full architecture — the analytics data model, the daily job, the nine `/api/admin/analytics` endpoints, and the `/admin` page — is designed at `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md`; read it before touching anything under `domain/analytics/` or the admin frontend. +- **Anything registered with `register_reaper_sweep` runs on the same 60-second tick** (`RUN_REAPER_INTERVAL_SECONDS`, `domain/runs/service.py`) — there is no per-sweep interval, so a sweep that does per-user work multiplies its cost by every tick, not by however slow that work actually is. The analytics snapshot-repair sweep registered there (`domain/analytics/maintenance.py::run_analytics_maintenance` → `states.py::repair_stale_snapshots`) was one of two September 2026 outage burners for exactly this reason; it is throttled to a 24-hour staleness window by PR 0, left on the reaper by PR A (which gives the daily-facts job its own worker thread, `domain/analytics/daily_job.py`, precisely so that job is *not* another reaper sweep, and moves `rollup_day` and the retention coordinator onto it), and deleted outright by PR B once every read path has moved to `user_daily_facts`. The admin layer's full architecture — the analytics data model, the daily job, the nine `/api/admin/analytics` endpoints, and the `/admin` page — is designed at `docs/superpowers/specs/2026-09-15-admin-layer-redesign-design.md`; read it before touching anything under `domain/analytics/` or the admin frontend. diff --git a/dashboard/backend/api/routers/admin_analytics.py b/dashboard/backend/api/routers/admin_analytics.py index ead325a3..50403683 100644 --- a/dashboard/backend/api/routers/admin_analytics.py +++ b/dashboard/backend/api/routers/admin_analytics.py @@ -15,7 +15,6 @@ AnalyticsActivityPage, AnalyticsOverview, AnalyticsQueryService, - AnalyticsUserFilters, get_analytics_query_service, get_value_analytics_query_service, ) @@ -51,8 +50,6 @@ _MODEL_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/\-:]{0,255}$") _POSITIVE_INTEGER_PATTERN = re.compile(r"^[0-9]+$") _USER_STATES = {"blocked", "needs_attention", "dormant", "onboarding", "active"} -_USER_SORTS = {"last_activity", "joined_at", "recent_runs", "recent_failures"} -_SORT_ORDERS = {"asc", "desc"} _ACTIVITY_SECTIONS = {"timeline", "runs", "usage", "sessions"} _LIFECYCLE_SEGMENTS = {"new", "onboarding", "growing", "core", "at_risk", "dormant"} _OPERATIONAL_STATES = {"blocked", "needs_attention", "healthy"} @@ -295,74 +292,6 @@ def _value_user_filters(request: Request) -> tuple[UserValueFilters, int, int]: return filters, limit, offset -def _user_filters(request: Request) -> tuple[AnalyticsUserFilters, int, int]: - values = _query_values( - request, - { - "q", - "status", - "last_activity_from", - "last_activity_to", - "sort", - "order", - "limit", - "offset", - "include_internal", - }, - ) - query = values.get("q") - if query is not None and len(query) > 100: - _invalid_query() - status = values.get("status") - if status is not None and status not in _USER_STATES: - _invalid_query() - sort = values.get("sort", "last_activity") - if sort not in _USER_SORTS: - _invalid_query() - order = values.get("order", "desc") - if order not in _SORT_ORDERS: - _invalid_query() - - from_date = ( - _parse_date(values["last_activity_from"]) - if "last_activity_from" in values - else None - ) - to_date = ( - _parse_date(values["last_activity_to"]) - if "last_activity_to" in values - else None - ) - if from_date is not None and to_date is not None and to_date < from_date: - _invalid_query() - activity_start = _utc_midnight(from_date) if from_date else None - activity_end = ( - _exclusive_date_end(to_date) - timedelta(microseconds=1) - if to_date - else None - ) - - try: - filters = AnalyticsUserFilters( - q=query, - status=status, - last_activity_from=activity_start, - last_activity_to=activity_end, - sort=sort, - order=order, - include_internal=( - _parse_bool(values["include_internal"]) - if "include_internal" in values - else False - ), - ) - except (ValidationError, ValueError): - _invalid_query() - limit = _parse_integer(values.get("limit", "50"), minimum=1, maximum=100) - offset = _parse_integer(values.get("offset", "0"), minimum=0) - return filters, limit, offset - - def _activity_query(request: Request) -> tuple[str, int, str | None]: values = _query_values(request, {"section", "limit", "cursor"}) section = values.get("section") @@ -380,7 +309,12 @@ def _raise_service_error(exc: Exception) -> Never: raise HTTPException(status_code=404, detail=_NOT_FOUND_DETAIL) from None if isinstance(exc, (ValidationError, ValueError)): raise HTTPException(status_code=422, detail=_INVALID_QUERY_DETAIL) from None - raise HTTPException(status_code=503, detail=_UNAVAILABLE_DETAIL) from None + # Category only, never the message: a psycopg OperationalError carries the + # DSN and a ValidationError carries field values. The class name is what + # tells "the pool is exhausted" from "a bad SQL statement" in prod logs, + # which the bare `from None` 503 never could (design D24, SS4.4). + print(f"ERROR: admin_analytics.unhandled category={type(exc).__name__[:80]}") + raise HTTPException(status_code=503, detail=_UNAVAILABLE_DETAIL) from exc def _record_access( diff --git a/dashboard/backend/api/routers/backtests.py b/dashboard/backend/api/routers/backtests.py index 51745cce..b90b7605 100644 --- a/dashboard/backend/api/routers/backtests.py +++ b/dashboard/backend/api/routers/backtests.py @@ -1576,6 +1576,7 @@ def run_backtest_background( # decoding it at timeout time would fail even if we had the key. Used only # to decide whether a timeout has a Credits cost worth reporting. billing_mode: Optional[str] = None, + owner_user_id: Optional[int] = None, ): """Run backtest in background thread. @@ -1731,6 +1732,8 @@ def run_backtest_background( # how that gap becomes its measured `starting` phase. "--launched-at", f"{launched_at:.3f}", ] + if owner_user_id is not None: + cmd += ["--owner-user-id", str(int(owner_user_id))] # Simulation capital is independent of the agent's portfolio sleeve. cmd += ["--initial-capital", str(resolve_initial_capital(initial_capital))] @@ -3522,6 +3525,10 @@ def run_backtest_endpoint( if execution_handoff_payload is not None and billing_mode is not None else None ), + # The caller's OWN account, the same user_id _backtest_owner_key + # bills the slot to -- never the session the results file under, + # which for a built-in agent is the agent's, not the caller's. + "owner_user_id": user_id, **({"universe_selection": universe_selection} if universe_selection is not None else {}), }, daemon=True diff --git a/dashboard/backend/app.py b/dashboard/backend/app.py index 3b2501b0..3d58f57e 100644 --- a/dashboard/backend/app.py +++ b/dashboard/backend/app.py @@ -314,18 +314,6 @@ def bar_cache_background(): except Exception as e: print(f"⚠️ legacy session sweep registration error: {e}") - try: - from dashboard.backend.domain.analytics.retention import ( - analytics_retention_coordinator, - ) - from dashboard.backend.domain.runs.service import register_reaper_sweep - register_reaper_sweep(analytics_retention_coordinator.run_if_due) - print("🧹 Analytics retention sweep registered with the reaper") - except Exception as e: - print( - "WARNING: analytics.retention_registration_failed " - f"category={type(e).__name__}" - ) try: from dashboard.backend.domain.analytics.maintenance import ( @@ -360,6 +348,25 @@ def bar_cache_background(): f"category={type(e).__name__}" ) + try: + # Admin layer redesign PR A (design D23, SS6.9): the daily-facts job + # runs on its own thread, not as a reaper sweep -- a whole-population + # batch across three databases on the heartbeat thread would let a + # slow analytics night mark live runs as orphaned. The worker also + # owns rollup_day and the retention coordinator now, and runs the + # idempotent user_activity seed + history copy once before ticking. + from dashboard.backend.domain.analytics.daily_job import ( + start_daily_facts_worker, + ) + from dashboard.backend.domain.analytics.facts_migration import ( + run_startup_migrations, + ) + run_startup_migrations() + start_daily_facts_worker() + print("📊 Analytics daily-facts worker started") + except Exception as e: + print(f"⚠️ Analytics daily-facts worker start error: {e}") + try: from dashboard.backend.domain.runs.service import start_reaper start_reaper() diff --git a/dashboard/backend/database.py b/dashboard/backend/database.py index a01930db..53ff8992 100644 --- a/dashboard/backend/database.py +++ b/dashboard/backend/database.py @@ -10,6 +10,7 @@ import json import os from pathlib import Path +from datetime import date, timedelta from typing import List, Dict, Optional, Any from dashboard.backend.paths import DEFAULT_DB_PATH @@ -146,6 +147,7 @@ def _init_schema(self): output_tokens INTEGER DEFAULT 0, est_cost_usd REAL DEFAULT 0, metadata TEXT, + owner_user_id INTEGER, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ) @@ -369,6 +371,11 @@ def _migrate_schema(self): "ALTER TABLE agent_runs ADD COLUMN est_cost_usd REAL DEFAULT 0"), ("metadata", "ALTER TABLE agent_runs ADD COLUMN metadata TEXT"), + # Analytics attribution (design §6.4): the authenticated caller + # who started a dashboard backtest. Nullable and never + # backfilled -- scheduled leaderboard deploys have no caller. + ("owner_user_id", + "ALTER TABLE agent_runs ADD COLUMN owner_user_id INTEGER"), ] for col_name, add_column_sql in token_columns: if col_name not in columns: @@ -642,7 +649,8 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, input_tokens: int = 0, output_tokens: int = 0, est_cost_usd: float = 0.0, - metadata: Optional[Dict[str, Any]] = None) -> None: + metadata: Optional[Dict[str, Any]] = None, + owner_user_id: Optional[int] = None) -> None: """Insert a new backtest run with session_id, LLM model and token-cost tracking. ``llm_calls`` and ``llm_decisions`` are not two spellings of one number: @@ -662,14 +670,15 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, initial_equity, final_equity, total_return, sharpe_ratio, max_drawdown, num_trades, llm_model, llm_calls, llm_decisions, input_tokens, output_tokens, - est_cost_usd, metadata) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + est_cost_usd, metadata, owner_user_id) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) """, (run_id, session_id, agent_name, mode, start_date, end_date, initial_equity, final_equity, total_return, sharpe_ratio, max_drawdown, num_trades, llm_model, llm_calls, llm_decisions, input_tokens, output_tokens, est_cost_usd, - json.dumps(metadata) if metadata is not None else None)) + json.dumps(metadata) if metadata is not None else None, + owner_user_id)) conn.commit() conn.close() @@ -920,6 +929,42 @@ def get_runs_by_mode(self, mode: str) -> List[Dict]: return [self._parse_run_row(dict(row)) for row in rows] + def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: + """Operator-funded model cost per owner for runs updated on ``day``, in micro-USD. + + The daily job's run step (design SS6.9 step 3): one statement grouped + by ``owner_user_id``, parameterised by two day bounds and nothing else. + Rows with a NULL owner -- everything before Task 2's column, and every + scheduled leaderboard deploy -- are skipped rather than attributed to + anyone. ``updated_at`` is CURRENT_TIMESTAMP text + ("YYYY-MM-DD HH:MM:SS", UTC) on both twins, so the bounds are text of + the same shape and compare correctly. + + ``est_cost_usd`` is a float; converted with ``round(value * 1_000_000)`` + and clamped at zero so a negative stored value cannot violate + ``user_daily_facts.operator_cost_micro``'s CHECK. + """ + start = f"{day.isoformat()} 00:00:00" + end = f"{(day + timedelta(days=1)).isoformat()} 00:00:00" + conn = self._get_connection() + cursor = conn.cursor() + cursor.execute( + """ + SELECT owner_user_id, COALESCE(SUM(est_cost_usd), 0) AS cost_usd + FROM agent_runs + WHERE owner_user_id IS NOT NULL + AND updated_at >= ? AND updated_at < ? + GROUP BY owner_user_id + """, + (start, end), + ) + rows = cursor.fetchall() + conn.close() + return { + int(row["owner_user_id"]): max(0, round(float(row["cost_usd"] or 0) * 1_000_000)) + for row in rows + } + def insert_trades(self, run_id: str, trades: List[Dict[str, Any]]) -> None: """Batch insert trade records for a backtest run.""" if not trades: diff --git a/dashboard/backend/database_postgres.py b/dashboard/backend/database_postgres.py index 80c1a895..e0197a17 100644 --- a/dashboard/backend/database_postgres.py +++ b/dashboard/backend/database_postgres.py @@ -42,6 +42,7 @@ from __future__ import annotations import json +from datetime import date, timedelta from typing import Any, Dict, List, Optional from dashboard.backend.database import ( @@ -126,6 +127,7 @@ def _init_schema(self) -> None: output_tokens INTEGER DEFAULT 0, est_cost_usd DOUBLE PRECISION DEFAULT 0, metadata TEXT, + owner_user_id INTEGER, created_at TEXT NOT NULL {created_at_default}, updated_at TEXT NOT NULL {created_at_default}, baseline_djia_run_id TEXT, @@ -275,6 +277,9 @@ def _init_schema(self) -> None: cur.execute( "ALTER TABLE agent_runs ADD COLUMN IF NOT EXISTS metadata TEXT" ) + cur.execute( + "ALTER TABLE agent_runs ADD COLUMN IF NOT EXISTS owner_user_id INTEGER" + ) cur.execute( "ALTER TABLE backtest_decisions ADD COLUMN IF NOT EXISTS " @@ -487,7 +492,8 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, input_tokens: int = 0, output_tokens: int = 0, est_cost_usd: float = 0.0, - metadata: Optional[Dict[str, Any]] = None) -> None: + metadata: Optional[Dict[str, Any]] = None, + owner_user_id: Optional[int] = None) -> None: """Insert or refresh a backtest run. Carries divergences 1-3 from the module docstring, all of them @@ -549,8 +555,8 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, initial_equity, final_equity, total_return, sharpe_ratio, max_drawdown, num_trades, llm_model, llm_calls, llm_decisions, input_tokens, output_tokens, - est_cost_usd, metadata) - VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) + est_cost_usd, metadata, owner_user_id) + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer) ON CONFLICT (run_id) DO UPDATE SET session_id = EXCLUDED.session_id, agent_name = EXCLUDED.agent_name, @@ -570,6 +576,7 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, output_tokens = EXCLUDED.output_tokens, est_cost_usd = EXCLUDED.est_cost_usd, metadata = EXCLUDED.metadata, + owner_user_id = EXCLUDED.owner_user_id, updated_at = to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD HH24:MI:SS') """, ( @@ -579,6 +586,7 @@ def insert_run(self, run_id: str, session_id: str, agent_name: str, mode: str, llm_calls, llm_decisions, input_tokens, output_tokens, est_cost_usd, json.dumps(metadata) if metadata is not None else None, + owner_user_id, ), ) @@ -1019,6 +1027,28 @@ def get_runs_by_mode(self, mode: str) -> List[Dict]: rows = cur.fetchall() return [BacktestDatabase._parse_run_row(row) for row in rows] + def aggregate_operator_cost_for_day(self, day: date) -> Dict[int, int]: + """See the SQLite twin.""" + start = f"{day.isoformat()} 00:00:00" + end = f"{(day + timedelta(days=1)).isoformat()} 00:00:00" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT owner_user_id, COALESCE(SUM(est_cost_usd), 0) AS cost_usd + FROM agent_runs + WHERE owner_user_id IS NOT NULL + AND updated_at >= %s AND updated_at < %s + GROUP BY owner_user_id + """, + (start, end), + ) + rows = cur.fetchall() + return { + int(row["owner_user_id"]): max(0, round(float(row["cost_usd"] or 0) * 1_000_000)) + for row in rows + } + def get_trades(self, run_id: str) -> List[Dict]: """Get all trades for a run. diff --git a/dashboard/backend/domain/agents/repository.py b/dashboard/backend/domain/agents/repository.py index 8aada3d9..5f53e0d8 100644 --- a/dashboard/backend/domain/agents/repository.py +++ b/dashboard/backend/domain/agents/repository.py @@ -16,7 +16,7 @@ import uuid from datetime import datetime, timezone from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Sequence from dashboard.backend.database import DB_PATH from dashboard.backend.db_url import describe_database_url @@ -702,6 +702,56 @@ def count_agents(self) -> int: conn.close() return int(row["n"] if row else 0) + + def list_agent_source_rows(self) -> List[Dict[str, Any]]: + """agent_id, session_id, owner_user_id, created_at for every agent. + + The analytics backfill's ownership source (design SS6.14): it used to + read these four columns through this store's connection. Guest agents + (NULL owner) are returned and skipped by the caller, so the caller can + count them as unmapped rather than silently losing them. + """ + conn = self._get_connection() + cursor = conn.cursor() + cursor.execute( + """ + SELECT agent_id, session_id, owner_user_id, created_at + FROM external_agents + ORDER BY created_at, agent_id + """ + ) + rows = cursor.fetchall() + conn.close() + return [dict(row) for row in rows] + + def list_agent_owners(self, user_ids: Sequence[int] | None = None) -> Dict[str, int]: + """agent_id -> owner_user_id for these owners (``None`` = every owned agent). + + `external_agents.owner_user_id` is indexed + (`idx_external_agents_owner_user`), and this table is in + CONTENT_DATABASE_URL while protocol_runs is in DATABASE_PATH, so this + mapping has to come back to Python before the runs can be counted. + Rows with a NULL owner are omitted rather than grouped. + """ + clause = "" + params: List[Any] = [] + if user_ids is not None: + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + if not ids: + return {} + clause = f" AND owner_user_id IN ({', '.join('?' for _ in ids)})" + params = ids + conn = self._get_connection() + cursor = conn.cursor() + cursor.execute( + "SELECT agent_id, owner_user_id FROM external_agents " + f"WHERE owner_user_id IS NOT NULL{clause}", + params, + ) + rows = cursor.fetchall() + conn.close() + return {str(row["agent_id"]): int(row["owner_user_id"]) for row in rows} + def list_owner_scope_agent_ids(self, agent_id: str) -> List[str]: """Agent ids that share an owner with ``agent_id``. diff --git a/dashboard/backend/domain/agents/repository_postgres.py b/dashboard/backend/domain/agents/repository_postgres.py index 54fd3aff..bd8b4933 100644 --- a/dashboard/backend/domain/agents/repository_postgres.py +++ b/dashboard/backend/domain/agents/repository_postgres.py @@ -13,7 +13,7 @@ import json import uuid -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Sequence from dashboard.backend.db_url import init_schema_unless_worker, require_postgres_url from dashboard.backend.domain.agents.repository import ( @@ -629,6 +629,37 @@ def count_agents(self) -> int: row = cur.fetchone() return int(row["n"] if row else 0) + + def list_agent_source_rows(self) -> List[Dict[str, Any]]: + """See the SQLite twin.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT agent_id, session_id, owner_user_id, created_at + FROM external_agents + ORDER BY created_at, agent_id + """ + ) + rows = cur.fetchall() + return [dict(row) for row in rows] + + def list_agent_owners(self, user_ids: Sequence[int] | None = None) -> Dict[str, int]: + """See the SQLite twin.""" + sql = "SELECT agent_id, owner_user_id FROM external_agents WHERE owner_user_id IS NOT NULL" + params: tuple[Any, ...] = () + if user_ids is not None: + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + if not ids: + return {} + sql += " AND owner_user_id = ANY(%s)" + params = (ids,) + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, params) + rows = cur.fetchall() + return {str(row["agent_id"]): int(row["owner_user_id"]) for row in rows} + def list_owner_scope_agent_ids(self, agent_id: str) -> List[str]: """Postgres twin of ``AgentStore.list_owner_scope_agent_ids``. diff --git a/dashboard/backend/domain/analytics/backfill.py b/dashboard/backend/domain/analytics/backfill.py index de510f99..bcfde8df 100644 --- a/dashboard/backend/domain/analytics/backfill.py +++ b/dashboard/backend/domain/analytics/backfill.py @@ -2,7 +2,7 @@ Only source records that prove an authenticated owner and a safe lifecycle outcome become Analytics events. Browser/session/network history is never -invented. The source adapter deliberately reads each database independently; +invented. The source adapter reads each database through the store that owns it and never opens another domain's connection; cross-database ownership is resolved in Python. """ @@ -116,19 +116,6 @@ class BackfillSource(Protocol): def collect(self, *, start: datetime, end: datetime) -> BackfillCollection: ... -def _query_all(store: Any, sql: str) -> list[dict[str, Any]]: - """Read safe source columns from either store twin without joining stores.""" - - if hasattr(store, "database_url"): - with store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute(sql) - return [dict(row) for row in cur.fetchall()] - conn = store._get_connection() - try: - return [dict(row) for row in conn.execute(sql).fetchall()] - finally: - conn.close() def _all_users(user_store: Any) -> list[dict[str, Any]]: @@ -148,25 +135,8 @@ def _credit_candidates( candidates: list[BackfillCandidate] = [] invalid = 0 try: - reservations = _query_all( - credits_store, - """ - SELECT reservation_id, user_id, run_id, call_index, - reserved_grant_micro, reserved_purchased_micro, - status, created_at, updated_at - FROM credit_llm_reservations - ORDER BY created_at, reservation_id - """, - ) - usage_entries = _query_all( - credits_store, - """ - SELECT id, user_id, reservation_id, run_id, call_index, - bucket, amount_micro, created_at - FROM credit_llm_usage_entries - ORDER BY created_at, id - """, - ) + reservations = credits_store.list_llm_reservation_rows() + usage_entries = credits_store.list_llm_usage_rows() except Exception: return [], 1 @@ -376,14 +346,7 @@ def collect(self, *, start: datetime, end: datetime) -> BackfillCollection: invalid += 1 try: - agents = _query_all( - self.agent_store, - """ - SELECT agent_id, session_id, owner_user_id, created_at - FROM external_agents - ORDER BY created_at, agent_id - """, - ) + agents = self.agent_store.list_agent_source_rows() except Exception: agents = [] invalid += 1 @@ -505,33 +468,9 @@ def collect(self, *, start: datetime, end: datetime) -> BackfillCollection: def _existing_source_event_ids(store: Any, source_ids: Iterable[str]) -> set[str]: values = sorted(set(source_ids)) - if not values or not hasattr(store, "_get_connection"): + if not values or not hasattr(store, "list_existing_source_event_ids"): return set() - existing: set[str] = set() - for offset in range(0, len(values), 500): - chunk = values[offset : offset + 500] - if hasattr(store, "database_url"): - with store._get_connection() as conn: - with conn.cursor() as cur: - cur.execute( - "SELECT source_event_id FROM analytics_events " - "WHERE source_event_id = ANY(%s)", - (chunk,), - ) - existing.update( - str(row["source_event_id"]) for row in cur.fetchall() - ) - else: - placeholders = ",".join("?" for _ in chunk) - with store._get_connection() as conn: - rows = conn.execute( - "SELECT source_event_id FROM analytics_events " - f"WHERE source_event_id IN ({placeholders})", - chunk, - ).fetchall() - existing.update(str(row["source_event_id"]) for row in rows) - return existing - + return store.list_existing_source_event_ids(values) def _event(candidate: BackfillCandidate, *, received_at: datetime) -> AnalyticsEventRecord: return AnalyticsEventRecord( diff --git a/dashboard/backend/domain/analytics/daily_facts.py b/dashboard/backend/domain/analytics/daily_facts.py new file mode 100644 index 00000000..9586736f --- /dev/null +++ b/dashboard/backend/domain/analytics/daily_facts.py @@ -0,0 +1,428 @@ +"""One set-based pass per UTC day over the whole user population. + +Every query here is parameterised by a date, never by a user id. That is the +property the read-budget test enforces, and it is the difference between this +module and the snapshot sweep it replaces: cost grows with days, not with +users times minutes (design SS6.9, SS6.12). + +Scheduling lives in ``daily_job.py`` (its own worker thread, D23). This module +is the tick: claim yesterday, compute it, release or complete it, recompute any +recent day whose evidence changed, and give the retention coordinator its turn. +""" + +from __future__ import annotations + +from datetime import date, datetime, time, timedelta, timezone +from typing import Any, Callable + +from pydantic import BaseModel, ConfigDict, Field + +from dashboard.backend.domain.user_groups import coerce_user_group + +from .lifecycle import ( + OperationalSignals, + calculate_lifecycle, + calculate_operational_state, + commercial_tier, +) +from .lifecycle_reads import build_lifecycle_inputs +from .value_repository import ( + ActivityUpdate, + LifecycleTransitionRow, + RecentFactTotals, + UserDailyFact, + _timestamp, +) + + +DAILY_FACTS_JOB = "analytics_daily_facts" +RECOMPUTE_LOOKBACK_DAYS = 6 +MAX_RECOMPUTES_PER_TICK = 2 +TRAILING_WINDOW_DAYS = 30 + + +class DailyFactsReport(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + snapshot_date: date | None = None + claimed: bool = False + users_written: int = Field(default=0, ge=0) + transitions_written: int = Field(default=0, ge=0) + partial: bool = False + failed_steps: tuple[str, ...] = () + # Earlier days this tick recomputed because late events landed in them. + # Reported rather than silent: a day that keeps reappearing here is a + # clock-skew or a replay problem, and the only place it is visible. + recomputed_dates: tuple[date, ...] = () + + +class _DayOutcome(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + users_written: int = 0 + transitions_written: int = 0 + failed_steps: tuple[str, ...] = () + + +def _utc_now(value: datetime | None) -> datetime: + current = value or datetime.now(timezone.utc) + if current.tzinfo is None or current.utcoffset() is None: + raise ValueError("now must include a timezone") + return current.astimezone(timezone.utc) + + +def _default_value_store(): + from .repository import analytics_store + from .value_repository import build_value_analytics_store + + return build_value_analytics_store(analytics_store) + + +def _default_run_history_store(): + from dashboard.backend.database import db + + return db + + +def _default_rollup() -> Callable[..., Any]: + from .rollups import rollup_day + + return rollup_day + + +def _default_retention(): + # One literal mention of the coordinator: the source-pin in + # test_daily_job.py counts it. + from .retention import analytics_retention_coordinator as _coordinator + + return _coordinator + + +def _end_of_day(day: date) -> datetime: + return datetime.combine(day, time(23, 59, 59, 999999), tzinfo=timezone.utc) + + +def _midnight(day: date) -> datetime: + return datetime.combine(day, time.min, tzinfo=timezone.utc) + + +def _count(totals: Any, field: str) -> int: + return int(getattr(totals, field)) if totals is not None else 0 + + +def _compute_day( + day: date, + *, + now: datetime, + store: Any, + run_history_store: Any, + rollup: Callable[..., Any], +) -> _DayOutcome: + """Steps 1-6 of design SS6.9 for one day, each wrapped individually. + + Step order differs from the design's numbering in one place: the eligible + population is read first, because the operational step (5) answers for + exactly those ids. Query count is unchanged. + """ + from .rollups import AnalyticsRollupStore + + analytics = store.analytics_base + end_of_day = _end_of_day(day) + failed: list[str] = [] + + def step(name: str, action: Callable[[], Any], default: Any) -> Any: + try: + return action() + except Exception as exc: + failed.append(name) + print( + f"WARNING: analytics.daily_facts.{name}_failed " + f"category={type(exc).__name__[:80]}" + ) + return default + + # 0. Who has a day D at all: every non-admin, non-excluded account created + # on or before the end of D (9a: an account created after D has no D). + def subjects_step() -> list[dict[str, Any]]: + eligible = [] + for subject in analytics.list_daily_subjects(): + created_at = _timestamp(subject["created_at"]) + if created_at <= end_of_day: + eligible.append({**subject, "created_at": created_at}) + return eligible + + subjects = step("subjects", subjects_step, None) + if subjects is None: + # Nothing can be written without the population; every later step + # would only mark a day partial that has no rows to be partial. + return _DayOutcome(failed_steps=tuple(failed)) + eligible_ids = [int(subject["id"]) for subject in subjects] + + # 1. Anonymous rollups for D (moved here from run_analytics_maintenance). + step( + "rollup", + lambda: rollup(day, store=AnalyticsRollupStore(analytics), now=_midnight(now.date())), + None, + ) + + # 2. One one-day event scan, then the idempotent activity correction. + def events_step() -> dict[int, Any]: + totals = store.aggregate_events_for_day(day) + store.record_activity_batch( + [ + ActivityUpdate( + user_id=item.user_id, + activated_at=item.first_success_at, + last_activity_at=item.last_activity_at, + ) + for item in totals.values() + ], + now=now, + ) + return totals + + events = step("events", events_step, {}) + + # 3. Operator-funded cost from the run-history store. + operator_cost = step( + "runs", lambda: run_history_store.aggregate_operator_cost_for_day(day), {} + ) + + # 4. Own spend and lifetime net purchases from the credits store, then the + # same activity correction from the ledger's own timestamps. + def ledger_step() -> dict[int, dict[str, Any]]: + totals = store.credits_base.aggregate_ledger_for_day(day) + store.record_activity_batch( + [ + ActivityUpdate( + user_id=user_id, + last_activity_at=_timestamp(row["last_activity_at"]), + ) + for user_id, row in totals.items() + if row.get("last_activity_at") + ], + now=now, + ) + return totals + + ledger = step("ledger", ledger_step, {}) + + # 5. Operational signals for the eligible population, as of the end of D, + # read population-wide so no statement carries an IN list. + signals = step( + "operational", + lambda: store.list_operational_signals( + eligible_ids, now=end_of_day, population_wide=True + ), + {}, + ) + + quality = "partial" if failed else "complete" + + # 6. The fact rows, then the transitions. + def facts_step() -> tuple[int, list[tuple[UserDailyFact, int]]]: + activity = store.list_activity(None) + window = store.sum_recent_facts( + None, + start=day - timedelta(days=TRAILING_WINDOW_DAYS - 1), + end=day - timedelta(days=1), + ) + rows_with_days: list[tuple[UserDailyFact, int]] = [] + for subject in subjects: + user_id = int(subject["id"]) + day_totals = events.get(user_id) + active = day_totals is not None and day_totals.active + prior = window.get(user_id, RecentFactTotals()) + # 9b: the window is D-29..D-1 in the table; D's own contribution + # comes from the events aggregate already in hand, so the stored + # segment equals what the read path computes tomorrow. + combined = RecentFactTotals( + active_days=prior.active_days + (1 if active else 0), + successful_backtests=prior.successful_backtests + + _count(day_totals, "runs_completed"), + runs_requested=prior.runs_requested + _count(day_totals, "runs_requested"), + runs_completed=prior.runs_completed + _count(day_totals, "runs_completed"), + runs_failed=prior.runs_failed + _count(day_totals, "runs_failed"), + runs_cancelled=prior.runs_cancelled + _count(day_totals, "runs_cancelled"), + operator_cost_micro=prior.operator_cost_micro, + own_spend_micro=prior.own_spend_micro, + days_present=prior.days_present + 1, + last_active_date=day if active else prior.last_active_date, + ) + # 9a: evidence is clamped to the end of D inside build_lifecycle_inputs; + # the day's own last activity is threaded in so a user whose only + # recent activity is on D is not read as dormant. + inputs = build_lifecycle_inputs( + user_id, + created_at=subject["created_at"], + activity=activity.get(user_id), + totals=combined, + as_of=end_of_day, + day_activity_at=day_totals.last_activity_at if day_totals is not None else None, + ) + lifecycle = calculate_lifecycle(inputs, end_of_day) + operational = calculate_operational_state( + signals.get(user_id) or OperationalSignals(user_id=user_id), end_of_day + ) + ledger_row = ledger.get(user_id, {}) + rows_with_days.append( + ( + UserDailyFact( + snapshot_date=day, + user_id=user_id, + lifecycle_segment=lifecycle.segment, + lifecycle_reason_code=lifecycle.reason_code, + operational_state=operational.state, + operational_reason_code=( + None if operational.state == "healthy" else operational.reason_code + ), + tier=commercial_tier( + int(ledger_row.get("lifetime_net_purchased_micro", 0)) + ), + user_group=coerce_user_group(subject.get("user_group")), + active=bool(active), + runs_requested=_count(day_totals, "runs_requested"), + runs_completed=_count(day_totals, "runs_completed"), + runs_failed=_count(day_totals, "runs_failed"), + runs_cancelled=_count(day_totals, "runs_cancelled"), + operator_cost_micro=int(operator_cost.get(user_id, 0)), + own_spend_micro=int(ledger_row.get("own_spend_micro", 0)), + data_quality=quality, + calculated_at=now, + ), + lifecycle.inactive_days, + ) + ) + written = store.upsert_daily_facts([row for row, _days in rows_with_days]) + return written, rows_with_days + + written, rows_with_days = step("facts", facts_step, (0, [])) + if "facts" in failed: + return _DayOutcome(failed_steps=tuple(failed)) + + def transitions_step() -> int: + previous = { + row.user_id: row for row in store.list_facts_for_date(day - timedelta(days=1)) + } + changed: list[LifecycleTransitionRow] = [] + for row, inactive_days in rows_with_days: + before = previous.get(row.user_id) + if before is None or before.lifecycle_segment == row.lifecycle_segment: + continue + changed.append( + LifecycleTransitionRow( + user_id=row.user_id, + snapshot_date=day, + from_segment=before.lifecycle_segment, + to_segment=row.lifecycle_segment, + inactive_days=inactive_days, + data_quality=quality, + created_at=now, + ) + ) + return store.append_lifecycle_transitions(changed) + + transitions = step("transitions", transitions_step, 0) + return _DayOutcome( + users_written=written, + transitions_written=transitions, + failed_steps=tuple(failed), + ) + + +def run_daily_facts( + *, + now: datetime | None = None, + value_store: Any = None, + run_history_store: Any = None, + rollup: Callable[..., Any] | None = None, + retention: Any = None, +) -> DailyFactsReport: + """One worker tick: yesterday if due, late arrivals, retention. + + The due check is the claim itself (Task 8): a tick on a day that is + already complete costs exactly one store call, the refused + compare-and-set. A claimed day whose steps all succeed is completed; a + claimed day with any failed step is written ``partial`` (the UI labels + such periods "Incomplete data") and **released**, so the next tick + retries the whole day and corrects the partial rows once the source is + back. Every retry is the same fixed set of set-based queries, so a source + that stays down costs one bounded batch per tick, never more. + + The late-arrival sweep (design SS6.9 step 7) runs only on a claimed tick, + for the six days before D; it does not touch the claim. The retention + coordinator runs on every tick -- its own clock makes that free when it + is not due, and calling it here rather than from the reaper is what keeps + the two from double-writing (design SS6.9 step 8). + """ + current = _utc_now(now) + store = value_store if value_store is not None else _default_value_store() + runs = run_history_store if run_history_store is not None else _default_run_history_store() + rollup_fn = rollup if rollup is not None else _default_rollup() + coordinator = retention if retention is not None else _default_retention() + target = current.date() - timedelta(days=1) + + claimed = store.claim_projection_day(DAILY_FACTS_JOB, day=target, now=current) + report = DailyFactsReport(snapshot_date=target, claimed=claimed) + if claimed: + outcome = _compute_day( + target, now=current, store=store, run_history_store=runs, rollup=rollup_fn + ) + if outcome.failed_steps: + store.release_projection_day(DAILY_FACTS_JOB, now=current) + else: + store.complete_projection_day(DAILY_FACTS_JOB, day=target, now=current) + + recomputed: list[date] = [] + try: + stale_days = store.list_days_needing_recompute( + since=target - timedelta(days=RECOMPUTE_LOOKBACK_DAYS), + until=target - timedelta(days=1), + ) + except Exception as exc: + stale_days = [] + print( + "WARNING: analytics.daily_facts.recompute_scan_failed " + f"category={type(exc).__name__[:80]}" + ) + # One store call after the loop, never inside it: the lease belongs to + # the job row, not the day, and the discipline guard forbids store + # calls inside loops (design SS6.11 rule 7). A failed stale day stays + # in list_days_needing_recompute, so the next tick retries it. + failed_recomputes: list[date] = [] + for stale_day in stale_days[:MAX_RECOMPUTES_PER_TICK]: + stale_outcome = _compute_day( + stale_day, now=current, store=store, run_history_store=runs, rollup=rollup_fn + ) + if stale_outcome.failed_steps: + failed_recomputes.append(stale_day) + print( + "WARNING: analytics.daily_facts.recompute_failed " + f"day={stale_day.isoformat()} steps={','.join(stale_outcome.failed_steps)}" + ) + continue + recomputed.append(stale_day) + if failed_recomputes: + store.release_projection_day(DAILY_FACTS_JOB, now=current) + + report = DailyFactsReport( + snapshot_date=target, + claimed=True, + users_written=outcome.users_written, + transitions_written=outcome.transitions_written, + partial=bool(outcome.failed_steps), + failed_steps=outcome.failed_steps, + recomputed_dates=tuple(recomputed), + ) + + try: + coordinator.run_if_due() + except Exception as exc: + print( + "WARNING: analytics.daily_facts.retention_failed " + f"category={type(exc).__name__[:80]}" + ) + return report + + +__all__ = ["DAILY_FACTS_JOB", "DailyFactsReport", "run_daily_facts"] diff --git a/dashboard/backend/domain/analytics/daily_job.py b/dashboard/backend/domain/analytics/daily_job.py new file mode 100644 index 00000000..0007d5db --- /dev/null +++ b/dashboard/backend/domain/analytics/daily_job.py @@ -0,0 +1,142 @@ +"""The daily-facts worker: its own thread, its own interval, its own stop event. + +Design D23 / SS6.9: the run reaper exists to keep ``heartbeat_at`` fresh against +``RUN_HEARTBEAT_STALE_SECONDS`` (300 s). A whole-population batch across three +databases on that thread would let a slow analytics night mark live runs as +orphaned, so the job is *not* a ``register_reaper_sweep`` step. It is still +inside the single web process (design SS2.3): no external scheduler, no second +service. + +Every tick is ``daily_facts.run_daily_facts``; the first thing the thread does, +once, is ``facts_migration.run_startup_migrations`` (seed ``user_activity``, +copy the legacy history). Both are idempotent, so a restart costs nothing. +""" + +from __future__ import annotations + +import os +import threading +from typing import Any, Callable + + +_DEFAULT_INTERVAL_SECONDS = 300 +_MIN_INTERVAL_SECONDS = 5 +_MAX_INTERVAL_SECONDS = 3600 +_ENV_NAME = "ANALYTICS_DAILY_JOB_INTERVAL_SECONDS" + + +def daily_job_interval_seconds() -> float: + """Seconds between worker ticks. Junk falls back to the default with a line. + + Same shape as ``MAX_ACTIVE_DASHBOARD_BACKTESTS`` in + ``api/routers/backtests.py``: an unparseable value read with a bare + ``int()`` at module scope once killed app boot, so a bad value here logs + and uses 300 rather than raising. + """ + raw = os.getenv(_ENV_NAME) + if raw is None or not str(raw).strip(): + return float(_DEFAULT_INTERVAL_SECONDS) + try: + value = int(str(raw).strip()) + except (TypeError, ValueError): + print( + f"{_ENV_NAME} is not an integer ({raw!r}); using {_DEFAULT_INTERVAL_SECONDS}", + flush=True, + ) + return float(_DEFAULT_INTERVAL_SECONDS) + if value < _MIN_INTERVAL_SECONDS or value > _MAX_INTERVAL_SECONDS: + print( + f"{_ENV_NAME} is out of range ({value}; allowed " + f"{_MIN_INTERVAL_SECONDS}-{_MAX_INTERVAL_SECONDS}); " + f"using {_DEFAULT_INTERVAL_SECONDS}", + flush=True, + ) + return float(_DEFAULT_INTERVAL_SECONDS) + return float(value) + + +_worker_lock = threading.Lock() +_worker_thread: threading.Thread | None = None +_worker_stop: threading.Event | None = None + + +def _default_tick() -> Any: + from .daily_facts import run_daily_facts + + return run_daily_facts() + + +def _default_prepare() -> Any: + from .facts_migration import run_startup_migrations + + return run_startup_migrations() + + +def start_daily_facts_worker( + interval_seconds: float | None = None, + stop_event: threading.Event | None = None, + *, + tick: Callable[[], Any] | None = None, + prepare: Callable[[], Any] | None = None, +) -> threading.Thread: + """Start the worker (idempotent -- a second call while it is alive no-ops). + + ``tick`` and ``prepare`` are injectable for tests; production leaves both + at their defaults. The loop waits ``interval_seconds`` *before* the first + tick, so a boot does not run the day's job on top of everything else the + startup hook is doing; the claim makes the first tick cheap when the day + is already done. + """ + global _worker_thread, _worker_stop + interval = ( + float(interval_seconds) if interval_seconds is not None else daily_job_interval_seconds() + ) + with _worker_lock: + if _worker_thread is not None and _worker_thread.is_alive(): + return _worker_thread + stop = stop_event if stop_event is not None else threading.Event() + tick_fn = tick if tick is not None else _default_tick + prepare_fn = prepare if prepare is not None else _default_prepare + + def _loop() -> None: + try: + prepare_fn() + except Exception as exc: + print( + "WARNING: analytics.facts_migration_failed " + f"category={type(exc).__name__[:80]}" + ) + while not stop.wait(interval): + try: + tick_fn() + except Exception as exc: + print( + "WARNING: analytics.daily_facts.tick_failed " + f"category={type(exc).__name__[:80]}" + ) + + thread = threading.Thread(target=_loop, daemon=True, name="analytics-daily-facts") + thread.start() + _worker_thread = thread + _worker_stop = stop + return thread + + +def stop_daily_facts_worker(timeout: float = 5.0) -> None: + """Stop the worker and wait for it; a no-op when none is running (tests).""" + global _worker_thread, _worker_stop + with _worker_lock: + thread, stop = _worker_thread, _worker_stop + _worker_thread = None + _worker_stop = None + if stop is not None: + stop.set() + if thread is not None and thread.is_alive(): + thread.join(timeout=timeout) + + +__all__ = [ + "daily_job_interval_seconds", + "start_daily_facts_worker", + "stop_daily_facts_worker", +] diff --git a/dashboard/backend/domain/analytics/facts_migration.py b/dashboard/backend/domain/analytics/facts_migration.py new file mode 100644 index 00000000..98d8fa95 --- /dev/null +++ b/dashboard/backend/domain/analytics/facts_migration.py @@ -0,0 +1,65 @@ +"""Startup migrations for the fact tables PR A creates. + +Both are idempotent and both run again on every boot; the worker thread in +``daily_job.py`` calls ``run_startup_migrations`` once before its first tick. +PR B re-runs the same seed immediately before dropping the legacy tables, +because the legacy snapshot row keeps being maintained between the two PRs +(design SS6.9 "Migration of existing history", SS11). +""" + +from __future__ import annotations + +from datetime import datetime, timedelta, timezone +from typing import Any + +from pydantic import BaseModel, ConfigDict, Field + + +HISTORY_COPY_DAYS = 56 + + +class StartupMigrationReport(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + activity_seeded: int = Field(default=0, ge=0) + history_copied: int = Field(default=0, ge=0) + + +def _utc_now(value: datetime | None) -> datetime: + current = value or datetime.now(timezone.utc) + if current.tzinfo is None or current.utcoffset() is None: + raise ValueError("now must include a timezone") + return current.astimezone(timezone.utc) + + +def migrate_lifecycle_history(*, value_store: Any, now: datetime) -> int: + """Copy the last eight weeks of legacy daily snapshots; returns rows inserted.""" + current = _utc_now(now) + return value_store.copy_daily_snapshot_history( + since=current.date() - timedelta(days=HISTORY_COPY_DAYS), now=current + ) + + +def run_startup_migrations( + *, + value_store: Any = None, + now: datetime | None = None, +) -> StartupMigrationReport: + """Seed ``user_activity`` from the legacy row, then copy the history.""" + current = _utc_now(now) + if value_store is None: + from .repository import analytics_store + from .value_repository import build_value_analytics_store + + value_store = build_value_analytics_store(analytics_store) + seeded = value_store.seed_activity_from_snapshots(now=current) + copied = migrate_lifecycle_history(value_store=value_store, now=current) + return StartupMigrationReport(activity_seeded=seeded, history_copied=copied) + + +__all__ = [ + "HISTORY_COPY_DAYS", + "StartupMigrationReport", + "migrate_lifecycle_history", + "run_startup_migrations", +] diff --git a/dashboard/backend/domain/analytics/lifecycle.py b/dashboard/backend/domain/analytics/lifecycle.py index f3404d29..59716ac6 100644 --- a/dashboard/backend/domain/analytics/lifecycle.py +++ b/dashboard/backend/domain/analytics/lifecycle.py @@ -346,6 +346,32 @@ def calculate_operational_state( ) +_FAILED_RUN_STATUSES = frozenset({"failed", "timed_out"}) + + +def consecutive_failed_terminal_runs(statuses: Sequence[str]) -> int: + """Count leading failures in a newest-first sequence of terminal statuses. + + The rule `calculate_operational_state` reports as + "three_consecutive_failed_runs", lifted out of `_run_health` so the live + profile and the daily job cannot answer it differently. A total count is + a different number -- failed, failed, succeeded, failed, failed is 2 here + and 4 to a `COUNT(*)` -- and the reason code the UI renders says + "consecutive", so the total would be wrong on screen as well as + inconsistent between the two paths. + + Pure and sequence-shaped rather than query-shaped on purpose: one caller + has rows from a store's list, the other from a cross-database fold, and + neither can express this in SQL over its own data alone. + """ + count = 0 + for status in statuses: + if status not in _FAILED_RUN_STATUSES: + break + count += 1 + return count + + def commercial_tier(net_purchased_micro: int) -> CommercialTier: """Classify lifetime net purchases; refunds cannot make revenue negative.""" @@ -383,5 +409,6 @@ def activation_cohort_week(activated_at: datetime) -> date: "calculate_lifecycle", "calculate_operational_state", "commercial_tier", + "consecutive_failed_terminal_runs", "is_lifecycle_activity", ] diff --git a/dashboard/backend/domain/analytics/lifecycle_reads.py b/dashboard/backend/domain/analytics/lifecycle_reads.py new file mode 100644 index 00000000..74fd7d45 --- /dev/null +++ b/dashboard/backend/domain/analytics/lifecycle_reads.py @@ -0,0 +1,111 @@ +"""Build lifecycle inputs from stored facts instead of an event scan. + +``calculate_lifecycle`` always took a struct of timestamps and two counts. +What changed is where the struct comes from: two indexed reads of +``user_activity`` and ``user_daily_facts`` rather than a 180-day scan of +``analytics_events`` per user (design SS6.6). + +Nothing in ``api/`` calls this module yet. PR B wires it into the one-user +profile route; the daily job (``daily_facts.py``) is its only caller in PR A. +""" + +from __future__ import annotations + +from datetime import datetime, time, timezone + +from .lifecycle import LifecycleInputs +from .repository_common import positive_user_id +from .value_repository import RecentFactTotals, UserActivity + + +def build_lifecycle_inputs( + user_id: int, + *, + created_at: datetime, + activity: UserActivity | None, + totals: RecentFactTotals | None, + as_of: datetime, + day_activity_at: datetime | None = None, +) -> LifecycleInputs: + """Assemble one user's lifecycle inputs from stored rows, as of ``as_of``. + + A user with no ``user_activity`` row has never done anything meaningful; + ``calculate_lifecycle`` anchors on ``created_at`` in that case, so a + brand-new account reads as New rather than as instantly Dormant. + + ``active_days_30d`` is capped at 30 because the model bounds it there and + a migrated or double-counted window must not raise a validation error on + a read path. + + **Every timestamp is clamped to ``as_of``, and that is not defensive + programming -- it is the only thing that lets this function serve a past + day at all.** ``calculate_lifecycle`` raises + ``ValueError("lifecycle evidence cannot occur after as_of")`` when its + anchor is newer than ``as_of`` (lifecycle.py:242-243), and + ``user_activity`` is a single row per user overwritten in place: it says + what is true *now*, never what was true at the end of some earlier day. + The live read path passes ``as_of=now`` and nothing clamps; the daily job + passes the end of the day it is computing, and without this every user + who acted after midnight would abort the whole population's fact write. + + ``activated_at`` newer than ``as_of`` becomes None rather than being + clamped to ``as_of``: "activated after this day" means "not activated as + of this day", and pretending they activated at midnight would put them in + the wrong retention cohort. + + ``last_meaningful_activity_at`` is only replaced when the stored value is + newer than ``as_of``, because when it is not, it is authoritative and more + precise than any fallback. The replacement walks newest-first: + + 1. ``day_activity_at`` -- the user's own last activity **inside the day + being computed**, from the daily job's one-day event aggregate. This + has to come first and it is the case a date-window fallback alone + silently gets wrong: a user whose only recent activity is on ``D`` + itself has no row in the ``D-29..D-1`` window at all, so skipping + straight to ``totals`` would read them as inactive for thirty days and + classify an active user as Dormant. + 2. ``totals.last_active_date`` -- the fact table *does* keep history, + which is the whole reason it exists. Midnight-start of that date is the + conservative choice; ``calculate_lifecycle`` counts ``inactive_days`` in + whole UTC dates, so the time of day is never read. + 3. None -- no activity anywhere in the window, so ``created_at`` anchors + the calculation, which is what a dormant classification needs. + + A user whose ``created_at`` is after ``as_of`` is **not** this function's + problem to clamp -- ``calculate_lifecycle`` would raise on + ``account_age_days < 0``, and rightly so, because an account that did not + exist on day D has no day D. The daily job excludes those users from the + eligible set instead; the live read path can never hit it. + """ + subject_id = positive_user_id(user_id) + counts = totals or RecentFactTotals() + boundary = as_of + + activated_at = activity.activated_at if activity is not None else None + if activated_at is not None and activated_at > boundary: + activated_at = None + + last_activity = ( + activity.last_meaningful_activity_at if activity is not None else None + ) + if last_activity is not None and last_activity > boundary: + if day_activity_at is not None and day_activity_at <= boundary: + last_activity = day_activity_at + elif counts.last_active_date is not None: + last_activity = datetime.combine( + counts.last_active_date, time.min, tzinfo=timezone.utc + ) + else: + last_activity = None + + return LifecycleInputs( + user_id=subject_id, + created_at=created_at, + first_successful_backtest_at=activated_at, + last_meaningful_activity_at=last_activity, + active_days_30d=min(30, counts.active_days), + successful_backtests_30d=counts.successful_backtests, + ) + + +__all__ = ["build_lifecycle_inputs"] diff --git a/dashboard/backend/domain/analytics/maintenance.py b/dashboard/backend/domain/analytics/maintenance.py index d4abb40a..ce877f67 100644 --- a/dashboard/backend/domain/analytics/maintenance.py +++ b/dashboard/backend/domain/analytics/maintenance.py @@ -1,9 +1,18 @@ -"""Bounded Analytics rollup and snapshot repair maintenance.""" +"""Throttled repair of the legacy Analytics snapshot rows. + +Admin layer redesign PR A moved the day rollup and the lifecycle history backfill off +this reaper tick: the daily-facts job (``daily_facts.py``, on its own worker +thread, ``daily_job.py``) owns the day rollup now, so the two can never write +the same rows (design SS6.9 step 1, D23), and the eight-week history copy in +``facts_migration.py`` replaces the history copy. What remains is the two +24-hour-throttled snapshot repairs PR 0 left, which keep +``user_analytics_snapshots`` maintained for the read paths that still use it. +PR B moves those read paths and deletes this module. +""" from __future__ import annotations -from datetime import date, datetime, timedelta, timezone -from threading import Lock +from datetime import datetime, timezone from typing import Any from pydantic import BaseModel, ConfigDict, Field @@ -12,21 +21,11 @@ class AnalyticsMaintenanceReport(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) - rollup_days: tuple[date, ...] - rollup_rebuilt: bool = False repaired_snapshots: int = Field(default=0, ge=0) repaired_value_snapshots: int = Field(default=0, ge=0) - backfilled_lifecycle_users: int = Field(default=0, ge=0) - backfilled_lifecycle_rows: int = Field(default=0, ge=0) - lifecycle_backfill_complete: bool = False - lifecycle_backfill_failures: int = Field(default=0, ge=0) failures: int = Field(default=0, ge=0) -_guard_lock = Lock() -_last_rollup_day: date | None = None - - def _current_utc(value: datetime | None) -> datetime: current = value or datetime.now(timezone.utc) if current.tzinfo is None or current.utcoffset() is None: @@ -34,38 +33,21 @@ def _current_utc(value: datetime | None) -> datetime: return current.astimezone(timezone.utc) -def reset_maintenance_guard_for_tests() -> None: - """Reset only the process-local scheduling guard.""" - - global _last_rollup_day - with _guard_lock: - _last_rollup_day = None - - def run_analytics_maintenance( *, now: datetime | None = None, snapshot_limit: int = 100, - rebuild_rollup: Any | None = None, repair_snapshots: Any | None = None, repair_value_snapshots: Any | None = None, - backfill_lifecycle: Any | None = None, ) -> AnalyticsMaintenanceReport: - """Rebuild yesterday and repair bounded dual-axis and legacy batches.""" + """Repair bounded batches of stale dual-axis and legacy snapshots.""" - global _last_rollup_day current = _current_utc(now) if isinstance(snapshot_limit, bool) or not isinstance(snapshot_limit, int): raise ValueError("snapshot_limit must be an integer") page_size = max(1, min(snapshot_limit, 100)) - completed_day = current.date() - timedelta(days=1) failures = 0 - rebuilt = False - - if rebuild_rollup is None: - from .rollups import rollup_day - rebuild_rollup = rollup_day if repair_snapshots is None: from .states import repair_stale_snapshots @@ -74,37 +56,6 @@ def run_analytics_maintenance( from .states import repair_stale_value_snapshots repair_value_snapshots = repair_stale_value_snapshots - if backfill_lifecycle is None: - from .lifecycle_backfill import run_lifecycle_backfill_batch - - backfill_lifecycle = run_lifecycle_backfill_batch - - with _guard_lock: - should_rebuild = _last_rollup_day != completed_day - if should_rebuild: - # Reserve the day before doing I/O so concurrent reaper passes do - # not rebuild it twice. A failure clears the reservation below. - _last_rollup_day = completed_day - if should_rebuild: - try: - rebuild_rollup( - completed_day, - now=datetime.combine( - current.date(), - datetime.min.time(), - tzinfo=timezone.utc, - ), - ) - rebuilt = True - except Exception as exc: - failures += 1 - with _guard_lock: - if _last_rollup_day == completed_day: - _last_rollup_day = None - print( - "WARNING: analytics.rollup_maintenance_failed " - f"category={type(exc).__name__[:80]}" - ) repaired_values = 0 try: @@ -136,41 +87,14 @@ def run_analytics_maintenance( f"category={type(exc).__name__[:80]}" ) - backfilled_users = 0 - backfilled_rows = 0 - backfill_complete = False - backfill_failures = 0 - try: - backfill_report = backfill_lifecycle( - now=current, - batch_size=page_size, - ) - backfilled_users = max(0, int(backfill_report.processed_users)) - backfilled_rows = max(0, int(backfill_report.written_rows)) - backfill_complete = bool(backfill_report.complete) - except Exception as exc: - failures += 1 - backfill_failures = 1 - print( - "WARNING: analytics.lifecycle_backfill_failed " - f"category={type(exc).__name__[:80]}" - ) - return AnalyticsMaintenanceReport( - rollup_days=(completed_day,), - rollup_rebuilt=rebuilt, repaired_snapshots=max(0, repaired), repaired_value_snapshots=max(0, repaired_values), - backfilled_lifecycle_users=backfilled_users, - backfilled_lifecycle_rows=backfilled_rows, - lifecycle_backfill_complete=backfill_complete, - lifecycle_backfill_failures=backfill_failures, failures=failures, ) __all__ = [ "AnalyticsMaintenanceReport", - "reset_maintenance_guard_for_tests", "run_analytics_maintenance", ] diff --git a/dashboard/backend/domain/analytics/query_service.py b/dashboard/backend/domain/analytics/query_service.py index c5381c36..9915f9f9 100644 --- a/dashboard/backend/domain/analytics/query_service.py +++ b/dashboard/backend/domain/analytics/query_service.py @@ -120,54 +120,8 @@ class AnalyticsUserListItem(BaseModel): profile_path: str -class AnalyticsUserFilters(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - q: str | None = Field(default=None, max_length=100) - status: str | None = None - last_activity_from: datetime | None = None - last_activity_to: datetime | None = None - sort: Literal[ - "last_activity", "joined_at", "recent_runs", "recent_failures" - ] = "last_activity" - order: Literal["asc", "desc"] = "desc" - include_internal: bool = False - - @field_validator("q") - @classmethod - def normalize_query(cls, value: str | None) -> str | None: - if value is None: - return None - normalized = value.strip() - return normalized or None - - @field_validator("status") - @classmethod - def validate_status(cls, value: str | None) -> str | None: - if value is not None and value not in _USER_STATES: - raise ValueError("status must be a supported Analytics user state") - return value - - @model_validator(mode="after") - def validate_activity_range(self) -> "AnalyticsUserFilters": - start = self.last_activity_from - end = self.last_activity_to - if start is not None: - object.__setattr__(self, "last_activity_from", _utc(start, "last_activity_from")) - if end is not None: - object.__setattr__(self, "last_activity_to", _utc(end, "last_activity_to")) - if start is not None and end is not None and _utc(end, "last_activity_to") < _utc(start, "last_activity_from"): - raise ValueError("last_activity_to cannot be before last_activity_from") - return self -class PaginatedUsers(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - - items: list[AnalyticsUserListItem] - total: int = Field(ge=0) - limit: int = Field(ge=1, le=100) - offset: int = Field(ge=0) class AnalyticsFootprintItem(BaseModel): @@ -958,116 +912,6 @@ def get_overview( availability=availability, ) - def list_users( - self, - *, - filters: AnalyticsUserFilters, - limit: int, - offset: int, - now: datetime | None = None, - ) -> PaginatedUsers: - if not isinstance(filters, AnalyticsUserFilters): - filters = AnalyticsUserFilters.model_validate(filters) - page_size = positive_limit(limit) - if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0: - raise ValueError("offset must be a non-negative integer") - current = _utc(now or datetime.now(timezone.utc), "now") - users = _all_users(self.user_store) - excluded = ( - set() - if filters.include_internal - else self.store.list_excluded_user_ids(include_admin_accounts=True) - ) - snapshots = self.query_store.list_snapshots() - events = self.query_store.rollups.list_events( - start=current - timedelta(days=30), - end=current + timedelta(microseconds=1), - include_internal=True, - ) - by_user: dict[int, list[AnalyticsEventRecord]] = defaultdict(list) - for event in events: - by_user[event.user_id].append(event) - - items: list[AnalyticsUserListItem] = [] - for user in users: - user_id = int(user["id"]) - if user_id in excluded: - continue - user_events = sorted( - by_user.get(user_id, []), - key=lambda event: event.occurred_at, - reverse=True, - ) - meaningful = [event for event in user_events if is_meaningful_event(event)] - last_activity = meaningful[0].occurred_at if meaningful else None - snapshot = snapshots.get(user_id) - if snapshot is None: - snapshot = calculate_user_state( - user_id, - now=current, - store=self.query_store.states, - ) - item = AnalyticsUserListItem( - user_id=user_id, - display_name=str(user.get("display_name") or ""), - email=str(user.get("email") or ""), - joined_at=_parse_timestamp(user["created_at"]), - status=snapshot.status, - reason_code=snapshot.reason_code, - human_readable_reason=snapshot.human_readable_reason, - last_meaningful_activity=last_activity, - recent_runs=sum(event.event_group == "run" for event in user_events), - recent_failures=sum( - event.event_name == "backtest_failed" for event in user_events - ), - profile_path=f"/admin/analytics/users/{user_id}", - ) - if filters.q is not None: - needle = filters.q.lower() - if needle not in item.email.lower() and needle not in item.display_name.lower(): - continue - if filters.status is not None and item.status != filters.status: - continue - if ( - filters.last_activity_from is not None - and ( - item.last_meaningful_activity is None - or item.last_meaningful_activity < filters.last_activity_from - ) - ): - continue - if ( - filters.last_activity_to is not None - and ( - item.last_meaningful_activity is None - or item.last_meaningful_activity > filters.last_activity_to - ) - ): - continue - items.append(item) - - def sort_value(item: AnalyticsUserListItem): - if filters.sort == "joined_at": - return item.joined_at - if filters.sort == "recent_runs": - return item.recent_runs - if filters.sort == "recent_failures": - return item.recent_failures - return item.last_meaningful_activity or datetime.min.replace( - tzinfo=timezone.utc - ) - - items.sort( - key=lambda item: (sort_value(item), item.user_id), - reverse=filters.order == "desc", - ) - return PaginatedUsers( - items=items[offset : offset + page_size], - total=len(items), - limit=page_size, - offset=offset, - ) - def get_user_profile( self, *, @@ -1327,11 +1171,9 @@ def get_value_analytics_query_service() -> ValueAnalyticsQueryService: "AnalyticsOverview", "AnalyticsQueryService", "AnalyticsStateSummary", - "AnalyticsUserFilters", "AnalyticsUserListItem", "AnalyticsUserProfile", "FailureCategoryCount", - "PaginatedUsers", "PanelAvailability", "get_analytics_query_service", "get_value_analytics_query_service", diff --git a/dashboard/backend/domain/analytics/repository.py b/dashboard/backend/domain/analytics/repository.py index 063e36bf..d2d38215 100644 --- a/dashboard/backend/domain/analytics/repository.py +++ b/dashboard/backend/domain/analytics/repository.py @@ -8,7 +8,7 @@ from contextlib import contextmanager from datetime import datetime from pathlib import Path -from typing import Any, Iterator +from typing import Any, Iterator, Sequence from dashboard.backend.database import DB_PATH from dashboard.backend.db_url import describe_database_url @@ -172,6 +172,69 @@ updated_at TEXT NOT NULL ); + +CREATE TABLE IF NOT EXISTS user_activity ( + user_id INTEGER PRIMARY KEY, + activated_at TEXT, + last_meaningful_activity_at TEXT, + updated_at TEXT NOT NULL, + FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE +); + +CREATE TABLE IF NOT EXISTS user_daily_facts ( + snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), + user_id INTEGER NOT NULL, + lifecycle_segment TEXT NOT NULL, + lifecycle_reason_code TEXT NOT NULL, + operational_state TEXT NOT NULL + CHECK (operational_state IN ('blocked', 'needs_attention', 'healthy')), + operational_reason_code TEXT, + tier TEXT NOT NULL + CHECK (tier IN ('unpaid', 'starter', 'invested', 'high_value')), + user_group TEXT NOT NULL DEFAULT 'unknown' + CHECK (user_group IN ( + 'internal', 'invited', 'organic', 'competition', 'partner', 'unknown' + )), + active INTEGER NOT NULL DEFAULT 0 CHECK (active IN (0, 1)), + runs_requested INTEGER NOT NULL DEFAULT 0 CHECK (runs_requested >= 0), + runs_completed INTEGER NOT NULL DEFAULT 0 CHECK (runs_completed >= 0), + runs_failed INTEGER NOT NULL DEFAULT 0 CHECK (runs_failed >= 0), + runs_cancelled INTEGER NOT NULL DEFAULT 0 CHECK (runs_cancelled >= 0), + operator_cost_micro INTEGER NOT NULL DEFAULT 0 + CHECK (operator_cost_micro >= 0), + own_spend_micro INTEGER NOT NULL DEFAULT 0 CHECK (own_spend_micro >= 0), + data_quality TEXT NOT NULL CHECK (data_quality IN ('complete', 'partial')), + calculated_at TEXT NOT NULL, + PRIMARY KEY (snapshot_date, user_id), + FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE +); +CREATE INDEX IF NOT EXISTS idx_daily_facts_segment + ON user_daily_facts(snapshot_date, lifecycle_segment); +CREATE INDEX IF NOT EXISTS idx_daily_facts_user + ON user_daily_facts(user_id, snapshot_date DESC); +CREATE INDEX IF NOT EXISTS idx_daily_facts_user_group + ON user_daily_facts(snapshot_date, user_group); +CREATE INDEX IF NOT EXISTS idx_daily_facts_tier + ON user_daily_facts(snapshot_date, tier); +CREATE INDEX IF NOT EXISTS idx_daily_facts_operational + ON user_daily_facts(snapshot_date, operational_state, operational_reason_code); + +CREATE TABLE IF NOT EXISTS lifecycle_transitions ( + transition_id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL, + snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), + from_segment TEXT NOT NULL, + to_segment TEXT NOT NULL, + inactive_days INTEGER NOT NULL DEFAULT 0 CHECK (inactive_days >= 0), + data_quality TEXT NOT NULL DEFAULT 'complete' + CHECK (data_quality IN ('complete', 'partial')), + created_at TEXT NOT NULL, + UNIQUE (user_id, snapshot_date), + FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE +); +CREATE INDEX IF NOT EXISTS idx_lifecycle_transitions_day + ON lifecycle_transitions(snapshot_date, to_segment); + CREATE TABLE IF NOT EXISTS analytics_subject_settings ( user_id INTEGER PRIMARY KEY, excluded INTEGER NOT NULL CHECK (excluded IN (0, 1)), @@ -562,6 +625,56 @@ def list_excluded_user_ids( excluded.update(int(row["id"]) for row in admin_rows) return excluded + + def list_existing_source_event_ids(self, source_ids: Sequence[str]) -> set[str]: + """Which of ``source_ids`` already have a row. Batched by 500.""" + values = sorted({str(value) for value in source_ids}) + existing: set[str] = set() + if not values: + return existing + with self._get_connection() as conn: + for offset in range(0, len(values), 500): + chunk = values[offset : offset + 500] + placeholders = ",".join("?" for _ in chunk) + rows = conn.execute( + "SELECT source_event_id FROM analytics_events " + f"WHERE source_event_id IN ({placeholders})", + chunk, + ).fetchall() + existing.update(str(row["source_event_id"]) for row in rows) + return existing + + def list_daily_subjects(self) -> list[dict[str, Any]]: + """Every non-admin, non-excluded account: id, user_group, created_at. + + The one place the daily job materialises the whole user list -- a + few hundred rows of three columns. The predicate is the one + ``list_stale_user_ids`` and the lifecycle backfill already use for + aggregate exclusion (design SS6.7). ``created_at`` comes back as the + raw text the users table holds (CURRENT_TIMESTAMP text on SQLite, + ISO-8601 on Postgres); the caller parses. + """ + with self._get_connection() as conn: + rows = conn.execute( + """ + SELECT users.id, users.user_group, users.created_at + FROM users + LEFT JOIN analytics_subject_settings AS settings + ON settings.user_id = users.id + WHERE users.role <> 'admin' + AND COALESCE(settings.excluded, 0) = 0 + ORDER BY users.id + """ + ).fetchall() + return [ + { + "id": int(row["id"]), + "user_group": row["user_group"], + "created_at": row["created_at"], + } + for row in rows + ] + def record_admin_access( self, admin_user_id: int, diff --git a/dashboard/backend/domain/analytics/repository_postgres.py b/dashboard/backend/domain/analytics/repository_postgres.py index 05d22b70..4373a6f0 100644 --- a/dashboard/backend/domain/analytics/repository_postgres.py +++ b/dashboard/backend/domain/analytics/repository_postgres.py @@ -3,7 +3,7 @@ from __future__ import annotations from datetime import datetime -from typing import Any +from typing import Any, Sequence import psycopg @@ -161,6 +161,66 @@ updated_at TEXT NOT NULL ); + +CREATE TABLE IF NOT EXISTS user_activity ( + user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, + activated_at TEXT, + last_meaningful_activity_at TEXT, + updated_at TEXT NOT NULL +); + +CREATE TABLE IF NOT EXISTS user_daily_facts ( + snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + lifecycle_segment TEXT NOT NULL, + lifecycle_reason_code TEXT NOT NULL, + operational_state TEXT NOT NULL + CHECK (operational_state IN ('blocked', 'needs_attention', 'healthy')), + operational_reason_code TEXT, + tier TEXT NOT NULL + CHECK (tier IN ('unpaid', 'starter', 'invested', 'high_value')), + user_group TEXT NOT NULL DEFAULT 'unknown' + CHECK (user_group IN ( + 'internal', 'invited', 'organic', 'competition', 'partner', 'unknown' + )), + active BOOLEAN NOT NULL DEFAULT FALSE, + runs_requested INTEGER NOT NULL DEFAULT 0 CHECK (runs_requested >= 0), + runs_completed INTEGER NOT NULL DEFAULT 0 CHECK (runs_completed >= 0), + runs_failed INTEGER NOT NULL DEFAULT 0 CHECK (runs_failed >= 0), + runs_cancelled INTEGER NOT NULL DEFAULT 0 CHECK (runs_cancelled >= 0), + operator_cost_micro BIGINT NOT NULL DEFAULT 0 + CHECK (operator_cost_micro >= 0), + own_spend_micro BIGINT NOT NULL DEFAULT 0 CHECK (own_spend_micro >= 0), + data_quality TEXT NOT NULL CHECK (data_quality IN ('complete', 'partial')), + calculated_at TEXT NOT NULL, + PRIMARY KEY (snapshot_date, user_id) +); +CREATE INDEX IF NOT EXISTS idx_daily_facts_segment + ON user_daily_facts(snapshot_date, lifecycle_segment); +CREATE INDEX IF NOT EXISTS idx_daily_facts_user + ON user_daily_facts(user_id, snapshot_date DESC); +CREATE INDEX IF NOT EXISTS idx_daily_facts_user_group + ON user_daily_facts(snapshot_date, user_group); +CREATE INDEX IF NOT EXISTS idx_daily_facts_tier + ON user_daily_facts(snapshot_date, tier); +CREATE INDEX IF NOT EXISTS idx_daily_facts_operational + ON user_daily_facts(snapshot_date, operational_state, operational_reason_code); + +CREATE TABLE IF NOT EXISTS lifecycle_transitions ( + transition_id BIGSERIAL PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + snapshot_date TEXT NOT NULL CHECK (length(snapshot_date) = 10), + from_segment TEXT NOT NULL, + to_segment TEXT NOT NULL, + inactive_days INTEGER NOT NULL DEFAULT 0 CHECK (inactive_days >= 0), + data_quality TEXT NOT NULL DEFAULT 'complete' + CHECK (data_quality IN ('complete', 'partial')), + created_at TEXT NOT NULL, + UNIQUE (user_id, snapshot_date) +); +CREATE INDEX IF NOT EXISTS idx_lifecycle_transitions_day + ON lifecycle_transitions(snapshot_date, to_segment); + CREATE TABLE IF NOT EXISTS analytics_subject_settings ( user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, excluded BOOLEAN NOT NULL, @@ -445,6 +505,50 @@ def list_excluded_user_ids( excluded.update(int(row["id"]) for row in cur.fetchall()) return excluded + + def list_existing_source_event_ids(self, source_ids: Sequence[str]) -> set[str]: + """See the SQLite twin.""" + values = sorted({str(value) for value in source_ids}) + existing: set[str] = set() + if not values: + return existing + with self._get_connection() as conn: + with conn.cursor() as cur: + for offset in range(0, len(values), 500): + chunk = values[offset : offset + 500] + cur.execute( + "SELECT source_event_id FROM analytics_events " + "WHERE source_event_id = ANY(%s)", + (chunk,), + ) + existing.update(str(row["source_event_id"]) for row in cur.fetchall()) + return existing + + def list_daily_subjects(self) -> list[dict[str, Any]]: + """See the SQLite twin.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT users.id, users.user_group, users.created_at + FROM users + LEFT JOIN analytics_subject_settings AS settings + ON settings.user_id = users.id + WHERE users.role <> 'admin' + AND COALESCE(settings.excluded, FALSE) = FALSE + ORDER BY users.id + """ + ) + rows = cur.fetchall() + return [ + { + "id": int(row["id"]), + "user_group": row["user_group"], + "created_at": row["created_at"], + } + for row in rows + ] + def record_admin_access( self, admin_user_id: int, diff --git a/dashboard/backend/domain/analytics/service.py b/dashboard/backend/domain/analytics/service.py index caf17ef0..5994b3d4 100644 --- a/dashboard/backend/domain/analytics/service.py +++ b/dashboard/backend/domain/analytics/service.py @@ -42,15 +42,21 @@ def __init__( state_store=None, value_store=None, project_snapshots: bool = False, + maintain_activity: bool = False, ): if not isinstance(project_snapshots, bool): raise TypeError("project_snapshots must be a boolean") + if not isinstance(maintain_activity, bool): + raise TypeError("maintain_activity must be a boolean") if project_snapshots and (state_store is None or value_store is None): raise ValueError("snapshot stores are required when projection is enabled") + if maintain_activity and value_store is None: + raise ValueError("a value store is required to maintain user activity") self.store = store self.state_store = state_store self.value_store = value_store self.project_snapshots = project_snapshots + self.maintain_activity = maintain_activity def _try_recalculate_snapshots( self, @@ -76,6 +82,35 @@ def _try_recalculate_snapshots( f"event={event_name} category={type(exc).__name__[:80]}" ) + def _try_record_activity( + self, + *, + user_id: int, + event: AnalyticsEventRecord, + now: datetime, + ) -> None: + """Advance the stored activity clock for one accepted event. + + Best-effort, like every other analytics write: a failure here must + never change the outcome of the operation that emitted the event. + This is the write that replaces the per-event history recompute + (design SS6.5): one upsert of two timestamps, no read. + """ + if not self.maintain_activity: + return + try: + self.value_store.record_activity( + user_id, + occurred_at=event.occurred_at, + activating=event.event_name == "backtest_completed", + now=now, + ) + except Exception as exc: + print( + "WARNING: analytics.activity_update_failed " + f"event={event.event_name} category={type(exc).__name__[:80]}" + ) + def accept_frontend_event( self, *, @@ -169,7 +204,10 @@ def record_server_event( result = self.store.append_event(event) from .lifecycle import is_lifecycle_activity - projection_relevant = is_lifecycle_activity(event) or event_name in { + lifecycle_activity = is_lifecycle_activity(event) + if result.created and lifecycle_activity: + self._try_record_activity(user_id=subject_id, event=event, now=received) + projection_relevant = lifecycle_activity or event_name in { "account_signed_up", "safe_error_recorded", } @@ -247,6 +285,9 @@ def _build_analytics_service() -> AnalyticsService: # which otherwise steps in for exactly this state (see that task -- # flipping this flag alone is not sufficient). project_snapshots=False, + # PR A replaces the per-event snapshot recompute with the activity + # upsert. PR B deletes the projection flag entirely. + maintain_activity=True, ) diff --git a/dashboard/backend/domain/analytics/value_repository.py b/dashboard/backend/domain/analytics/value_repository.py index ad968b0e..59d0b5f3 100644 --- a/dashboard/backend/domain/analytics/value_repository.py +++ b/dashboard/backend/domain/analytics/value_repository.py @@ -27,8 +27,11 @@ from .lifecycle import ( CommercialTier, LifecycleSegment, + OperationalSignals, OperationalState, + _LIFECYCLE_ACTIVITY_EVENTS, commercial_tier, + consecutive_failed_terminal_runs, ) from .repository import analytics_store from .repository_common import positive_limit, positive_user_id, utc_iso @@ -126,6 +129,453 @@ def require_timezone(cls, value: datetime) -> datetime: return _utc(value) +class UserActivity(BaseModel): + """One user's current activity clock. One row, overwritten in place. + + Design SS6.5: ``activated_at`` is the first accepted ``backtest_completed`` + and is set once; ``last_meaningful_activity_at`` is the greatest + ``occurred_at`` of any accepted event in the lifecycle activity set. It is + current state, not history, and is never swept (SS11). + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + activated_at: datetime | None = None + last_meaningful_activity_at: datetime | None = None + updated_at: datetime + + @field_validator("activated_at", "last_meaningful_activity_at", "updated_at") + @classmethod + def require_timezone(cls, value: datetime | None) -> datetime | None: + return _utc(value) if value is not None else None + + +def _activity_from_row(row: Any) -> UserActivity: + """Shared by both twins.""" + return UserActivity( + user_id=int(_row_value(row, "user_id")), + activated_at=_optional_timestamp(_row_value(row, "activated_at")), + last_meaningful_activity_at=_optional_timestamp( + _row_value(row, "last_meaningful_activity_at") + ), + updated_at=_timestamp(_row_value(row, "updated_at")), + ) + + +class RecentFactTotals(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + active_days: int = Field(default=0, ge=0) + successful_backtests: int = Field(default=0, ge=0) + runs_requested: int = Field(default=0, ge=0) + runs_completed: int = Field(default=0, ge=0) + runs_failed: int = Field(default=0, ge=0) + runs_cancelled: int = Field(default=0, ge=0) + operator_cost_micro: int = Field(default=0, ge=0) + own_spend_micro: int = Field(default=0, ge=0) + # How many of the requested dates actually have a row. Fewer than + # requested means the window is incomplete, which the UI labels rather + # than rendering as zero. + days_present: int = Field(default=0, ge=0) + # The latest date inside the window on which this user was active, or + # None. The daily job needs it to answer "what did this user's activity + # look like as of the end of day D" -- `user_activity` holds one row + # overwritten in place and cannot answer a question about the past. + last_active_date: date | None = None + + +_RECENT_FACTS_SQL = """ + SELECT user_id, + SUM(CASE WHEN active THEN 1 ELSE 0 END) AS active_days, + SUM(runs_completed) AS successful_backtests, + SUM(runs_requested) AS runs_requested, + SUM(runs_failed) AS runs_failed, + SUM(runs_cancelled) AS runs_cancelled, + SUM(operator_cost_micro) AS operator_cost_micro, + SUM(own_spend_micro) AS own_spend_micro, + COUNT(*) AS days_present, + MAX(CASE WHEN active THEN snapshot_date END) AS last_active_date + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date <= {p}{user_clause} + GROUP BY user_id +""" + + +def _recent_totals_from_row(row: Any) -> RecentFactTotals: + """Shared by both twins.""" + last_active = _row_value(row, "last_active_date") + completed = int(_row_value(row, "successful_backtests", 0) or 0) + return RecentFactTotals( + active_days=int(_row_value(row, "active_days", 0) or 0), + successful_backtests=completed, + runs_requested=int(_row_value(row, "runs_requested", 0) or 0), + runs_completed=completed, + runs_failed=int(_row_value(row, "runs_failed", 0) or 0), + runs_cancelled=int(_row_value(row, "runs_cancelled", 0) or 0), + operator_cost_micro=int(_row_value(row, "operator_cost_micro", 0) or 0), + own_spend_micro=int(_row_value(row, "own_spend_micro", 0) or 0), + days_present=int(_row_value(row, "days_present", 0) or 0), + last_active_date=( + date.fromisoformat(str(last_active)) if last_active else None + ), + ) + + +class DayEventTotals(BaseModel): + """One user's run outcomes and activity marks inside one UTC day.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + runs_requested: int = Field(default=0, ge=0) + runs_completed: int = Field(default=0, ge=0) + runs_failed: int = Field(default=0, ge=0) + runs_cancelled: int = Field(default=0, ge=0) + first_success_at: datetime | None = None + last_activity_at: datetime | None = None + + @property + def active(self) -> bool: + return self.last_activity_at is not None + + +class ActivityUpdate(BaseModel): + """One row of a batched ``record_activity``; either timestamp may be absent.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + activated_at: datetime | None = None + last_activity_at: datetime | None = None + + +class UserDailyFact(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + snapshot_date: date + user_id: int = Field(gt=0) + lifecycle_segment: LifecycleSegment + lifecycle_reason_code: str = Field(min_length=1, max_length=100) + operational_state: OperationalState + operational_reason_code: str | None = Field(default=None, max_length=100) + tier: CommercialTier + user_group: str = Field(min_length=1, max_length=32) + active: bool = False + runs_requested: int = Field(default=0, ge=0) + runs_completed: int = Field(default=0, ge=0) + runs_failed: int = Field(default=0, ge=0) + runs_cancelled: int = Field(default=0, ge=0) + operator_cost_micro: int = Field(default=0, ge=0) + own_spend_micro: int = Field(default=0, ge=0) + data_quality: Literal["complete", "partial"] + calculated_at: datetime + + @field_validator("calculated_at") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + return _utc(value) + + +class LifecycleTransitionRow(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + user_id: int = Field(gt=0) + snapshot_date: date + from_segment: LifecycleSegment + to_segment: LifecycleSegment + inactive_days: int = Field(default=0, ge=0) + data_quality: Literal["complete", "partial"] = "complete" + created_at: datetime + + @field_validator("created_at") + @classmethod + def require_timezone(cls, value: datetime) -> datetime: + return _utc(value) + + +_DAILY_FACT_COLUMNS = ( + "snapshot_date", "user_id", "lifecycle_segment", "lifecycle_reason_code", + "operational_state", "operational_reason_code", "tier", "user_group", "active", + "runs_requested", "runs_completed", "runs_failed", "runs_cancelled", + "operator_cost_micro", "own_spend_micro", "data_quality", "calculated_at", +) +_DAILY_FACT_UPDATES = ", ".join( + f"{column} = excluded.{column}" for column in _DAILY_FACT_COLUMNS[2:] +) +_TRANSITION_COLUMNS = ( + "user_id", "snapshot_date", "from_segment", "to_segment", "inactive_days", + "data_quality", "created_at", +) +_TRANSITION_UPDATES = ", ".join( + f"{column} = excluded.{column}" for column in _TRANSITION_COLUMNS[2:] +) +_ACTIVITY_EVENT_NAMES = tuple(sorted(_LIFECYCLE_ACTIVITY_EVENTS)) +_EVENTS_FOR_DAY_SQL = """ + SELECT user_id, + SUM(CASE WHEN event_name = 'backtest_requested' THEN 1 ELSE 0 END) + AS runs_requested, + SUM(CASE WHEN event_name = 'backtest_completed' THEN 1 ELSE 0 END) + AS runs_completed, + SUM(CASE WHEN event_name = 'backtest_failed' THEN 1 ELSE 0 END) + AS runs_failed, + SUM(CASE WHEN event_name = 'backtest_cancelled' THEN 1 ELSE 0 END) + AS runs_cancelled, + MIN(CASE WHEN event_name = 'backtest_completed' THEN occurred_at END) + AS first_success_at, + MAX(CASE WHEN event_name IN ({activity}) THEN occurred_at END) + AS last_activity_at + FROM analytics_events + WHERE occurred_at >= {p} AND occurred_at < {p} + GROUP BY user_id +""" +_RECOMPUTE_EVENTS_SQL = """ + SELECT substr(occurred_at, 1, 10) AS day, MAX(received_at) AS last_received + FROM analytics_events + WHERE occurred_at >= {p} AND occurred_at < {p} + GROUP BY substr(occurred_at, 1, 10) +""" +_RECOMPUTE_FACTS_SQL = """ + SELECT snapshot_date, MIN(calculated_at) AS calculated_at + FROM user_daily_facts + WHERE snapshot_date >= {p} AND snapshot_date <= {p} + GROUP BY snapshot_date +""" + + +def _day_bounds_iso(day: date) -> tuple[str, str]: + start = datetime.combine(day, datetime.min.time(), tzinfo=timezone.utc) + return utc_iso(start), utc_iso(start + timedelta(days=1)) + + +def _events_totals_from_row(row: Any) -> DayEventTotals: + """Shared by both twins.""" + return DayEventTotals( + user_id=int(_row_value(row, "user_id")), + runs_requested=int(_row_value(row, "runs_requested", 0) or 0), + runs_completed=int(_row_value(row, "runs_completed", 0) or 0), + runs_failed=int(_row_value(row, "runs_failed", 0) or 0), + runs_cancelled=int(_row_value(row, "runs_cancelled", 0) or 0), + first_success_at=_optional_timestamp(_row_value(row, "first_success_at")), + last_activity_at=_optional_timestamp(_row_value(row, "last_activity_at")), + ) + + +def _fact_from_row(row: Any) -> UserDailyFact: + """Shared by both twins.""" + return UserDailyFact( + snapshot_date=date.fromisoformat(str(_row_value(row, "snapshot_date"))), + user_id=int(_row_value(row, "user_id")), + lifecycle_segment=_row_value(row, "lifecycle_segment"), + lifecycle_reason_code=_row_value(row, "lifecycle_reason_code"), + operational_state=_row_value(row, "operational_state"), + operational_reason_code=_row_value(row, "operational_reason_code"), + tier=_row_value(row, "tier"), + user_group=_row_value(row, "user_group"), + active=bool(_row_value(row, "active", 0)), + runs_requested=int(_row_value(row, "runs_requested", 0) or 0), + runs_completed=int(_row_value(row, "runs_completed", 0) or 0), + runs_failed=int(_row_value(row, "runs_failed", 0) or 0), + runs_cancelled=int(_row_value(row, "runs_cancelled", 0) or 0), + operator_cost_micro=int(_row_value(row, "operator_cost_micro", 0) or 0), + own_spend_micro=int(_row_value(row, "own_spend_micro", 0) or 0), + data_quality=_row_value(row, "data_quality"), + calculated_at=_timestamp(_row_value(row, "calculated_at")), + ) + + +def _fact_values(row: UserDailyFact, *, active_value: Any) -> tuple[Any, ...]: + """Shared by both twins; ``active_value`` is int on SQLite, bool on Postgres.""" + return ( + row.snapshot_date.isoformat(), + row.user_id, + row.lifecycle_segment, + row.lifecycle_reason_code, + row.operational_state, + row.operational_reason_code, + row.tier, + row.user_group, + active_value, + row.runs_requested, + row.runs_completed, + row.runs_failed, + row.runs_cancelled, + row.operator_cost_micro, + row.own_spend_micro, + row.data_quality, + utc_iso(row.calculated_at), + ) + + +def _transition_values(row: LifecycleTransitionRow) -> tuple[Any, ...]: + return ( + row.user_id, + row.snapshot_date.isoformat(), + row.from_segment, + row.to_segment, + row.inactive_days, + row.data_quality, + utc_iso(row.created_at), + ) + + +def _activity_update_values(update: ActivityUpdate, stamp: str) -> tuple[Any, ...]: + return ( + update.user_id, + utc_iso(update.activated_at) if update.activated_at is not None else None, + utc_iso(update.last_activity_at) if update.last_activity_at is not None else None, + stamp, + ) + + +def _recompute_days(event_rows: Sequence[Any], fact_rows: Sequence[Any]) -> list[date]: + """Shared by both twins: days whose newest event landed after their facts.""" + calculated = { + str(_row_value(row, "snapshot_date")): str(_row_value(row, "calculated_at")) + for row in fact_rows + } + stale: list[date] = [] + for row in event_rows: + day = str(_row_value(row, "day")) + last_received = _row_value(row, "last_received") + if day in calculated and last_received is not None and str(last_received) > calculated[day]: + stale.append(date.fromisoformat(day)) + return sorted(stale, reverse=True) + + +def _platform_lane_open(providers: Sequence[Mapping[str, Any]], platform_statuses: Mapping[str, str]) -> bool: + """The population-wide half of ``platform_credits_available``. + + Mirrors ModelProviderService.list_execution_options (service.py:175-186): + an enabled, platform-enabled provider whose platform credential is + verified or whose deployment secret is set. No user appears in this + expression, which is the whole reason the lane is batchable at all. + """ + from dashboard.backend.domain.model_providers.service import ( + _environment_platform_secret, + ) + + for provider in providers: + provider_id = str(provider.get("provider_id")) + if provider.get("status") != "enabled" or not provider.get("platform_enabled"): + continue + if platform_statuses.get(provider_id) == "verified": + return True + if _environment_platform_secret(provider_id): + return True + return False + + +def _population_operational_signals( + store: Any, + user_ids: Sequence[int], + *, + now: datetime, + population_wide: bool, +) -> dict[int, OperationalSignals]: + """Shared by both twins: seven owning-store calls, one fold, no per-user query. + + ``user_ids`` is always the set answered for. ``population_wide`` decides + how the sources are *read*: False passes the ids as an IN list (the + live/batched shape, capped by ``_ids``); True passes ``None`` so every + statement is population-wide and the same at any user count -- the daily + job's shape. Either way a user with no row in any source is computed from + the model's defaults and a zero balance, exactly as ``get_operational_facts`` + computes them, rather than being dropped. + """ + current = _utc(now, "now") + if population_wide: + if not isinstance(user_ids, (list, tuple)): + raise ValueError("user_ids must be a list or tuple") + ids = list(dict.fromkeys(positive_user_id(item) for item in user_ids)) + query_ids = None + else: + ids = _ids(user_ids) + query_ids = ids + if not ids: + return {} + + def batched(base: Any, method: str, *args: Any, **kwargs: Any) -> Any: + if not hasattr(base, method): + return {} + result = getattr(base, method)(*args, **kwargs) + if not result: + # Fail-visible: every OperationalSignals default is permissive, so + # a source that silently answers nothing reads as "everyone + # healthy". A fresh deploy may legitimately have no rows, which is + # why this is a line and not an exception. + print(f"WARNING: analytics.operational_signals_empty source={method}") + return result + + balances = batched(store.credits_base, "get_balance_projections", query_ids) # 1 + billing = batched(store.credits_base, "list_account_billing_states", query_ids) # 2 + credentials = batched(store.provider_base, "list_default_credential_facts", query_ids) # 3 + providers = ( + list(store.provider_base.list_all_providers()) # 4 + if hasattr(store.provider_base, "list_all_providers") + else [] + ) + platform_statuses = batched(store.provider_base, "list_platform_credential_statuses") # 5 + owners = batched(store.agent_base, "list_agent_owners", query_ids) # 6 + runs = ( + store.run_base.list_terminal_runs_since(since=current - timedelta(hours=24)) # 7 + if hasattr(store.run_base, "list_terminal_runs_since") + else {} + ) + + providers_by_id = {str(row.get("provider_id")): row for row in providers} + platform_open = _platform_lane_open(providers, platform_statuses) + agents_by_owner: dict[int, list[str]] = {} + for agent_id, owner in owners.items(): + agents_by_owner.setdefault(int(owner), []).append(agent_id) + + signals: dict[int, OperationalSignals] = {} + for user_id in ids: + facts = credentials.get(user_id) + default_ids = facts.default_provider_ids if facts is not None else frozenset() + verified_counts = facts.verified_default_provider_counts if facts is not None else {} + selected_enabled = all( + providers_by_id.get(provider_id, {}).get("status") == "enabled" + for provider_id in default_ids + ) + verified_byok = any( + row.get("status") == "enabled" + and bool(row.get("byok_enabled")) + and verified_counts.get(str(row.get("provider_id")), 0) == 1 + for row in providers + ) + total_available = int( + _object_value(balances.get(user_id, {}), "total_available_micro") + ) + platform_lane = total_available > 0 and platform_open + pooled = sorted( + ( + pair + for agent_id in agents_by_owner.get(user_id, ()) + for pair in runs.get(agent_id, ()) + if pair[0] <= current + ), + key=lambda pair: pair[0], + reverse=True, + ) + signals[user_id] = OperationalSignals( + user_id=user_id, + account_restricted=( + billing.get(user_id, {}).get("account_status") == "restricted" + ), + usable_billing_lane=platform_lane or verified_byok, + selected_provider_enabled=selected_enabled, + default_credential_status=facts.status if facts is not None else "missing", + failed_terminal_runs_24h=consecutive_failed_terminal_runs( + [status for _stamp, status in pooled] + ), + # A live condition about a run happening right now; a fact row for + # a completed day cannot meaningfully carry it. + run_beyond_safe_deadline=False, + ) + return signals + + class CommercialValueFact(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) @@ -271,28 +721,6 @@ def seq(name: str) -> tuple[str, ...]: ) -def _fetchall(conn: Any, postgres: bool, sql: str, params: Sequence[Any]): - """Shared by both twins. - - Still dialect-parameterised: it serves ``list_commercial_values``/ - ``list_credit_activity``, which branch on the *credits* store's dialect, - never on this class's own -- see the module docstring. - """ - if postgres: - with conn.cursor() as cur: - cur.execute(sql, params) - return cur.fetchall() - return conn.execute(sql, params).fetchall() - - -def _user_clause(ids: list[int], postgres: bool) -> tuple[str, list[Any]]: - """Shared by both twins; see ``_fetchall`` above.""" - if postgres: - return "user_id = ANY(%s)", [ids] - placeholders = ", ".join("?" for _ in ids) - return f"user_id IN ({placeholders})", list(ids) - - def _projection_job_name(value: object) -> str: """Shared by both twins; see ``_current_snapshot_from_row`` above.""" if ( @@ -679,6 +1107,193 @@ def has_daily_before(self, before: date) -> bool: row = conn.execute(sql, (before.isoformat(),)).fetchone() return row is not None + def record_activity( + self, + user_id: int, + *, + occurred_at: datetime, + activating: bool, + now: datetime, + ) -> None: + """Advance one user's activity timestamps. One statement, no read. + + The two columns move in opposite directions, on purpose. + + ``activated_at`` keeps the **earliest** success, because activation + is defined as the first server-authoritative ``backtest_completed`` + by ``occurred_at`` -- not by arrival order. Ingestion does not see + events in occurred_at order: a completion can be appended late, + replayed, or backdated up to the 24 hours ``service.py:96-97`` + accepts. A plain ``COALESCE(stored, incoming)`` would freeze + whichever row happened to land first and make the value + uncorrectable afterwards, which would also silently turn the daily + job's ``events`` step repair into a no-op. A non-activating event + passes NULL and the COALESCE pair leaves the stored value alone. + + ``last_meaningful_activity_at`` only ever advances, so a + late-arriving old event cannot make a user look more dormant than + they are. + + Timestamps are ISO-8601 UTC text throughout this schema, which orders + lexicographically, so MIN and MAX over the text are the same + comparisons as over the instants. + + SQLite's scalar ``max(X, Y)`` and ``min(X, Y)`` both return NULL if + **either** argument is NULL, which is why each stored value is + COALESCEd against the incoming one before the comparison rather than + passed raw. + """ + subject_id = positive_user_id(user_id) + occurred = utc_iso(_utc(occurred_at, "occurred_at")) + values = ( + subject_id, + occurred if activating else None, + occurred, + utc_iso(_utc(now, "now")), + ) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) VALUES (?, ?, ?, ?) + ON CONFLICT(user_id) DO UPDATE SET + activated_at = MIN( + COALESCE(user_activity.activated_at, excluded.activated_at), + COALESCE(excluded.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = MAX( + COALESCE( + user_activity.last_meaningful_activity_at, + excluded.last_meaningful_activity_at + ), + excluded.last_meaningful_activity_at + ), + updated_at = excluded.updated_at + """ + with self._analytics_connection() as conn: + conn.execute(sql, values) + + def get_activity(self, user_id: int) -> UserActivity | None: + """One user's stored activity row, or None if they have none yet.""" + subject_id = positive_user_id(user_id) + with self._analytics_connection() as conn: + row = conn.execute( + "SELECT * FROM user_activity WHERE user_id = ?", (subject_id,) + ).fetchone() + return _activity_from_row(row) if row is not None else None + + def list_activity( + self, + user_ids: Sequence[int] | None = None, + ) -> dict[int, UserActivity]: + """Activity rows for many users, or for everyone when ``user_ids`` is None. + + ``None`` is the daily job's shape: one statement over the whole + population, parameterised by nothing. A sequence is batched by + ``MAX_USER_BATCH`` the way ``list_current_snapshots`` already is. + """ + result: dict[int, UserActivity] = {} + if user_ids is None: + with self._analytics_connection() as conn: + rows = conn.execute( + "SELECT * FROM user_activity ORDER BY user_id" + ).fetchall() + for row in rows: + activity = _activity_from_row(row) + result[activity.user_id] = activity + return result + ids = _ids(user_ids) + if not ids: + return {} + for offset in range(0, len(ids), MAX_USER_BATCH): + chunk = ids[offset : offset + MAX_USER_BATCH] + clause = f"user_id IN ({', '.join('?' for _ in chunk)})" + with self._analytics_connection() as conn: + rows = conn.execute( + f"SELECT * FROM user_activity WHERE {clause} ORDER BY user_id", + chunk, + ).fetchall() + for row in rows: + activity = _activity_from_row(row) + result[activity.user_id] = activity + return result + + def seed_activity_from_snapshots(self, *, now: datetime) -> int: + """Copy ``activated_at`` / ``last_meaningful_activity_at`` from the legacy row. + + Design SS11: ``user_analytics_snapshots`` is the only table that knows + when an existing user first activated, and it is dropped in PR B. + Ingestion only maintains ``user_activity`` for events arriving after + this deploy, so without this copy every pre-existing activation date + would vanish on the night of the drop. + + Idempotent by construction -- the same MIN/MAX upsert + ``record_activity`` uses -- so PR B can re-run it immediately before + the drop (the legacy row keeps being maintained by the throttled + repair between the two PRs). Returns the number of rows touched. + """ + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) + SELECT user_id, activated_at, last_meaningful_activity_at, ? + FROM user_analytics_snapshots + WHERE activated_at IS NOT NULL + OR last_meaningful_activity_at IS NOT NULL + ON CONFLICT(user_id) DO UPDATE SET + activated_at = MIN( + COALESCE(user_activity.activated_at, excluded.activated_at), + COALESCE(excluded.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = MAX( + COALESCE( + user_activity.last_meaningful_activity_at, + excluded.last_meaningful_activity_at + ), + COALESCE( + excluded.last_meaningful_activity_at, + user_activity.last_meaningful_activity_at + ) + ), + updated_at = excluded.updated_at + """ + with self._analytics_connection() as conn: + cursor = conn.execute(sql, (stamp,)) + return max(0, int(cursor.rowcount)) + + def sum_recent_facts( + self, + user_ids: Sequence[int] | None, + *, + start: date, + end: date, + ) -> dict[int, RecentFactTotals]: + """Trailing-window totals for many users in one query. + + ``start`` and ``end`` are inclusive UTC dates. ``None`` for + ``user_ids`` means the whole population -- the daily job's shape, + parameterised by two dates and nothing else. One statement for the + whole batch: a per-user loop here is the shape that caused the + outage, and the read-budget test fails on it. + """ + if end < start: + raise ValueError("end must not precede start") + params: list[Any] = [start.isoformat(), end.isoformat()] + user_clause = "" + if user_ids is not None: + ids = _ids(user_ids) + if not ids: + return {} + user_clause = f" AND user_id IN ({', '.join('?' for _ in ids)})" + params.extend(ids) + sql = _RECENT_FACTS_SQL.format(p="?", user_clause=user_clause) + with self._analytics_connection() as conn: + rows = conn.execute(sql, params).fetchall() + return { + int(_row_value(row, "user_id")): _recent_totals_from_row(row) + for row in rows + } + def list_commercial_values( self, user_ids: Sequence[int], @@ -691,92 +1306,16 @@ def list_commercial_values( if not ids: return {} - lifetime_by_user: dict[int, tuple[int, int]] = {} - period_by_user: dict[int, tuple[int, int, int]] = {} - usage_by_user: dict[int, int] = {} - if hasattr(self.credits_base, "_get_connection"): - postgres = hasattr(self.credits_base, "database_url") - user_clause, user_params = _user_clause(ids, postgres) - placeholder = "%s" if postgres else "?" - window_params = [ - *user_params, - utc_iso(window_start), - utc_iso(window_end), - ] - with self.credits_base._get_connection() as conn: - lifetime_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(CASE WHEN entry_type = 'purchase' - THEN amount_micro ELSE 0 END), 0) AS purchased_micro, - COALESCE(SUM(CASE WHEN entry_type = 'refund' - THEN -amount_micro ELSE 0 END), 0) AS refunded_micro - FROM credit_ledger_entries - WHERE {user_clause} - AND entry_type IN ('purchase', 'refund') - GROUP BY user_id - """, - user_params, - ) - period_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(CASE WHEN entry_type = 'purchase' - THEN amount_micro ELSE 0 END), 0) AS purchased_micro, - COALESCE(SUM(CASE WHEN entry_type = 'refund' - THEN -amount_micro ELSE 0 END), 0) AS refunded_micro, - COALESCE(SUM(CASE - WHEN entry_type = 'admin_grant_assign' THEN amount_micro - WHEN entry_type = 'admin_grant_reclaim' THEN -amount_micro - ELSE 0 END), 0) AS grant_activity_micro - FROM credit_ledger_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - GROUP BY user_id - """, - window_params, - ) - usage_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(-amount_micro), 0) AS consumed_micro - FROM credit_llm_usage_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - GROUP BY user_id - """, - window_params, - ) - lifetime_by_user = { - int(_row_value(row, "user_id")): ( - max(int(_row_value(row, "purchased_micro", 0)), 0), - max(int(_row_value(row, "refunded_micro", 0)), 0), - ) - for row in lifetime_rows - } - period_by_user = { - int(_row_value(row, "user_id")): ( - max(int(_row_value(row, "purchased_micro", 0)), 0), - max(int(_row_value(row, "refunded_micro", 0)), 0), - max(int(_row_value(row, "grant_activity_micro", 0)), 0), - ) - for row in period_rows - } - usage_by_user = { - int(_row_value(row, "user_id")): max( - int(_row_value(row, "consumed_micro", 0)), 0 - ) - for row in usage_rows - } - + # The ledger is read through the credits domain (design SS6.14). The + # guard mirrors the old ``hasattr(self.credits_base, "_get_connection")`` + # one: retention.py constructs this store with ``credits_base=object()``. + ledger = ( + self.credits_base.aggregate_commercial_ledger( + ids, start=window_start, end=window_end + ) + if hasattr(self.credits_base, "aggregate_commercial_ledger") + else {} + ) balances = ( self.credits_base.get_balance_projections(ids) if hasattr(self.credits_base, "get_balance_projections") @@ -784,10 +1323,13 @@ def list_commercial_values( ) result: dict[int, CommercialValueFact] = {} for user_id in ids: - lifetime_purchased, lifetime_refunded = lifetime_by_user.get( - user_id, (0, 0) - ) - purchased, refunded, grant_activity = period_by_user.get(user_id, (0, 0, 0)) + totals = ledger.get(user_id, {}) + lifetime_purchased = max(int(totals.get("lifetime_purchased_micro", 0)), 0) + lifetime_refunded = max(int(totals.get("lifetime_refunded_micro", 0)), 0) + purchased = max(int(totals.get("purchased_micro", 0)), 0) + refunded = max(int(totals.get("refunded_micro", 0)), 0) + grant_activity = max(int(totals.get("grant_activity_micro", 0)), 0) + consumed = max(int(totals.get("consumed_micro", 0)), 0) net_purchased = max(lifetime_purchased - lifetime_refunded, 0) balance = balances.get(user_id, {}) result[user_id] = CommercialValueFact( @@ -796,7 +1338,7 @@ def list_commercial_values( commercial_tier=commercial_tier(net_purchased), purchased_micro=purchased, refunded_micro=refunded, - consumed_micro=usage_by_user.get(user_id, 0), + consumed_micro=consumed, admin_grant_activity_micro=grant_activity, grant_available_micro=max( int(_object_value(balance, "grant_available_micro")), 0 @@ -809,7 +1351,6 @@ def list_commercial_values( ), ) return result - def list_credit_activity( self, user_ids: Sequence[int], @@ -819,48 +1360,15 @@ def list_credit_activity( ) -> dict[int, Sequence[datetime]]: ids = _ids(user_ids) window_start, window_end = _validate_window(start, end) - result: dict[int, list[datetime]] = {user_id: [] for user_id in ids} - if not ids or not hasattr(self.credits_base, "_get_connection"): + if not ids or not hasattr(self.credits_base, "list_credit_activity_timestamps"): return {user_id: () for user_id in ids} - - postgres = hasattr(self.credits_base, "database_url") - user_clause, user_params = _user_clause(ids, postgres) - placeholder = "%s" if postgres else "?" - params = [*user_params, utc_iso(window_start), utc_iso(window_end)] - with self.credits_base._get_connection() as conn: - purchase_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, created_at - FROM credit_ledger_entries - WHERE {user_clause} - AND entry_type = 'purchase' - AND created_at >= {placeholder} - AND created_at < {placeholder} - """, - params, - ) - usage_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, created_at - FROM credit_llm_usage_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - """, - params, - ) - for row in (*purchase_rows, *usage_rows): - user_id = int(_row_value(row, "user_id")) - if user_id in result: - result[user_id].append(_timestamp(_row_value(row, "created_at"))) + stamps = self.credits_base.list_credit_activity_timestamps( + ids, start=window_start, end=window_end + ) return { - user_id: tuple(sorted(timestamps)) for user_id, timestamps in result.items() + user_id: tuple(sorted(_timestamp(value) for value in stamps.get(user_id, ()))) + for user_id in ids } - def _run_health(self, user_id: int, now: datetime) -> tuple[int, bool]: if self.agent_base is None or self.run_base is None: return 0, False @@ -893,11 +1401,9 @@ def _run_health(self, user_id: int, now: datetime) -> tuple[int, bool]: ) and now - timedelta(hours=24) <= timestamp <= now ] - consecutive_failures = 0 - for run in terminal_24h: - if str(run.get("status")) not in {"failed", "timed_out"}: - break - consecutive_failures += 1 + consecutive_failures = consecutive_failed_terminal_runs( + [str(run.get("status")) for run in terminal_24h] + ) beyond_deadline = False for run in ordered: @@ -1002,6 +1508,27 @@ def provider_supports(row: Mapping[str, Any], mode: str) -> bool: run_beyond_safe_deadline=beyond_deadline, ) + def list_operational_signals( + self, + user_ids: Sequence[int], + *, + now: datetime, + population_wide: bool = False, + ) -> dict[int, OperationalSignals]: + """Operational signals for many users, at a fixed query count. + + The per-user twin, ``get_operational_facts``, fans out into roughly + five store calls plus one per agent the user owns. That is right for + one profile and catastrophic for a population, so the daily job uses + this instead, with ``population_wide=True`` so no statement carries an + IN list that grows with the population. Both must produce the same + answer; the equivalence is pinned by + tests/domain/analytics/test_operational_signals.py. + """ + return _population_operational_signals( + self, user_ids, now=now, population_wide=population_wide + ) + def get_projection_job(self, job_name: str) -> ProjectionJob | None: name = _projection_job_name(job_name) with self._analytics_connection() as conn: @@ -1045,6 +1572,262 @@ def save_projection_job(self, job: ProjectionJob) -> ProjectionJob: conn.execute(sql, values) return job + def claim_projection_day( + self, + job_name: str, + *, + day: date, + now: datetime, + stale_after: timedelta = timedelta(hours=2), + ) -> bool: + """Take the lease on ``day`` for ``job_name``; True for the caller that won. + + Two fields, two questions (design SS6.9). ``cursor`` answers "which + day is done" and only ``complete_projection_day`` moves it, so a day + is never run twice once it succeeded. ``status`` answers "is someone + running it now": the compare-and-set below moves it from + ``pending``/``complete`` to ``running`` and stamps ``updated_at``; a + second process whose UPDATE matches nothing gets False. A ``running`` + lease older than ``stale_after`` is a crash, not a worker, and may be + taken over -- which is how a job killed mid-day retries on the next + tick instead of never. + + The cursor comparison is lexicographic on ISO dates. ``cursor >= day`` + means the day (or a later one) already completed and the claim is + refused without a write. + """ + name = _projection_job_name(job_name) + target = day.isoformat() + stamp = utc_iso(_utc(now, "now")) + stale_before = utc_iso(_utc(now, "now") - stale_after) + with self._analytics_connection() as conn: + conn.execute( + """ + INSERT INTO analytics_projection_jobs ( + job_name, window_start, window_end, cursor, status, updated_at + ) VALUES (?, ?, ?, NULL, 'pending', ?) + ON CONFLICT(job_name) DO NOTHING + """, + (name, target, target, stamp), + ) + cursor = conn.execute( + """ + UPDATE analytics_projection_jobs + SET status = 'running', window_end = ?, updated_at = ? + WHERE job_name = ? + AND (cursor IS NULL OR cursor < ?) + AND ( + status IN ('pending', 'complete') + OR (status = 'running' AND updated_at < ?) + ) + """, + (target, stamp, name, target, stale_before), + ) + return cursor.rowcount == 1 + + def complete_projection_day(self, job_name: str, *, day: date, now: datetime) -> None: + """Record ``day`` as done: cursor -> day, status -> complete.""" + name = _projection_job_name(job_name) + with self._analytics_connection() as conn: + conn.execute( + """ + UPDATE analytics_projection_jobs + SET cursor = ?, status = 'complete', updated_at = ? + WHERE job_name = ? + """, + (day.isoformat(), utc_iso(_utc(now, "now")), name), + ) + + def release_projection_day(self, job_name: str, *, now: datetime) -> None: + """Give a failed day back: status -> pending, cursor untouched. + + Called when a step failed, so the next tick's claim retries the same + day at once instead of waiting out the stale window. + """ + name = _projection_job_name(job_name) + with self._analytics_connection() as conn: + conn.execute( + """ + UPDATE analytics_projection_jobs + SET status = 'pending', updated_at = ? + WHERE job_name = ? AND status = 'running' + """, + (utc_iso(_utc(now, "now")), name), + ) + + def aggregate_events_for_day(self, day: date) -> dict[int, DayEventTotals]: + """Per-user outcome counts and activity marks for one UTC day. + + The one-day scan the event-log discipline permits (design SS6.11 rule + 4); it takes no user id. The fifteen activity names are bound as + parameters rather than interpolated. + """ + start, end = _day_bounds_iso(day) + activity = ", ".join("?" for _ in _ACTIVITY_EVENT_NAMES) + sql = _EVENTS_FOR_DAY_SQL.format(activity=activity, p="?") + with self._analytics_connection() as conn: + rows = conn.execute(sql, [*_ACTIVITY_EVENT_NAMES, start, end]).fetchall() + return { + int(_row_value(row, "user_id")): _events_totals_from_row(row) for row in rows + } + + def record_activity_batch( + self, + updates: Sequence[ActivityUpdate], + *, + now: datetime, + ) -> int: + """Many ``record_activity`` corrections in one ``executemany``. + + Same MIN/MAX upsert as ``record_activity`` and ``seed_activity_from_snapshots``, + with both incoming columns nullable: the daily job's ledger step knows + a user's last credit activity but nothing about activation, and its + events step knows both. A per-user ``record_activity`` loop would be a + query count that grows with the population (design SS6.12). Returns + the number of updates submitted; an empty batch touches nothing. + """ + if not updates: + return 0 + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) VALUES (?, ?, ?, ?) + ON CONFLICT(user_id) DO UPDATE SET + activated_at = MIN( + COALESCE(user_activity.activated_at, excluded.activated_at), + COALESCE(excluded.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = MAX( + COALESCE( + user_activity.last_meaningful_activity_at, + excluded.last_meaningful_activity_at + ), + COALESCE( + excluded.last_meaningful_activity_at, + user_activity.last_meaningful_activity_at + ) + ), + updated_at = excluded.updated_at + """ + with self._analytics_connection() as conn: + conn.executemany( + sql, [_activity_update_values(update, stamp) for update in updates] + ) + return len(updates) + + def upsert_daily_facts(self, rows: Sequence[UserDailyFact]) -> int: + """Write one batch of fact rows. One executemany, not a loop.""" + if not rows: + return 0 + columns = ", ".join(_DAILY_FACT_COLUMNS) + placeholders = ", ".join("?" for _ in _DAILY_FACT_COLUMNS) + sql = f""" + INSERT INTO user_daily_facts ({columns}) + VALUES ({placeholders}) + ON CONFLICT(snapshot_date, user_id) DO UPDATE SET {_DAILY_FACT_UPDATES} + """ + with self._analytics_connection() as conn: + conn.executemany( + sql, [_fact_values(row, active_value=int(row.active)) for row in rows] + ) + return len(rows) + + def append_lifecycle_transitions( + self, rows: Sequence[LifecycleTransitionRow] + ) -> int: + """Write segment changes for one day, correcting any already there. + + ``ON CONFLICT (user_id, snapshot_date) DO UPDATE`` -- **not** DO + NOTHING. The day this table is rewritten is exactly the day something + went wrong: a step failed and the row was derived from a missing + source, or a late event changed the day's totals. DO NOTHING would + freeze that first, weakest answer forever while the retry corrected + ``user_daily_facts`` (which upserts), leaving the two tables in + permanent disagreement (design SS6.8). The UNIQUE constraint's job is + to stop a *duplicate*, which DO UPDATE does equally well. + """ + if not rows: + return 0 + columns = ", ".join(_TRANSITION_COLUMNS) + placeholders = ", ".join("?" for _ in _TRANSITION_COLUMNS) + sql = f""" + INSERT INTO lifecycle_transitions ({columns}) + VALUES ({placeholders}) + ON CONFLICT(user_id, snapshot_date) DO UPDATE SET {_TRANSITION_UPDATES} + """ + with self._analytics_connection() as conn: + conn.executemany(sql, [_transition_values(row) for row in rows]) + return len(rows) + + def list_facts_for_date(self, day: date) -> list[UserDailyFact]: + """Every fact row for one date. Used for the previous day's segments.""" + with self._analytics_connection() as conn: + rows = conn.execute( + "SELECT * FROM user_daily_facts WHERE snapshot_date = ? ORDER BY user_id", + (day.isoformat(),), + ).fetchall() + return [_fact_from_row(row) for row in rows] + + def list_days_needing_recompute(self, *, since: date, until: date) -> list[date]: + """Past days whose events arrived after their facts were computed. + + Two statements, neither parameterised by a user: events grouped by + UTC day with ``MAX(received_at)``, and facts grouped by date with + ``MIN(calculated_at)``; a day is returned, newest first, when the + former exceeds the latter. This exists because + ``aggregate_events_for_day`` filters on ``occurred_at`` while the job + runs minutes after midnight: the frontend route accepts ``occurred_at`` + up to 24 hours old (``service.py:96-97``), and a server event for a run + that finished at 23:59 can be appended after the aggregate was taken. + Without this sweep every such row would be silently dropped from + ``active``, DAU and the run counts, permanently and with no signal. + """ + if until < since: + raise ValueError("until must not precede since") + start, _unused = _day_bounds_iso(since) + _unused, end = _day_bounds_iso(until) + with self._analytics_connection() as conn: + event_rows = conn.execute( + _RECOMPUTE_EVENTS_SQL.format(p="?"), (start, end) + ).fetchall() + fact_rows = conn.execute( + _RECOMPUTE_FACTS_SQL.format(p="?"), + (since.isoformat(), until.isoformat()), + ).fetchall() + return _recompute_days(event_rows, fact_rows) + + def copy_daily_snapshot_history(self, *, since: date, now: datetime) -> int: + """Copy legacy ``user_lifecycle_daily_snapshots`` rows into the fact table. + + Run, cost and tier columns are filled with their neutral values and + every copied row is ``partial``, so the UI labels the period + "Incomplete data" instead of charting zeros as fact (design SS6.7). + ``tier='unpaid'`` is a placeholder, not a claim, for the same reason. + ``user_group`` is read from ``users`` at the copy (D9). ``ON CONFLICT + DO NOTHING`` makes it both idempotent and unable to clobber a row the + daily job already computed. Returns the number of rows inserted. + """ + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, operational_reason_code, tier, user_group, active, + runs_requested, runs_completed, runs_failed, runs_cancelled, + operator_cost_micro, own_spend_micro, data_quality, calculated_at + ) + SELECT s.snapshot_date, s.user_id, s.lifecycle_segment, + s.lifecycle_reason_code, 'healthy', NULL, 'unpaid', + users.user_group, 0, 0, 0, 0, 0, 0, 0, 'partial', ? + FROM user_lifecycle_daily_snapshots AS s + JOIN users ON users.id = s.user_id + WHERE s.snapshot_date >= ? + ON CONFLICT(snapshot_date, user_id) DO NOTHING + """ + with self._analytics_connection() as conn: + cursor = conn.execute(sql, (stamp, since.isoformat())) + return max(0, int(cursor.rowcount)) + def build_value_analytics_store( analytics_base: Any | None = None, @@ -1087,6 +1870,12 @@ def build_value_analytics_store( "CommercialValueFact", "CurrentOperationalFacts", "ProjectionJob", + "ActivityUpdate", + "DayEventTotals", + "RecentFactTotals", + "LifecycleTransitionRow", + "UserActivity", + "UserDailyFact", "UserLifecycleDailySnapshot", "UserValueSnapshot", "ValueAnalyticsStore", diff --git a/dashboard/backend/domain/analytics/value_repository_postgres.py b/dashboard/backend/domain/analytics/value_repository_postgres.py index dadae100..038da4b9 100644 --- a/dashboard/backend/domain/analytics/value_repository_postgres.py +++ b/dashboard/backend/domain/analytics/value_repository_postgres.py @@ -18,17 +18,26 @@ from .value_repository import ( _ACTIVE_RUN_STATUSES, _TERMINAL_RUN_STATUSES, + _RECENT_FACTS_SQL, + _activity_from_row, + _population_operational_signals, + _recent_totals_from_row, LIFECYCLE_ROLLUP_METRICS, LIFECYCLE_SEGMENTS, MAX_USER_BATCH, RUN_SAFE_DEADLINE, CommercialValueFact, CurrentOperationalFacts, + ActivityUpdate, + DayEventTotals, + LifecycleTransitionRow, ProjectionJob, + RecentFactTotals, + UserActivity, + UserDailyFact, UserLifecycleDailySnapshot, UserValueSnapshot, _current_snapshot_from_row, - _fetchall, _ids, _legacy_seed, _object_value, @@ -36,12 +45,15 @@ _projection_job_name, _row_value, _timestamp, - _user_clause, _utc, _validate_window, analytics_store, commercial_tier, ) +from .lifecycle import ( + OperationalSignals, + consecutive_failed_terminal_runs, +) class PostgresValueAnalyticsStore: @@ -431,6 +443,304 @@ def has_daily_before(self, before: date) -> bool: row = cur.fetchone() return row is not None + def record_activity( + self, + user_id: int, + *, + occurred_at: datetime, + activating: bool, + now: datetime, + ) -> None: + """See the SQLite twin. Postgres ``LEAST``/``GREATEST`` skip NULLs, so + the COALESCE pair is redundant here but harmless; keeping both dialects + spelled the same way is worth more than saving two lines. ``%s::text`` + on the nullable ``activated_at`` gives psycopg a type for the ``None`` + it would otherwise send as OID 0. + """ + subject_id = positive_user_id(user_id) + occurred = utc_iso(_utc(occurred_at, "occurred_at")) + values = ( + subject_id, + occurred if activating else None, + occurred, + utc_iso(_utc(now, "now")), + ) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) VALUES (%s, %s::text, %s, %s) + ON CONFLICT(user_id) DO UPDATE SET + activated_at = LEAST( + COALESCE(user_activity.activated_at, EXCLUDED.activated_at), + COALESCE(EXCLUDED.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = GREATEST( + COALESCE( + user_activity.last_meaningful_activity_at, + EXCLUDED.last_meaningful_activity_at + ), + EXCLUDED.last_meaningful_activity_at + ), + updated_at = EXCLUDED.updated_at + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, values) + + def get_activity(self, user_id: int) -> UserActivity | None: + """One user's stored activity row, or None if they have none yet.""" + subject_id = positive_user_id(user_id) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "SELECT * FROM user_activity WHERE user_id = %s", (subject_id,) + ) + row = cur.fetchone() + return _activity_from_row(row) if row is not None else None + + def list_activity( + self, + user_ids: Sequence[int] | None = None, + ) -> dict[int, UserActivity]: + """See the SQLite twin.""" + result: dict[int, UserActivity] = {} + if user_ids is None: + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute("SELECT * FROM user_activity ORDER BY user_id") + rows = cur.fetchall() + for row in rows: + activity = _activity_from_row(row) + result[activity.user_id] = activity + return result + ids = _ids(user_ids) + if not ids: + return {} + for offset in range(0, len(ids), MAX_USER_BATCH): + chunk = ids[offset : offset + MAX_USER_BATCH] + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "SELECT * FROM user_activity WHERE user_id = ANY(%s) " + "ORDER BY user_id", + (chunk,), + ) + rows = cur.fetchall() + for row in rows: + activity = _activity_from_row(row) + result[activity.user_id] = activity + return result + + def seed_activity_from_snapshots(self, *, now: datetime) -> int: + """See the SQLite twin.""" + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) + SELECT user_id, activated_at, last_meaningful_activity_at, %s + FROM user_analytics_snapshots + WHERE activated_at IS NOT NULL + OR last_meaningful_activity_at IS NOT NULL + ON CONFLICT(user_id) DO UPDATE SET + activated_at = LEAST( + COALESCE(user_activity.activated_at, EXCLUDED.activated_at), + COALESCE(EXCLUDED.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = GREATEST( + COALESCE( + user_activity.last_meaningful_activity_at, + EXCLUDED.last_meaningful_activity_at + ), + COALESCE( + EXCLUDED.last_meaningful_activity_at, + user_activity.last_meaningful_activity_at + ) + ), + updated_at = EXCLUDED.updated_at + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, (stamp,)) + return max(0, int(cur.rowcount)) + + def sum_recent_facts( + self, + user_ids: Sequence[int] | None, + *, + start: date, + end: date, + ) -> dict[int, RecentFactTotals]: + """See the SQLite twin.""" + if end < start: + raise ValueError("end must not precede start") + params: list[Any] = [start.isoformat(), end.isoformat()] + user_clause = "" + if user_ids is not None: + ids = _ids(user_ids) + if not ids: + return {} + user_clause = " AND user_id = ANY(%s)" + params.append(ids) + sql = _RECENT_FACTS_SQL.format(p="%s", user_clause=user_clause) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, params) + rows = cur.fetchall() + return { + int(_row_value(row, "user_id")): _recent_totals_from_row(row) + for row in rows + } + + def aggregate_events_for_day(self, day: date) -> dict[int, DayEventTotals]: + """See the SQLite twin.""" + start, end = _day_bounds_iso(day) + activity = ", ".join("%s" for _ in _ACTIVITY_EVENT_NAMES) + sql = _EVENTS_FOR_DAY_SQL.format(activity=activity, p="%s") + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, [*_ACTIVITY_EVENT_NAMES, start, end]) + rows = cur.fetchall() + return { + int(_row_value(row, "user_id")): _events_totals_from_row(row) for row in rows + } + + def record_activity_batch( + self, + updates: Sequence[ActivityUpdate], + *, + now: datetime, + ) -> int: + """See the SQLite twin.""" + if not updates: + return 0 + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_activity ( + user_id, activated_at, last_meaningful_activity_at, updated_at + ) VALUES (%s, %s::text, %s::text, %s) + ON CONFLICT(user_id) DO UPDATE SET + activated_at = LEAST( + COALESCE(user_activity.activated_at, EXCLUDED.activated_at), + COALESCE(EXCLUDED.activated_at, user_activity.activated_at) + ), + last_meaningful_activity_at = GREATEST( + COALESCE( + user_activity.last_meaningful_activity_at, + EXCLUDED.last_meaningful_activity_at + ), + COALESCE( + EXCLUDED.last_meaningful_activity_at, + user_activity.last_meaningful_activity_at + ) + ), + updated_at = EXCLUDED.updated_at + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.executemany( + sql, [_activity_update_values(update, stamp) for update in updates] + ) + return len(updates) + + def upsert_daily_facts(self, rows: Sequence[UserDailyFact]) -> int: + """See the SQLite twin. ``active`` is BOOLEAN here.""" + if not rows: + return 0 + columns = ", ".join(_DAILY_FACT_COLUMNS) + placeholders = ", ".join("%s" for _ in _DAILY_FACT_COLUMNS) + sql = f""" + INSERT INTO user_daily_facts ({columns}) + VALUES ({placeholders}) + ON CONFLICT(snapshot_date, user_id) DO UPDATE SET {_DAILY_FACT_UPDATES} + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.executemany( + sql, [_fact_values(row, active_value=bool(row.active)) for row in rows] + ) + return len(rows) + + def append_lifecycle_transitions( + self, rows: Sequence[LifecycleTransitionRow] + ) -> int: + """See the SQLite twin.""" + if not rows: + return 0 + columns = ", ".join(_TRANSITION_COLUMNS) + placeholders = ", ".join("%s" for _ in _TRANSITION_COLUMNS) + sql = f""" + INSERT INTO lifecycle_transitions ({columns}) + VALUES ({placeholders}) + ON CONFLICT(user_id, snapshot_date) DO UPDATE SET {_TRANSITION_UPDATES} + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.executemany(sql, [_transition_values(row) for row in rows]) + return len(rows) + + def list_facts_for_date(self, day: date) -> list[UserDailyFact]: + """See the SQLite twin.""" + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "SELECT * FROM user_daily_facts WHERE snapshot_date = %s ORDER BY user_id", + (day.isoformat(),), + ) + rows = cur.fetchall() + return [_fact_from_row(row) for row in rows] + + def list_days_needing_recompute(self, *, since: date, until: date) -> list[date]: + """See the SQLite twin.""" + if until < since: + raise ValueError("until must not precede since") + start, _unused = _day_bounds_iso(since) + _unused, end = _day_bounds_iso(until) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(_RECOMPUTE_EVENTS_SQL.format(p="%s"), (start, end)) + event_rows = cur.fetchall() + cur.execute( + _RECOMPUTE_FACTS_SQL.format(p="%s"), + (since.isoformat(), until.isoformat()), + ) + fact_rows = cur.fetchall() + return _recompute_days(event_rows, fact_rows) + + def copy_daily_snapshot_history(self, *, since: date, now: datetime) -> int: + """See the SQLite twin. ``active`` is BOOLEAN here.""" + stamp = utc_iso(_utc(now, "now")) + sql = """ + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, operational_reason_code, tier, user_group, active, + runs_requested, runs_completed, runs_failed, runs_cancelled, + operator_cost_micro, own_spend_micro, data_quality, calculated_at + ) + SELECT s.snapshot_date, s.user_id, s.lifecycle_segment, + s.lifecycle_reason_code, 'healthy', NULL, 'unpaid', + users.user_group, FALSE, 0, 0, 0, 0, 0, 0, 'partial', %s + FROM user_lifecycle_daily_snapshots AS s + JOIN users ON users.id = s.user_id + WHERE s.snapshot_date >= %s + ON CONFLICT(snapshot_date, user_id) DO NOTHING + """ + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, (stamp, since.isoformat())) + return max(0, int(cur.rowcount)) + + def list_operational_signals( + self, + user_ids: Sequence[int], + *, + now: datetime, + population_wide: bool = False, + ) -> dict[int, OperationalSignals]: + """See the SQLite twin.""" + return _population_operational_signals( + self, user_ids, now=now, population_wide=population_wide + ) def list_commercial_values( self, user_ids: Sequence[int], @@ -443,92 +753,16 @@ def list_commercial_values( if not ids: return {} - lifetime_by_user: dict[int, tuple[int, int]] = {} - period_by_user: dict[int, tuple[int, int, int]] = {} - usage_by_user: dict[int, int] = {} - if hasattr(self.credits_base, "_get_connection"): - postgres = hasattr(self.credits_base, "database_url") - user_clause, user_params = _user_clause(ids, postgres) - placeholder = "%s" if postgres else "?" - window_params = [ - *user_params, - utc_iso(window_start), - utc_iso(window_end), - ] - with self.credits_base._get_connection() as conn: - lifetime_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(CASE WHEN entry_type = 'purchase' - THEN amount_micro ELSE 0 END), 0) AS purchased_micro, - COALESCE(SUM(CASE WHEN entry_type = 'refund' - THEN -amount_micro ELSE 0 END), 0) AS refunded_micro - FROM credit_ledger_entries - WHERE {user_clause} - AND entry_type IN ('purchase', 'refund') - GROUP BY user_id - """, - user_params, - ) - period_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(CASE WHEN entry_type = 'purchase' - THEN amount_micro ELSE 0 END), 0) AS purchased_micro, - COALESCE(SUM(CASE WHEN entry_type = 'refund' - THEN -amount_micro ELSE 0 END), 0) AS refunded_micro, - COALESCE(SUM(CASE - WHEN entry_type = 'admin_grant_assign' THEN amount_micro - WHEN entry_type = 'admin_grant_reclaim' THEN -amount_micro - ELSE 0 END), 0) AS grant_activity_micro - FROM credit_ledger_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - GROUP BY user_id - """, - window_params, - ) - usage_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, - COALESCE(SUM(-amount_micro), 0) AS consumed_micro - FROM credit_llm_usage_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - GROUP BY user_id - """, - window_params, - ) - lifetime_by_user = { - int(_row_value(row, "user_id")): ( - max(int(_row_value(row, "purchased_micro", 0)), 0), - max(int(_row_value(row, "refunded_micro", 0)), 0), - ) - for row in lifetime_rows - } - period_by_user = { - int(_row_value(row, "user_id")): ( - max(int(_row_value(row, "purchased_micro", 0)), 0), - max(int(_row_value(row, "refunded_micro", 0)), 0), - max(int(_row_value(row, "grant_activity_micro", 0)), 0), - ) - for row in period_rows - } - usage_by_user = { - int(_row_value(row, "user_id")): max( - int(_row_value(row, "consumed_micro", 0)), 0 - ) - for row in usage_rows - } - + # The ledger is read through the credits domain (design SS6.14). The + # guard mirrors the old ``hasattr(self.credits_base, "_get_connection")`` + # one: retention.py constructs this store with ``credits_base=object()``. + ledger = ( + self.credits_base.aggregate_commercial_ledger( + ids, start=window_start, end=window_end + ) + if hasattr(self.credits_base, "aggregate_commercial_ledger") + else {} + ) balances = ( self.credits_base.get_balance_projections(ids) if hasattr(self.credits_base, "get_balance_projections") @@ -536,10 +770,13 @@ def list_commercial_values( ) result: dict[int, CommercialValueFact] = {} for user_id in ids: - lifetime_purchased, lifetime_refunded = lifetime_by_user.get( - user_id, (0, 0) - ) - purchased, refunded, grant_activity = period_by_user.get(user_id, (0, 0, 0)) + totals = ledger.get(user_id, {}) + lifetime_purchased = max(int(totals.get("lifetime_purchased_micro", 0)), 0) + lifetime_refunded = max(int(totals.get("lifetime_refunded_micro", 0)), 0) + purchased = max(int(totals.get("purchased_micro", 0)), 0) + refunded = max(int(totals.get("refunded_micro", 0)), 0) + grant_activity = max(int(totals.get("grant_activity_micro", 0)), 0) + consumed = max(int(totals.get("consumed_micro", 0)), 0) net_purchased = max(lifetime_purchased - lifetime_refunded, 0) balance = balances.get(user_id, {}) result[user_id] = CommercialValueFact( @@ -548,7 +785,7 @@ def list_commercial_values( commercial_tier=commercial_tier(net_purchased), purchased_micro=purchased, refunded_micro=refunded, - consumed_micro=usage_by_user.get(user_id, 0), + consumed_micro=consumed, admin_grant_activity_micro=grant_activity, grant_available_micro=max( int(_object_value(balance, "grant_available_micro")), 0 @@ -561,7 +798,6 @@ def list_commercial_values( ), ) return result - def list_credit_activity( self, user_ids: Sequence[int], @@ -571,48 +807,15 @@ def list_credit_activity( ) -> dict[int, Sequence[datetime]]: ids = _ids(user_ids) window_start, window_end = _validate_window(start, end) - result: dict[int, list[datetime]] = {user_id: [] for user_id in ids} - if not ids or not hasattr(self.credits_base, "_get_connection"): + if not ids or not hasattr(self.credits_base, "list_credit_activity_timestamps"): return {user_id: () for user_id in ids} - - postgres = hasattr(self.credits_base, "database_url") - user_clause, user_params = _user_clause(ids, postgres) - placeholder = "%s" if postgres else "?" - params = [*user_params, utc_iso(window_start), utc_iso(window_end)] - with self.credits_base._get_connection() as conn: - purchase_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, created_at - FROM credit_ledger_entries - WHERE {user_clause} - AND entry_type = 'purchase' - AND created_at >= {placeholder} - AND created_at < {placeholder} - """, - params, - ) - usage_rows = _fetchall( - conn, - postgres, - f""" - SELECT user_id, created_at - FROM credit_llm_usage_entries - WHERE {user_clause} - AND created_at >= {placeholder} - AND created_at < {placeholder} - """, - params, - ) - for row in (*purchase_rows, *usage_rows): - user_id = int(_row_value(row, "user_id")) - if user_id in result: - result[user_id].append(_timestamp(_row_value(row, "created_at"))) + stamps = self.credits_base.list_credit_activity_timestamps( + ids, start=window_start, end=window_end + ) return { - user_id: tuple(sorted(timestamps)) for user_id, timestamps in result.items() + user_id: tuple(sorted(_timestamp(value) for value in stamps.get(user_id, ()))) + for user_id in ids } - def _run_health(self, user_id: int, now: datetime) -> tuple[int, bool]: if self.agent_base is None or self.run_base is None: return 0, False @@ -645,11 +848,9 @@ def _run_health(self, user_id: int, now: datetime) -> tuple[int, bool]: ) and now - timedelta(hours=24) <= timestamp <= now ] - consecutive_failures = 0 - for run in terminal_24h: - if str(run.get("status")) not in {"failed", "timed_out"}: - break - consecutive_failures += 1 + consecutive_failures = consecutive_failed_terminal_runs( + [str(run.get("status")) for run in terminal_24h] + ) beyond_deadline = False for run in ordered: @@ -799,3 +1000,70 @@ def save_projection_job(self, job: ProjectionJob) -> ProjectionJob: with conn.cursor() as cur: cur.execute(sql, values) return job + + def claim_projection_day( + self, + job_name: str, + *, + day: date, + now: datetime, + stale_after: timedelta = timedelta(hours=2), + ) -> bool: + """See the SQLite twin.""" + name = _projection_job_name(job_name) + target = day.isoformat() + stamp = utc_iso(_utc(now, "now")) + stale_before = utc_iso(_utc(now, "now") - stale_after) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + INSERT INTO analytics_projection_jobs ( + job_name, window_start, window_end, cursor, status, updated_at + ) VALUES (%s, %s, %s, NULL, 'pending', %s) + ON CONFLICT(job_name) DO NOTHING + """, + (name, target, target, stamp), + ) + cur.execute( + """ + UPDATE analytics_projection_jobs + SET status = 'running', window_end = %s, updated_at = %s + WHERE job_name = %s + AND (cursor IS NULL OR cursor < %s) + AND ( + status IN ('pending', 'complete') + OR (status = 'running' AND updated_at < %s) + ) + """, + (target, stamp, name, target, stale_before), + ) + return cur.rowcount == 1 + + def complete_projection_day(self, job_name: str, *, day: date, now: datetime) -> None: + """See the SQLite twin.""" + name = _projection_job_name(job_name) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + UPDATE analytics_projection_jobs + SET cursor = %s, status = 'complete', updated_at = %s + WHERE job_name = %s + """, + (day.isoformat(), utc_iso(_utc(now, "now")), name), + ) + + def release_projection_day(self, job_name: str, *, now: datetime) -> None: + """See the SQLite twin.""" + name = _projection_job_name(job_name) + with self._analytics_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + UPDATE analytics_projection_jobs + SET status = 'pending', updated_at = %s + WHERE job_name = %s AND status = 'running' + """, + (utc_iso(_utc(now, "now")), name), + ) diff --git a/dashboard/backend/domain/backtesting/engine.py b/dashboard/backend/domain/backtesting/engine.py index 4cffcf6f..dbb60eac 100644 --- a/dashboard/backend/domain/backtesting/engine.py +++ b/dashboard/backend/domain/backtesting/engine.py @@ -231,6 +231,7 @@ def __init__( model: str = None, pipeline: list = None, live_run_id: str = None, + owner_user_id: Optional[int] = None, progress_file: str = None, data_source: str = ALPACA, initial_capital: float = None, @@ -280,6 +281,7 @@ def __init__( self.model = model or default_model_name() self.execution_client = execution_client self.live_run_id = (live_run_id or "").strip() or None + self.owner_user_id = int(owner_user_id) if owner_user_id is not None else None self.progress_file = (progress_file or "").strip() or None self._init_progress_phases(launched_at, startup_clock) self.data_source = data_source @@ -2187,6 +2189,7 @@ def run_agent_backtest(self) -> Tuple[str, List[Dict]]: output_tokens=manager.output_tokens, est_cost_usd=est_cost, metadata=self._agent_run_metadata(), + owner_user_id=self.owner_user_id, ) db.insert_equity_points(run_id, equity_curve) @@ -2294,6 +2297,7 @@ def run_buyhold_baseline(self) -> Tuple[str, List[Dict]]: costs_applied=profile.transaction_cost_profile is not None, baseline_allocation=baseline_allocation, ), + owner_user_id=self.owner_user_id, ) db.insert_equity_points(run_id, equity_history) @@ -2372,6 +2376,7 @@ def run_djia_baseline(self) -> Tuple[str, List[Dict]]: max_drawdown=self._calc_max_dd(equity_history), num_trades=0, metadata=self._run_metadata(), + owner_user_id=self.owner_user_id, ) db.insert_equity_points(run_id, equity_history) diff --git a/dashboard/backend/domain/credits/repository.py b/dashboard/backend/domain/credits/repository.py index b93b6497..433ef342 100644 --- a/dashboard/backend/domain/credits/repository.py +++ b/dashboard/backend/domain/credits/repository.py @@ -5,9 +5,9 @@ import json import os import sqlite3 -from collections.abc import Iterator +from collections.abc import Iterator, Sequence from contextlib import contextmanager -from datetime import datetime, timedelta +from datetime import date, datetime, timedelta, timezone from pathlib import Path from typing import Any @@ -30,6 +30,12 @@ _required_text, _utcnow_iso, _validate_amount_pair, + _assemble_billing_states, + _assemble_commercial_ledger, + _assemble_ledger_day, + _day_bounds, + _unique_user_ids, + _utc_text, ) from dashboard.backend.domain.model_providers.repository_common import ( validate_provider_id, @@ -902,16 +908,30 @@ def get_balance_projection(self, user_id: int) -> dict[str, int]: return self._balance_projection_in_transaction(conn, user_id) def get_balance_projections( - self, user_ids: list[int] | tuple[int, ...] + self, user_ids: list[int] | tuple[int, ...] | None ) -> dict[int, dict[str, int]]: - if not isinstance(user_ids, (list, tuple)): - raise ValueError("user_ids must be a list or tuple") - validated = [_positive_integer(user_id, "user_id") for user_id in user_ids] - if not validated: - return {} + """Balances for many accounts; ``None`` means every account with a row. - unique_ids = list(dict.fromkeys(validated)) - placeholders = ", ".join("?" for _ in unique_ids) + ``None`` is the daily job's shape (design SS6.12): the four statements + below then carry no ``WHERE user_id IN`` clause at all, so the query + count and shape are the same at 200 and 20,000 users. + """ + if user_ids is None: + unique_ids = None + where = "" + params: list[Any] = [] + else: + if not isinstance(user_ids, (list, tuple)): + raise ValueError("user_ids must be a list or tuple") + validated = [_positive_integer(user_id, "user_id") for user_id in user_ids] + if not validated: + return {} + unique_ids = list(dict.fromkeys(validated)) + where = f"WHERE user_id IN ({', '.join('?' for _ in unique_ids)})" + params = list(unique_ids) + reservation_filter = ( + f"{where} AND status = 'open'" if where else "WHERE status = 'open'" + ) with self._get_connection() as conn: rows = conn.execute( f""" @@ -924,20 +944,20 @@ def get_balance_projections( CASE WHEN bucket = 'purchased' THEN amount_micro ELSE 0 END ), 0) AS purchased_committed_micro FROM credit_ledger_entries - WHERE user_id IN ({placeholders}) + {where} GROUP BY user_id """, - unique_ids, + params, ).fetchall() promotion_rows = conn.execute( f""" SELECT user_id, COALESCE(SUM(amount_micro), 0) AS grant_micro FROM credit_promotion_grants - WHERE user_id IN ({placeholders}) + {where} GROUP BY user_id """, - unique_ids, + params, ).fetchall() usage_rows = conn.execute( @@ -949,10 +969,10 @@ def get_balance_projections( COALESCE(SUM(CASE WHEN bucket = 'purchased' THEN amount_micro ELSE 0 END), 0) AS purchased_usage_micro FROM credit_llm_usage_entries - WHERE user_id IN ({placeholders}) + {where} GROUP BY user_id """, - unique_ids, + params, ).fetchall() reservation_rows = conn.execute( f""" @@ -961,10 +981,10 @@ def get_balance_projections( COALESCE(SUM(reserved_grant_micro), 0) AS reserved_grant_micro, COALESCE(SUM(reserved_purchased_micro), 0) AS reserved_purchased_micro FROM credit_llm_reservations - WHERE user_id IN ({placeholders}) AND status = 'open' + {reservation_filter} GROUP BY user_id """, - unique_ids, + params, ).fetchall() amounts = { @@ -993,6 +1013,8 @@ def get_balance_projections( for row in reservation_rows } projections: dict[int, dict[str, int]] = {} + if unique_ids is None: + unique_ids = sorted(set(amounts) | set(usage_amounts) | set(reserved_amounts)) for user_id in unique_ids: grant_micro, purchased_micro = amounts.get(user_id, (0, 0)) grant_usage, purchased_usage = usage_amounts.get(user_id, (0, 0)) @@ -1021,6 +1043,237 @@ def list_user_ids(self) -> list[int]: rows = conn.execute("SELECT id FROM users ORDER BY id").fetchall() return [int(row["id"]) for row in rows] + def aggregate_commercial_ledger( + self, + user_ids: Sequence[int], + *, + start: datetime, + end: datetime, + ) -> dict[int, dict[str, int]]: + """Lifetime and windowed ledger totals for many users, three statements. + + The exact SQL ``domain/analytics/value_repository.py::list_commercial_values`` + ran against this store's connection until PR A; moved here so the + analytics package reads the ledger through the credits domain instead + of opening its connection (design SS6.14). The window is + ``[start, end)`` in ISO-8601 UTC text, the format ``created_at`` holds. + """ + ids = _unique_user_ids(user_ids) + if not ids: + return {} + placeholders = ", ".join("?" for _ in ids) + window = [_utc_text(start, "start"), _utc_text(end, "end")] + with self._get_connection() as conn: + lifetime_rows = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro + FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) + AND entry_type IN ('purchase', 'refund') + GROUP BY user_id + """, + ids, + ).fetchall() + period_rows = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro, + COALESCE(SUM(CASE + WHEN entry_type = 'admin_grant_assign' THEN amount_micro + WHEN entry_type = 'admin_grant_reclaim' THEN -amount_micro + ELSE 0 END), 0) AS grant_activity_micro + FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) + AND created_at >= ? + AND created_at < ? + GROUP BY user_id + """, + [*ids, *window], + ).fetchall() + usage_rows = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM(-amount_micro), 0) AS consumed_micro + FROM credit_llm_usage_entries + WHERE user_id IN ({placeholders}) + AND created_at >= ? + AND created_at < ? + GROUP BY user_id + """, + [*ids, *window], + ).fetchall() + return _assemble_commercial_ledger(ids, lifetime_rows, period_rows, usage_rows) + + def list_credit_activity_timestamps( + self, + user_ids: Sequence[int], + *, + start: datetime, + end: datetime, + ) -> dict[int, list[str]]: + """``created_at`` of every purchase and consumption inside ``[start, end)``. + + Purchases and model consumption are the two ledger movements that count + as meaningful activity (design SS15.1); refunds and admin grants are + not the user's own action. Text, not datetimes: the caller parses. + """ + ids = _unique_user_ids(user_ids) + result: dict[int, list[str]] = {user_id: [] for user_id in ids} + if not ids: + return result + placeholders = ", ".join("?" for _ in ids) + params = [*ids, _utc_text(start, "start"), _utc_text(end, "end")] + with self._get_connection() as conn: + purchase_rows = conn.execute( + f""" + SELECT user_id, created_at + FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) + AND entry_type = 'purchase' + AND created_at >= ? + AND created_at < ? + """, + params, + ).fetchall() + usage_rows = conn.execute( + f""" + SELECT user_id, created_at + FROM credit_llm_usage_entries + WHERE user_id IN ({placeholders}) + AND created_at >= ? + AND created_at < ? + """, + params, + ).fetchall() + for row in (*purchase_rows, *usage_rows): + result[int(row["user_id"])].append(str(row["created_at"])) + return result + + def aggregate_ledger_for_day(self, day: date) -> dict[int, dict[str, Any]]: + """Own spend on ``day`` plus lifetime net purchases, per user. + + The daily job's ledger step (design SS6.9 step 4). Three statements, + none parameterised by a user id: the day's consumption grouped by + user, the lifetime purchase-minus-refund total grouped by user (what + ``commercial_tier()`` consumes, so the stored tier is correct as of the + end of the day), and the latest purchase inside the day. The + ``last_activity_at`` it yields is how a dropped ``credits_settled`` + event is prevented from leaving a paying user looking inactive. + """ + day_start, day_end = _day_bounds(day) + with self._get_connection() as conn: + usage_rows = conn.execute( + """ + SELECT user_id, + COALESCE(SUM(-amount_micro), 0) AS consumed_micro, + MAX(created_at) AS last_usage_at + FROM credit_llm_usage_entries + WHERE created_at >= ? AND created_at < ? + GROUP BY user_id + """, + (day_start, day_end), + ).fetchall() + lifetime_rows = conn.execute( + """ + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro + FROM credit_ledger_entries + WHERE entry_type IN ('purchase', 'refund') + GROUP BY user_id + """ + ).fetchall() + purchase_rows = conn.execute( + """ + SELECT user_id, MAX(created_at) AS last_purchase_at + FROM credit_ledger_entries + WHERE entry_type = 'purchase' + AND created_at >= ? AND created_at < ? + GROUP BY user_id + """, + (day_start, day_end), + ).fetchall() + return _assemble_ledger_day(usage_rows, lifetime_rows, purchase_rows) + + def list_account_billing_states( + self, + user_ids: Sequence[int] | None = None, + ) -> dict[int, dict[str, Any]]: + """Billing state for many accounts (``None`` = every account) in two queries. + + The batched twin of ``get_account_billing_state``: same columns, same + restriction-reason normalisation, no account row created as a side + effect. A user with no ``credit_accounts`` row is simply absent, which + the caller reads as an unrestricted account -- the same answer the + single-user reader gives after it lazily creates the row. + """ + clause = "" + params: list[Any] = [] + if user_ids is not None: + ids = _unique_user_ids(user_ids) + if not ids: + return {} + clause = f" WHERE user_id IN ({', '.join('?' for _ in ids)})" + params = list(ids) + with self._get_connection() as conn: + account_rows = conn.execute( + f"SELECT user_id, status, restriction_reason FROM credit_accounts{clause}", + params, + ).fetchall() + outstanding_rows = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM( + MAX(outstanding_micro - outstanding_recovered_micro, 0) + ), 0) AS outstanding_micro + FROM credit_llm_reservations + WHERE status = 'settled'{clause.replace(' WHERE ', ' AND ')} + GROUP BY user_id + """, + params, + ).fetchall() + return _assemble_billing_states(account_rows, outstanding_rows) + + def list_llm_reservation_rows(self) -> list[dict[str, Any]]: + """Every LLM reservation, oldest first, for the analytics backfill. + + Safe columns only -- no evidence, no digests. ``backfill.py`` read this + table through this store's connection until PR A (design SS6.14). + """ + with self._get_connection() as conn: + rows = conn.execute( + """ + SELECT reservation_id, user_id, run_id, call_index, + reserved_grant_micro, reserved_purchased_micro, + status, created_at, updated_at + FROM credit_llm_reservations + ORDER BY created_at, reservation_id + """ + ).fetchall() + return [dict(row) for row in rows] + + def list_llm_usage_rows(self) -> list[dict[str, Any]]: + """Every LLM usage entry, oldest first, for the analytics backfill.""" + with self._get_connection() as conn: + rows = conn.execute( + """ + SELECT id, user_id, reservation_id, run_id, call_index, + bucket, amount_micro, created_at + FROM credit_llm_usage_entries + ORDER BY created_at, id + """ + ).fetchall() + return [dict(row) for row in rows] + def grant_promotion_credits( self, *, diff --git a/dashboard/backend/domain/credits/repository_common.py b/dashboard/backend/domain/credits/repository_common.py index 1c58b5e0..e6bb72c5 100644 --- a/dashboard/backend/domain/credits/repository_common.py +++ b/dashboard/backend/domain/credits/repository_common.py @@ -7,7 +7,118 @@ import hashlib import json from collections.abc import Iterable, Mapping -from datetime import datetime, timezone +from datetime import date, datetime, timedelta, timezone + + +def _utc_text(value: datetime, name: str) -> str: + """ISO-8601 UTC text, the format every ``created_at`` in this ledger uses.""" + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError(f"{name} must include a timezone") + return value.astimezone(timezone.utc).isoformat() + + +def _day_bounds(day: date) -> tuple[str, str]: + start = datetime.combine(day, datetime.min.time(), tzinfo=timezone.utc) + return _utc_text(start, "day"), _utc_text(start + timedelta(days=1), "day") + + +def _unique_user_ids(user_ids: Sequence[int]) -> list[int]: + if not isinstance(user_ids, (list, tuple)): + raise ValueError("user_ids must be a list or tuple") + return list( + dict.fromkeys(_positive_integer(user_id, "user_id") for user_id in user_ids) + ) + + +def _assemble_commercial_ledger( + ids: list[int], lifetime_rows, period_rows, usage_rows +) -> dict[int, dict[str, int]]: + lifetime = { + int(row["user_id"]): ( + int(row["purchased_micro"] or 0), + int(row["refunded_micro"] or 0), + ) + for row in lifetime_rows + } + period = { + int(row["user_id"]): ( + int(row["purchased_micro"] or 0), + int(row["refunded_micro"] or 0), + int(row["grant_activity_micro"] or 0), + ) + for row in period_rows + } + usage = {int(row["user_id"]): int(row["consumed_micro"] or 0) for row in usage_rows} + result: dict[int, dict[str, int]] = {} + for user_id in ids: + lifetime_purchased, lifetime_refunded = lifetime.get(user_id, (0, 0)) + purchased, refunded, grant_activity = period.get(user_id, (0, 0, 0)) + result[user_id] = { + "lifetime_purchased_micro": lifetime_purchased, + "lifetime_refunded_micro": lifetime_refunded, + "purchased_micro": purchased, + "refunded_micro": refunded, + "grant_activity_micro": grant_activity, + "consumed_micro": usage.get(user_id, 0), + } + return result + + +def _assemble_ledger_day(usage_rows, lifetime_rows, purchase_rows) -> dict[int, dict[str, Any]]: + result: dict[int, dict[str, Any]] = {} + + def entry(user_id: int) -> dict[str, Any]: + return result.setdefault( + user_id, + { + "own_spend_micro": 0, + "lifetime_net_purchased_micro": 0, + "last_activity_at": None, + }, + ) + + def later(current: str | None, candidate: Any) -> str | None: + if candidate is None: + return current + text = str(candidate) + return text if current is None or text > current else current + + for row in usage_rows: + record = entry(int(row["user_id"])) + record["own_spend_micro"] = max(0, int(row["consumed_micro"] or 0)) + record["last_activity_at"] = later(record["last_activity_at"], row["last_usage_at"]) + for row in lifetime_rows: + record = entry(int(row["user_id"])) + record["lifetime_net_purchased_micro"] = max( + 0, int(row["purchased_micro"] or 0) - int(row["refunded_micro"] or 0) + ) + for row in purchase_rows: + record = entry(int(row["user_id"])) + record["last_activity_at"] = later( + record["last_activity_at"], row["last_purchase_at"] + ) + return result + + +def _assemble_billing_states(account_rows, outstanding_rows) -> dict[int, dict[str, Any]]: + outstanding = { + int(row["user_id"]): int(row["outstanding_micro"] or 0) for row in outstanding_rows + } + result: dict[int, dict[str, Any]] = {} + for row in account_rows: + user_id = int(row["user_id"]) + reason = row["restriction_reason"] + if row["status"] == "restricted" and reason not in { + "llm_overage", + "refund_reconciliation", + }: + reason = "refund_reconciliation" + result[user_id] = { + "account_status": row["status"], + "restriction_reason": reason, + "outstanding_credits_micro": outstanding.get(user_id, 0), + } + return result class CreditsStoreError(RuntimeError): @@ -238,4 +349,4 @@ def normalize_activity_item( ) item.pop("reservation_id", None) item.pop("call_index", None) - return item + return item \ No newline at end of file diff --git a/dashboard/backend/domain/credits/repository_postgres.py b/dashboard/backend/domain/credits/repository_postgres.py index 1dab9029..678ef62b 100644 --- a/dashboard/backend/domain/credits/repository_postgres.py +++ b/dashboard/backend/domain/credits/repository_postgres.py @@ -3,7 +3,8 @@ from __future__ import annotations import json -from datetime import datetime, timedelta +from collections.abc import Sequence +from datetime import date, datetime, timedelta from typing import Any import psycopg @@ -27,6 +28,14 @@ _utcnow_iso, _validate_amount_pair, ) +from dashboard.backend.domain.credits.repository import ( + _assemble_billing_states, + _assemble_commercial_ledger, + _assemble_ledger_day, + _day_bounds, + _unique_user_ids, + _utc_text, +) from dashboard.backend.domain.model_providers.repository_common import ( validate_provider_id, ) @@ -754,19 +763,34 @@ def get_balance_projection(self, user_id: int) -> dict[str, int]: return self._balance_projection_in_transaction(cur, user_id) def get_balance_projections( - self, user_ids: list[int] | tuple[int, ...] + self, user_ids: list[int] | tuple[int, ...] | None ) -> dict[int, dict[str, int]]: - if not isinstance(user_ids, (list, tuple)): - raise ValueError("user_ids must be a list or tuple") - validated = [_positive_integer(user_id, "user_id") for user_id in user_ids] - if not validated: - return {} + """Balances for many accounts; ``None`` means every account with a row. - unique_ids = list(dict.fromkeys(validated)) + ``None`` is the daily job's shape (design SS6.12): the four statements + below then carry no ``WHERE user_id = ANY(%s)`` clause at all, so the + query count and shape are the same at 200 and 20,000 users. + """ + if user_ids is None: + unique_ids = None + where = "" + params: tuple[Any, ...] = () + else: + if not isinstance(user_ids, (list, tuple)): + raise ValueError("user_ids must be a list or tuple") + validated = [_positive_integer(user_id, "user_id") for user_id in user_ids] + if not validated: + return {} + unique_ids = list(dict.fromkeys(validated)) + where = "WHERE user_id = ANY(%s)" + params = (unique_ids,) + reservation_filter = ( + f"{where} AND status = 'open'" if where else "WHERE status = 'open'" + ) with self._get_connection() as conn: with conn.cursor() as cur: cur.execute( - """ + f""" SELECT user_id, COALESCE(SUM( @@ -776,24 +800,24 @@ def get_balance_projections( CASE WHEN bucket = 'purchased' THEN amount_micro ELSE 0 END ), 0) AS purchased_committed_micro FROM credit_ledger_entries - WHERE user_id = ANY(%s) + {where} GROUP BY user_id """, - (unique_ids,), + params, ) rows = cur.fetchall() cur.execute( - """ + f""" SELECT user_id, COALESCE(SUM(amount_micro), 0) AS grant_micro FROM credit_promotion_grants - WHERE user_id = ANY(%s) + {where} GROUP BY user_id """, - (unique_ids,), + params, ) promotion_rows = cur.fetchall() cur.execute( - """ + f""" SELECT user_id, COALESCE(SUM(CASE WHEN bucket = 'grant' THEN amount_micro ELSE 0 END), 0) @@ -801,23 +825,23 @@ def get_balance_projections( COALESCE(SUM(CASE WHEN bucket = 'purchased' THEN amount_micro ELSE 0 END), 0) AS purchased_usage_micro FROM credit_llm_usage_entries - WHERE user_id = ANY(%s) + {where} GROUP BY user_id """, - (unique_ids,), + params, ) usage_rows = cur.fetchall() cur.execute( - """ + f""" SELECT user_id, COALESCE(SUM(reserved_grant_micro), 0) AS reserved_grant_micro, COALESCE(SUM(reserved_purchased_micro), 0) AS reserved_purchased_micro FROM credit_llm_reservations - WHERE user_id = ANY(%s) AND status = 'open' + {reservation_filter} GROUP BY user_id """, - (unique_ids,), + params, ) reservation_rows = cur.fetchall() @@ -847,6 +871,8 @@ def get_balance_projections( for row in reservation_rows } projections: dict[int, dict[str, int]] = {} + if unique_ids is None: + unique_ids = sorted(set(amounts) | set(usage_amounts) | set(reserved_amounts)) for user_id in unique_ids: grant_micro, purchased_micro = amounts.get(user_id, (0, 0)) grant_usage, purchased_usage = usage_amounts.get(user_id, (0, 0)) @@ -873,6 +899,217 @@ def list_user_ids(self) -> list[int]: rows = cur.fetchall() return [int(row["id"]) for row in rows] + def aggregate_commercial_ledger( + self, + user_ids: Sequence[int], + *, + start: datetime, + end: datetime, + ) -> dict[int, dict[str, int]]: + """See the SQLite twin.""" + ids = _unique_user_ids(user_ids) + if not ids: + return {} + window = (_utc_text(start, "start"), _utc_text(end, "end")) + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro + FROM credit_ledger_entries + WHERE user_id = ANY(%s) + AND entry_type IN ('purchase', 'refund') + GROUP BY user_id + """, + (ids,), + ) + lifetime_rows = cur.fetchall() + cur.execute( + """ + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro, + COALESCE(SUM(CASE + WHEN entry_type = 'admin_grant_assign' THEN amount_micro + WHEN entry_type = 'admin_grant_reclaim' THEN -amount_micro + ELSE 0 END), 0) AS grant_activity_micro + FROM credit_ledger_entries + WHERE user_id = ANY(%s) + AND created_at >= %s + AND created_at < %s + GROUP BY user_id + """, + (ids, *window), + ) + period_rows = cur.fetchall() + cur.execute( + """ + SELECT user_id, + COALESCE(SUM(-amount_micro), 0) AS consumed_micro + FROM credit_llm_usage_entries + WHERE user_id = ANY(%s) + AND created_at >= %s + AND created_at < %s + GROUP BY user_id + """, + (ids, *window), + ) + usage_rows = cur.fetchall() + return _assemble_commercial_ledger(ids, lifetime_rows, period_rows, usage_rows) + + def list_credit_activity_timestamps( + self, + user_ids: Sequence[int], + *, + start: datetime, + end: datetime, + ) -> dict[int, list[str]]: + """See the SQLite twin.""" + ids = _unique_user_ids(user_ids) + result: dict[int, list[str]] = {user_id: [] for user_id in ids} + if not ids: + return result + params = (ids, _utc_text(start, "start"), _utc_text(end, "end")) + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT user_id, created_at + FROM credit_ledger_entries + WHERE user_id = ANY(%s) + AND entry_type = 'purchase' + AND created_at >= %s + AND created_at < %s + """, + params, + ) + purchase_rows = cur.fetchall() + cur.execute( + """ + SELECT user_id, created_at + FROM credit_llm_usage_entries + WHERE user_id = ANY(%s) + AND created_at >= %s + AND created_at < %s + """, + params, + ) + usage_rows = cur.fetchall() + for row in (*purchase_rows, *usage_rows): + result[int(row["user_id"])].append(str(row["created_at"])) + return result + + def aggregate_ledger_for_day(self, day: date) -> dict[int, dict[str, Any]]: + """See the SQLite twin.""" + day_start, day_end = _day_bounds(day) + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT user_id, + COALESCE(SUM(-amount_micro), 0) AS consumed_micro, + MAX(created_at) AS last_usage_at + FROM credit_llm_usage_entries + WHERE created_at >= %s AND created_at < %s + GROUP BY user_id + """, + (day_start, day_end), + ) + usage_rows = cur.fetchall() + cur.execute( + """ + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro + FROM credit_ledger_entries + WHERE entry_type IN ('purchase', 'refund') + GROUP BY user_id + """ + ) + lifetime_rows = cur.fetchall() + cur.execute( + """ + SELECT user_id, MAX(created_at) AS last_purchase_at + FROM credit_ledger_entries + WHERE entry_type = 'purchase' + AND created_at >= %s AND created_at < %s + GROUP BY user_id + """, + (day_start, day_end), + ) + purchase_rows = cur.fetchall() + return _assemble_ledger_day(usage_rows, lifetime_rows, purchase_rows) + + def list_account_billing_states( + self, + user_ids: Sequence[int] | None = None, + ) -> dict[int, dict[str, Any]]: + """See the SQLite twin.""" + account_sql = "SELECT user_id, status, restriction_reason FROM credit_accounts" + outstanding_sql = """ + SELECT user_id, + COALESCE(SUM( + GREATEST(outstanding_micro - outstanding_recovered_micro, 0) + ), 0) AS outstanding_micro + FROM credit_llm_reservations + WHERE status = 'settled' + """ + params: tuple[Any, ...] = () + if user_ids is not None: + ids = _unique_user_ids(user_ids) + if not ids: + return {} + account_sql += " WHERE user_id = ANY(%s)" + outstanding_sql += " AND user_id = ANY(%s)" + params = (ids,) + outstanding_sql += " GROUP BY user_id" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute(account_sql, params) + account_rows = cur.fetchall() + cur.execute(outstanding_sql, params) + outstanding_rows = cur.fetchall() + return _assemble_billing_states(account_rows, outstanding_rows) + + def list_llm_reservation_rows(self) -> list[dict[str, Any]]: + """See the SQLite twin.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT reservation_id, user_id, run_id, call_index, + reserved_grant_micro, reserved_purchased_micro, + status, created_at, updated_at + FROM credit_llm_reservations + ORDER BY created_at, reservation_id + """ + ) + rows = cur.fetchall() + return [dict(row) for row in rows] + + def list_llm_usage_rows(self) -> list[dict[str, Any]]: + """See the SQLite twin.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + """ + SELECT id, user_id, reservation_id, run_id, call_index, + bucket, amount_micro, created_at + FROM credit_llm_usage_entries + ORDER BY created_at, id + """ + ) + rows = cur.fetchall() + return [dict(row) for row in rows] + def grant_promotion_credits( self, *, diff --git a/dashboard/backend/domain/model_providers/repository.py b/dashboard/backend/domain/model_providers/repository.py index a5477619..58a19cb2 100644 --- a/dashboard/backend/domain/model_providers/repository.py +++ b/dashboard/backend/domain/model_providers/repository.py @@ -2,6 +2,7 @@ from __future__ import annotations +from collections.abc import Sequence import os import json import sqlite3 @@ -23,6 +24,8 @@ serialize_capabilities, validate_adapter_type, validate_approved_origin, + DefaultCredentialFacts, + fold_default_credential_rows, ) from .models import ProviderRecord @@ -416,6 +419,47 @@ def list_all_providers(self) -> list[dict[str, Any]]: conn.close() return [self._public_provider(row) for row in rows] + def list_default_credential_facts( + self, user_ids: Sequence[int] | None = None + ) -> dict[int, DefaultCredentialFacts]: + """Each user's default credentials as one row per user (``None`` = everyone). + + Same predicate ``list_user_credentials`` applies (``status <> 'revoked'``) + narrowed to ``is_default = 1``, so the per-user reader and this batched + one see identical rows. A user with no default credential is absent; + the caller reads absence as ``missing``. + """ + clause = "" + params: list[Any] = [] + if user_ids is not None: + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + if not ids: + return {} + clause = f" AND user_id IN ({', '.join('?' for _ in ids)})" + params = ids + conn = self._get_connection() + rows = conn.execute( + "SELECT user_id, provider_id, status FROM user_model_credentials " + f"WHERE is_default = 1 AND status <> 'revoked'{clause}", + params, + ).fetchall() + conn.close() + return fold_default_credential_rows(rows) + + def list_platform_credential_statuses(self) -> dict[str, str]: + """Every provider's platform-credential status. Takes no user id. + + The population-wide half of ``platform_credits_available``: it depends + only on the provider row, its platform credential and the environment + secret, never on who is asking. One call for the whole job. + """ + conn = self._get_connection() + rows = conn.execute( + "SELECT provider_id, status FROM platform_model_credentials" + ).fetchall() + conn.close() + return {str(row["provider_id"]): str(row["status"]) for row in rows} + def record_admin_operation(self, **values: Any) -> None: conn = self._get_connection() try: diff --git a/dashboard/backend/domain/model_providers/repository_common.py b/dashboard/backend/domain/model_providers/repository_common.py index a7bfcb73..44e693e0 100644 --- a/dashboard/backend/domain/model_providers/repository_common.py +++ b/dashboard/backend/domain/model_providers/repository_common.py @@ -2,6 +2,8 @@ from __future__ import annotations +from dataclasses import dataclass +from typing import Mapping from urllib.parse import urlsplit import base64 import hashlib @@ -39,6 +41,56 @@ class InvalidProviderOriginError(ModelProviderStoreError): pass + +@dataclass(frozen=True) +class DefaultCredentialFacts: + """One user's default credentials, folded to what operational state needs. + + ``status`` follows get_operational_facts' worst-first precedence: invalid + beats verification_unavailable beats verified. ``default_provider_ids`` + carries every provider a default credential points at, and + ``verified_default_provider_counts`` the per-provider count of *verified* + defaults -- the ``== 1`` test in ModelProviderService.list_execution_options + (service.py:194) is a count, not a boolean, and collapsing it to one loses + the two-defaults case. + """ + + status: str + default_provider_ids: frozenset[str] + verified_default_provider_counts: Mapping[str, int] + + +def fold_default_credential_rows(rows) -> dict[int, DefaultCredentialFacts]: + """Shared by both twins: rows of (user_id, provider_id, status) -> facts.""" + statuses: dict[int, set[str]] = {} + providers: dict[int, set[str]] = {} + verified: dict[int, dict[str, int]] = {} + for row in rows: + user_id = int(row["user_id"]) + provider_id = str(row["provider_id"]) + status = str(row["status"]) + statuses.setdefault(user_id, set()).add(status) + providers.setdefault(user_id, set()).add(provider_id) + if status == "verified": + counts = verified.setdefault(user_id, {}) + counts[provider_id] = counts.get(provider_id, 0) + 1 + result: dict[int, DefaultCredentialFacts] = {} + for user_id, seen in statuses.items(): + if "invalid" in seen: + status = "invalid" + elif "verification_unavailable" in seen: + status = "verification_unavailable" + elif "verified" in seen: + status = "verified" + else: + status = "missing" + result[user_id] = DefaultCredentialFacts( + status=status, + default_provider_ids=frozenset(providers[user_id]), + verified_default_provider_counts=dict(verified.get(user_id, {})), + ) + return result + def canonical_request_digest(payload: Mapping[str, object]) -> str: """Return a stable digest for an admin mutation without retaining secrets.""" diff --git a/dashboard/backend/domain/model_providers/repository_postgres.py b/dashboard/backend/domain/model_providers/repository_postgres.py index 20a6218d..31960a46 100644 --- a/dashboard/backend/domain/model_providers/repository_postgres.py +++ b/dashboard/backend/domain/model_providers/repository_postgres.py @@ -2,6 +2,7 @@ from __future__ import annotations +from collections.abc import Sequence import uuid import json import os @@ -24,6 +25,8 @@ serialize_capabilities, validate_adapter_type, validate_approved_origin, + DefaultCredentialFacts, + fold_default_credential_rows, ) @@ -298,6 +301,35 @@ def list_all_providers(self) -> list[dict[str, Any]]: rows = cur.fetchall() return [_public_provider(row) for row in rows] + def list_default_credential_facts( + self, user_ids: Sequence[int] | None = None + ) -> dict[int, DefaultCredentialFacts]: + """See the SQLite twin.""" + sql = ( + "SELECT user_id, provider_id, status FROM user_model_credentials " + "WHERE is_default = TRUE AND status <> 'revoked'" + ) + params: tuple[Any, ...] = () + if user_ids is not None: + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + if not ids: + return {} + sql += " AND user_id = ANY(%s)" + params = (ids,) + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute(sql, params) + rows = cur.fetchall() + return fold_default_credential_rows(rows) + + def list_platform_credential_statuses(self) -> dict[str, str]: + """See the SQLite twin.""" + with self._get_connection() as conn: + with conn.cursor() as cur: + cur.execute("SELECT provider_id, status FROM platform_model_credentials") + rows = cur.fetchall() + return {str(row["provider_id"]): str(row["status"]) for row in rows} + def record_admin_operation(self, **values: Any) -> None: with self._get_connection() as conn: with conn.cursor() as cur: diff --git a/dashboard/backend/domain/runs/repository.py b/dashboard/backend/domain/runs/repository.py index ee996907..53b9a8d9 100644 --- a/dashboard/backend/domain/runs/repository.py +++ b/dashboard/backend/domain/runs/repository.py @@ -19,7 +19,7 @@ import uuid from datetime import datetime, timedelta, timezone from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Tuple from dashboard.backend.database import DB_PATH, enable_wal @@ -61,6 +61,19 @@ def _public_run(row: sqlite3.Row | Dict[str, Any]) -> Dict[str, Any]: "updated_at": data.get("updated_at"), } +def _parse_run_timestamp(value: object) -> Optional[datetime]: + """Both shapes protocol_runs holds: CURRENT_TIMESTAMP text and ISO-8601.""" + if value in (None, ""): + return None + try: + parsed = datetime.fromisoformat(str(value)) + except (TypeError, ValueError): + return None + if parsed.tzinfo is None or parsed.utcoffset() is None: + parsed = parsed.replace(tzinfo=timezone.utc) + return parsed.astimezone(timezone.utc) + + class RunStore: """Persist protocol run metadata and engine/result linkage.""" @@ -265,6 +278,58 @@ def list_runs(self, agent_id: str) -> List[Dict[str, Any]]: conn.close() return [_public_run(row) for row in rows] + _TERMINAL_STATUSES = ("completed", "failed", "cancelled", "closed", "timed_out") + + def list_terminal_runs_since( + self, + *, + since: datetime, + ) -> Dict[str, Tuple[Tuple[datetime, str], ...]]: + """Terminal runs for every agent since ``since``, newest first per agent. + + Returns `(effective_time, status)` pairs per agent, where the effective + time is `COALESCE(updated_at, created_at)` -- the same ordering key + `_run_health` uses. Bounded by construction: a 24-hour window of + terminal runs across the whole population is a small result, and the + caller pools and re-sorts them per owner because the consecutive rule + runs over an owner's agents together, not per agent. + + The two timestamp columns hold mixed shapes (``CURRENT_TIMESTAMP`` + text on old rows, ISO-8601 with an offset from ``update_run``), so the + SQL bounds the scan to calendar days by the 10-character date prefix + and the exact comparison happens in Python. + """ + if since.tzinfo is None or since.utcoffset() is None: + raise ValueError("since must include a timezone") + floor = since.astimezone(timezone.utc).date().isoformat() + placeholders = ", ".join("?" for _ in self._TERMINAL_STATUSES) + conn = self._get_connection() + cursor = conn.cursor() + cursor.execute( + f""" + SELECT agent_id, status, created_at, updated_at + FROM protocol_runs + WHERE agent_id IS NOT NULL + AND status IN ({placeholders}) + AND substr(COALESCE(updated_at, created_at), 1, 10) >= ? + """, + (*self._TERMINAL_STATUSES, floor), + ) + rows = cursor.fetchall() + conn.close() + grouped: Dict[str, List[Tuple[datetime, str]]] = {} + for row in rows: + stamp = _parse_run_timestamp(row["updated_at"]) or _parse_run_timestamp( + row["created_at"] + ) + if stamp is None or stamp < since: + continue + grouped.setdefault(str(row["agent_id"]), []).append((stamp, str(row["status"]))) + return { + agent_id: tuple(sorted(pairs, key=lambda pair: pair[0], reverse=True)) + for agent_id, pairs in grouped.items() + } + # Runs that are still consuming resources (not yet terminal). _ACTIVE_STATUSES = ("created", "loading", "running") diff --git a/dashboard/backend/tests/conftest.py b/dashboard/backend/tests/conftest.py index ae82990c..9a9ee3f1 100644 --- a/dashboard/backend/tests/conftest.py +++ b/dashboard/backend/tests/conftest.py @@ -48,7 +48,6 @@ # A developer's deployment pseudonymization secret must never affect tests. os.environ.pop("ANALYTICS_PSEUDONYMIZATION_KEY", None) - # Hermetic HMAC key for session-token digests (see session_tokens.py). os.environ["SESSION_HASH_SECRET"] = "test-session-hash-secret" @@ -91,6 +90,7 @@ os.environ.pop("AGENT_AUTH_CACHE_TTL_SECONDS", None) os.environ.pop("MAX_LEGACY_ACTIVE_PER_SESSION", None) os.environ.pop("MAX_LEGACY_ACTIVE_GLOBAL", None) +os.environ.pop("ANALYTICS_DAILY_JOB_INTERVAL_SECONDS", None) # A shell that exported the backtest-worker flag would make every Postgres # twin the suite constructs skip DDL and fail on its first query. os.environ.pop("ATL_BACKTEST_WORKER", None) diff --git a/dashboard/backend/tests/domain/analytics/_store_spies.py b/dashboard/backend/tests/domain/analytics/_store_spies.py new file mode 100644 index 00000000..48d819c0 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/_store_spies.py @@ -0,0 +1,91 @@ +"""Call-counting wrappers for the read-budget tests. + +Not a test module (no ``test_`` prefix): imported by +test_operational_signals.py and test_read_budget.py. Wraps any store so every +public method call is recorded as ``(name, args, kwargs)`` and delegates to the +real object, so the SQL still runs against real SQLite -- the budget is +measured on real behaviour, not on a stub that agrees with everything. +""" + +from __future__ import annotations + +import inspect +from typing import Any + + +class CountingSpy: + def __init__(self, target: Any, name: str) -> None: + self._target = target + self.name = name + self.calls: list[tuple[str, tuple, dict]] = [] + + def __getattr__(self, attr: str) -> Any: + value = getattr(self._target, attr) + if attr.startswith("_") or not callable(value): + # Private helpers (``_get_connection``) and plain attributes + # (``analytics_base``, ``credits_base``) pass through unchanged. + return value + + def wrapped(*args: Any, **kwargs: Any) -> Any: + self.calls.append((attr, args, kwargs)) + return value(*args, **kwargs) + + return wrapped + + def reset(self) -> None: + self.calls.clear() + + @property + def total_calls(self) -> int: + return len(self.calls) + + @property + def calls_with_scalar_user_id(self) -> list[str]: + """Calls that bound an ``int`` to a parameter named ``user_id``. + + A batched call passes a sequence (or None) and does not match. Binding + through the real signature rather than eyeballing ``args[0]`` means a + positional ``user_id`` is caught as surely as a keyword one. + """ + offenders: list[str] = [] + for name, args, kwargs in self.calls: + try: + signature = inspect.signature(getattr(self._target, name)) + bound = signature.bind_partial(*args, **kwargs) + except (TypeError, ValueError): + continue + if isinstance(bound.arguments.get("user_id"), int): + offenders.append(f"{self.name}.{name}") + return offenders + + +class SpyBundle: + """Every spy the daily job touches, addressable by store name.""" + + def __init__(self, **spies: CountingSpy) -> None: + self.spies = spies + + def __getattr__(self, name: str) -> CountingSpy: + try: + return self.spies[name] + except KeyError as exc: + raise AttributeError(name) from exc + + def reset(self) -> None: + for spy in self.spies.values(): + spy.reset() + + @property + def total_calls(self) -> int: + return sum(spy.total_calls for spy in self.spies.values()) + + @property + def calls_with_scalar_user_id(self) -> list[str]: + return [ + offender + for spy in self.spies.values() + for offender in spy.calls_with_scalar_user_id + ] + + def calls_by_store(self) -> dict[str, int]: + return {name: spy.total_calls for name, spy in self.spies.items()} diff --git a/dashboard/backend/tests/domain/analytics/test_backfill.py b/dashboard/backend/tests/domain/analytics/test_backfill.py index b1e48d25..26907f40 100644 --- a/dashboard/backend/tests/domain/analytics/test_backfill.py +++ b/dashboard/backend/tests/domain/analytics/test_backfill.py @@ -323,14 +323,53 @@ def test_backfill_rejects_invalid_day_bounds(tmp_path, days): ) -class SqliteRows: +class FakeAgentRows: + """What AgentStore.list_agent_source_rows() answers, without a store.""" + def __init__(self, path): self.path = path - def _get_connection(self): + def list_agent_source_rows(self): conn = sqlite3.connect(self.path) conn.row_factory = sqlite3.Row - return conn + try: + return [ + dict(row) + for row in conn.execute( + "SELECT agent_id, session_id, owner_user_id, created_at " + "FROM external_agents ORDER BY created_at, agent_id" + ).fetchall() + ] + finally: + conn.close() + + +class FakeCreditRows: + """What CreditsStore.list_llm_*_rows() answer, without a store.""" + + def __init__(self, path): + self.path = path + + def _rows(self, sql): + conn = sqlite3.connect(self.path) + conn.row_factory = sqlite3.Row + try: + return [dict(row) for row in conn.execute(sql).fetchall()] + finally: + conn.close() + + def list_llm_reservation_rows(self): + return self._rows( + "SELECT reservation_id, user_id, run_id, call_index, " + "reserved_grant_micro, reserved_purchased_micro, status, created_at, " + "updated_at FROM credit_llm_reservations ORDER BY created_at, reservation_id" + ) + + def list_llm_usage_rows(self): + return self._rows( + "SELECT id, user_id, reservation_id, run_id, call_index, bucket, " + "amount_micro, created_at FROM credit_llm_usage_entries ORDER BY created_at, id" + ) class FakeUsers: @@ -472,10 +511,10 @@ def test_authoritative_source_combines_safe_independent_store_evidence(tmp_path) collection = AuthoritativeBackfillSource( user_store=FakeUsers(), - agent_store=SqliteRows(content_path), + agent_store=FakeAgentRows(content_path), protocol_run_store=FakeProtocolRuns(), run_history_store=FakeRunHistory(), - credits_store=SqliteRows(credits_path), + credits_store=FakeCreditRows(credits_path), ).collect(start=NOW - timedelta(days=180), end=NOW) by_source_id = {item.source_event_id: item for item in collection.candidates} diff --git a/dashboard/backend/tests/domain/analytics/test_daily_facts.py b/dashboard/backend/tests/domain/analytics/test_daily_facts.py new file mode 100644 index 00000000..3413853a --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_daily_facts.py @@ -0,0 +1,449 @@ +"""One set-based pass per UTC day: correctness properties of the daily job.""" + +from __future__ import annotations + +import sqlite3 +from datetime import date, datetime, time, timedelta, timezone + +import pytest +from cryptography.fernet import Fernet + +from dashboard.backend.database import BacktestDatabase +from dashboard.backend.domain.agents.repository import AgentStore +from dashboard.backend.domain.analytics.daily_facts import ( + DAILY_FACTS_JOB, + DailyFactsReport, + run_daily_facts, +) +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.service import AnalyticsService +from dashboard.backend.domain.analytics.value_repository import ( + UserDailyFact, + build_value_analytics_store, +) +from dashboard.backend.domain.brokers import repository as broker_repository +from dashboard.backend.domain.credits.repository import CreditsStore +from dashboard.backend.domain.model_providers.repository import ModelProviderStore +from dashboard.backend.domain.runs.repository import RunStore +from dashboard.backend.users import UserStore + + +D = date(2026, 9, 11) +NOW = datetime(2026, 9, 12, 0, 5, tzinfo=timezone.utc) # the tick after midnight +END_OF_D = datetime.combine(D, time(23, 59, 59, 999999), tzinfo=timezone.utc) + + +@pytest.fixture(autouse=True) +def _encryption_key(monkeypatch): + monkeypatch.setenv("BROKER_TOKEN_ENCRYPTION_KEY", Fernet.generate_key().decode()) + monkeypatch.setattr(broker_repository, "_fernet_instance", None) + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + monkeypatch.delenv("COMMONSTACK_API_KEY", raising=False) + + +class _NoRetention: + def __init__(self): + self.calls = 0 + + def run_if_due(self): + self.calls += 1 + return None + + +class DailyFixture: + def __init__(self, path): + self.path = path / "daily.db" + self.runs_path = path / "runs.db" + UserStore(db_path=self.path) + self.analytics = AnalyticsStore(self.path) + self.credits = CreditsStore(self.path) + self.providers = ModelProviderStore(self.path) + self.agents = AgentStore(self.path) + self.protocol_runs = RunStore(self.path) + self.run_history = BacktestDatabase(self.runs_path) + self.store = build_value_analytics_store( + self.analytics, + credits_base=self.credits, + provider_base=self.providers, + agent_base=self.agents, + run_base=self.protocol_runs, + ) + self.service = AnalyticsService( + self.analytics, value_store=self.store, maintain_activity=True + ) + self.retention = _NoRetention() + self._next_id = 1 + self.admin_id = self.create_user_at(NOW - timedelta(days=90), role="admin") + self.excluded_id = self.create_user_at(NOW - timedelta(days=90)) + self.analytics.set_subject_exclusion( + self.excluded_id, excluded=True, actor_user_id=self.admin_id, reason="test" + ) + self.active_id = self.create_user_at(NOW - timedelta(days=60), user_group="organic") + self.at_risk_id = self.create_user_at(NOW - timedelta(days=60)) + self.created_today_id = None + self._seed() + + # -- helpers ------------------------------------------------------------- + + def create_user_at(self, when, *, role="user", user_group="unknown"): + user_id = self._next_id + self._next_id += 1 + with sqlite3.connect(self.path) as conn: + conn.execute( + "INSERT INTO users (id, email, display_name, password_hash, role, " + "user_group, created_at) VALUES (?, ?, ?, 'x', ?, ?, ?)", + (user_id, f"user{user_id}@example.test", f"User {user_id}", role, + user_group, when.isoformat()), + ) + if when > END_OF_D: + self.created_today_id = user_id + return user_id + + def append_event(self, user_id, *, event_name, occurred_at, received_at, index=None): + suffix = index if index is not None else occurred_at.isoformat() + return self.service.record_server_event( + event_name=event_name, + user_id=user_id, + source_event_id=f"run:{event_name}:{user_id}:{suffix}", + source_record_type="run", + source_record_id=f"run-{user_id}-{suffix}", + occurred_at=occurred_at, + received_at=received_at, + ) + + def touch_activity(self, user_id, *, at): + self.store.record_activity(user_id, occurred_at=at, activating=False, now=at) + + def seed_previous_segment(self, segment, *, user_id=None): + self.store.upsert_daily_facts( + [ + UserDailyFact( + snapshot_date=D - timedelta(days=1), + user_id=user_id or self.at_risk_id, + lifecycle_segment=segment, + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="healthy", + operational_reason_code=None, + tier="unpaid", + user_group="unknown", + active=False, + data_quality="complete", + calculated_at=NOW - timedelta(days=1), + ) + ] + ) + + def seed_fact(self, day, user_id, *, active, runs_completed): + self.store.upsert_daily_facts( + [ + UserDailyFact( + snapshot_date=day, + user_id=user_id, + lifecycle_segment="growing", + lifecycle_reason_code="growing_activated_below_core_threshold", + operational_state="healthy", + operational_reason_code=None, + tier="unpaid", + user_group="unknown", + active=active, + runs_requested=runs_completed, + runs_completed=runs_completed, + data_quality="complete", + calculated_at=NOW - timedelta(days=1), + ) + ] + ) + + def seed_two_of_three_before_d(self): + user_id = self.create_user_at(NOW - timedelta(days=60)) + self.touch_activity(user_id, at=NOW - timedelta(days=20)) + self.store.record_activity( + user_id, + occurred_at=NOW - timedelta(days=20), + activating=True, + now=NOW - timedelta(days=20), + ) + self.seed_fact(D - timedelta(days=5), user_id, active=True, runs_completed=1) + self.seed_fact(D - timedelta(days=3), user_id, active=True, runs_completed=1) + return user_id + + def seed_success_on(self, day, user_id): + at = datetime.combine(day, time(15, 0), tzinfo=timezone.utc) + self.append_event( + user_id, event_name="backtest_completed", occurred_at=at, + received_at=at + timedelta(seconds=1), + ) + + def break_step(self, name): + assert name == "ledger", "only the ledger step is breakable in this fixture" + + def _broken(*_args, **_kwargs): + raise RuntimeError("private ledger detail") + + self.credits.aggregate_ledger_for_day = _broken + + def repair_step(self, name): + assert name == "ledger" + del self.credits.aggregate_ledger_for_day + + def fact_for(self, user_id, day=D): + rows = {row.user_id: row for row in self.store.list_facts_for_date(day)} + return rows[user_id] + + def transitions_for(self, user_id): + with self.analytics._get_connection() as conn: + rows = conn.execute( + "SELECT * FROM lifecycle_transitions WHERE user_id = ? ORDER BY snapshot_date", + (user_id,), + ).fetchall() + return [dict(row) for row in rows] + + def run(self, *, now=NOW): + return run_daily_facts( + now=now, + value_store=self.store, + run_history_store=self.run_history, + retention=self.retention, + ) + + # -- seed -------------------------------------------------------------- + + def _seed(self): + active_day = datetime.combine(D, time(10, 0), tzinfo=timezone.utc) + for index, name in enumerate( + ("backtest_requested", "backtest_completed", "backtest_requested", "backtest_failed") + ): + self.append_event( + self.active_id, + event_name=name, + occurred_at=active_day + timedelta(minutes=index), + received_at=active_day + timedelta(minutes=index, seconds=1), + index=index, + ) + self.run_history.insert_run( + run_id="run-active", + session_id="session-active", + agent_name="Agent", + mode="backtest", + start_date="2026-09-01", + end_date="2026-09-02", + initial_equity=100000.0, + est_cost_usd=1.25, + owner_user_id=self.active_id, + ) + conn = self.run_history._get_connection() + try: + conn.execute( + "UPDATE agent_runs SET updated_at = ? WHERE run_id = ?", + (f"{D.isoformat()} 10:30:00", "run-active"), + ) + conn.commit() + finally: + conn.close() + # at_risk: activated long ago, last meaningful activity ten days before D. + self.store.record_activity( + self.at_risk_id, + occurred_at=NOW - timedelta(days=40), + activating=True, + now=NOW - timedelta(days=40), + ) + self.touch_activity(self.at_risk_id, at=datetime.combine(D - timedelta(days=10), time(9), tzinfo=timezone.utc)) + + +@pytest.fixture +def daily_fixture(tmp_path): + return DailyFixture(tmp_path) + + +def test_a_day_is_written_once_and_only_once(daily_fixture): + """The second call in the same UTC day does nothing.""" + first = daily_fixture.run() + second = daily_fixture.run() + + assert isinstance(first, DailyFactsReport) + assert first.claimed is True + assert first.snapshot_date == D + assert second.claimed is False + assert second.users_written == 0 + job = daily_fixture.store.get_projection_job(DAILY_FACTS_JOB) + assert job.cursor == D.isoformat() + assert job.status == "complete" + + +def test_admins_and_excluded_users_get_no_rows(daily_fixture): + daily_fixture.run() + written = {row.user_id for row in daily_fixture.store.list_facts_for_date(D)} + + assert daily_fixture.admin_id not in written + assert daily_fixture.excluded_id not in written + assert daily_fixture.active_id in written + assert daily_fixture.at_risk_id in written + + +def test_run_outcomes_cost_group_and_states_land_on_the_row(daily_fixture): + daily_fixture.run() + row = daily_fixture.fact_for(daily_fixture.active_id) + + assert row.runs_requested == 2 + assert row.runs_completed == 1 + assert row.runs_failed == 1 + assert row.runs_cancelled == 0 + assert row.operator_cost_micro == 1_250_000 + assert row.own_spend_micro == 0 + assert row.active is True + assert row.user_group == "organic" + assert row.tier == "unpaid" + assert row.lifecycle_segment == "growing" + assert row.operational_state == "blocked" # no balance, no BYOK credential + assert row.operational_reason_code == "billing_lane_unavailable" + assert row.data_quality == "complete" + + +def test_the_at_risk_user_reads_from_the_clock_not_the_events(daily_fixture): + daily_fixture.run() + row = daily_fixture.fact_for(daily_fixture.at_risk_id) + + assert row.active is False + assert row.lifecycle_segment == "at_risk" + assert row.lifecycle_reason_code == "at_risk_previously_activated" + + +def test_a_segment_change_appends_exactly_one_transition(daily_fixture): + daily_fixture.seed_previous_segment("growing") + daily_fixture.run() + daily_fixture.run() + + transitions = daily_fixture.transitions_for(daily_fixture.at_risk_id) + assert len(transitions) == 1 + assert transitions[0]["from_segment"] == "growing" + assert transitions[0]["to_segment"] == "at_risk" + assert transitions[0]["inactive_days"] == 10 + assert transitions[0]["data_quality"] == "complete" + + +def test_no_previous_row_means_no_transition(daily_fixture): + daily_fixture.run() + + assert daily_fixture.transitions_for(daily_fixture.active_id) == [] + + +def test_a_failed_step_marks_the_day_partial_and_retries_it(daily_fixture, capsys): + daily_fixture.break_step("ledger") + first = daily_fixture.run() + printed = capsys.readouterr().out + + assert first.claimed is True + assert first.partial is True + assert "ledger" in first.failed_steps + assert "WARNING: analytics.daily_facts.ledger_failed category=RuntimeError" in printed + assert "private ledger detail" not in printed + assert daily_fixture.fact_for(daily_fixture.active_id).data_quality == "partial" + job = daily_fixture.store.get_projection_job(DAILY_FACTS_JOB) + assert job.status == "pending" # released, not completed + assert job.cursor is None + + daily_fixture.repair_step("ledger") + retry = daily_fixture.run(now=NOW + timedelta(minutes=5)) + assert retry.claimed is True + assert retry.partial is False + assert daily_fixture.fact_for(daily_fixture.active_id).data_quality == "complete" + + +def test_a_retry_corrects_the_transition_the_partial_day_wrote(daily_fixture): + """DO UPDATE, not DO NOTHING. + + The day a transition is rewritten is the day the first attempt was + wrong. Freezing it leaves `lifecycle_transitions` permanently + disagreeing with the `user_daily_facts` row the retry did correct. + """ + daily_fixture.seed_previous_segment("growing") + daily_fixture.break_step("ledger") + daily_fixture.run() + first = daily_fixture.transitions_for(daily_fixture.at_risk_id)[0] + + daily_fixture.repair_step("ledger") + daily_fixture.run(now=NOW + timedelta(minutes=5)) + corrected = daily_fixture.transitions_for(daily_fixture.at_risk_id) + + assert len(corrected) == 1 + assert first["data_quality"] == "partial" + assert corrected[0]["data_quality"] == "complete" + + +def test_a_user_active_after_midnight_does_not_abort_the_day(daily_fixture): + """The regression that would have fired on the first night in prod. + + `user_activity` is overwritten in place, so a user who acts at 00:02 + has a `last_meaningful_activity_at` newer than the end of D. Fed to + `calculate_lifecycle(as_of=)` unclamped that raises, and the + single try/except around the step turns one such user into zero fact + rows for the entire population. + """ + daily_fixture.touch_activity(daily_fixture.active_id, at=NOW) + daily_fixture.create_user_at(NOW) + + report = daily_fixture.run() + + assert report.partial is False + assert report.failed_steps == () + assert report.users_written >= 2 + written = {row.user_id for row in daily_fixture.store.list_facts_for_date(D)} + assert daily_fixture.active_id in written + # Created after D ended: no day D to describe. + assert daily_fixture.created_today_id not in written + + +def test_the_stored_segment_counts_its_own_day(daily_fixture): + """The trailing window is 30 dates ending at D, D included. + + Sitting at two active days and two successes before D, plus an active + day with a success on D, is the `core` threshold exactly. Summing only + D-29..D-1 writes `growing` and the read path answers `core` the next + morning. + """ + user_id = daily_fixture.seed_two_of_three_before_d() + daily_fixture.seed_success_on(D, user_id) + + daily_fixture.run() + + assert daily_fixture.fact_for(user_id).lifecycle_segment == "core" + + +def test_an_event_arriving_after_the_day_was_written_is_picked_up(daily_fixture): + """Late arrivals are recomputed, not lost. + + A run finishing at 23:59 can be appended minutes later, and the + frontend route accepts `occurred_at` up to 24 hours old. Both land + inside D after D's aggregate was taken, and the cursor has already + moved past D. The next day's job finds them by `received_at`. + """ + daily_fixture.run() + before = daily_fixture.fact_for(daily_fixture.active_id).runs_completed + + daily_fixture.append_event( + daily_fixture.active_id, + event_name="backtest_completed", + occurred_at=datetime(2026, 9, 11, 23, 59, tzinfo=timezone.utc), + received_at=NOW + timedelta(minutes=5), + index="late", + ) + report = daily_fixture.run(now=NOW + timedelta(days=1)) + + assert report.snapshot_date == D + timedelta(days=1) + assert D in report.recomputed_dates + assert daily_fixture.fact_for(daily_fixture.active_id).runs_completed == before + 1 + + +def test_a_settled_day_is_not_recomputed_forever(daily_fixture): + """The sweep is driven by evidence, not by a timer.""" + daily_fixture.run() + report = daily_fixture.run(now=NOW + timedelta(days=1)) + + assert report.recomputed_dates == () + + +def test_the_retention_coordinator_runs_on_every_tick(daily_fixture): + daily_fixture.run() + daily_fixture.run(now=NOW + timedelta(minutes=5)) # idle tick + + assert daily_fixture.retention.calls == 2 diff --git a/dashboard/backend/tests/domain/analytics/test_daily_job.py b/dashboard/backend/tests/domain/analytics/test_daily_job.py new file mode 100644 index 00000000..ccea126e --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_daily_job.py @@ -0,0 +1,118 @@ +"""The daily-facts worker owns its schedule (design D23, SS6.9).""" + +from __future__ import annotations + +import inspect +import threading + +import pytest + +import dashboard.backend.app as app_module +from dashboard.backend.domain.analytics import daily_facts as daily_facts_module +from dashboard.backend.domain.analytics import daily_job + + +@pytest.fixture(autouse=True) +def _stop_worker(): + yield + daily_job.stop_daily_facts_worker() + + +def test_interval_defaults_to_five_minutes(monkeypatch): + monkeypatch.delenv("ANALYTICS_DAILY_JOB_INTERVAL_SECONDS", raising=False) + assert daily_job.daily_job_interval_seconds() == 300.0 + + +@pytest.mark.parametrize("raw", ["junk", "", " ", "4", "3601", "-1"]) +def test_bad_intervals_fall_back_with_a_line_instead_of_raising(monkeypatch, capsys, raw): + monkeypatch.setenv("ANALYTICS_DAILY_JOB_INTERVAL_SECONDS", raw) + assert daily_job.daily_job_interval_seconds() == 300.0 + if raw.strip(): + assert "ANALYTICS_DAILY_JOB_INTERVAL_SECONDS" in capsys.readouterr().out + + +def test_a_valid_interval_is_honoured(monkeypatch): + monkeypatch.setenv("ANALYTICS_DAILY_JOB_INTERVAL_SECONDS", "60") + assert daily_job.daily_job_interval_seconds() == 60.0 + + +def test_the_worker_prepares_once_then_ticks_until_stopped(): + stop = threading.Event() + calls: list[str] = [] + + def prepare(): + calls.append("prepare") + + def tick(): + calls.append("tick") + if calls.count("tick") >= 3: + stop.set() + + thread = daily_job.start_daily_facts_worker( + 0.01, stop, tick=tick, prepare=prepare + ) + thread.join(timeout=5) + + assert not thread.is_alive() + assert calls[0] == "prepare" + assert calls.count("prepare") == 1 + assert calls.count("tick") >= 3 + assert thread.daemon is True + assert thread.name == "analytics-daily-facts" + + +def test_a_failing_tick_is_logged_and_the_loop_continues(capsys): + stop = threading.Event() + ticks: list[int] = [] + + def tick(): + ticks.append(len(ticks)) + if len(ticks) == 1: + raise RuntimeError("private tick detail") + if len(ticks) >= 2: + stop.set() + + daily_job.start_daily_facts_worker(0.01, stop, tick=tick, prepare=lambda: None).join(5) + printed = capsys.readouterr().out + + assert len(ticks) >= 2 + assert "WARNING: analytics.daily_facts.tick_failed category=RuntimeError" in printed + assert "private tick detail" not in printed + + +def test_a_failing_prepare_is_logged_and_the_worker_still_ticks(capsys): + stop = threading.Event() + + def prepare(): + raise RuntimeError("private migration detail") + + daily_job.start_daily_facts_worker(0.01, stop, tick=stop.set, prepare=prepare).join(5) + printed = capsys.readouterr().out + + assert "WARNING: analytics.facts_migration_failed category=RuntimeError" in printed + assert "private migration detail" not in printed + + +def test_starting_twice_returns_the_live_thread(): + stop = threading.Event() + first = daily_job.start_daily_facts_worker(60, stop, tick=lambda: None, prepare=lambda: None) + second = daily_job.start_daily_facts_worker(60, stop, tick=lambda: None, prepare=lambda: None) + + assert first is second + daily_job.stop_daily_facts_worker() + assert not first.is_alive() + + +def test_startup_starts_the_worker_and_leaves_the_reaper_to_heartbeats(): + source = inspect.getsource(app_module.startup_event) + + assert source.count("start_daily_facts_worker()") == 1 + assert "register_reaper_sweep(analytics_retention_coordinator.run_if_due)" not in source + assert "register_reaper_sweep(run_daily_facts" not in source + # The throttled snapshot repairs stay on the reaper until PR B deletes them. + assert "register_reaper_sweep(run_analytics_maintenance)" in source + + +def test_the_job_owns_the_retention_coordinator_now(): + source = inspect.getsource(daily_facts_module) + assert source.count("analytics_retention_coordinator") == 1 diff --git a/dashboard/backend/tests/domain/analytics/test_facts_migration.py b/dashboard/backend/tests/domain/analytics/test_facts_migration.py new file mode 100644 index 00000000..05d01bf8 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_facts_migration.py @@ -0,0 +1,139 @@ +"""Existing lifecycle history is carried into the fact table as labelled partial rows.""" + +from __future__ import annotations + +from datetime import date, datetime, timedelta, timezone + +from dashboard.backend.domain.analytics.facts_migration import ( + HISTORY_COPY_DAYS, + StartupMigrationReport, + migrate_lifecycle_history, + run_startup_migrations, +) +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.value_repository import ( + UserDailyFact, + UserLifecycleDailySnapshot, + build_value_analytics_store, +) +from dashboard.backend.tests.domain.analytics.test_value_repository import ( + _value_snapshot, +) +from dashboard.backend.users import UserStore + + +NOW = datetime(2026, 9, 12, 0, 5, tzinfo=timezone.utc) + + +class MigrationFixture: + def __init__(self, tmp_path): + path = tmp_path / "migration.db" + users = UserStore(db_path=path) + self.user_id = int(users.create_user("m@example.test", "M", "SecurePass1!")["id"]) + users.apply_admin_patch(self.user_id, user_group="partner") + self.analytics = AnalyticsStore(db_path=path) + self.store = build_value_analytics_store( + self.analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + self.legacy_dates = [ + date(2026, 8, 20), + date(2026, 9, 1), + date(2026, 9, 10), + ] + for day in self.legacy_dates + [date(2026, 6, 1)]: # June is older than 56 days + self.store.upsert_daily_snapshot( + UserLifecycleDailySnapshot( + snapshot_date=day, + user_id=self.user_id, + lifecycle_segment="growing", + lifecycle_reason_code="growing_activated_below_core_threshold", + data_quality="complete", + calculated_at=datetime.combine( + day + timedelta(days=1), datetime.min.time(), tzinfo=timezone.utc + ), + ) + ) + self.legacy_row_count = len(self.legacy_dates) + + def seed_complete_fact(self, day): + self.store.upsert_daily_facts( + [ + UserDailyFact( + snapshot_date=day, + user_id=self.user_id, + lifecycle_segment="core", + lifecycle_reason_code="core_repeated_value", + operational_state="healthy", + operational_reason_code=None, + tier="starter", + user_group="partner", + active=True, + runs_completed=3, + data_quality="complete", + calculated_at=NOW, + ) + ] + ) + + def fact_for(self, day): + return {row.user_id: row for row in self.store.list_facts_for_date(day)}[self.user_id] + + +def test_history_is_copied_as_partial(tmp_path): + """Movement charts keep their history; the missing columns stay honest.""" + fixture = MigrationFixture(tmp_path) + + copied = migrate_lifecycle_history(value_store=fixture.store, now=NOW) + + assert copied == fixture.legacy_row_count + for day in fixture.legacy_dates: + row = fixture.fact_for(day) + assert row.data_quality == "partial" + assert row.lifecycle_segment == "growing" + assert row.runs_completed == 0 + assert row.operator_cost_micro == 0 + assert row.tier == "unpaid" + assert row.operational_state == "healthy" + assert row.operational_reason_code is None + assert row.user_group == "partner" # read from users at the copy (D9) + assert row.active is False + assert fixture.store.list_facts_for_date(date(2026, 6, 1)) == [] + assert (NOW.date() - timedelta(days=HISTORY_COPY_DAYS)) > date(2026, 6, 1) + + +def test_migration_is_idempotent(tmp_path): + fixture = MigrationFixture(tmp_path) + migrate_lifecycle_history(value_store=fixture.store, now=NOW) + + second = migrate_lifecycle_history(value_store=fixture.store, now=NOW) + + assert second == 0 + + +def test_migration_never_overwrites_a_complete_row(tmp_path): + """A day the new job already computed wins over a migrated stub.""" + fixture = MigrationFixture(tmp_path) + fixture.seed_complete_fact(date(2026, 9, 10)) + + migrate_lifecycle_history(value_store=fixture.store, now=NOW) + row = fixture.fact_for(date(2026, 9, 10)) + + assert row.data_quality == "complete" + assert row.lifecycle_segment == "core" + assert row.tier == "starter" + + +def test_startup_migrations_seed_activity_and_copy_history(tmp_path): + fixture = MigrationFixture(tmp_path) + fixture.store.upsert_current_snapshot(_value_snapshot(fixture.user_id)) + + report = run_startup_migrations(value_store=fixture.store, now=NOW) + again = run_startup_migrations(value_store=fixture.store, now=NOW) + + assert report == StartupMigrationReport(activity_seeded=1, history_copied=3) + assert again == StartupMigrationReport(activity_seeded=1, history_copied=0) + assert fixture.store.get_activity(fixture.user_id).activated_at is not None diff --git a/dashboard/backend/tests/domain/analytics/test_lifecycle_reads.py b/dashboard/backend/tests/domain/analytics/test_lifecycle_reads.py new file mode 100644 index 00000000..3a34ab44 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_lifecycle_reads.py @@ -0,0 +1,250 @@ +"""Lifecycle from stored facts, with the rules unchanged.""" + +from __future__ import annotations + +from datetime import date, datetime, timedelta, timezone + +from dashboard.backend.domain.analytics.lifecycle import calculate_lifecycle +from dashboard.backend.domain.analytics.lifecycle_reads import build_lifecycle_inputs +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.value_repository import ( + RecentFactTotals, + UserActivity, + build_value_analytics_store, +) +from dashboard.backend.users import UserStore + + +NOW = datetime(2026, 9, 12, 12, 0, tzinfo=timezone.utc) + + +def _totals(**overrides): + base = dict( + active_days=0, + successful_backtests=0, + runs_requested=0, + runs_completed=0, + runs_failed=0, + runs_cancelled=0, + operator_cost_micro=0, + own_spend_micro=0, + days_present=30, + ) + base.update(overrides) + return RecentFactTotals(**base) + + +def test_core_needs_three_active_days_and_three_successes(): + activity = UserActivity( + user_id=1, + activated_at=NOW - timedelta(days=20), + last_meaningful_activity_at=NOW - timedelta(days=1), + updated_at=NOW, + ) + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=60), + activity=activity, + totals=_totals(active_days=3, successful_backtests=3), + as_of=NOW, + ) + + assert calculate_lifecycle(inputs, NOW).segment == "core" + + below = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=60), + activity=activity, + totals=_totals(active_days=3, successful_backtests=2), + as_of=NOW, + ) + assert calculate_lifecycle(below, NOW).segment == "growing" + + +def test_a_user_with_no_activity_row_falls_back_to_signup(): + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=2), + activity=None, + totals=None, + as_of=NOW, + ) + result = calculate_lifecycle(inputs, NOW) + + assert result.segment == "new" + assert result.active_days_30d == 0 + + +def test_evidence_newer_than_as_of_is_clamped_not_raised(): + """Serving a past day from a table that only knows the present. + + `user_activity` is one row per user, overwritten in place. Asked for + the end of yesterday while holding a timestamp from this morning, + `calculate_lifecycle` raises rather than guessing -- so the clamp has to + happen here, and the fact table is where the answer for that past day + actually lives. + """ + end_of_yesterday = datetime(2026, 9, 11, 23, 59, 59, tzinfo=timezone.utc) + activity = UserActivity( + user_id=1, + activated_at=NOW, # activated today + last_meaningful_activity_at=NOW, # active today + updated_at=NOW, + ) + + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=60), + activity=activity, + totals=_totals(active_days=1, last_active_date=date(2026, 9, 3)), + as_of=end_of_yesterday, + ) + + # Not activated as of yesterday, and last active on the 3rd -- eight + # whole UTC days before the day being computed, i.e. genuinely at risk. + # (The plan drafted this fixture with the 4th, which is only seven days + # back and classifies onboarding under lifecycle.py's >=8 threshold.) + assert inputs.first_successful_backtest_at is None + assert inputs.last_meaningful_activity_at == datetime( + 2026, 9, 3, tzinfo=timezone.utc + ) + result = calculate_lifecycle(inputs, end_of_yesterday) + assert result.segment == "at_risk" + + +def test_activity_inside_the_computed_day_beats_the_window_fallback(): + """A user active only on D is not dormant, and the window cannot say so. + + The daily job narrows the fact window to D-29..D-1, so a user whose first + activity in a month lands on D has no row in it. Falling straight through + to `last_active_date` reads None and classifies them Dormant on the very + day they came back. + """ + end_of_yesterday = datetime(2026, 9, 11, 23, 59, 59, tzinfo=timezone.utc) + activity = UserActivity( + user_id=1, + activated_at=NOW - timedelta(days=200), + last_meaningful_activity_at=NOW, # newer than as_of, so clamped + updated_at=NOW, + ) + + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=300), + activity=activity, + totals=_totals(active_days=1, last_active_date=None), + as_of=end_of_yesterday, + day_activity_at=datetime(2026, 9, 11, 18, 0, tzinfo=timezone.utc), + ) + + assert inputs.last_meaningful_activity_at == datetime( + 2026, 9, 11, 18, 0, tzinfo=timezone.utc + ) + assert calculate_lifecycle(inputs, end_of_yesterday).segment == "growing" + + +def test_the_live_path_clamps_nothing(): + """as_of=now is the identity case, and must stay cost-free.""" + activity = UserActivity( + user_id=1, + activated_at=NOW - timedelta(days=3), + last_meaningful_activity_at=NOW, + updated_at=NOW, + ) + + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=60), + activity=activity, + totals=_totals(active_days=5, successful_backtests=5), + as_of=NOW, + ) + + assert inputs.first_successful_backtest_at == activity.activated_at + assert inputs.last_meaningful_activity_at == activity.last_meaningful_activity_at + + +def test_inactivity_crosses_at_risk_without_any_new_evidence(): + """Time alone moves the segment. That is why nothing needs recomputing.""" + activity = UserActivity( + user_id=1, + activated_at=NOW - timedelta(days=40), + last_meaningful_activity_at=NOW - timedelta(days=7), + updated_at=NOW, + ) + inputs = build_lifecycle_inputs( + 1, + created_at=NOW - timedelta(days=90), + activity=activity, + totals=_totals(), + as_of=NOW, + ) + + assert calculate_lifecycle(inputs, NOW).segment == "growing" + assert calculate_lifecycle(inputs, NOW + timedelta(days=1)).segment == "at_risk" + assert calculate_lifecycle(inputs, NOW + timedelta(days=23)).segment == "dormant" + + +def _fact_store(tmp_path): + path = tmp_path / "facts.db" + users = UserStore(db_path=path) + first = int(users.create_user("a@example.test", "A", "SecurePass1!")["id"]) + second = int(users.create_user("b@example.test", "B", "SecurePass1!")["id"]) + analytics = AnalyticsStore(db_path=path) + store = build_value_analytics_store( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + with analytics._get_connection() as conn: + conn.executemany( + """ + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, tier, user_group, active, runs_requested, + runs_completed, runs_failed, runs_cancelled, operator_cost_micro, + own_spend_micro, data_quality, calculated_at + ) VALUES (?, ?, 'growing', 'growing_activated_below_core_threshold', + 'healthy', 'unpaid', 'unknown', ?, ?, ?, ?, ?, ?, ?, + 'complete', '2026-09-12T00:05:00+00:00') + """, + [ + ("2026-09-01", first, 1, 2, 1, 1, 0, 1_000_000, 0), + ("2026-09-05", first, 1, 1, 1, 0, 0, 0, 250_000), + ("2026-09-09", first, 0, 0, 0, 0, 0, 0, 0), + ("2026-09-11", first, 1, 3, 2, 0, 1, 500_000, 0), + ("2026-09-11", second, 1, 1, 1, 0, 0, 0, 0), + ("2026-08-01", first, 1, 9, 9, 0, 0, 0, 0), # outside the window + ], + ) + return store, first, second + + +def test_sum_recent_facts_groups_the_window_in_one_statement(tmp_path): + store, first, second = _fact_store(tmp_path) + + totals = store.sum_recent_facts( + [first, second], start=date(2026, 8, 13), end=date(2026, 9, 11) + ) + + assert totals[first] == RecentFactTotals( + active_days=3, + successful_backtests=4, + runs_requested=6, + runs_completed=4, + runs_failed=1, + runs_cancelled=1, + operator_cost_micro=1_500_000, + own_spend_micro=250_000, + days_present=4, + last_active_date=date(2026, 9, 11), + ) + assert totals[second].active_days == 1 + assert totals[second].days_present == 1 + # None means the whole population -- the daily job's shape. + assert store.sum_recent_facts( + None, start=date(2026, 8, 13), end=date(2026, 9, 11) + ) == totals + assert store.sum_recent_facts([], start=date(2026, 8, 13), end=date(2026, 9, 11)) == {} diff --git a/dashboard/backend/tests/domain/analytics/test_operational_signals.py b/dashboard/backend/tests/domain/analytics/test_operational_signals.py new file mode 100644 index 00000000..aabec183 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_operational_signals.py @@ -0,0 +1,359 @@ +"""The batched operational read agrees with the per-user one. + +Two implementations of one rule set drift silently. This test is the only +thing that keeps the daily job's answer equal to the profile's answer. +""" + +from __future__ import annotations + +import sqlite3 +from datetime import datetime, timedelta, timezone + +import pytest +from cryptography.fernet import Fernet + +from dashboard.backend.domain.agents.repository import AgentStore +from dashboard.backend.domain.analytics.lifecycle import ( + calculate_operational_state, + consecutive_failed_terminal_runs, +) +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.value_repository import ( + build_value_analytics_store, +) +from dashboard.backend.domain.brokers import repository as broker_repository +from dashboard.backend.domain.credits.repository import CreditsStore +from dashboard.backend.domain.model_providers.repository import ModelProviderStore +from dashboard.backend.domain.runs.repository import RunStore +from dashboard.backend.tests.domain.analytics._store_spies import CountingSpy, SpyBundle +from dashboard.backend.tests.test_credits_ledger_aggregates import _insert, _ledger_row +from dashboard.backend.users import UserStore + + +NOW = datetime(2026, 9, 12, 12, 0, tzinfo=timezone.utc) + + +@pytest.fixture(autouse=True) +def _encryption_key(monkeypatch): + monkeypatch.setenv("BROKER_TOKEN_ENCRYPTION_KEY", Fernet.generate_key().decode()) + monkeypatch.setattr(broker_repository, "_fernet_instance", None) + # The platform lane must be decided by the stored credential alone. + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + monkeypatch.delenv("COMMONSTACK_API_KEY", raising=False) + + +class OperationalFixture: + def __init__(self, path, *, users): + self.path = path + UserStore(db_path=path) # creates the users table + with sqlite3.connect(path) as conn: + conn.executemany( + "INSERT INTO users (id, email, display_name, password_hash, role, " + "user_group, created_at) VALUES (?, ?, ?, 'x', 'user', 'unknown', ?)", + [ + (index, f"user{index}@example.test", f"User {index}", + (NOW - timedelta(days=30)).isoformat()) + for index in range(1, users + 1) + ], + ) + self.user_ids = list(range(1, users + 1)) + self.credits = CreditsStore(path) + self.providers = ModelProviderStore(path) + self.agents = AgentStore(path) + self.runs = RunStore(path) + self.analytics = AnalyticsStore(path) + self.spies = SpyBundle( + credits=CountingSpy(self.credits, "credits"), + providers=CountingSpy(self.providers, "providers"), + agents=CountingSpy(self.agents, "agents"), + runs=CountingSpy(self.runs, "runs"), + ) + self.store = build_value_analytics_store( + self.analytics, + credits_base=self.spies.credits, + provider_base=self.spies.providers, + agent_base=self.spies.agents, + run_base=self.spies.runs, + ) + self.interleaved_failures_id = 4 + self._seed() + + # -- seeding ----------------------------------------------------------- + + def _provider(self, provider_id, *, status="enabled", platform_enabled=None): + conn = self.providers._get_connection() + try: + conn.execute( + "UPDATE provider_registry SET status = ? WHERE provider_id = ?", + (status, provider_id), + ) + if platform_enabled is not None: + conn.execute( + "UPDATE provider_registry SET platform_enabled = ? WHERE provider_id = ?", + (int(platform_enabled), provider_id), + ) + conn.commit() + finally: + conn.close() + + def _default_credential(self, user_id, provider_id, *, status="verified"): + created = self.providers.create_user_credential( + user_id=user_id, + provider_id=provider_id, + label=f"{provider_id}-{user_id}", + secret="sk-synthetic-secret-1234", + status="verified", + set_default=True, + ) + if status != "verified": + # The store clears is_default when a credential stops being + # verified, so an *invalid default* -- the exact state + # ``default_credential_status == "invalid"`` describes -- can only + # be reached by a row that went invalid underneath its flag. Write + # that row directly; both readers see the same table. + conn = self.providers._get_connection() + try: + conn.execute( + "UPDATE user_model_credentials SET status = ? WHERE credential_id = ?", + (status, created["credential_id"]), + ) + conn.commit() + finally: + conn.close() + + def _run(self, agent, status, *, hours_ago): + run = self.runs.create_run( + agent_id=agent["agent_id"], + agent_version_id=None, + session_id=agent["session_id"], + environment_id=None, + environment_type="backtest", + config={}, + status=status, + ) + stamp = (NOW - timedelta(hours=hours_ago)).isoformat() + conn = self.runs._get_connection() + try: + conn.execute( + "UPDATE protocol_runs SET created_at = ?, updated_at = ? WHERE run_id = ?", + (stamp, stamp, run["run_id"]), + ) + conn.commit() + finally: + conn.close() + + def _grant(self, user_id, amount_micro, *, key): + _insert( + self.path, + "credit_ledger_entries", + [_ledger_row(user_id, "admin_grant_assign", amount_micro, NOW - timedelta(days=1), key=key)], + ) + + def _seed(self): + # One platform lane is open for everyone with a balance. + self._provider("openrouter", status="enabled", platform_enabled=True) + self.providers.upsert_platform_credential( + provider_id="openrouter", secret="sk-platform-secret-9999", status="verified" + ) + for user_id in self.user_ids: + self._grant(user_id, 1_000_000, key=f"grant-{user_id}") + # 1: restricted account (blocked, highest precedence). + self.credits.restrict_account(1, reason="llm_overage") + # 2: invalid default credential (needs_attention). + self._default_credential(2, "openai", status="invalid") + # 3: three consecutive failed terminal runs inside 24h (needs_attention). + agent_three = self.agents.create_agent(name="Three", owner_user_id=3) + for hours in (1, 2, 3): + self._run(agent_three, "failed", hours_ago=hours) + # 4: failed, failed, succeeded, failed, failed -> consecutive count 2. + agent_four = self.agents.create_agent(name="Four", owner_user_id=4) + for status, hours in (("failed", 1), ("failed", 2), ("completed", 3), ("failed", 4), ("failed", 5)): + self._run(agent_four, status, hours_ago=hours) + # 5: failures spread across two agents; pooled newest-first is what counts. + agent_five_a = self.agents.create_agent(name="Five A", owner_user_id=5) + agent_five_b = self.agents.create_agent(name="Five B", owner_user_id=5) + self._run(agent_five_a, "failed", hours_ago=1) + self._run(agent_five_b, "failed", hours_ago=2) + self._run(agent_five_a, "completed", hours_ago=3) + self._run(agent_five_b, "failed", hours_ago=30) # outside 24h + # 6: default credential on a provider that is then disabled (blocked). + # Order matters: create_user_credential refuses a disabled provider, + # so the credential is created first and the provider disabled after. + # "anthropic" is one of the four SEEDED_PROVIDERS (repository_common.py) + # and is BYOK-enabled; nobody else in this fixture uses it. + self._default_credential(6, "anthropic") + self._provider("anthropic", status="disabled") + # 7: no usable lane -- remove the grant so the balance is zero (blocked). + with sqlite3.connect(self.path) as conn: + conn.execute("DELETE FROM credit_ledger_entries WHERE user_id = 7") + # 8+: clean accounts (healthy). + + def reset(self): + self.spies.reset() + + @property + def total_calls(self): + return self.spies.total_calls + + @property + def calls_with_scalar_user_id(self): + return self.spies.calls_with_scalar_user_id + + +def _operational_fixture(path, *, users): + return OperationalFixture(path / "operational.db", users=users) + + +@pytest.fixture +def operational_fixture(tmp_path): + return _operational_fixture(tmp_path, users=8) + + +def test_batched_signals_match_the_per_user_facts(operational_fixture): + """Same users, same instant, same answer -- and every seeded state is hit.""" + store, user_ids = operational_fixture.store, operational_fixture.user_ids + + batched = store.list_operational_signals(user_ids, now=NOW) + + states = {} + for user_id in user_ids: + facts = store.get_operational_facts(user_id, now=NOW) + signals = batched[user_id] + assert signals.account_restricted == facts.account_restricted, user_id + assert signals.usable_billing_lane == facts.usable_billing_lane, user_id + assert signals.selected_provider_enabled == facts.selected_provider_enabled, user_id + assert signals.default_credential_status == facts.default_credential_status, user_id + assert signals.failed_terminal_runs_24h == facts.failed_terminal_runs_24h, user_id + states[user_id] = calculate_operational_state(signals, NOW) + assert states[user_id].state == calculate_operational_state( + type(signals)( + user_id=user_id, + account_restricted=facts.account_restricted, + usable_billing_lane=facts.usable_billing_lane, + selected_provider_enabled=facts.selected_provider_enabled, + default_credential_status=facts.default_credential_status, + failed_terminal_runs_24h=facts.failed_terminal_runs_24h, + run_beyond_safe_deadline=False, + ), + NOW, + ).state + assert states[1].reason_code == "account_restricted" + assert states[2].reason_code == "invalid_default_credential" + assert states[3].reason_code == "three_consecutive_failed_runs" + assert states[4].state == "healthy" + assert states[6].reason_code == "provider_disabled" + assert states[7].reason_code == "billing_lane_unavailable" + assert states[8].state == "healthy" + + +def test_the_population_path_answers_for_the_given_ids_without_an_in_list(operational_fixture): + store = operational_fixture.store + ids = operational_fixture.user_ids + + wide = store.list_operational_signals(ids, now=NOW, population_wide=True) + listed = store.list_operational_signals(ids, now=NOW) + + assert set(wide) == set(ids) + assert wide == listed + # User 7 has no row in any source; both modes still answer for them. + assert wide[7].usable_billing_lane is False + assert store.list_operational_signals([], now=NOW, population_wide=True) == {} + + +def test_batched_signals_do_not_query_per_user(tmp_path): + """Query count is fixed, not proportional to the population. + + Asserting a literal ceiling pins an implementation detail this task + cannot honour: the run count alone needs two statements, because + protocol_runs and external_agents are in different databases. The + property that matters is that neither number moves when the user count + does. + """ + small = _operational_fixture(tmp_path / "small", users=10) + large = _operational_fixture(tmp_path / "large", users=40) + + small.reset() + small.store.list_operational_signals(small.user_ids, now=NOW, population_wide=True) + small_calls = small.total_calls + + large.reset() + large.store.list_operational_signals(large.user_ids, now=NOW, population_wide=True) + + assert large.calls_with_scalar_user_id == [] + assert large.total_calls == small_calls + # A loose absolute bound as well, so "zero queries because it silently + # returned defaults" cannot pass the equality above. + assert 4 <= small_calls <= 8 + + +def test_the_batched_count_is_consecutive_not_total(operational_fixture): + """failed, failed, succeeded, failed, failed inside 24h scores 2. + + This is the one place the two implementations could disagree while + every other assertion stayed green, because both numbers are plausible + and only one of them matches the reason code the UI renders. + """ + user_id = operational_fixture.interleaved_failures_id + + signals = operational_fixture.store.list_operational_signals( + [user_id], now=NOW + )[user_id] + + assert signals.failed_terminal_runs_24h == 2 + assert calculate_operational_state(signals, NOW).state == "healthy" + + +def test_runs_are_pooled_across_an_owners_agents_before_counting(operational_fixture): + signals = operational_fixture.store.list_operational_signals([5], now=NOW)[5] + + # Newest-first across both agents: failed(A,-1h), failed(B,-2h), completed(A,-3h). + assert signals.failed_terminal_runs_24h == 2 + + +def test_a_user_with_no_rows_agrees_with_the_per_user_reader(operational_fixture): + """Absence is computed, not defaulted. + + A user with no ledger row, no credential and no agent has a zero balance + and no BYOK lane, so both readers say `billing_lane_unavailable`. The + superseded plan asserted `healthy` here against synthetic stores whose + defaults were permissive; against real stores the honest answer is + "blocked", and it must be the *same* honest answer on both paths. + """ + store = operational_fixture.store + absent = max(operational_fixture.user_ids) + 1 + # The user must exist -- get_account_billing_state lazily creates the + # credit_accounts row and its FK points at users. "Absent" here means + # exists as an account with no ledger row, no credential, no agent. + import sqlite3 as _sqlite3 + with _sqlite3.connect(operational_fixture.path) as conn: + conn.execute( + "INSERT INTO users (id, email, display_name, password_hash, role, " + "user_group, created_at) VALUES (?, 'absent@example.test', 'Absent', " + "'x', 'user', 'unknown', ?)", + (absent, (NOW - timedelta(days=30)).isoformat()), + ) + + signals = store.list_operational_signals([absent], now=NOW)[absent] + facts = store.get_operational_facts(absent, now=NOW) + + assert signals.default_credential_status == "missing" + assert signals.failed_terminal_runs_24h == 0 + assert signals.usable_billing_lane is facts.usable_billing_lane is False + assert ( + calculate_operational_state(signals, NOW).reason_code + == "billing_lane_unavailable" + ) + + +@pytest.mark.parametrize( + "statuses,expected", + [ + ((), 0), + (("failed", "failed", "failed"), 3), + (("failed", "timed_out", "completed", "failed"), 2), + (("completed", "failed", "failed"), 0), + (("cancelled", "failed"), 0), + ], +) +def test_consecutive_failed_terminal_runs_counts_leading_failures(statuses, expected): + assert consecutive_failed_terminal_runs(statuses) == expected \ No newline at end of file diff --git a/dashboard/backend/tests/domain/analytics/test_read_budget.py b/dashboard/backend/tests/domain/analytics/test_read_budget.py new file mode 100644 index 00000000..5c020f78 --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_read_budget.py @@ -0,0 +1,221 @@ +"""The architectural claim of the design, stated as tests (design SS6.12). + +Every store the daily job touches is wrapped in a counting spy and the job is +driven against 200 and then 400 synthetic users. If the query count moves, +someone has put a user id into a query inside the job; if the wall-clock ratio +moves, a constant-count step hides a scan that grows with the population. +""" + +from __future__ import annotations + +import sqlite3 +import time +from datetime import date, datetime, time as clock_time, timedelta, timezone + +import pytest +from cryptography.fernet import Fernet + +from dashboard.backend.database import BacktestDatabase +from dashboard.backend.domain.agents.repository import AgentStore +from dashboard.backend.domain.analytics.daily_facts import run_daily_facts +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.service import AnalyticsService +from dashboard.backend.domain.analytics.value_repository import ( + build_value_analytics_store, +) +from dashboard.backend.domain.brokers import repository as broker_repository +from dashboard.backend.domain.credits.repository import CreditsStore +from dashboard.backend.domain.model_providers.repository import ModelProviderStore +from dashboard.backend.domain.runs.repository import RunStore +from dashboard.backend.tests.domain.analytics._store_spies import CountingSpy, SpyBundle +from dashboard.backend.tests.test_credits_ledger_aggregates import _insert, _ledger_row +from dashboard.backend.users import UserStore + + +D = date(2026, 9, 11) +NOW = datetime(2026, 9, 12, 0, 5, tzinfo=timezone.utc) + +# The constant the daily job commits to, per claimed tick, in public store-method +# calls (the unit CountingSpy measures). Task 9's table in the PR A plan is the +# source of this number; if it moves, either the job or the table drifted, and +# the table is authoritative until the design changes. +# value store 12 claim, aggregate_events, record_activity_batch x2, +# list_operational_signals, list_activity, sum_recent_facts, +# upsert_daily_facts, list_facts_for_date, +# append_lifecycle_transitions, list_days_needing_recompute, +# complete_projection_day +# analytics 2 list_excluded_user_ids (inside rollup_day), list_daily_subjects +# credits 3 aggregate_ledger_for_day, get_balance_projections, +# list_account_billing_states +# providers 3 list_default_credential_facts, list_all_providers, +# list_platform_credential_statuses +# agents 1 list_agent_owners +# protocol runs 1 list_terminal_runs_since +# run history 1 aggregate_operator_cost_for_day +DAILY_JOB_CLAIMED_TICK_CALLS = 23 +WALL_CLOCK_BUDGET_SECONDS_AT_200 = 20.0 +WALL_CLOCK_RATIO_LIMIT = 2.5 +# A ratio over tiny absolute times is noise, not a scan; floor the denominator. +WALL_CLOCK_FLOOR_SECONDS = 0.25 + + +@pytest.fixture(autouse=True) +def _encryption_key(monkeypatch): + monkeypatch.setenv("BROKER_TOKEN_ENCRYPTION_KEY", Fernet.generate_key().decode()) + monkeypatch.setattr(broker_repository, "_fernet_instance", None) + monkeypatch.delenv("OPENROUTER_API_KEY", raising=False) + monkeypatch.delenv("COMMONSTACK_API_KEY", raising=False) + + +class _NoRetention: + def run_if_due(self): + return None + + +class _DailyFixture: + def __init__(self, path, *, users): + path.mkdir(parents=True, exist_ok=True) + self.path = path / "budget.db" + UserStore(db_path=self.path) + with sqlite3.connect(self.path) as conn: + conn.executemany( + "INSERT INTO users (id, email, display_name, password_hash, role, " + "user_group, created_at) VALUES (?, ?, ?, 'x', 'user', 'unknown', ?)", + [ + (index, f"user{index}@example.test", f"User {index}", + (NOW - timedelta(days=60)).isoformat()) + for index in range(1, users + 1) + ], + ) + self.user_ids = list(range(1, users + 1)) + analytics = AnalyticsStore(self.path) + credits = CreditsStore(self.path) + providers = ModelProviderStore(self.path) + agents = AgentStore(self.path) + protocol_runs = RunStore(self.path) + run_history = BacktestDatabase(path / "runs.db") + self.spies = SpyBundle( + analytics=CountingSpy(analytics, "analytics"), + credits=CountingSpy(credits, "credits"), + providers=CountingSpy(providers, "providers"), + agents=CountingSpy(agents, "agents"), + runs=CountingSpy(protocol_runs, "runs"), + run_history=CountingSpy(run_history, "run_history"), + ) + real_value_store = build_value_analytics_store( + self.spies.analytics, + credits_base=self.spies.credits, + provider_base=self.spies.providers, + agent_base=self.spies.agents, + run_base=self.spies.runs, + ) + self.value_store = CountingSpy(real_value_store, "value_store") + self.spies.spies["value_store"] = self.value_store + # Seed through the real (unspied) stores so setup is not counted. + service = AnalyticsService(analytics, value_store=real_value_store, maintain_activity=True) + day_start = datetime.combine(D, clock_time(9, 0), tzinfo=timezone.utc) + for user_id in self.user_ids: + service.record_server_event( + event_name="backtest_requested", + user_id=user_id, + source_event_id=f"run:backtest_requested:{user_id}", + source_record_type="run", + source_record_id=f"run-{user_id}", + occurred_at=day_start + timedelta(seconds=user_id), + received_at=day_start + timedelta(seconds=user_id + 1), + ) + if user_id % 5 == 0: + service.record_server_event( + event_name="backtest_completed", + user_id=user_id, + source_event_id=f"run:backtest_completed:{user_id}", + source_record_type="run", + source_record_id=f"run-{user_id}", + occurred_at=day_start + timedelta(minutes=1, seconds=user_id), + received_at=day_start + timedelta(minutes=1, seconds=user_id + 1), + ) + if user_id % 2 == 0: + agents.create_agent(name=f"Agent {user_id}", owner_user_id=user_id) + _insert( + self.path, + "credit_ledger_entries", + [ + _ledger_row(user_id, "admin_grant_assign", 1_000_000, NOW - timedelta(days=2), key=f"g{user_id}") + for user_id in self.user_ids + if user_id % 3 == 0 + ], + ) + self.spies.reset() + + def run(self, *, now=NOW): + return run_daily_facts( + now=now, + value_store=self.value_store, + run_history_store=self.spies.run_history, + retention=_NoRetention(), + ) + + +def _timed(action): + started = time.perf_counter() + result = action() + return result, time.perf_counter() - started + + +def test_the_daily_job_costs_the_same_at_any_user_count(tmp_path): + """Doubling the population changes nothing about the query count. + + This is the architectural claim of the whole design, stated as a test. + If it ever fails, someone has put a user id into a query inside the job. + """ + small = _DailyFixture(tmp_path / "small", users=200) + large = _DailyFixture(tmp_path / "large", users=400) + + small_report = small.run() + large_report = large.run() + + assert small_report.claimed and large_report.claimed + assert small_report.partial is False and large_report.partial is False + assert small_report.users_written == 200 + assert large_report.users_written == 400 + assert small.spies.total_calls == large.spies.total_calls, ( + small.spies.calls_by_store(), + large.spies.calls_by_store(), + ) + assert small.spies.total_calls == DAILY_JOB_CLAIMED_TICK_CALLS, small.spies.calls_by_store() + assert small.spies.calls_with_scalar_user_id == [] + assert large.spies.calls_with_scalar_user_id == [] + + +def test_the_daily_job_stays_inside_the_wall_clock_budget(tmp_path): + """A constant query count can still hide a scan (design SS12 item 5).""" + small = _DailyFixture(tmp_path / "small", users=200) + large = _DailyFixture(tmp_path / "large", users=400) + + _report, small_elapsed = _timed(small.run) + _report, large_elapsed = _timed(large.run) + + assert small_elapsed < WALL_CLOCK_BUDGET_SECONDS_AT_200, small_elapsed + assert large_elapsed <= WALL_CLOCK_RATIO_LIMIT * max(small_elapsed, WALL_CLOCK_FLOOR_SECONDS), ( + small_elapsed, + large_elapsed, + ) + + +def test_an_idle_tick_costs_one_query(tmp_path): + """Most ticks in a day do nothing. They must cost (almost) nothing.""" + fixture = _DailyFixture(tmp_path, users=50) + fixture.run() + fixture.spies.reset() + + for tick in range(60): + report = fixture.run(now=NOW + timedelta(minutes=5 * tick)) + assert report.claimed is False + + assert fixture.spies.total_calls == 60 + assert fixture.spies.calls_by_store()["value_store"] == 60 + assert all( + count == 0 + for name, count in fixture.spies.calls_by_store().items() + if name != "value_store" + ) diff --git a/dashboard/backend/tests/domain/analytics/test_repository_contract.py b/dashboard/backend/tests/domain/analytics/test_repository_contract.py index 2d5befbe..98134d1b 100644 --- a/dashboard/backend/tests/domain/analytics/test_repository_contract.py +++ b/dashboard/backend/tests/domain/analytics/test_repository_contract.py @@ -15,7 +15,6 @@ ) from dashboard.backend.domain.analytics.query_service import ( AnalyticsQueryService, - AnalyticsUserFilters, ) from dashboard.backend.domain.analytics.repository import ( ANALYTICS_SQLITE_DDL, @@ -294,12 +293,7 @@ def assert_pr2_query_contract(store, user_id): ), now=NOW, ) - users = service.list_users( - filters=AnalyticsUserFilters(), - limit=10, - offset=0, - now=NOW, - ) + profile = service.get_user_profile(user_id=user_id, now=NOW) activity = service.get_user_activity( user_id=user_id, @@ -328,8 +322,6 @@ def assert_pr2_query_contract(store, user_id): assert overview.completed_runs == 1 assert overview.failed_runs == 1 assert overview.platform_model_cost_usd == 0.25 - assert users.total == 1 - assert users.items[0].status == "active" assert profile.state.status == "active" assert completed.event_id in profile.state.evidence_event_ids assert profile.input_tokens == 120 @@ -729,3 +721,156 @@ def test_foreign_keys_reject_missing_users(sqlite_contract): store.append_event(event_record(999_999)) with pytest.raises((AnalyticsStoreError, sqlite3.IntegrityError)): store.record_admin_access(999_998, 999_999, "overview") + + +def test_sqlite_declares_the_daily_fact_tables(sqlite_contract): + store, _admin_id, _user_id = sqlite_contract + + with store._get_connection() as conn: + names = { + row[0] + for row in conn.execute( + "SELECT name FROM sqlite_master WHERE type = 'table'" + ).fetchall() + } + fact_columns = { + row[1] + for row in conn.execute("PRAGMA table_info(user_daily_facts)").fetchall() + } + activity_columns = { + row[1] + for row in conn.execute("PRAGMA table_info(user_activity)").fetchall() + } + transition_columns = { + row[1] + for row in conn.execute( + "PRAGMA table_info(lifecycle_transitions)" + ).fetchall() + } + + assert { + "user_activity", + "user_daily_facts", + "lifecycle_transitions", + } <= names + assert fact_columns == { + "snapshot_date", + "user_id", + "lifecycle_segment", + "lifecycle_reason_code", + "operational_state", + "operational_reason_code", + "tier", + "user_group", + "active", + "runs_requested", + "runs_completed", + "runs_failed", + "runs_cancelled", + "operator_cost_micro", + "own_spend_micro", + "data_quality", + "calculated_at", + } + # D9: the struck axis must not come back under its old name. + assert "cohort" not in fact_columns + assert activity_columns == { + "user_id", + "activated_at", + "last_meaningful_activity_at", + "updated_at", + } + assert transition_columns == { + "transition_id", + "user_id", + "snapshot_date", + "from_segment", + "to_segment", + "inactive_days", + "data_quality", + "created_at", + } + + +def test_user_group_on_facts_defaults_to_unknown_and_is_constrained(sqlite_contract): + store, _admin_id, user_id = sqlite_contract + + with store._get_connection() as conn: + conn.execute( + """ + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, tier, data_quality, calculated_at + ) VALUES ( + '2026-09-11', ?, 'growing', 'growing_activated_below_core_threshold', + 'healthy', 'unpaid', 'complete', '2026-09-12T00:05:00+00:00' + ) + """, + (user_id,), + ) + row = conn.execute( + "SELECT user_group, operational_reason_code FROM user_daily_facts" + ).fetchone() + assert row["user_group"] == "unknown" + assert row["operational_reason_code"] is None + with pytest.raises(sqlite3.IntegrityError): + conn.execute( + """ + INSERT INTO user_daily_facts ( + snapshot_date, user_id, lifecycle_segment, lifecycle_reason_code, + operational_state, tier, user_group, data_quality, calculated_at + ) VALUES ( + '2026-09-10', ?, 'growing', 'growing_activated_below_core_threshold', + 'healthy', 'unpaid', 'lab', 'complete', '2026-09-11T00:05:00+00:00' + ) + """, + (user_id,), + ) + + +def test_lifecycle_transitions_are_unique_per_user_per_day(sqlite_contract): + """A retried daily job must not append the same transition twice.""" + store, _admin_id, user_id = sqlite_contract + + with store._get_connection() as conn: + for _ in range(2): + conn.execute( + """ + INSERT OR IGNORE INTO lifecycle_transitions ( + user_id, snapshot_date, from_segment, to_segment, + inactive_days, created_at + ) VALUES (?, '2026-09-11', 'growing', 'at_risk', 8, + '2026-09-12T00:05:00+00:00') + """, + (user_id,), + ) + count = conn.execute( + "SELECT COUNT(*) FROM lifecycle_transitions" + ).fetchone()[0] + + assert count == 1 + + +def test_existing_source_event_ids_are_looked_up_in_batches(sqlite_contract): + store, _admin_id, user_id = sqlite_contract + for suffix in ("a", "b"): + store.append_event( + event_record( + user_id, + event_name="backtest_completed", + event_group="run", + event_source="server", + source_event_id=f"run:backtest_completed:run-{suffix}", + page_view=None, + device_category=None, + browser_family=None, + ) + ) + + found = store.list_existing_source_event_ids( + ["run:backtest_completed:run-a", "run:backtest_completed:run-zzz"] + + [f"run:backtest_completed:filler-{i}" for i in range(600)] + ) + + assert found == {"run:backtest_completed:run-a"} + assert store.list_existing_source_event_ids([]) == set() diff --git a/dashboard/backend/tests/domain/analytics/test_repository_postgres.py b/dashboard/backend/tests/domain/analytics/test_repository_postgres.py index 699ec7ac..fd288463 100644 --- a/dashboard/backend/tests/domain/analytics/test_repository_postgres.py +++ b/dashboard/backend/tests/domain/analytics/test_repository_postgres.py @@ -279,3 +279,24 @@ def test_postgres_user_value_projection_round_trip( ) == [NOW.date()] assert value_store.delete_daily_snapshots_for_date(NOW.date()) == 1 assert value_store.has_daily_before(NOW.date() + timedelta(days=1)) is False + + +def test_postgres_ddl_declares_the_daily_fact_tables(): + ddl = pg_module.ANALYTICS_POSTGRES_DDL + assert "CREATE TABLE IF NOT EXISTS user_activity" in ddl + assert "CREATE TABLE IF NOT EXISTS user_daily_facts" in ddl + assert "CREATE TABLE IF NOT EXISTS lifecycle_transitions" in ddl + for column in ( + "operator_cost_micro", + "own_spend_micro", + "runs_completed", + "data_quality", + "operational_reason_code", + "user_group TEXT NOT NULL DEFAULT 'unknown'", + "tier", + ): + assert column in ddl + assert "cohort" not in ddl + # Wide counters are BIGINT on Postgres, matching analytics_daily_rollups. + assert "operator_cost_micro BIGINT" in ddl + assert "own_spend_micro BIGINT" in ddl diff --git a/dashboard/backend/tests/domain/analytics/test_retention.py b/dashboard/backend/tests/domain/analytics/test_retention.py index 4c576163..0ad20796 100644 --- a/dashboard/backend/tests/domain/analytics/test_retention.py +++ b/dashboard/backend/tests/domain/analytics/test_retention.py @@ -355,3 +355,54 @@ def test_lifecycle_history_is_aggregated_before_user_rows_are_deleted(tmp_path): "growing", 1, ) in [tuple(row) for row in rows] + + +def test_the_retention_sweep_never_touches_user_activity(tmp_path): + """Design doc SS11: user_activity is current state, not history. + + Every other user-keyed analytics table is retained 180 days. This one + holds the only record of when a user first activated, on a row that is + overwritten in place; sweeping it would delete every lifetime metric with + nothing else failing. A future "complete the 180-day rule" tidy-up must + turn this red. + """ + import inspect + import sqlite3 + + from dashboard.backend.domain.analytics import retention as retention_module + from dashboard.backend.domain.analytics.value_repository import ( + build_value_analytics_store, + ) + + path = tmp_path / "retention.db" + UserStore(db_path=path) + with sqlite3.connect(path) as conn: + conn.executemany( + "INSERT INTO users (id, email, display_name, password_hash, role, user_group, created_at) " + "VALUES (?, ?, ?, 'x', 'user', 'unknown', ?)", + [ + (1, "one@example.test", "One", (NOW - timedelta(days=500)).isoformat()), + (2, "two@example.test", "Two", (NOW - timedelta(days=500)).isoformat()), + ], + ) + analytics = AnalyticsStore(path) + value_store = build_value_analytics_store( + analytics, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + ancient = NOW - timedelta(days=RAW_EVENT_RETENTION_DAYS + 200) + for user_id in (1, 2): + value_store.record_activity(user_id, occurred_at=ancient, activating=True, now=ancient) + analytics.append_event(_event(user_id, ancient)) + before = {user_id: value_store.get_activity(user_id) for user_id in (1, 2)} + + result = AnalyticsRetentionService(store=analytics, value_store=value_store).run_once(NOW) + + assert result.raw_events_deleted == 2 + assert {user_id: value_store.get_activity(user_id) for user_id in (1, 2)} == before + assert before[1].activated_at == ancient + assert "user_activity" not in inspect.getsource(retention_module) + diff --git a/dashboard/backend/tests/domain/analytics/test_service.py b/dashboard/backend/tests/domain/analytics/test_service.py index eac2e27b..72b40def 100644 --- a/dashboard/backend/tests/domain/analytics/test_service.py +++ b/dashboard/backend/tests/domain/analytics/test_service.py @@ -237,3 +237,19 @@ def test_built_singleton_does_not_synchronously_project_snapshots(monkeypatch): assert result.created is True assert calls == [] + + +def test_built_singleton_maintains_user_activity(monkeypatch): + from dashboard.backend.domain.analytics import service as service_module + + monkeypatch.setattr(service_module, "analytics_store", RecordingStore()) + + built = service_module._build_analytics_service() + + assert built.maintain_activity is True + assert built.project_snapshots is False + + +def test_maintaining_activity_requires_a_value_store(): + with pytest.raises(ValueError): + AnalyticsService(RecordingStore(), maintain_activity=True) diff --git a/dashboard/backend/tests/domain/analytics/test_user_activity.py b/dashboard/backend/tests/domain/analytics/test_user_activity.py new file mode 100644 index 00000000..49f3799d --- /dev/null +++ b/dashboard/backend/tests/domain/analytics/test_user_activity.py @@ -0,0 +1,222 @@ +"""Ingestion maintains one current-state row per user and never reads history.""" + +from __future__ import annotations + +import sqlite3 +from datetime import datetime, timedelta, timezone + + +from dashboard.backend.domain.analytics import rollups as rollups_module +from dashboard.backend.domain.analytics.repository import AnalyticsStore +from dashboard.backend.domain.analytics.service import AnalyticsService +from dashboard.backend.domain.analytics.value_repository import ( + build_value_analytics_store, +) +from dashboard.backend.tests.domain.analytics.test_value_repository import ( + _value_snapshot, +) + + +NOW = datetime(2026, 9, 12, 12, 0, tzinfo=timezone.utc) + + +class CountingAnalyticsStore(AnalyticsStore): + """The real SQLite store, counting every per-user history read.""" + + def __init__(self, path): + super().__init__(path) + self.event_reads = 0 + + def list_user_events(self, *args, **kwargs): + self.event_reads += 1 + return super().list_user_events(*args, **kwargs) + + +def _fixture(tmp_path, monkeypatch): + path = tmp_path / "activity.db" + with sqlite3.connect(path) as conn: + conn.execute( + """ + CREATE TABLE users ( + id INTEGER PRIMARY KEY, + email TEXT NOT NULL, + display_name TEXT NOT NULL, + password_hash TEXT NOT NULL, + role TEXT NOT NULL, + user_group TEXT NOT NULL DEFAULT 'unknown', + created_at TEXT NOT NULL + ) + """ + ) + conn.execute( + "INSERT INTO users VALUES (1, 'user@example.test', 'User', 'x', 'user', " + "'unknown', ?)", + ((NOW - timedelta(days=60)).isoformat(),), + ) + store = CountingAnalyticsStore(path) + + def _no_history_scan(*_args, **_kwargs): + raise AssertionError("ingestion must not scan analytics_events") + + monkeypatch.setattr( + rollups_module.AnalyticsRollupStore, "list_events", _no_history_scan + ) + value_store = build_value_analytics_store( + store, + credits_base=object(), + provider_base=object(), + agent_base=object(), + run_base=object(), + ) + service = AnalyticsService( + store, + value_store=value_store, + maintain_activity=True, + ) + return service, store, value_store + + +def _emit(service, index, occurred_at, *, name="backtest_requested"): + return service.record_server_event( + event_name=name, + user_id=1, + source_event_id=f"run:{name}:run-{index}", + source_record_type="run", + source_record_id=f"run-{index}", + occurred_at=occurred_at, + received_at=NOW, + ) + + +def test_ingestion_never_reads_a_users_history(tmp_path, monkeypatch): + """Accepting an event costs one upsert, not a scan. + + The 2026-09-11 outage was this path: every accepted lifecycle event + recomputed a label from a 180-day read of the log it had just appended to. + """ + service, store, _value_store = _fixture(tmp_path, monkeypatch) + for index in range(20): + _emit(service, index, NOW - timedelta(seconds=index)) + + assert store.event_reads == 0 + + +def test_activated_at_holds_the_earliest_success(tmp_path, monkeypatch): + """Activation is the *first* success, in occurred_at order.""" + service, _store, value_store = _fixture(tmp_path, monkeypatch) + + _emit(service, "a", NOW - timedelta(days=3), name="backtest_completed") + first = value_store.get_activity(1).activated_at + + _emit(service, "b", NOW, name="backtest_completed") + activity = value_store.get_activity(1) + + assert first == NOW - timedelta(days=3) + assert activity.activated_at == first + assert activity.last_meaningful_activity_at == NOW + + +def test_an_out_of_order_success_moves_activated_at_earlier(tmp_path, monkeypatch): + """The newest write does not own the value; the earliest instant does. + + Ingestion sees events in arrival order, not occurred_at order: a + `backtest_completed` can be appended late, replayed from the queue, or + backdated by the 24-hour acceptance window in `service.py:96-97`. If + activation were pinned by whichever row landed first, this user's + activation cohort week -- and therefore their whole column of the + retention grid -- would be permanently wrong, with nothing able to + correct it. + """ + service, _store, value_store = _fixture(tmp_path, monkeypatch) + + for suffix, occurred in (("late", NOW), ("early", NOW - timedelta(days=5))): + _emit(service, suffix, occurred, name="backtest_completed") + + activity = value_store.get_activity(1) + + assert activity.activated_at == NOW - timedelta(days=5) + # The activity clock still only advances. + assert activity.last_meaningful_activity_at == NOW + + +def test_page_views_and_signups_do_not_advance_the_activity_clock( + tmp_path, monkeypatch +): + """Only the meaningful-activity set counts. A visit is not activity.""" + service, _store, value_store = _fixture(tmp_path, monkeypatch) + + service.record_server_event( + event_name="account_signed_up", + user_id=1, + source_event_id="account:account_signed_up:1", + source_record_type="user", + source_record_id="1", + occurred_at=NOW, + received_at=NOW, + ) + + assert value_store.get_activity(1) is None + + +def test_a_replayed_event_does_not_touch_the_row(tmp_path, monkeypatch): + """`created=False` means the log already had it; the clock must not move.""" + service, _store, value_store = _fixture(tmp_path, monkeypatch) + + _emit(service, "once", NOW - timedelta(days=1)) + before = value_store.get_activity(1) + replay = _emit(service, "once", NOW - timedelta(days=1)) + + assert replay.created is False + assert value_store.get_activity(1) == before + + +def test_list_activity_returns_every_row_when_no_ids_are_given(tmp_path, monkeypatch): + service, _store, value_store = _fixture(tmp_path, monkeypatch) + _emit(service, "x", NOW - timedelta(hours=1)) + + everyone = value_store.list_activity() + some = value_store.list_activity([1]) + none = value_store.list_activity([]) + + assert set(everyone) == {1} + assert some == everyone + assert none == {} + + +def test_seed_copies_the_legacy_timestamps_and_is_idempotent(tmp_path, monkeypatch): + """Design doc SS11: the migration must carry ``activated_at`` across. + + Ingestion only writes ``user_activity`` for events that arrive after the + deploy, and the daily job's one-day scan cannot see an activation from + last year. Without this seed every existing user's activation date would + be lost on the night PR B drops ``user_analytics_snapshots``. + """ + _service, _store, value_store = _fixture(tmp_path, monkeypatch) + legacy = _value_snapshot(1) # activated NOW-20d, last activity NOW-2d + value_store.upsert_current_snapshot(legacy) + + first = value_store.seed_activity_from_snapshots(now=NOW) + seeded = value_store.get_activity(1) + second = value_store.seed_activity_from_snapshots(now=NOW + timedelta(hours=1)) + reseeded = value_store.get_activity(1) + + assert first == 1 + assert seeded.activated_at == legacy.activated_at + assert seeded.last_meaningful_activity_at == legacy.last_meaningful_activity_at + assert second == 1 # the upsert touches the row again ... + assert reseeded.activated_at == seeded.activated_at # ... and changes nothing + assert reseeded.last_meaningful_activity_at == seeded.last_meaningful_activity_at + + +def test_seed_never_regresses_a_row_ingestion_already_advanced(tmp_path, monkeypatch): + """MIN on activation, MAX on activity: the seed can only fill gaps.""" + service, _store, value_store = _fixture(tmp_path, monkeypatch) + value_store.upsert_current_snapshot(_value_snapshot(1)) # NOW-20d / NOW-2d + _emit(service, "earlier", NOW - timedelta(days=30), name="backtest_completed") + _emit(service, "today", NOW) + + value_store.seed_activity_from_snapshots(now=NOW) + activity = value_store.get_activity(1) + + assert activity.activated_at == NOW - timedelta(days=30) + assert activity.last_meaningful_activity_at == NOW diff --git a/dashboard/backend/tests/domain/analytics/test_value_repository.py b/dashboard/backend/tests/domain/analytics/test_value_repository.py index 30a36b1b..64caa04b 100644 --- a/dashboard/backend/tests/domain/analytics/test_value_repository.py +++ b/dashboard/backend/tests/domain/analytics/test_value_repository.py @@ -93,6 +93,101 @@ def get_account_billing_state(self, user_id): ) + def aggregate_commercial_ledger(self, user_ids, *, start, end): + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + placeholders = ", ".join("?" for _ in ids) + window = [start.isoformat(), end.isoformat()] + with self._get_connection() as conn: + lifetime = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro + FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) AND entry_type IN ('purchase', 'refund') + GROUP BY user_id + """, + ids, + ).fetchall() + period = conn.execute( + f""" + SELECT user_id, + COALESCE(SUM(CASE WHEN entry_type = 'purchase' + THEN amount_micro ELSE 0 END), 0) AS purchased_micro, + COALESCE(SUM(CASE WHEN entry_type = 'refund' + THEN -amount_micro ELSE 0 END), 0) AS refunded_micro, + COALESCE(SUM(CASE + WHEN entry_type = 'admin_grant_assign' THEN amount_micro + WHEN entry_type = 'admin_grant_reclaim' THEN -amount_micro + ELSE 0 END), 0) AS grant_activity_micro + FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) AND created_at >= ? AND created_at < ? + GROUP BY user_id + """, + [*ids, *window], + ).fetchall() + usage = conn.execute( + f""" + SELECT user_id, COALESCE(SUM(-amount_micro), 0) AS consumed_micro + FROM credit_llm_usage_entries + WHERE user_id IN ({placeholders}) AND created_at >= ? AND created_at < ? + GROUP BY user_id + """, + [*ids, *window], + ).fetchall() + by_user = { + user_id: { + "lifetime_purchased_micro": 0, + "lifetime_refunded_micro": 0, + "purchased_micro": 0, + "refunded_micro": 0, + "grant_activity_micro": 0, + "consumed_micro": 0, + } + for user_id in ids + } + for row in lifetime: + by_user[int(row["user_id"])].update( + lifetime_purchased_micro=int(row["purchased_micro"]), + lifetime_refunded_micro=int(row["refunded_micro"]), + ) + for row in period: + by_user[int(row["user_id"])].update( + purchased_micro=int(row["purchased_micro"]), + refunded_micro=int(row["refunded_micro"]), + grant_activity_micro=int(row["grant_activity_micro"]), + ) + for row in usage: + by_user[int(row["user_id"])]["consumed_micro"] = int(row["consumed_micro"]) + return by_user + + def list_credit_activity_timestamps(self, user_ids, *, start, end): + ids = list(dict.fromkeys(int(user_id) for user_id in user_ids)) + placeholders = ", ".join("?" for _ in ids) + params = [*ids, start.isoformat(), end.isoformat()] + result = {user_id: [] for user_id in ids} + with self._get_connection() as conn: + rows = conn.execute( + f""" + SELECT user_id, created_at FROM credit_ledger_entries + WHERE user_id IN ({placeholders}) AND entry_type = 'purchase' + AND created_at >= ? AND created_at < ? + """, + params, + ).fetchall() + rows += conn.execute( + f""" + SELECT user_id, created_at FROM credit_llm_usage_entries + WHERE user_id IN ({placeholders}) AND created_at >= ? AND created_at < ? + """, + params, + ).fetchall() + for row in rows: + result[int(row["user_id"])].append(str(row["created_at"])) + return result + class SyntheticProviderStore: def __init__(self, credentials, providers): self.credentials = credentials @@ -475,3 +570,69 @@ def test_non_default_credential_does_not_create_a_usable_billing_lane(tmp_path): assert facts.usable_billing_lane is False assert facts.default_credential_status == "missing" + + +JOB = "analytics_daily_facts" +DAY = date(2026, 9, 11) +CLAIM_AT = datetime(2026, 9, 12, 0, 5, tzinfo=timezone.utc) + + +def test_only_one_claim_of_a_day_succeeds(tmp_path): + _user_id, analytics, credits = _stores(tmp_path) + store = _value_store(analytics, credits) + + first = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT) + second = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(minutes=1)) + job = store.get_projection_job(JOB) + + assert first is True + assert second is False + assert job.status == "running" + assert job.cursor is None # the cursor moves only when the day completes + + +def test_a_crashed_claim_is_reclaimable_after_two_hours(tmp_path): + _user_id, analytics, credits = _stores(tmp_path) + store = _value_store(analytics, credits) + store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT) + + too_soon = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(hours=1)) + reclaimed = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(hours=2, minutes=1)) + + assert too_soon is False + assert reclaimed is True + + +def test_a_completed_day_is_never_run_again(tmp_path): + _user_id, analytics, credits = _stores(tmp_path) + store = _value_store(analytics, credits) + store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT) + store.complete_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(minutes=2)) + + again = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(hours=5)) + earlier = store.claim_projection_day(JOB, day=DAY - timedelta(days=1), now=CLAIM_AT + timedelta(hours=5)) + next_day = store.claim_projection_day( + JOB, day=DAY + timedelta(days=1), now=CLAIM_AT + timedelta(days=1) + ) + job = store.get_projection_job(JOB) + + assert again is False + assert earlier is False + assert next_day is True + assert job.cursor == DAY.isoformat() + assert job.status == "running" + assert job.window_end == DAY + timedelta(days=1) + + +def test_a_released_claim_can_be_retried_at_once(tmp_path): + _user_id, analytics, credits = _stores(tmp_path) + store = _value_store(analytics, credits) + store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT) + + store.release_projection_day(JOB, now=CLAIM_AT + timedelta(minutes=1)) + retried = store.claim_projection_day(JOB, day=DAY, now=CLAIM_AT + timedelta(minutes=2)) + job = store.get_projection_job(JOB) + + assert retried is True + assert job.cursor is None + assert job.status == "running" diff --git a/dashboard/backend/tests/test_admin_analytics_api.py b/dashboard/backend/tests/test_admin_analytics_api.py index 7977b96f..b633d3f0 100644 --- a/dashboard/backend/tests/test_admin_analytics_api.py +++ b/dashboard/backend/tests/test_admin_analytics_api.py @@ -17,7 +17,6 @@ from dashboard.backend.domain.analytics.metrics import AnalyticsMetricFilters from dashboard.backend.domain.analytics.query_service import ( AnalyticsQueryService, - AnalyticsUserFilters, get_analytics_query_service, get_value_analytics_query_service, ) @@ -329,17 +328,10 @@ def test_user_list_and_profile_are_display_safe(tmp_path): recalculate_user_snapshot(1, now=NOW, store=states) service = AnalyticsQueryService(store=analytics, user_store=users) - listing = service.list_users( - filters=AnalyticsUserFilters(), - limit=25, - offset=0, - now=NOW, - ) + profile = service.get_user_profile(user_id=1, now=NOW) serialized = profile.model_dump(mode="json") - assert listing.total == 1 - assert listing.items[0].user_id == 1 assert profile.state.status == "active" assert success.event_id in profile.state.evidence_event_ids assert profile.input_tokens == 120 diff --git a/dashboard/backend/tests/test_admin_analytics_hygiene.py b/dashboard/backend/tests/test_admin_analytics_hygiene.py new file mode 100644 index 00000000..20a163ac --- /dev/null +++ b/dashboard/backend/tests/test_admin_analytics_hygiene.py @@ -0,0 +1,51 @@ +"""D24 hygiene on the admin analytics router: loud 503s, no dead users-list stack.""" + +from __future__ import annotations + +import inspect + +import pytest +from fastapi import HTTPException + +from dashboard.backend.api.routers import admin_analytics +from dashboard.backend.domain.analytics import query_service + + +def test_an_unrecognised_exception_is_logged_before_it_becomes_a_503(capsys): + """A bad SQL statement and an exhausted pool used to be indistinguishable + in prod logs: the router had no print and raised `from None`.""" + with pytest.raises(HTTPException) as info: + admin_analytics._raise_service_error(RuntimeError("secret-canary")) + + printed = capsys.readouterr().out + assert info.value.status_code == 503 + assert isinstance(info.value.__cause__, RuntimeError) + assert "ERROR: admin_analytics.unhandled category=RuntimeError" in printed + assert "secret-canary" not in printed + # The display-safe detail is unchanged. + assert info.value.detail == "Analytics is temporarily unavailable." + + +@pytest.mark.parametrize( + "exc,status", + [(LookupError("missing"), 404), (ValueError("bad"), 422)], +) +def test_recognised_exceptions_keep_their_quiet_mapping(capsys, exc, status): + with pytest.raises(HTTPException) as info: + admin_analytics._raise_service_error(exc) + + assert info.value.status_code == status + assert info.value.__cause__ is None + assert capsys.readouterr().out == "" + + +def test_the_dead_users_list_stack_is_gone(): + assert not hasattr(admin_analytics, "_user_filters") + assert not hasattr(admin_analytics, "_USER_SORTS") + assert not hasattr(query_service, "AnalyticsUserFilters") + assert not hasattr(query_service, "PaginatedUsers") + assert not hasattr(query_service.AnalyticsQueryService, "list_users") + assert "AnalyticsUserFilters" not in query_service.__all__ + assert "PaginatedUsers" not in query_service.__all__ + # The live users route is untouched: it still answers from the value stack. + assert "_value_user_filters" in inspect.getsource(admin_analytics) diff --git a/dashboard/backend/tests/test_agent_source_rows.py b/dashboard/backend/tests/test_agent_source_rows.py new file mode 100644 index 00000000..53f9ad3c --- /dev/null +++ b/dashboard/backend/tests/test_agent_source_rows.py @@ -0,0 +1,32 @@ +"""Agent ownership rows for the analytics backfill come from the agent store.""" + +from __future__ import annotations + +from pathlib import Path + +from dashboard.backend.domain.agents.repository import AgentStore + + +def test_list_agent_source_rows_returns_ownership_in_creation_order(tmp_path): + store = AgentStore(tmp_path / "agents.db") + owned = store.create_agent(name="Owned", owner_user_id=1, session_id="session-1") + guest = store.create_agent(name="Guest", owner_browser_session="browser-1") + # created_at has second resolution and the SQL tie-breaks on the random + # agent_id, so pin the ordering the store is being asked to guarantee. + import sqlite3 + with sqlite3.connect(Path(tmp_path) / "agents.db") as conn: + conn.execute( + "UPDATE external_agents SET created_at = ? WHERE agent_id = ?", + ("2026-09-01T00:00:00", owned["agent_id"]), + ) + conn.execute( + "UPDATE external_agents SET created_at = ? WHERE agent_id = ?", + ("2026-09-02T00:00:00", guest["agent_id"]), + ) + + rows = store.list_agent_source_rows() + + assert [row["agent_id"] for row in rows] == [owned["agent_id"], guest["agent_id"]] + assert set(rows[0]) == {"agent_id", "session_id", "owner_user_id", "created_at"} + assert rows[0]["owner_user_id"] == 1 + assert rows[1]["owner_user_id"] is None diff --git a/dashboard/backend/tests/test_analytics_maintenance.py b/dashboard/backend/tests/test_analytics_maintenance.py index efc5df10..2391d119 100644 --- a/dashboard/backend/tests/test_analytics_maintenance.py +++ b/dashboard/backend/tests/test_analytics_maintenance.py @@ -14,119 +14,56 @@ NOW = datetime(2026, 8, 26, 12, 0, tzinfo=timezone.utc) -def test_maintenance_rebuilds_one_day_and_bounds_snapshot_repairs(): - maintenance.reset_maintenance_guard_for_tests() - rollup_calls = [] +def test_maintenance_repairs_bounded_batches_and_nothing_else(): repair_limits = [] value_repair_limits = [] - backfill_limits = [] repair_order = [] - repair_results = iter([25, 0]) - value_repair_results = iter([12, 0]) - - def rebuild(day, **kwargs): - rollup_calls.append((day, kwargs["now"])) def repair(**kwargs): repair_order.append("legacy") repair_limits.append(kwargs["limit"]) - return next(repair_results) + return 25 def repair_values(**kwargs): repair_order.append("value") value_repair_limits.append(kwargs["limit"]) - return next(value_repair_results) - - def backfill(**kwargs): - repair_order.append("backfill") - backfill_limits.append(kwargs["batch_size"]) - return SimpleNamespace( - processed_users=4, - written_rows=224, - complete=len(backfill_limits) > 1, - ) - - first = maintenance.run_analytics_maintenance( - now=NOW, - snapshot_limit=250, - rebuild_rollup=rebuild, - repair_snapshots=repair, - repair_value_snapshots=repair_values, - backfill_lifecycle=backfill, - ) - second = maintenance.run_analytics_maintenance( + return 12 + + report = maintenance.run_analytics_maintenance( now=NOW, snapshot_limit=250, - rebuild_rollup=rebuild, repair_snapshots=repair, repair_value_snapshots=repair_values, - backfill_lifecycle=backfill, ) - assert first.rollup_days == (date(2026, 8, 25),) - assert second.rollup_days == first.rollup_days - assert first.rollup_rebuilt is True - assert second.rollup_rebuilt is False - assert first.repaired_snapshots == 25 - assert second.repaired_snapshots == 0 - assert first.repaired_value_snapshots == 12 - assert second.repaired_value_snapshots == 0 - assert first.backfilled_lifecycle_users == 4 - assert first.backfilled_lifecycle_rows == 224 - assert first.lifecycle_backfill_complete is False - assert second.lifecycle_backfill_complete is True - assert repair_limits == [100, 100] - assert value_repair_limits == [100, 100] - assert backfill_limits == [100, 100] - assert repair_order == [ - "value", - "legacy", - "backfill", - "value", - "legacy", - "backfill", - ] - assert len(rollup_calls) == 1 - - -def test_maintenance_isolates_rollup_and_snapshot_failures(): - maintenance.reset_maintenance_guard_for_tests() - - def fail_rollup(*_args, **_kwargs): - raise RuntimeError("private database detail") + assert report.repaired_snapshots == 25 + assert report.repaired_value_snapshots == 12 + assert report.failures == 0 + assert repair_limits == [100] + assert value_repair_limits == [100] + assert repair_order == ["value", "legacy"] + +def test_maintenance_isolates_repair_failures(): def fail_repair(**_kwargs): raise RuntimeError("private user detail") def fail_value_repair(**_kwargs): raise RuntimeError("private value projection detail") - def fail_backfill(**_kwargs): - raise RuntimeError("private backfill detail") - report = maintenance.run_analytics_maintenance( now=NOW, - rebuild_rollup=fail_rollup, repair_snapshots=fail_repair, repair_value_snapshots=fail_value_repair, - backfill_lifecycle=fail_backfill, ) - assert report.rollup_rebuilt is False assert report.repaired_snapshots == 0 assert report.repaired_value_snapshots == 0 - assert report.lifecycle_backfill_failures == 1 - assert report.failures == 4 - - -def test_app_registers_analytics_maintenance_through_reaper(): - app_file = Path(__file__).resolve().parents[1] / "app.py" - source = app_file.read_text(encoding="utf-8") - - assert "register_reaper_sweep(run_analytics_maintenance)" in source - assert "analytics.maintenance_registration_failed" in source + assert report.failures == 2 -def test_startup_disables_synchronous_snapshot_projection(): - source = inspect.getsource(app_module.startup_event) - assert source.count("disable_synchronous_projection()") == 1 +def test_maintenance_no_longer_owns_rollups_or_the_lifecycle_backfill(): + """Design SS6.9 step 1 / D23: the daily job owns rollup_day now, so the + reaper tick must not be able to write the same rollup rows.""" + parameters = inspect.signature(maintenance.run_analytics_maintenance).parameters + source = inspect.getsource(maintenance) diff --git a/dashboard/backend/tests/test_architecture_boundaries.py b/dashboard/backend/tests/test_architecture_boundaries.py index 9f228e41..d73e23b7 100644 --- a/dashboard/backend/tests/test_architecture_boundaries.py +++ b/dashboard/backend/tests/test_architecture_boundaries.py @@ -550,3 +550,169 @@ def test_portfolio_manager_construction_always_declares_settlement(): "PortfolioManager built without an explicit t_plus_one_enabled at: " + ", ".join(offenders) ) + + +# --------------------------------------------------------------------------- +# Event-log discipline (admin layer redesign design doc SS6.11, rules 1-7) +# --------------------------------------------------------------------------- + +_ANALYTICS = _BACKEND / "domain" / "analytics" +_EVENT_READ_NAMES = {"list_events", "list_user_events", "list_metric_events"} +# Files that may call a raw-event reader, each with the reason. Rules 3-5: the +# paginated timeline and the overview's one-day scan are the only permitted raw +# reads; everything else reads rollups or user_daily_facts. Every entry must +# still contain a hit (the stale check below), so PR B removes the last two as +# it deletes the files and narrows the others as it moves the reads. +_EVENT_READ_ALLOWLIST = { + "dashboard/backend/domain/analytics/rollups.py": ( + "defines AnalyticsRollupStore.list_events; rollup_day / rollup_current_day " + "are the day rollups and the overview's current-day scan (rule 5), " + "narrowed to one UTC day in PR B" + ), + "dashboard/backend/domain/analytics/query_service.py": ( + "the paginated timeline (rule 3) and the overview's current-day read " + "(rule 5); PR B narrows the overview read and gives sessions a 30-day window" + ), + "dashboard/backend/domain/analytics/value_queries.py": ( + "the retention grid's per-activation-week scan and the one-user profile " + "scan; PR B moves both onto user_daily_facts / user_activity" + ), + "dashboard/backend/domain/analytics/states.py": ( + "legacy five-state calculator: full-history per-user reads. DELETED IN " + "PR B (design SS13 row B); allowlisted, not fixed, because PR A creates " + "and PR B drops" + ), + "dashboard/backend/domain/analytics/lifecycle_backfill.py": ( + "historical eight-week reconstruction, per-user history reads. DELETED " + "IN PR B; the copy in facts_migration.py carries the history now" + ), +} +_OWN_CONNECTION_RECEIVERS = {"self", "self.analytics_base", "self.base_store"} +_OWN_DIALECT_RECEIVERS = { + "base_store", + "self.base_store", + "analytics_base", + "self.analytics_base", + "resolved_analytics_base", +} +# Rule 7 tolerates nothing after PR A. A future entry needs a file path and a +# reason, exactly like _EVENT_READ_ALLOWLIST -- and a design-doc amendment, +# because the rule it relaxes is SS6.14's first row. +_CROSS_DOMAIN_READ_ALLOWLIST: dict[str, str] = {} +_PER_USER_QUERY_PREFIXES = ( + "aggregate_", "list_", "upsert_", "record_", "sum_", "claim_", + "complete_", "release_", "get_", "append_", "copy_", "seed_", +) + + +def _analytics_sources(): + for path in sorted(_ANALYTICS.glob("*.py")): + yield path.relative_to(_REPO_ROOT).as_posix(), ast.parse( + path.read_text(encoding="utf-8") + ) + + +def _dotted(node) -> str | None: + if isinstance(node, ast.Name): + return node.id + if isinstance(node, ast.Attribute): + base = _dotted(node.value) + return f"{base}.{node.attr}" if base else None + return None + + +def test_event_log_rule_1_and_2_constants_hold(): + from dashboard.backend.domain.analytics import models, retention + + assert models.MAX_PROPERTIES_BYTES == 1024 + assert isinstance(models.ALLOWED_SERVER_EVENT_NAMES, (set, frozenset)) + assert retention.RAW_EVENT_RETENTION_DAYS == 180 + mutators = [] + for relative, tree in _analytics_sources(): + for node in ast.walk(tree): + if ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr in {"add", "update", "discard", "remove"} + and isinstance(node.func.value, ast.Name) + and node.func.value.id.startswith("ALLOWED_") + ): + mutators.append((relative, node.lineno)) + assert mutators == [], f"the event allowlist is mutated at runtime: {mutators}" + + +def test_raw_event_reads_stay_inside_the_allowlist(): + """No new caller may read the event log (rules 3, 4, 5). + + On 2026-09-11 a maintenance sweep read every user's 180-day history + every fifteen minutes, exhausted a 5 GB monthly egress allowance, and + 500'd login for five hours. Nothing in the suite noticed, because every + behavioural assertion still passed. This is the assertion that would have. + """ + hits: dict[str, list[tuple[str, int]]] = {} + for relative, tree in _analytics_sources(): + for node in ast.walk(tree): + if ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr in _EVENT_READ_NAMES + ): + hits.setdefault(relative, []).append((node.func.attr, node.lineno)) + unapproved = {path: calls for path, calls in hits.items() if path not in _EVENT_READ_ALLOWLIST} + assert unapproved == {}, f"unapproved raw-event reads: {unapproved}" + stale = sorted(set(_EVENT_READ_ALLOWLIST) - set(hits)) + assert stale == [], ( + "allowlist entries with no raw-event read left in them -- delete the entry " + f"so the exemption cannot be inherited by the next reader: {stale}" + ) + + +def test_the_daily_job_never_loops_over_users(): + """A for-loop issuing a query per user is the shape that caused the outage (rule 6).""" + source = (_ANALYTICS / "daily_facts.py").read_text(encoding="utf-8") + offenders = [] + for node in ast.walk(ast.parse(source)): + if not isinstance(node, (ast.For, ast.AsyncFor)): + continue + # The loop's own iterable may be one set-based read; its body may not + # issue any. + for statement in node.body: + for inner in ast.walk(statement): + if ( + isinstance(inner, ast.Call) + and isinstance(inner.func, ast.Attribute) + and inner.func.attr.startswith(_PER_USER_QUERY_PREFIXES) + ): + offenders.append((inner.func.attr, inner.lineno)) + assert offenders == [], f"store calls inside a loop in daily_facts.py: {offenders}" + + +def test_the_analytics_package_opens_only_its_own_connection(): + """Rule 7 (design SS6.11, SS6.14 row 1): `_get_connection()` is called only + on the analytics domain's own store, and no analytics module sniffs another + store's dialect. Other domains' tables are read through public methods on + the store that owns them -- CreditsStore.aggregate_ledger_for_day, not SQL + against credit_ledger_entries from inside this package.""" + offenders = [] + for relative, tree in _analytics_sources(): + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + if isinstance(node.func, ast.Attribute) and node.func.attr == "_get_connection": + receiver = _dotted(node.func.value) + if receiver not in _OWN_CONNECTION_RECEIVERS: + offenders.append((relative, node.lineno, f"{receiver}._get_connection()")) + if ( + isinstance(node.func, ast.Name) + and node.func.id == "hasattr" + and len(node.args) == 2 + and isinstance(node.args[1], ast.Constant) + and node.args[1].value == "database_url" + ): + receiver = _dotted(node.args[0]) + if receiver not in _OWN_DIALECT_RECEIVERS: + offenders.append((relative, node.lineno, f'hasattr({receiver}, "database_url")')) + unlisted = [entry for entry in offenders if entry[0] not in _CROSS_DOMAIN_READ_ALLOWLIST] + assert unlisted == [], f"cross-domain connection or dialect sniff in domain/analytics: {unlisted}" + stale = sorted(set(_CROSS_DOMAIN_READ_ALLOWLIST) - {entry[0] for entry in offenders}) + assert stale == [], f"stale rule-7 allowlist entries: {stale}" diff --git a/dashboard/backend/tests/test_backtest_owner_attribution.py b/dashboard/backend/tests/test_backtest_owner_attribution.py new file mode 100644 index 00000000..187023df --- /dev/null +++ b/dashboard/backend/tests/test_backtest_owner_attribution.py @@ -0,0 +1,77 @@ +"""agent_runs rows carry the authenticated caller who started them. + +Operator-funded model cost is in ``est_cost_usd`` on this row and nowhere +else, so without an owner column there is no per-user cost at all. +""" + +from __future__ import annotations + +import inspect + +from dashboard.backend.database import BacktestDatabase + + +def test_insert_run_accepts_an_owner(tmp_path): + db = BacktestDatabase(tmp_path / "runs.db") + db.insert_run( + run_id="run-owned", + session_id="session-1", + agent_name="Agent", + mode="backtest", + start_date="2026-09-01", + end_date="2026-09-02", + initial_equity=100000.0, + est_cost_usd=1.25, + owner_user_id=7, + ) + + conn = db._get_connection() + try: + row = conn.execute( + "SELECT owner_user_id, est_cost_usd FROM agent_runs WHERE run_id = ?", + ("run-owned",), + ).fetchone() + finally: + conn.close() + + assert row["owner_user_id"] == 7 + assert row["est_cost_usd"] == 1.25 + + +def test_owner_is_optional_and_defaults_to_null(tmp_path): + """A scheduled leaderboard deploy has no caller and must still insert.""" + db = BacktestDatabase(tmp_path / "runs.db") + db.insert_run( + run_id="run-unowned", + session_id="session-1", + agent_name="Agent", + mode="backtest", + start_date="2026-09-01", + end_date="2026-09-02", + initial_equity=100000.0, + ) + + conn = db._get_connection() + try: + row = conn.execute( + "SELECT owner_user_id FROM agent_runs WHERE run_id = ?", + ("run-unowned",), + ).fetchone() + finally: + conn.close() + + assert row["owner_user_id"] is None + + +def test_the_owner_reaches_the_subprocess_and_the_engine(): + """The four hops from the route to the row, pinned by source shape.""" + from dashboard.backend.api.routers import backtests + from dashboard.backend.domain.backtesting import engine + + assert "owner_user_id" in inspect.signature( + backtests.run_backtest_background + ).parameters + assert "--owner-user-id" in inspect.getsource(backtests.run_backtest_background) + assert "owner_user_id" in inspect.signature( + engine.HourlyBacktester.__init__ + ).parameters diff --git a/dashboard/backend/tests/test_credits_ledger_aggregates.py b/dashboard/backend/tests/test_credits_ledger_aggregates.py new file mode 100644 index 00000000..0554ed68 --- /dev/null +++ b/dashboard/backend/tests/test_credits_ledger_aggregates.py @@ -0,0 +1,230 @@ +"""The credits domain answers the analytics domain's ledger questions itself. + +Design SS6.14: a domain reads another domain's *service*, never its tables. +Until PR A, ``value_repository.py`` opened ``credits_store._get_connection()`` +and ran its own SQL against the ledger; these four methods are that SQL, moved +onto the store that owns the tables. +""" + +from __future__ import annotations + +import sqlite3 +from datetime import date, datetime, timedelta, timezone + +from dashboard.backend.domain.credits.repository import CreditsStore +from dashboard.backend.users import UserStore + + +NOW = datetime(2026, 9, 12, 12, 0, tzinfo=timezone.utc) +DAY = date(2026, 9, 11) + + +def _iso(value: datetime) -> str: + return value.astimezone(timezone.utc).replace(microsecond=0).isoformat() + + +def _ledger_row(user_id, entry_type, amount_micro, created_at, *, key): + """One row satisfying credit_ledger_entries' CHECK for its entry_type.""" + base = { + "user_id": user_id, + "entry_type": entry_type, + "amount_micro": amount_micro, + "operation_key": f"op:{key}", + "operation_id": f"operation-{key}", + "idempotency_key": f"idem:{key}", + "source": "test", + "reason": "ledger aggregate test", + "created_at": _iso(created_at), + "payment_order_id": None, + "refund_request_id": None, + "stripe_event_id": None, + "request_digest": None, + "actor_user_id": None, + "reference_type": None, + "reference_id": None, + } + if entry_type == "purchase": + base.update( + bucket="purchased", payment_order_id=f"order-{key}", + stripe_event_id=f"evt-{key}", + ) + elif entry_type == "refund": + base.update( + bucket="purchased", payment_order_id=f"order-{key}", + refund_request_id=f"refund-{key}", stripe_event_id=f"evt-{key}", + ) + else: # admin_grant_assign / admin_grant_reclaim + base.update( + bucket="grant", request_digest="digest", actor_user_id=1, + reference_type="grant_pool", reference_id="default", + ) + return base + + +def _insert(path, table, rows): + # A bare connection: FKs are off by default in sqlite3, so the payment + # order / refund / stripe event FKs need no parent rows here. The CHECK + # constraints still apply, which is why _ledger_row fills every column the + # constraint for its entry_type inspects. + with sqlite3.connect(path) as conn: + for row in rows: + columns = ", ".join(row) + marks = ", ".join("?" for _ in row) + conn.execute( + f"INSERT INTO {table} ({columns}) VALUES ({marks})", + tuple(row.values()), + ) + + +def _usage_row(user_id, amount_micro, created_at, *, key, bucket="grant"): + return { + "user_id": user_id, + "reservation_id": f"res-{key}", + "run_id": f"run-{key}", + "call_index": 0, + "bucket": bucket, + "amount_micro": amount_micro, + "operation_key": f"settle:{key}", + "evidence_json": "{}", + "created_at": _iso(created_at), + } + + +def _store(tmp_path): + path = tmp_path / "credits.db" + users = UserStore(db_path=path) + alice = int(users.create_user("alice@example.test", "Alice", "SecurePass1!")["id"]) + bob = int(users.create_user("bob@example.test", "Bob", "SecurePass1!")["id"]) + store = CreditsStore(path) + day_start = datetime.combine(DAY, datetime.min.time(), tzinfo=timezone.utc) + _insert( + path, + "credit_ledger_entries", + [ + _ledger_row(alice, "purchase", 10_000_000, NOW - timedelta(days=40), key="a1"), + _ledger_row(alice, "purchase", 3_000_000, day_start + timedelta(hours=1), key="a2"), + _ledger_row(alice, "refund", -6_000_000, day_start + timedelta(hours=2), key="a3"), + _ledger_row(alice, "admin_grant_assign", 1_500_000, day_start + timedelta(hours=3), key="a4"), + _ledger_row(alice, "admin_grant_reclaim", -500_000, day_start + timedelta(hours=4), key="a5"), + _ledger_row(bob, "purchase", 2_000_000, NOW - timedelta(days=3), key="b1"), + ], + ) + _insert( + path, + "credit_llm_usage_entries", + [ + _usage_row(alice, -400_000, day_start + timedelta(hours=5), key="a6"), + _usage_row(alice, -100_000, day_start + timedelta(hours=6), key="a7", bucket="purchased"), + _usage_row(alice, -50_000, day_start + timedelta(days=1, hours=1), key="a8"), + _usage_row(bob, -25_000, day_start + timedelta(hours=9), key="b2"), + ], + ) + return store, alice, bob, day_start + + +def test_aggregate_commercial_ledger_matches_the_reader_it_replaces(tmp_path): + store, alice, bob, day_start = _store(tmp_path) + + totals = store.aggregate_commercial_ledger( + [alice, bob], start=day_start, end=day_start + timedelta(days=1) + ) + + assert totals[alice] == { + "lifetime_purchased_micro": 13_000_000, + "lifetime_refunded_micro": 6_000_000, + "purchased_micro": 3_000_000, + "refunded_micro": 6_000_000, + "grant_activity_micro": 2_000_000, + "consumed_micro": 500_000, + } + # Bob bought outside the window and consumed inside it. + assert totals[bob]["lifetime_purchased_micro"] == 2_000_000 + assert totals[bob]["purchased_micro"] == 0 + assert totals[bob]["consumed_micro"] == 25_000 + assert store.aggregate_commercial_ledger([], start=day_start, end=NOW) == {} + + +def test_list_credit_activity_timestamps_returns_purchases_and_consumption_only(tmp_path): + store, alice, _bob, day_start = _store(tmp_path) + + stamps = store.list_credit_activity_timestamps( + [alice], start=day_start, end=day_start + timedelta(days=1) + ) + + assert sorted(stamps[alice]) == [ + _iso(day_start + timedelta(hours=1)), # the purchase + _iso(day_start + timedelta(hours=5)), # consumption + _iso(day_start + timedelta(hours=6)), # consumption + ] + + +def test_aggregate_ledger_for_day_takes_no_user_id(tmp_path): + store, alice, bob, day_start = _store(tmp_path) + + totals = store.aggregate_ledger_for_day(DAY) + + assert totals[alice]["own_spend_micro"] == 500_000 + assert totals[alice]["lifetime_net_purchased_micro"] == 7_000_000 + assert totals[alice]["last_activity_at"] == _iso(day_start + timedelta(hours=6)) + assert totals[bob]["own_spend_micro"] == 25_000 + assert totals[bob]["lifetime_net_purchased_micro"] == 2_000_000 + assert totals[bob]["last_activity_at"] == _iso(day_start + timedelta(hours=9)) + # The day after has only Alice's 50_000 consumption and no purchases. + next_day = store.aggregate_ledger_for_day(DAY + timedelta(days=1)) + assert next_day[alice]["own_spend_micro"] == 50_000 + assert next_day[alice]["lifetime_net_purchased_micro"] == 7_000_000 + + +def test_list_account_billing_states_agrees_with_the_single_user_reader(tmp_path): + store, alice, bob, _day_start = _store(tmp_path) + store.restrict_account(alice, reason="llm_overage") + store.ensure_account(bob) + + batched = store.list_account_billing_states([alice, bob]) + everyone = store.list_account_billing_states() + + for user_id in (alice, bob): + assert batched[user_id] == store.get_account_billing_state(user_id) + assert everyone == batched + assert batched[alice]["account_status"] == "restricted" + assert batched[alice]["restriction_reason"] == "llm_overage" + assert store.list_account_billing_states([]) == {} + + +def test_backfill_source_rows_come_from_the_credits_store(tmp_path): + store, alice, _bob, day_start = _store(tmp_path) + _insert( + tmp_path / "credits.db", + "credit_llm_reservations", + [ + { + "reservation_id": "res-a6", + "user_id": alice, + "run_id": "run-a6", + "call_index": 0, + "reserved_micro": 500_000, + "reserved_grant_micro": 500_000, + "reserved_purchased_micro": 0, + "status": "settled", + "operation_key": "reserve:a6", + "request_digest": "digest", + "created_at": _iso(day_start + timedelta(hours=4)), + "updated_at": _iso(day_start + timedelta(hours=5)), + } + ], + ) + + reservations = store.list_llm_reservation_rows() + usage = store.list_llm_usage_rows() + + assert [row["reservation_id"] for row in reservations] == ["res-a6"] + assert set(reservations[0]) == { + "reservation_id", "user_id", "run_id", "call_index", + "reserved_grant_micro", "reserved_purchased_micro", "status", + "created_at", "updated_at", + } + assert [row["reservation_id"] for row in usage] == ["res-a6", "res-a7", "res-b2", "res-a8"] + assert set(usage[0]) == { + "id", "user_id", "reservation_id", "run_id", "call_index", "bucket", + "amount_micro", "created_at", + } diff --git a/dashboard/backend/tests/test_event_loop_threadpool.py b/dashboard/backend/tests/test_event_loop_threadpool.py index 1da5af15..8d55c8d2 100644 --- a/dashboard/backend/tests/test_event_loop_threadpool.py +++ b/dashboard/backend/tests/test_event_loop_threadpool.py @@ -35,6 +35,7 @@ "dashboard.backend.api.routers.backtests", "dashboard.backend.api.routers.admin", "dashboard.backend.api.routers.admin_users", + "dashboard.backend.api.routers.admin_analytics", "dashboard.backend.api.routers.discord", "dashboard.backend.api.routers.external_backtest", "dashboard.backend.api.v2.leaderboard", diff --git a/dashboard/backend/tests/test_run_history_aggregates.py b/dashboard/backend/tests/test_run_history_aggregates.py new file mode 100644 index 00000000..afbcc9ee --- /dev/null +++ b/dashboard/backend/tests/test_run_history_aggregates.py @@ -0,0 +1,53 @@ +"""Operator-funded model cost per owner comes from the run-history store.""" + +from __future__ import annotations + +from datetime import date + +from dashboard.backend.database import BacktestDatabase + + +def _seed(db, run_id, *, owner, cost, day): + db.insert_run( + run_id=run_id, + session_id="session-1", + agent_name="Agent", + mode="backtest", + start_date="2026-09-01", + end_date="2026-09-02", + initial_equity=100000.0, + est_cost_usd=cost, + owner_user_id=owner, + ) + conn = db._get_connection() + try: + # created_at/updated_at are CURRENT_TIMESTAMP text ("YYYY-MM-DD HH:MM:SS"); + # pin the row onto the day under test. + conn.execute( + "UPDATE agent_runs SET updated_at = ? WHERE run_id = ?", + (f"{day} 15:30:00", run_id), + ) + conn.commit() + finally: + conn.close() + + +def test_operator_cost_is_grouped_by_owner_for_one_day(tmp_path): + db = BacktestDatabase(tmp_path / "runs.db") + _seed(db, "a", owner=7, cost=1.25, day="2026-09-11") + _seed(db, "b", owner=7, cost=0.5, day="2026-09-11") + _seed(db, "c", owner=9, cost=0.1, day="2026-09-11") + _seed(db, "d", owner=7, cost=3.0, day="2026-09-10") # another day + _seed(db, "e", owner=None, cost=2.0, day="2026-09-11") # unattributed + + totals = db.aggregate_operator_cost_for_day(date(2026, 9, 11)) + + assert totals == {7: 1_750_000, 9: 100_000} + assert db.aggregate_operator_cost_for_day(date(2026, 9, 12)) == {} + + +def test_a_negative_stored_cost_cannot_violate_the_fact_check(tmp_path): + db = BacktestDatabase(tmp_path / "runs.db") + _seed(db, "neg", owner=3, cost=-0.75, day="2026-09-11") + + assert db.aggregate_operator_cost_for_day(date(2026, 9, 11)) == {3: 0} diff --git a/dashboard/backend/tests/test_run_lifecycle_unification.py b/dashboard/backend/tests/test_run_lifecycle_unification.py index c3a75689..2e427be3 100644 --- a/dashboard/backend/tests/test_run_lifecycle_unification.py +++ b/dashboard/backend/tests/test_run_lifecycle_unification.py @@ -183,10 +183,13 @@ def test_reap_runs_invokes_registered_sweeps(monkeypatch): assert calls == ["swept"] -def test_startup_registers_analytics_retention_sweep_once(): +def test_startup_leaves_analytics_retention_to_the_daily_job(): + """Admin layer redesign PR A: the retention coordinator runs from the + daily-facts worker (domain/analytics/daily_facts.py), not the reaper.""" source = inspect.getsource(app_module.startup_event) call = "register_reaper_sweep(analytics_retention_coordinator.run_if_due)" - assert source.count(call) == 1 + assert source.count(call) == 0 + assert source.count("start_daily_facts_worker()") == 1 def test_reap_runs_survives_a_raising_analytics_retention_sweep(monkeypatch): diff --git a/dashboard/backend/tests/test_store_twin_parity.py b/dashboard/backend/tests/test_store_twin_parity.py index 4b020cae..5183763a 100644 --- a/dashboard/backend/tests/test_store_twin_parity.py +++ b/dashboard/backend/tests/test_store_twin_parity.py @@ -249,29 +249,13 @@ def test_every_postgres_twin_module_is_registered(): ) _DIALECT_BRANCH_ALLOWLIST: dict[str, str] = { - "dashboard/backend/domain/analytics/value_repository.py": ( - "list_commercial_values and list_credit_activity branch on the " - "injected credits_base's own dialect (hasattr(self.credits_base, " - "'database_url')), not on ValueAnalyticsStore's -- a caller can (and " - "in tests does) pair either analytics_base with either credits_base. " - "PostgresValueAnalyticsStore carries the identical branch for the " - "same reason (see value_repository_postgres.py's module docstring); " - "both are correct, not a missing extraction. A third hit lives in " - "build_value_analytics_store's own selection branch " - "(hasattr(resolved_analytics_base, 'database_url')): that factory's " - "whole job is to pick SQLite vs. Postgres, and it must do so by " - "reading the resolved analytics_base object rather than an " - "os.getenv(...) check, because it receives an already-constructed " - "base and must follow that object's dialect -- reading the " - "environment instead would hand a caller who injects a Postgres " - "base the SQLite twin whenever the var happens to be unset. That " - "makes this a dialect selection, the same idiom every other " - "store's _build_*_store() performs on os.getenv(...) directly, not " - "a missing extraction. A fourth hit is ValueAnalyticsStore.__init__'s " - "guard (PR #498 review), which refuses a Postgres analytics_base " - "rather than silently emitting `?` placeholders against it; it reads " - "the same attribute the factory dispatches on, on purpose, so guard " - "and factory cannot disagree about which twin a base belongs to." + "dashboard/backend/domain/analytics/value_repository.py": ( + "build_value_analytics_store() dispatches on hasattr(resolved_analytics_base, " + "'database_url') to pick the twin, mirroring repository.py's " + "_build_analytics_store(). That is the factory's job, not an inline " + "dialect branch: every method on both twins has exactly one code path. " + "The two credit readers that used to branch here moved onto " + "CreditsStore / PostgresCreditsStore in admin layer redesign PR A." ), "dashboard/backend/domain/analytics/value_repository_postgres.py": ( "The Postgres twin's two hits are the same credits_base branch its " @@ -322,16 +306,6 @@ def test_every_postgres_twin_module_is_registered(): "job. Not a store of its own; admin layer redesign PR A's daily job " "and migration (design doc §6.9) replace this reconstruction path." ), - "dashboard/backend/domain/analytics/backfill.py": ( - "_query_all dialect-branches over injected credits/agent stores for " - "the authoritative-history backfill; admin layer redesign PR A " - "moves those two ledger and run reads onto CreditsStore/" - "PostgresCreditsStore and the run-history store (design doc §12 " - "item 1), which removes this branch. _existing_source_event_ids " - "separately dialect-branches over the already-twinned analytics_store " - "to deduplicate against analytics_events, the analytics domain's own " - "table -- a different case, out of scope for PR T." - ), } diff --git a/dashboard/scripts/backtest_hourly_agent.py b/dashboard/scripts/backtest_hourly_agent.py index 917fe7bd..5c2cdf3c 100644 --- a/dashboard/scripts/backtest_hourly_agent.py +++ b/dashboard/scripts/backtest_hourly_agent.py @@ -269,6 +269,13 @@ def main(): help="Read one signed, secret-free execution handoff from stdin", ) parser.add_argument("--run-id", default=None, help="Preset run id (used for live progress + DB row)") + + parser.add_argument( + "--owner-user-id", + type=int, + default=None, + help="Authenticated caller who started this run (analytics attribution)", + ) parser.add_argument("--progress-file", default=None, help="Path to write incremental equity snapshots for live dashboard charting") parser.add_argument( "--launched-at", @@ -508,6 +515,7 @@ def main(): model=args.model, pipeline=pipeline, live_run_id=args.run_id, + owner_user_id=args.owner_user_id, progress_file=args.progress_file, data_source=args.data_source, initial_capital=capital,