feat(b2b_learner_records): serve activity from the learner-records MVs

docs/b2b-learner-records-design.md

@@ -58,9 +58,14 @@ The second is more work but produces a correct `user_fk`-keyed
 `(learner × courserun × day)` fact that the aggregate MVs would also benefit
 from. Recommend the second.
 
-Until one lands, `last_active_on`, `days_active` and the per-run activity
-counters ship as `null`. The spec marks them `x-data-readiness: pending-model`
-so a partner sizing the integration knows which columns to expect empty.
+Resolved by the second: `afact_learner_courserun_daily_activity`
+(ol-data-platform#2672) feeds `last_active_on`, `days_active` and the per-run
+counters in both learner-records MVs (ol-data-platform#2693). The counters sum
+the fact's per-day distinct counts, so a block used on two days counts twice,
+as the `b2b_dashboard` totals do. Activity does not move `record_updated_on`: a
+day's activity first appears at a refresh after that day began, so a cursor
+taken from it would already sort below the `updated_since` a partner passes.
+Partners pick activity up from a full reload.
 
 **3. `organization_key` is unreliable for activity attribution.** In
 `organization_administration_report` it is

docs/b2b-learner-records-onepager.md

@@ -232,8 +232,10 @@ Options: an end-date claim on the client, or a warehouse check against
 Degrading, not blocking — the service ships without these and fills in as they
 land: the learner-consent field, without which every record reads
 `outcomes_shared: false`; a per-learner activity model, without which "last
-active" and the engagement counters are null; and learner removal on
-`mv_b2b_learner`, without which `is_current` is always true.
+active" and the engagement counters are null (wired in tenant-side, gated on
+ol-data-platform#2693 merging and the MVs rebuilding with the new columns);
+and learner removal on `mv_b2b_learner`, without which `is_current` is always
+true.
 
 Blocking the first partner: the per-contract Keycloak client template and a
 bearer-only gateway route. No partner can authenticate without both.

docs/openapi/b2b-learner-records-v1.yaml

@@ -33,8 +33,9 @@ info:
       `outcomes_shared: false` and outcome fields nulled, a deactivated
       enrollment with `enrollment_is_active: false`, and a learner who left the
       organization with `is_current: false`. A client that upserts drops what
-      it held. Activity changes don't arrive on sync; reload in full
-      periodically.
+      it held. Activity (`last_active_on`, `days_active`, the counters, and
+      the `in_progress` status and count they drive) does not move
+      `updated_since`; a full reload picks it up.
     * **Paging is stable within one refresh.** Every collection has a unique
       order, but a refresh between two pages can shift rows across the offset.
       If `as_of` changes between pages, restart from offset 0.
@@ -593,12 +594,21 @@ components:
         last_active_on:
           type: [string, 'null']
           format: date
-          description: Most recent day with recorded course activity. A date, not a timestamp — activity is aggregated per day. Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: >-
+            Most recent day with tracked course activity (video play, problem
+            check, navigation, discussion or chatbot submit) across the
+            enrollments the request covers: active ones only, unless
+            `include_inactive=true`. The course platform's local day, not a
+            UTC day. Null with no activity.
+          x-data-readiness: available
         courses_in_progress:
           type: [integer, 'null']
           minimum: 0
-          x-data-readiness: pending-model
+          description: >-
+            Distinct course runs whose `completion_status` is `in_progress`:
+            not passed or certified, with a nonzero grade or any tracked
+            activity. Active enrollments only, unless `include_inactive=true`.
+          x-data-readiness: derived
         courses_passed:
           type: [integer, 'null']
           minimum: 0
@@ -717,7 +727,10 @@ components:
           oneOf:
             - $ref: '#/components/schemas/CompletionStatus'
             - type: 'null'
-          description: Single derived answer per row. Null when outcomes are withheld.
+          description: >-
+            Single derived answer per row. Null when outcomes are withheld.
+            `in_progress` means not passed or certified, with a nonzero grade
+            or any tracked activity in the run; `not_started` means neither.
           x-data-readiness: derived
         is_passing:
           type: [boolean, 'null']
@@ -747,28 +760,36 @@ components:
         last_active_on:
           type: [string, 'null']
           format: date
-          description: Most recent day with recorded activity in this course run. Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: >-
+            Most recent day with tracked activity in this course run. The
+            course platform's local day, not a UTC day. Null with no activity.
+          x-data-readiness: available
         days_active:
           type: [integer, 'null']
           minimum: 0
-          description: Distinct days with recorded activity in this course run. Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: Distinct days with tracked activity in this course run.
+          x-data-readiness: available
         videos_watched:
           type: [integer, 'null']
           minimum: 0
-          description: Distinct video blocks played. Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: >-
+            Video blocks played, counted once per day: a block played on two
+            days counts twice.
+          x-data-readiness: available
         problems_attempted:
           type: [integer, 'null']
           minimum: 0
-          description: Distinct problem blocks attempted. Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: >-
+            Problem blocks checked, counted once per day: a block attempted on
+            two days counts twice. Viewing an answer is not an attempt.
+          x-data-readiness: available
         chatbot_interactions:
           type: [integer, 'null']
           minimum: 0
-          description: Changes to it do not move `updated_since`; a full reload picks them up.
-          x-data-readiness: pending-model
+          description: >-
+            Chatbot submits in this course run, counting each (session, block)
+            once per day.
+          x-data-readiness: available
 
     CourseRun:
       type: object

openapi/specs/b2b_learner_records.yaml

@@ -447,10 +447,9 @@ components:
           anyOf:
           - $ref: '#/components/schemas/CompletionStatus'
           - type: 'null'
-          description: 'Single derived answer per row. Null when outcomes are withheld.
-            Until activity data lands, not_started and in_progress come from the grade
-            alone: in_progress means a nonzero grade, so a learner active in the run
-            with no graded work yet reads not_started.'
+          description: Single derived answer per row. Null when outcomes are withheld.
+            in_progress means not passed or certified, with a nonzero grade or any
+            tracked activity in the run; not_started means neither.
         is_passing:
           anyOf:
           - type: boolean
@@ -490,41 +489,40 @@ components:
             format: date
           - type: 'null'
           title: Last Active On
-          description: 'Most recent day with recorded activity in this course run.
-            Not yet populated upstream: null for every record until activity data
-            lands, whatever outcomes_shared says.'
+          description: Most recent day with tracked activity in this course run. A
+            date in the course platform's local day, not a UTC day. Null with no activity.
+            Changes to it do not move updated_since; a full reload picks them up.
         days_active:
           anyOf:
           - type: integer
           - type: 'null'
           title: Days Active
-          description: 'Distinct days with recorded activity in this course run. Not
-            yet populated upstream: null for every record until activity data lands,
-            whatever outcomes_shared says.'
+          description: Distinct days with tracked activity in this course run. Changes
+            to it do not move updated_since; a full reload picks them up.
         videos_watched:
           anyOf:
           - type: integer
           - type: 'null'
           title: Videos Watched
-          description: 'Distinct video blocks played. Not yet populated upstream:
-            null for every record until activity data lands, whatever outcomes_shared
-            says.'
+          description: 'Video blocks played, counted once per day: a block played
+            on two days counts twice. Changes to it do not move updated_since; a full
+            reload picks them up.'
         problems_attempted:
           anyOf:
           - type: integer
           - type: 'null'
           title: Problems Attempted
-          description: 'Distinct problem blocks attempted. Not yet populated upstream:
-            null for every record until activity data lands, whatever outcomes_shared
-            says.'
+          description: 'Problem blocks checked, counted once per day: a block attempted
+            on two days counts twice. Viewing an answer is not an attempt. Changes
+            to it do not move updated_since; a full reload picks them up.'
         chatbot_interactions:
           anyOf:
           - type: integer
           - type: 'null'
           title: Chatbot Interactions
-          description: 'Chatbot interactions recorded for this course run. Not yet
-            populated upstream: null for every record until activity data lands, whatever
-            outcomes_shared says.'
+          description: Chatbot submits in this course run, counting each (session,
+            block) once per day. Changes to it do not move updated_since; a full reload
+            picks them up.
       type: object
       required:
       - learner_id
@@ -638,18 +636,21 @@ components:
             format: date
           - type: 'null'
           title: Last Active On
-          description: 'Most recent day with recorded course activity. A date, not
-            a timestamp: activity is aggregated per day. Not yet populated upstream:
-            null for every record until activity data lands, whatever outcomes_shared
-            says.'
+          description: 'Most recent day with tracked course activity (video play,
+            problem check, navigation, discussion or chatbot submit) across the enrollments
+            the request covers: active ones only, unless include_inactive is set.
+            A date in the course platform''s local day, not a UTC day. Null with no
+            activity. Changes to it do not move updated_since; a full reload picks
+            them up.'
         courses_in_progress:
           anyOf:
           - type: integer
           - type: 'null'
           title: Courses In Progress
-          description: 'Distinct course runs the learner has started but not yet passed
-            or certified. Not yet populated upstream: null for every record until
-            activity data lands, whatever outcomes_shared says.'
+          description: 'Distinct course runs whose completion_status is in_progress:
+            not passed or certified, with a nonzero grade or any tracked activity.
+            Active enrollments only, unless include_inactive is set. Changes to it
+            do not move updated_since; a full reload picks them up.'
         courses_passed:
           anyOf:
           - type: integer

src/ol_analytics_api/tenants/b2b_learner_records/models.py

@@ -9,9 +9,9 @@
 NULL for them (see queries._outcomes_shared). This is a second check, so a
 query change that projects a raw outcome column still can't disclose it.
 
-Fields the warehouse doesn't carry yet (consent date, activity) are projected
-as NULL and ship null until upstream models land. They have no default, so
-the generated schema lists them as required and nullable, as the contract does.
+The consent date isn't in the warehouse yet, so it is projected as NULL and
+ships null until the upstream model lands. It has no default, so the generated
+schema lists it as required and nullable, as the contract does.
 """
 
 from __future__ import annotations
@@ -34,13 +34,10 @@ def _assume_utc(value: datetime.datetime) -> datetime.datetime:
 
 UtcDatetime = Annotated[datetime.datetime, AfterValidator(_assume_utc)]
 
-# Activity is a separate gap from consent. These fields are hardcoded NULL in
-# the queries until the activity fact is wired into the MVs, so they stay null
-# even for a record whose outcomes are shared.
-_ACTIVITY_PENDING = (
-    "Not yet populated upstream: null for every record until activity data lands, "
-    "whatever outcomes_shared says."
-)
+# record_updated_on carries no activity: a day's activity first appears at a
+# refresh after that day began, so a cursor derived from it would already sort
+# below the updated_since a partner passes and the change would never be sent.
+_ACTIVITY_NOT_SYNCED = "Changes to it do not move updated_since; a full reload picks them up."
 
 
 def _withhold_outcomes[RecordT: BaseModel](record: RecordT, fields: tuple[str, ...]) -> RecordT:
@@ -157,14 +154,18 @@ class Learner(BaseModel):
     )
     last_active_on: datetime.date | None = Field(
         description=(
-            "Most recent day with recorded course activity. A date, not a timestamp: "
-            f"activity is aggregated per day. {_ACTIVITY_PENDING}"
+            "Most recent day with tracked course activity (video play, problem check, "
+            "navigation, discussion or chatbot submit) across the enrollments the request "
+            "covers: active ones only, unless include_inactive is set. A date in the course "
+            "platform's local day, not a UTC day. Null with no activity. "
+            f"{_ACTIVITY_NOT_SYNCED}"
         )
     )
     courses_in_progress: int | None = Field(
         description=(
-            "Distinct course runs the learner has started but not yet passed or certified. "
-            f"{_ACTIVITY_PENDING}"
+            "Distinct course runs whose completion_status is in_progress: not passed or "
+            "certified, with a nonzero grade or any tracked activity. Active enrollments "
+            f"only, unless include_inactive is set. {_ACTIVITY_NOT_SYNCED}"
         )
     )
     courses_passed: int | None = Field(
@@ -238,10 +239,9 @@ class Enrollment(BaseModel):
     )
     completion_status: CompletionStatus | None = Field(
         description=(
-            "Single derived answer per row. Null when outcomes are withheld. Until activity "
-            "data lands, not_started and in_progress come from the grade alone: in_progress "
-            "means a nonzero grade, so a learner active in the run with no graded work yet "
-            "reads not_started."
+            "Single derived answer per row. Null when outcomes are withheld. in_progress "
+            "means not passed or certified, with a nonzero grade or any tracked activity in "
+            "the run; not_started means neither."
         )
     )
     is_passing: bool | None = Field(description="Null where no grade has been computed.")
@@ -261,20 +261,32 @@ class Enrollment(BaseModel):
     )
     last_active_on: datetime.date | None = Field(
         description=(
-            f"Most recent day with recorded activity in this course run. {_ACTIVITY_PENDING}"
+            "Most recent day with tracked activity in this course run. A date in the course "
+            f"platform's local day, not a UTC day. Null with no activity. {_ACTIVITY_NOT_SYNCED}"
         )
     )
     days_active: int | None = Field(
-        description=f"Distinct days with recorded activity in this course run. {_ACTIVITY_PENDING}"
+        description=(
+            f"Distinct days with tracked activity in this course run. {_ACTIVITY_NOT_SYNCED}"
+        )
     )
     videos_watched: int | None = Field(
-        description=f"Distinct video blocks played. {_ACTIVITY_PENDING}"
+        description=(
+            "Video blocks played, counted once per day: a block played on two days counts "
+            f"twice. {_ACTIVITY_NOT_SYNCED}"
+        )
     )
     problems_attempted: int | None = Field(
-        description=f"Distinct problem blocks attempted. {_ACTIVITY_PENDING}"
+        description=(
+            "Problem blocks checked, counted once per day: a block attempted on two days "
+            f"counts twice. Viewing an answer is not an attempt. {_ACTIVITY_NOT_SYNCED}"
+        )
     )
     chatbot_interactions: int | None = Field(
-        description=f"Chatbot interactions recorded for this course run. {_ACTIVITY_PENDING}"
+        description=(
+            "Chatbot submits in this course run, counting each (session, block) once per day. "
+            f"{_ACTIVITY_NOT_SYNCED}"
+        )
     )
 
     @model_validator(mode="after")