Optimize Constellation and Journal with incremental history

README.md

@@ -232,6 +232,8 @@ If saving shows a warning, follow it before assuming your change is active. If c
 
 Open timeline settings with the **sliders icon**. Choose Followers or Paid subscribers, filter All/Gained/Lost, or switch between **Constellation** (one star per recorded action) and **Journal** (every recorded event). These viewing choices do not change what you record.
 
+XFlog loads the newest activity first. Scroll toward the older edge or choose **Load older activity** to bring in another page. Both layouts show how many events and people are loaded alongside the totals for the selected filters and time range.
+
 Each star represents one recorded action. Repeated actions from the same person stay as separate entries; new activity does not replace an earlier action. In Constellation, scroll up for older history and down for newer history. The person nearest the center becomes highlighted as you move, and their details appear in the readout. You can also click any visible star or name to select and center that person; they stay selected until you move through history again. Click the selected person or their readout for details. Interaction details include a linked post excerpt of up to 280 characters when text is already saved; no extra X request is made. The side arrows move one recorded action at a time. Use the bottom sliders to move through history or change zoom, which starts at **80%**. You can also hold the middle mouse button and scroll up to zoom in, or down to zoom out. Journal uses normal vertical scrolling and hides the sliders.
 
 <details>
@@ -268,7 +270,7 @@ Use either **clock button** to change the shared time range for Constellation, J
 | **Verification** | Verified/unverified and blue, business or government badge types when supplied. |
 | **Profile location** | Similar location text, tidied for spacing, capitalization and clear aliases such as NYC → New York City and Bangalore → Bengaluru. |
 
-Zoom out to see every available group in Constellation. On smaller screens, scroll sideways or use the history slider to reach groups that do not fit. Journal uses collapsible sections. A person can appear in several groups: a later like or reply does not hide their earlier follow or unfollow. Group badges and legend counts show total actions within the selected range. Hover over a group badge or legend count for the number of unique people; screen readers announce both counts. Each matching action remains a separate star.
+Zoom out to see groups with loaded events in Constellation. The overview also reports how many groups have events still to load. On smaller screens, scroll sideways or use the history slider to reach groups that do not fit. Journal uses collapsible sections and shows a summary for groups whose events have yet to load. A person can appear in several groups. Group badges and legend counts show total actions within the selected range. Hover over a group badge or legend count for the number of unique people. Screen readers announce both counts. Each loaded action has its own star.
 
 Account size, age, verification and location use only details delivered with new events while enabled. Old missing details stay Unknown; turning a group off keeps saved history. Profile facts describe the recorded event, not a live profile check. When space allows, enabled Verification, Account size and Account age add facts to the selected-person readout. Known verified accounts get a checkmark beside their name in Constellation and the readout when Verification is enabled.
 

src/xflog/app.py

@@ -14,11 +14,12 @@
 from . import audience_groups
 from .config import ASSETS, Config
 from .errors import SetupError
-from .history.demo import demo_activity, demo_history
+from .history.demo import demo_activity, demo_history, demo_timeline
 from .history.demo import encounter_context as demo_encounter_context
 from .history.filters import valid_filters
 from .history.summary import SUBJECT_TYPES, SUMMARY_PERIODS, activity_window, all_readings, readings
-from .history.timeline import Timeline, decode_cursor
+from .history.timeline import Timeline
+from .history.timeline_cursor import InvalidTimelineRequest
 from .interactions import TYPES, catalog
 from .interactions.history import ThreadOptions
 from .profile_names import ProfileNames
@@ -37,6 +38,19 @@
 from .web.ui import mount_ui as mount_browser_ui
 
 
+def _history_scope(kind, subject, since, limit, lens):
+    if not valid_filters(kind, subject, 0):
+        raise HTTPException(422, "Invalid timeline filter.")
+    if lens != "none" and lens not in audience_groups.MODULES:
+        raise HTTPException(422, "Invalid audience group.")
+    if not 1 <= limit <= 50:
+        raise HTTPException(422, "History page size must be between 1 and 50.")
+    try:
+        return timestamp(since) if since else ""
+    except (ValueError, OverflowError):
+        raise HTTPException(422, "Invalid history range.") from None
+
+
 def create_app(config: Config | None = None, *, transport=None, mount_ui=True):
     config = config or Config.from_env()
     runtime = Runtime(config, transport=transport)
@@ -207,36 +221,33 @@ async def timeline_history(
         since: str = "",
         local_only: bool = False,
         limit: int = 25,
+        lens: str = "none",
+        after_revision: str = "",
+        selected: str = "",
+        encounter_revision: str = "",
     ):
-        try:
-            decode_cursor(before)
-        except ValueError:
-            raise HTTPException(422, "Invalid timeline cursor.") from None
-        if not valid_filters(kind, subject, 0):
-            raise HTTPException(422, "Invalid timeline filter.")
-        try:
-            since = timestamp(since) if since else ""
-        except (ValueError, OverflowError):
-            raise HTTPException(422, "Invalid history range.") from None
-        if not 1 <= limit <= 50:
-            raise HTTPException(422, "History page size must be between 1 and 50.")
-        account = await store.setting("account")
-        if not account:
-            return {"events": [], "next": None}
+        since = _history_scope(kind, subject, since, limit, lens)
         query = {
             "kind": kind,
             "subject": subject,
             "before": before,
             "since": since,
             "limit": limit,
+            "lens": lens,
+            "after_revision": after_revision,
+            "selected": selected,
+            "encounter_revision": encounter_revision,
         }
-        result = await timeline.history(account["id"], query)
-        if not local_only and await profile_names.ensure(result["events"]):
-            result = await timeline.history(account["id"], query)
-        groups = await store.setting("audience_groups", {})
-        result["events"] = await audience_groups.decorate_history(
-            result["events"], groups, timeline.encounter_context
-        )
+        try:
+            result = await timeline.current_history(query)
+            if before in {"", "0"} and not local_only and result["events"]:
+                fresh = [
+                    event for event in result["events"] if not event.get("encounter_correction")
+                ]
+                if fresh and await profile_names.ensure(fresh):
+                    result = await timeline.current_history(query)
+        except InvalidTimelineRequest as error:
+            raise HTTPException(422, str(error)) from None
         return result
 
     @app.get("/api/events")
@@ -256,11 +267,40 @@ async def history(kind: str = "all", subject: str = "followers", before: int = 0
         return result
 
     @app.get("/api/demo")
-    async def demo(kind: str = "all", subject: str = "followers", before: int = 0):
-        if not valid_filters(kind, subject, before):
-            raise HTTPException(422, "Invalid demo filter.")
-        result = demo_history({"kind": kind, "subject": subject, "before": before})
+    async def demo(
+        kind: str = "all",
+        subject: str = "followers",
+        before: str = "0",
+        since: str = "",
+        limit: int | None = None,
+        lens: str = "none",
+    ):
+        since = _history_scope(kind, subject, since, 25 if limit is None else limit, lens)
         groups = await store.setting("audience_groups", {})
+        if (
+            limit is not None
+            or since
+            or lens != "none"
+            or not before.isascii()
+            or not before.isdecimal()
+        ):
+            try:
+                return await demo_timeline(
+                    {
+                        "kind": kind,
+                        "subject": subject,
+                        "before": before,
+                        "since": since,
+                        "limit": 25 if limit is None else limit,
+                        "lens": lens,
+                    },
+                    groups,
+                )
+            except InvalidTimelineRequest as error:
+                raise HTTPException(422, str(error)) from None
+        if len(before) > 19:
+            raise HTTPException(422, "Invalid demo cursor.")
+        result = demo_history({"kind": kind, "subject": subject, "before": int(before)})
         result["events"] = await audience_groups.decorate_history(
             result["events"], groups, demo_encounter_context
         )