diff --git a/docs/admin-guide.md b/docs/admin-guide.md index 22e0394e..228ccfac 100644 --- a/docs/admin-guide.md +++ b/docs/admin-guide.md @@ -567,6 +567,20 @@ Setup seeds the vault with three things: the classification ontology (the topics The wiki maintains itself. A curator sidecar (`stack-memory-curator`) watches the vault: when new filings settle it regenerates the pages of the family members involved plus the home page (a couple of LLM calls, a few minutes after the burst), and once a night it rebuilds everything — topic pages, cross-references, the lot — while the GPU has nothing better to do. To bring the wiki up to date now instead of waiting, `./stack memory wiki update` regenerates what the latest filings touched and `./stack memory wiki update --all` runs the whole nightly rebuild; both return when the pages are written. `--member`, `--topic` and `--home` regenerate just those pages, and `[memory]` in `stack.toml` holds the knobs (see [Configuration](#configuration)). Edits happen in Forgejo (every wiki page links to its source), so the commit log doubles as the household's learning history. +**Who the family is.** Every person the family knows has a file of their own in the vault, `family/people/.md`: the full name, the other names the family uses, and the chat account if they have one. + +```markdown +--- +type: person +title: Bart Simpson +aliases: + - Bartholomew +account: bart +--- +``` + +Each `./stack up memory` adds a file for every account in `users.toml` that has none, so an update reaches an existing install with a restart of the memory stacklet. An existing file is never rewritten: it is the family's. Add someone without an account, such as a baby, by adding a file with `type`, `title` and any `aliases`, and add the names a person goes by to their file. A person with an account is a member of the household. The wiki gives every person a page under the names their file gives, and its home page lists the members. The archivist files a document under a person by any of their names ("Margaret" on a birth certificate is Maggie) and creates a `Person:` tag for a person without an account the first time a document names them. Transcription and the diary listen for every name in the files. A file the wiki cannot read is named in the curator's log (`docker logs stack-memory-curator`) and left out, and no page is removed until it reads again. A change to any file rebuilds the person pages within a few minutes, or right away with `./stack memory wiki update --home --members`. Edit the files in Forgejo, in Obsidian, or with `./stack memory write family/people/.md --by --from `. + **The family diary.** Everything posted in `#Memories` becomes a diary entry. Each night the curator transcribes the new recordings, has the AI read each entry (a title, a summary, the people, topics from the ontology, facts), writes one card per entry to the vault under `family/diary/entries/`, and rebuilds the diary pages in the wiki from those cards. At 07:30 local time the archivist posts each new card as a quiet notice in the thread under its memory; the first time, one notice in the room says how to correct a card, and after two weeks without a new memory one notice invites the next, once per quiet stretch (`diary_job_at` in the archivist's `bot.toml` sets the time; it catches up after a night the Mac slept through). A family member corrects a card by replying in its thread, typed or spoken; the correction is committed to the vault under their name, and a corrected or hand-edited card is never overwritten by a later run. The archivist says nothing else in `#Memories`. The user guide has the family's side: [The Memories Room](user-guide.md#the-memories-room). | | | @@ -630,7 +644,7 @@ Key things to know: ### users.toml -Your family members. Generated by the installer. User accounts are seeded on the first `./stack up` of each stacklet. +The accounts: who can sign in, to what. Generated by the installer. User accounts are seeded on the first `./stack up` of each stacklet. The people the family knows are a separate list, the memory stacklet's `family/people/` (see [Memory](#memory-memory-optional)): a baby has no account but is family. ```toml [[users]] diff --git a/docs/adr/adr-011-vault-brain-projection.md b/docs/adr/adr-011-vault-brain-projection.md index d2552dac..36eec3d5 100644 --- a/docs/adr/adr-011-vault-brain-projection.md +++ b/docs/adr/adr-011-vault-brain-projection.md @@ -118,3 +118,28 @@ room) or edited by hand is theirs: the compile never overwrites it, and the vault commit that changed it names who did. The diary is therefore the one chat filing not rebuilt by replaying its thread (see `docs/agent/dev.md`, "Reprocessing replays the source"). + +## Update 2026-10-09: compiled projections are the agent's fast path + +The rule above keeps the agent off projections so that a stale page +never becomes a wrong answer. It assumed the agent could afford to +read the records for every question. With a local model it cannot: +reading dozens of records per question is slow, and the compiled pages +exist precisely to have read them once. + +The vault stays the truth. The agent may answer from a compiled +projection when two things hold: + +- every statement it uses cites the records it came from, as the home + page's numbered references and the person pages' `sources` do, so + the answer can name its source and a person can check it; +- it also reads the records filed since that projection was last + compiled, so nothing newer is missed. + +What this does not remove is a projection that is wrong rather than +stale. The citations make that checkable, family corrections fix it, +and a nightly plausibility pass (the "dream cycle" in +`docs/design/brain/domain-model.md`) is meant to find it. + +The agent already reads the brain (`MEMORY_VAULT_DIR` points at it), +so this update describes what it does and puts conditions on it. diff --git a/docs/design/brain/domain-model.md b/docs/design/brain/domain-model.md index 3e65e7d2..d30baa71 100644 --- a/docs/design/brain/domain-model.md +++ b/docs/design/brain/domain-model.md @@ -35,17 +35,40 @@ placement stays the deletion test: does deleting it lose information? ## Aggregates in the knowledge core -**Household** — the instance itself. Owns Members, the shared bucket -slug, language, and the scope rules. `shared_bucket` being +**Household** — the instance itself. Owns its People, the shared +bucket slug, language, and the scope rules. `shared_bucket` being configurable is the tell that the real concept is a circle of people sharing an archive; a family is the primary instantiation, a non-family deployment is the same aggregate with different vocabulary. -**Member** — identity is the triple equality -`matrix localpart == bucket slug == git author`. That equality is -load-bearing: it is what makes attribution (`--by homer`), scoping, -and the vault chronicle line up. A Member owns a personal bucket with -the same shape as the shared one. +**Person** — someone the family knows, declared in one file each: +`family/people/.md` (vault-format §7). Identity is the file's id +(`maggie`). The file holds the full name, the other names the family +uses for them, and the person's chat account if they have one. It is a +state document: the family writes and corrects it, by hand or through +the CLI, and `stack up memory` adds one for every account that has +none. The wiki's person page (`maggie/about.md`) is the compiled +projection of it: the declared names win, the compiled detail fills in +the rest. One entity, two sources, so the family's corrections to +what the stack learned ("Ed is Ned", "Grampa is Homer's father") are +edits to the same file, never a second format. + +**Member** — a person who belongs to the household. Not a separate +entity and not a declared field: membership is derived, and the stack +makes an opinionated guess. In the first iteration a person is a +member when they have an account. For a member with an account the +triple equality `matrix localpart == bucket slug == git author` holds, +and it is what makes attribution (`--by homer`), scoping, and the +vault chronicle line up; such a member owns a personal bucket with the +same shape as the shared one. A person without an account (a baby, a +grandparent) is still declared, so the classifier, the transcription +vocabulary and the diary know them by every name, and the wiki gives +them a page; they are not listed among the members. The next iteration +suggests membership from the records when they make it likely (Maggie +lives in the house, the records say so), and asks the family before +it counts. Deriving members from folders is what put `media/` on the +wiki as a family member, which is why the guess starts from accounts +and nothing weaker. **Record** — the generalization that ate "document". A capture, a document mirror, an email thread, a voice memo. Identity is @@ -89,7 +112,11 @@ from frontmatter; it wants to live in a registry (below). Scope (the privacy rule as a type: shared bucket or one member's bucket, derived from room membership). SourceContent (text + title + origin URI, the ingestion contract). Classification (correspondent, -doctype, persons, topics, summary, facts, action items). Provenance. +doctype, persons, mentions, topics, summary, facts, action items). +`persons` are the members a record belongs to; `mentions` are the +people outside the household it names, extracted in the same model +call that classifies it, so a record can be replayed to recompute +them like every other part of its classification. Provenance. Attribution. CaptureHash / PaperlessId / ThreadRoot. VaultPath (the deterministic path builders are its factories). SummaryCallout with Facts and ActionItems. PermaLink (`/go/docs|topic|person`, the @@ -110,6 +137,22 @@ connects Conversation to Knowledge (this room feeds that topic). identity authority. This is also the foundation for typed entities inside standing topics: a Policy, an Account, a Vehicle, each linked to a Correspondent and to source Records. + *Being built, people first.* A person entry is compiled from the + `mentions` of every record: canonical name, aliases, the kind of + relationship to the household (relative, friend, neighbor, teacher, + doctor, ...) and the stated relation, how present the person is + (records, months, first and last seen), and the records it cites. + It is a projection; the family's corrections ("Ed is Ned", "Grampa + is Homer's father") are the person's declared file (Person, above), + which wins over it and survives every rebuild. Correspondents become a second kind in the + same registry, and `pet`, `vehicle`, `place` after them, with the + same aliases, citations and corrections rather than a mechanism + each. The wiki's people tiers (family, broader family, close, + other) are a projection of the members plus the registry: tier + from the relationship kind, order within a tier from presence. + Evidence: `tools/family-memories/spec.people.en.yaml` and the + people-tiers prototype (2026-10-08). Relationship kind put 21 of 22 + people in the right tier; mention counts alone, 14. - **FactStore** — `facts.toml` holds facts (rule | fact | habit) with no aggregate managing them. Typed facts with as-of validity are what turn a standing topic from a folder into knowledge: policy number, @@ -117,6 +160,17 @@ connects Conversation to Knowledge (this room feeds that topic). - **Reminder** — time-bound intent attached to a Member or Topic. Standing-topic facts with dates (renewals, expiries) feed it almost for free. +- **Dream cycle** — a nightly pass after the compile that checks what + was compiled for plausibility: one person with two fathers, a + relation that contradicts another source, near-duplicate entries, + dates that cannot both be true. Cheap deterministic checks select + the suspects; the model rereads only their cited records. It never + rewrites the family's own documents: a contradiction the sources + cannot settle becomes a question to the family in chat, and the + answer becomes a correction. Majority voting is not enough: in the + prototype Grampa was "Marge's father" in 3 records and "Homer's + father" in 2, because Marge writes "Grampa" from the children's + side. Earlier sketch: `ontology-design.md`, "Maintenance". ## Rules of thumb @@ -127,4 +181,8 @@ connects Conversation to Knowledge (this room feeds that topic). 3. New containers are topics until proven otherwise; prefer a provenance flag over a new mechanism. 4. Facts cite Records. A fact without a source link is a rumor. -5. The agent is a client of the database, never of the projections. +5. The vault is the truth; compiled projections are the agent's fast + path. The agent may answer from a projection whose statements cite + their records, and reads the records filed since that projection + was last compiled. A slow local model cannot search every record + for every question. (ADR-011, update 2026-10-09.) diff --git a/docs/design/brain/family-ontology.md b/docs/design/brain/family-ontology.md index 699e4eba..ae9c950f 100644 --- a/docs/design/brain/family-ontology.md +++ b/docs/design/brain/family-ontology.md @@ -1,6 +1,15 @@ # Family ontology: declared entities priming classification -Status: design draft, 2026-06-12. Implementation slice of +Status: design draft, 2026-06-12. Partly superseded 2026-10-09: the +people pages below are built (`vault-format.md` §5, `person` +(declared)), with `type: person` and `title` instead of a folder-derived +kind and `canonical`; membership is derived (an account, in the first +iteration), not a `member:` field; people outside the household are +compiled from the `mentions` on records into the person registry +(`domain-model.md`, EntityRegistry); relations are a `relation_kind` +plus words, not a `role`. Onboarding, the write-path contract and +experiment round 6 +still stand. Implementation slice of `ontology-design.md` (2026-04-14), which designs the full living ontology (persons, organizations, relations, growth loop, dream-cycle maintenance, search expansion). Companion to `wiki-page-anatomy.md` and diff --git a/docs/design/brain/vault-format.md b/docs/design/brain/vault-format.md index 37bbc5e0..23053100 100644 --- a/docs/design/brain/vault-format.md +++ b/docs/design/brain/vault-format.md @@ -82,9 +82,18 @@ cheap). - `diary` — one entry from the memories room (a recording, photo or note, with its joined fragments and replies). +**State documents** (the family's current truth; mutable through the +CLI or by hand; never carry `generated`; see §7): +- `todos` — a topic's todo list. +- `person` (declared) — someone the family knows, one file each under + `/people/`. + **Entities & structure** (generated projections; MUST carry `generated: true`): -- `person` — a household member or known person page. +- `person` — the wiki page for a person: the declared file compiled + with what the records say, and later anyone the records mention (the + person registry). `person` is the one type that is both: declared in + `people/`, generated everywhere else, told apart by `generated`. - `correspondent` — an external party page. - `topic` — a topic overview page. - `index` — a folder navigation page. @@ -97,6 +106,14 @@ Legend: **R** required, **O** optional (present-when-nonempty — absence is a signal, so never emit an empty list/string). List fields are one-deep string lists per §2. +Two person fields recur on records and mean different things: +`persons` lists the household members the record belongs to, by +display name; `mentions` lists the people outside the household it +names, each by the most specific name the record gives, or a +description through a member when it gives none ("Marge's mother"). +Both come from the call that classifies the record. The person +registry is compiled from `mentions`; the record never links to it. + ### `document` | Field | | Notes | |---|---|---| @@ -111,6 +128,7 @@ one-deep string lists per §2. | `document_type` | O | Paperless subtype (invoice, contract) — a different axis from `type`. | | `category` | O | | | `persons` | O | list. | +| `mentions` | O | list. | | `tags` | O | list. | | `paperless_url` | O | base URL. | | `processing` | O | `ai_formatted` \| `ocr` \| `original`. | @@ -124,6 +142,7 @@ one-deep string lists per §2. | `title` | R | | | `timestamp` | R | | | `persons` | O | list. | +| `mentions` | O | list. | | `filed_by` | O | Matrix localpart of the filer (mirrors the git author). | | `tags` | O | list. | | `resource` | O | source URL (OKF `resource`; bookmarks always have one). | @@ -160,6 +179,7 @@ commit (`correct: `, authored by whoever wrote it), not on the card; its event id joins `event_ids`. | `description` | O | one sentence. | | `persons` | O | list. | +| `mentions` | O | list. | | `tags` | O | list. | | `filed_by` | O | Matrix localpart of the sender. | | `addressee` | O | who the message is spoken to, as it names them; a correction can change it. | @@ -177,16 +197,51 @@ and last the family's own words under `## Transcript`, `## Text` or `## Caption`. The words come last so nothing in them can be mistaken for a section. +### `person` (declared) +The family's file for one person, `<shared_bucket>/people/<id>.md`. The +file name is the id. + +| Field | | Notes | +|---|---|---| +| `type` | R | `person` | +| `title` | R | the full name (OKF `title`). | +| `aliases` | O | list; every other name the family uses (Obsidian reads the same key). | +| `account` | O | the person's chat user name (Matrix localpart), when they have one. | + +``` +--- +type: person +title: Maggie Simpson +aliases: + - Margaret +--- +``` + +A member of the household is a person with an `account` (first +iteration; `domain-model.md`, Member). Membership is derived, never a +field the family sets. A file that breaks the §2 subset or this table +is reported by path and left out, and the wiki retires no member page +while any file is reported. + ### `person` (generated) | Field | | Notes | |---|---|---| | `type` | R | `person` | | `generated` | R | `true` | | `title` | R | canonical display name (OKF `title`). | -| `slug` | R | bucket slug (== Matrix localpart for members). | +| `slug` | R | the person's id: the declared file's name (== Matrix localpart and bucket slug for a member); otherwise derived from the canonical name. | | `canonical` | R | canonical name (our H1/canonicalization logic). | | `aliases` | O | list of known surface forms (the EntityRegistry coupling field). | -| `role`, `member`, `birthday`, `employer`, `owners`, `relation` | O | family-semantic custom fields. | +| `relation_kind` | O | `relative` \| `friend` \| `neighbor` \| `teacher` \| `doctor` \| `employer` \| `colleague` \| `service` \| `other`; absent for members. | +| `relation` | O | the relation in words, through a member ("Marge's sister"). | +| `first_seen`, `last_seen` | O | dates of the first and last record that mentions the person. | +| `sources` | O | list of the vault paths of the records it cites. | +| `birthday`, `employer`, `owners` | O | family-semantic custom fields. | + +Whether someone is a member is never a field: it is derived from the +declared file's `account`. The tier the wiki shows (broader family, close, other) is +not stored either; the wiki derives it from `relation_kind` and +presence. ### `correspondent` (generated) | Field | | Notes | @@ -225,6 +280,16 @@ carries `generated`. It is a markdown checklist; frontmatter is optional and, if present, uses `type: todos`. State documents get read-your-writes through the CLI (ADR-011); they are not regenerated. +`<shared_bucket>/people/<id>.md` is a declared person (§5): the +family's own facts about someone, starting with their names. On every +start `stack up memory` adds one for each account that has none, as +a commit by the memory bot, and never rewrites an existing file. +A family member adds a person without an account (a baby) or an alias +by writing the file in Obsidian, Forgejo or with `stack memory write`; +a hand edit is a commit like every other. The family's corrections to +what the registry compiles ("Ed is Ned", "Grampa is Homer's father") +go into the same file when the registry is built. + `ontology.toml` / `facts.toml` are TOML config at the vault root, outside this markdown-frontmatter spec. diff --git a/lib/stack/frontmatter.py b/lib/stack/frontmatter.py index 7f00cf61..7f466fbc 100644 --- a/lib/stack/frontmatter.py +++ b/lib/stack/frontmatter.py @@ -13,7 +13,7 @@ from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, field from typing import Optional @@ -40,9 +40,10 @@ class FrontmatterError(ValueError): @dataclass class TypeSchema: - """Schema for a single type: required fields and optional list fields.""" + """Schema for a single type: required fields, list fields, single-value fields.""" required: set[str] list_fields: set[str] + scalar_fields: set[str] = field(default_factory=set) # Build the type-to-schema mapping from §5 of the spec. @@ -86,6 +87,18 @@ class TypeSchema: } +# Types the family may also declare by hand, as a state document without +# `generated`: the family's own file for an entity, which the generated +# page of the same type is compiled from (§5, `person` (declared)). +DECLARED_SCHEMAS: dict[str, TypeSchema] = { + "person": TypeSchema( + required={"type", "title"}, + list_fields={"aliases"}, + scalar_fields={"title", "account"}, + ), +} + + # ── Parser ──────────────────────────────────────────────────────────── def parse(text: str) -> dict: @@ -323,7 +336,9 @@ def validate(fm: dict) -> list[str]: - `type` is present and in the closed vocabulary - Required fields for the type are present - List fields are actually lists (or can be coerced) - - `generated` presence matches record vs. projection class + - Single-value fields are not lists + - `generated` presence matches record vs. projection class; a + declarable type without it is checked as the family's own file Args: fm: Parsed frontmatter dict @@ -349,7 +364,11 @@ def validate(fm: dict) -> list[str]: ) return errors - schema = SCHEMAS[entry_type] + # A declarable type without the marker is the family's own file. + is_projection = entry_type in PROJECTION_TYPES + has_generated = fm.get("generated") is True + declared = entry_type in DECLARED_SCHEMAS and not has_generated + schema = DECLARED_SCHEMAS[entry_type] if declared else SCHEMAS[entry_type] # Check required fields for req_field in schema.required: @@ -365,11 +384,13 @@ def validate(fm: dict) -> list[str]: f"field `{list_field}` must be a list, got {type(val).__name__}" ) - # Check `generated` invariant (§6) - is_projection = entry_type in PROJECTION_TYPES - has_generated = fm.get("generated") is True + # Check single-value fields are not lists + for scalar_field in schema.scalar_fields: + if isinstance(fm.get(scalar_field), list): + errors.append(f"field `{scalar_field}` must be a single value, not a list") - if is_projection and not has_generated: + # Check `generated` invariant (§6) + if is_projection and not has_generated and not declared: errors.append( f"projection type `{entry_type}` must have `generated: true`" ) diff --git a/lib/stack/people.py b/lib/stack/people.py new file mode 100644 index 00000000..7cb67dae --- /dev/null +++ b/lib/stack/people.py @@ -0,0 +1,238 @@ +"""The family's people: one declared file each. + +Every person the family knows has a file in the shared bucket, +`people/<id>.md`, in the vault's frontmatter format: + + --- + type: person + title: Maggie Simpson + aliases: + - Margaret + --- + +The file name is the id. `title` is the full name, `aliases` the other +names the family uses, and `account` the chat user name of a person who +has one. The schema lives with the others in `stack.frontmatter` +(`DECLARED_SCHEMAS`), and a file that breaks it is reported by path +and left out; nothing here guesses at what a broken file meant. + +A member of the household is a person with an account. That is the +first, opinionated guess at membership (docs/design/brain/domain-model.md, +Member); it is derived, never a field the family sets. A person +without an account (a baby) is still known by every name: the +classifier, the transcription vocabulary and the diary read all +people, the wiki's member list reads the members. + +The readers return `None` while no person file exists, so a caller can +keep its previous behaviour until `stack up memory` has written them. + +Pure functions and file reads. No Forgejo, no git: the memory stacklet +writes the files (`ensure_people` in its library). Lives in the +framework because the readers span stacklets: the wiki, the diary, the +archivist, transcription. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Iterable + +from .frontmatter import FrontmatterError, dump, parse, validate +from .vault import DEFAULT_SHARED_BUCKET, slug + +PEOPLE_DIR = "people" + +# An id as the vault uses it for buckets and Matrix for localparts. A +# file name that is not one is slugged ("Grampa Simpson.md" is +# `grampa-simpson`), so a file made in Obsidian still names a person. +_ID = re.compile(r"^[a-z0-9][a-z0-9._-]*$") + +Report = Callable[[str], None] + + +@dataclass(frozen=True) +class Person: + """One person the family declared. + + >>> p = Person("maggie", "Maggie Simpson", ("Margaret",)) + >>> p.all_names() + ['Maggie Simpson', 'Maggie', 'Margaret'] + >>> p.member + False + """ + + id: str + name: str + aliases: tuple[str, ...] = field(default_factory=tuple) + account: str = "" + + @property + def member(self) -> bool: + """Whether the person belongs to the household: they have an account.""" + return bool(self.account) + + @property + def first_name(self) -> str: + return self.name.split()[0] if self.name.split() else self.id + + def all_names(self) -> list[str]: + """Full name, first name, then the aliases; each once, in that order.""" + out: list[str] = [] + for name in (self.name, self.first_name, *self.aliases): + if name and name.lower() not in {n.lower() for n in out}: + out.append(name) + return out + + +class PersonError(ValueError): + """A person file that does not follow the format; the message says how.""" + + +# ── Reading ────────────────────────────────────────────────────────────── + +def people_dir(root: Path, shared_bucket: str = DEFAULT_SHARED_BUCKET) -> Path: + """Where the person files live under a vault or brain working copy.""" + return Path(root) / shared_bucket / PEOPLE_DIR + + +def person_path(person_id: str, shared_bucket: str = DEFAULT_SHARED_BUCKET) -> str: + """The repo-relative path of one person's file.""" + return f"{shared_bucket}/{PEOPLE_DIR}/{person_id}.md" + + +def person_id(file_stem: str) -> str: + """The id a person file's name gives.""" + return file_stem if _ID.match(file_stem) else slug(file_stem) + + +def read_person(file_stem: str, text: str) -> Person: + """The person one file declares, or `PersonError` saying what is wrong.""" + try: + fm = parse(text) + except FrontmatterError as e: + raise PersonError(f"frontmatter does not parse: {e}") from e + if fm.get("type") != "person": + raise PersonError("not a person file: it needs `type: person`") + if fm.get("generated") is True: + raise PersonError("a generated page does not belong in people/") + if errors := validate(fm): + raise PersonError("; ".join(errors)) + return Person( + id=person_id(file_stem), + name=str(fm["title"]).strip(), + aliases=tuple(str(a).strip() for a in fm.get("aliases") or [] if str(a).strip()), + account=str(fm.get("account") or "").strip(), + ) + + +def load_people( + root: Path | None, + shared_bucket: str = DEFAULT_SHARED_BUCKET, + *, + report: Report | None = None, +) -> list[Person] | None: + """Every person declared under `root`, members first, or `None` if none is. + + A file that cannot be read as a person is passed to `report` with + its path and the reason, and left out. `None` covers no folder, an + empty folder and a folder of broken files alike: the caller keeps + its previous behaviour rather than act on nobody. + """ + if root is None: + return None + folder = people_dir(root, shared_bucket) + people: dict[str, Person] = {} + for path in sorted(folder.glob("*.md")): + rel = f"{shared_bucket}/{PEOPLE_DIR}/{path.name}" + try: + person = read_person(path.stem, path.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError) as e: + _report(report, f"{rel}: cannot be read ({e})") + continue + except PersonError as e: + _report(report, f"{rel}: {e}") + continue + if person.id in people: + _report(report, f"{rel}: {person.id!r} is declared twice; the first file counts") + continue + people[person.id] = person + if not people: + return None + return sorted(people.values(), key=lambda p: (not p.member, p.name.lower())) + + +def find_people( + roots: Iterable, + shared_bucket: str = DEFAULT_SHARED_BUCKET, + *, + report: Report | None = None, +) -> list[Person] | None: + """The people from the first of `roots` that declares any. + + A process often sees two copies of the vault: the working copy the + family's edits land in, and the brain the curator mirrors it to. + Callers list the fresher one first. Empty and None roots are skipped. + """ + for root in roots: + if root and (people := load_people(Path(root), shared_bucket, report=report)) is not None: + return people + return None + + +def members(people: list[Person]) -> list[Person]: + """The people who belong to the household.""" + return [p for p in people if p.member] + + +def resolve(name: str, people: list[Person]) -> Person | None: + """The person a name means, by any name they go by, or None. + + >>> resolve("Margaret", [Person("maggie", "Maggie Simpson", ("Margaret",))]).id + 'maggie' + """ + wanted = name.strip().lower() + for person in people: + if wanted == person.id or wanted in (n.lower() for n in person.all_names()): + return person + return None + + +def _report(report: Report | None, message: str) -> None: + if report is not None: + report(message) + + +# ── Writing ────────────────────────────────────────────────────────────── + +def people_from_accounts(users: list[dict]) -> list[Person]: + """A person for everyone with an account, as `users.toml` lists them. + + The id follows `users.user_id` (an explicit `id`, else the first name + lowercased), so it is the person's Matrix localpart and their bucket. + Entries without a name are skipped. Bots never appear in `users.toml`. + """ + people: list[Person] = [] + seen: set[str] = set() + for user in users: + name = str(user.get("name") or "").strip() + if not name: + continue + account = str(user.get("id") or name.split()[0].lower()) + if account in seen: + continue + seen.add(account) + people.append(Person(account, name, account=account)) + return people + + +def render_person(person: Person) -> str: + """A person's file as famstack first writes it.""" + fm = dump({ + "type": "person", + "title": person.name, + "aliases": list(person.aliases), + "account": person.account, + }) + return f"---\n{fm}\n---\n\n# {person.name}\n" diff --git a/stacklets/docs/bot/archivist.py b/stacklets/docs/bot/archivist.py index 999ed406..e4ba8ea3 100644 --- a/stacklets/docs/bot/archivist.py +++ b/stacklets/docs/bot/archivist.py @@ -557,6 +557,7 @@ def _build_services(self) -> ArchivistServices: capture_tag_prompt_size=self.capture_tag_prompt_size, vision_max_pdf_pages=self.vision_max_pdf_pages, llm=classifier.llm if classifier is not None else None, + vault=vault, ) return ArchivistServices( paperless=paperless, diff --git a/stacklets/docs/bot/capture_pipeline.py b/stacklets/docs/bot/capture_pipeline.py index f34ecd08..839afb84 100644 --- a/stacklets/docs/bot/capture_pipeline.py +++ b/stacklets/docs/bot/capture_pipeline.py @@ -165,8 +165,12 @@ def __init__( capture_tag_prompt_size: int, vision_max_pdf_pages: int = DEFAULT_VISION_MAX_PDF_PAGES, llm=None, + vault=None, ): self._url_extractor = url_extractor + # The memory vault, for the person files. None in callers + # that run without one; the Person tags stand in for them then. + self._vault = vault self._text_extractor = text_extractor self._classifier = classifier self._mirror = mirror @@ -772,6 +776,17 @@ async def _publish( scope=entity_slug, ) + async def _family_names(self) -> list[str]: + """The family's first names: the person files, else the Person tags.""" + people = self._vault.people() if self._vault is not None else None + if people is not None: + return [p.first_name for p in people] + person_tags = await self._paperless.get_tags() + return [ + t.replace("Person: ", "") for t in person_tags + if t.startswith("Person: ") + ] + async def _classify( self, source, sender_name: str, *, images: list[ImageAttachment] | None = None, @@ -795,11 +810,7 @@ async def _classify( # Bot was brought up without AI configured; fall through to the # minimal classification below so the capture still files. return {} - person_tags = await self._paperless.get_tags() - person_names = [ - t.replace("Person: ", "") for t in person_tags - if t.startswith("Person: ") - ] + person_names = await self._family_names() existing_tags = ( self._capture_tags.top(self.capture_tag_prompt_size) if self._capture_tags else [] diff --git a/stacklets/docs/bot/document_pipeline.py b/stacklets/docs/bot/document_pipeline.py index c03044b2..8d2c966d 100644 --- a/stacklets/docs/bot/document_pipeline.py +++ b/stacklets/docs/bot/document_pipeline.py @@ -378,6 +378,7 @@ async def _enrich( ontology_section=ontology.classifier_prompt_section(self.language), correspondents_section=self._vault.correspondents_section(), persons_section=self._vault.persons_section(), + people=self._vault.people(), ontology=ontology, lang=self.language, date_filed=date_filed, @@ -465,6 +466,7 @@ async def reprocess( ontology_section=ontology.classifier_prompt_section(self.language), correspondents_section=self._vault.correspondents_section(), persons_section=self._vault.persons_section(), + people=self._vault.people(), ontology=ontology, lang=self.language, is_reprocess=True, diff --git a/stacklets/docs/bot/matching.py b/stacklets/docs/bot/matching.py index 47ab6d11..804558b1 100644 --- a/stacklets/docs/bot/matching.py +++ b/stacklets/docs/bot/matching.py @@ -188,6 +188,38 @@ def match_persons(names: str | list | None, tags: dict[str, Any]) -> list[str]: return matched_tags +def declared_person_names(names: str | list | None, people) -> list[str]: + """LLM-returned person names, each as its declared person's first name. + + The first name is what a person's `Person:` tag carries. A name the + family declared under any spelling ("Margaret", "Maggie Simpson", + "Person: Maggie") becomes that person's first name; any other name + is passed on unchanged for `match_persons` to try, so a document + still matches by the tags alone, as it did before the person files. + + >>> from stack.people import Person + >>> people = [Person("maggie", "Maggie Simpson", ("Margaret",))] + >>> declared_person_names(["Margaret", "Homer"], people) + ['Maggie', 'Homer'] + """ + from stack.people import resolve + + if isinstance(names, str): + names = [names] + if not isinstance(names, list): + return [] + out: list[str] = [] + for name in names: + if not isinstance(name, str) or _is_empty(name): + continue + clean = name.replace("Person: ", "").strip() + person = resolve(clean, people) + resolved = person.first_name if person else clean + if resolved and resolved not in out: + out.append(resolved) + return out + + def submitter_person_tag( mxid: str | None, tags: dict[str, Any], ) -> str | None: diff --git a/stacklets/docs/bot/pipeline.py b/stacklets/docs/bot/pipeline.py index 14b7f4c9..f807cf0b 100644 --- a/stacklets/docs/bot/pipeline.py +++ b/stacklets/docs/bot/pipeline.py @@ -42,6 +42,7 @@ MAX_TITLE_LENGTH, _is_empty, fuzzy_match_entity, + declared_person_names, match_persons, match_topics, submitter_person_tag, @@ -78,6 +79,7 @@ ImageAttachment = LLMImage if TYPE_CHECKING: + from stack.people import Person from stack.ontology import Ontology @@ -1637,6 +1639,8 @@ def extract_bot_summary(doc: dict) -> str: _ISO_DATE = re.compile(r"^\d{4}-\d{2}-\d{2}$") _NEW_TOPIC_COLOR = "#4caf50" +# Same colour `seed.py` gives the person tags it creates for accounts. +_PERSON_TAG_COLOR = "#2196f3" async def enrich_document( @@ -1657,6 +1661,7 @@ async def enrich_document( initial_classification: dict | None = None, submitter_mxid: str | None = None, write_in: str | None = None, + people: "list[Person] | None" = None, ) -> EnrichResult: """Classify a doc, reconcile entities, PATCH Paperless. Pure data out. @@ -1758,6 +1763,19 @@ async def enrich_document( # tag name; we strip the prefix for the resolved-name list callers # want to render. persons_raw = classification.get("persons") or classification.get("person") + if people is not None: + # The person files name who the family knows, accounts or not: + # each name resolves through the person's aliases, and a person + # who has no `Person:` tag yet (no account, so none was seeded) + # gets one the first time a document names them. + persons_raw = declared_person_names(persons_raw, people) + firsts = {p.first_name for p in people} + for name in persons_raw: + tag = f"Person: {name}" + if name in firsts and tag not in tags: + if new_id := await paperless.create_tag(tag, _PERSON_TAG_COLOR): + tags[tag] = new_id + result.created_new.append(f'tag "{tag}"') for pt in match_persons(persons_raw, tags): tag_ids.append(tags[pt]) result.resolved_persons.append(pt.replace("Person: ", "")) diff --git a/stacklets/docs/bot/vault_context.py b/stacklets/docs/bot/vault_context.py index 61111600..da4c821e 100644 --- a/stacklets/docs/bot/vault_context.py +++ b/stacklets/docs/bot/vault_context.py @@ -15,13 +15,17 @@ import os from pathlib import Path +from loguru import logger + from memory.lib import ( correspondents_prompt_section, get_ontology, load_correspondents_from_vault, load_persons_from_vault, + people_prompt_section, persons_prompt_section, ) +from stack.people import Person, load_people class VaultContext: @@ -65,15 +69,30 @@ def correspondents_section(self) -> str: correspondents = load_correspondents_from_vault(path, self.shared_bucket) return correspondents_prompt_section(correspondents) + def people(self) -> list[Person] | None: + """The family's declared people; None until a person file exists. + + A file that does not read is logged with its path and left out. + """ + return load_people( + self._vault_path(), self.shared_bucket, + report=lambda problem: logger.warning("[vault] person file skipped: {}", problem), + ) + def persons_section(self) -> str: - """Build the persons block from `<slug>/about.md` across the vault. + """The persons block: the person files, else `<slug>/about.md` pages. - Empty when the vault isn't seeded yet — the prompt then falls back - to the flat Paperless Person roster (canonical first names only, - no synonym signal). + With person files the block lists every declared person with every + name they go by, accounts or not. Without them it is built from + person pages in the vault, and empty when there are none -- the prompt + then falls back to the flat Paperless Person roster (canonical + first names only, no synonym signal). """ path = self._vault_path() if path is None: return "" + people = self.people() + if people is not None: + return people_prompt_section(people) persons = load_persons_from_vault(path, self.shared_bucket) return persons_prompt_section(persons) diff --git a/stacklets/memory/bot/cli/diary.py b/stacklets/memory/bot/cli/diary.py index 70d37b44..03a50bed 100644 --- a/stacklets/memory/bot/cli/diary.py +++ b/stacklets/memory/bot/cli/diary.py @@ -81,12 +81,14 @@ from stack.forgejo import FileChange, ForgejoClient, ForgejoError # noqa: E402 from stack.ai import transcripts # noqa: E402 from stack import media # noqa: E402 +from stack.people import Person as DeclaredPerson, find_people # noqa: E402 from memory.lib import ( # noqa: E402 BOT_EMAIL, BOT_USERNAME, REPO_NAME, REPO_OWNER, + Person, get_ontology, load_persons_from_vault, persons_prompt_section, @@ -290,15 +292,55 @@ def _length(ms: int | None) -> str: return f"{total // 60}:{total % 60:02d}" +def _declared() -> list[DeclaredPerson] | None: + """The family's person files; None until `stack up memory` wrote them. + + A file that does not read is logged with its path and left out. + """ + return find_people( + (os.environ.get("MEMORY_VAULT_DIR"), os.environ.get("BRAIN_REPO_DIR")), + os.environ.get("SHARED_BUCKET") or "family", + report=lambda problem: _err(f"person file skipped: {problem}"), + ) + + def _household_people() -> list[str]: - """The household's names: everyone in users.toml, then the wiki's person pages. + """The household's names: the person files, then the wiki's person pages. - The pages carry the family's own spelling of each name and its - variants. Best-effort: no brain means the users.toml names alone. + The person files declare everyone the family knows, a baby without + an account too, with every name the family uses for them. Until they + exist the accounts stand in (FAMILY_NAMES, from users.toml). The + pages add the spellings documents used. Best-effort: no brain means + the declared names alone. """ brain = Path(os.environ.get("BRAIN_REPO_DIR", "")) pages = [_frontmatter(about) for about in sorted(brain.glob("*/about.md"))] if brain.is_dir() else [] - return diary.household(os.environ.get("FAMILY_NAMES", ""), pages) + declared = _declared() + if declared is None: + return diary.household(os.environ.get("FAMILY_NAMES", ""), pages) + ids = {p.id for p in declared} + names = ",".join(n for p in declared for n in p.all_names()) + return diary.household(names, [p for p in pages if p.get("slug") in ids]) + + +def _declared_persons(declared: list[DeclaredPerson], learned: list[Person]) -> list[Person]: + """The person files as the classifier's persons: first name canonical. + + The first name is the name cards carry and the `Person:` tag the + archivist files under, so a card about "Margaret" lists Maggie the + same way a document about her is tagged. The full name, the + family's aliases and the spellings documents used are the synonyms. + """ + by_slug = {p.slug: p for p in learned} + persons = [] + for d in declared: + names = [n for n in d.all_names() if n != d.first_name] + seen = by_slug.get(d.id) + for n in seen.all_known_names() if seen else []: + if n.lower() not in {x.lower() for x in [d.first_name, *names]}: + names.append(n) + persons.append(Person(canonical=d.first_name, slug=d.id, synonyms=names)) + return persons def _household_vocabulary() -> str: @@ -1314,6 +1356,9 @@ def _vocabulary(bucket: str): # Person pages are generated, so they live in the brain, not the vault. brain_dir = Path(os.environ.get("BRAIN_REPO_DIR", "")) persons = load_persons_from_vault(brain_dir, bucket) if brain_dir.is_dir() else [] + declared = _declared() + if declared is not None: + persons = _declared_persons(declared, persons) people = diary.card_people(persons, _household_people()) # The pages describe the people they know; the rest of the household # joins the list by name, so the model can still say a note is about them. diff --git a/stacklets/memory/bot/cli/wiki.py b/stacklets/memory/bot/cli/wiki.py index ae4765c3..b6b1e178 100644 --- a/stacklets/memory/bot/cli/wiki.py +++ b/stacklets/memory/bot/cli/wiki.py @@ -6,6 +6,7 @@ (no flags) home + members + topics: the curator's nightly pass --home just the household home page --member homer just Homer's page + --members every person's page --topic camping just one topic's page --topics every topic page, no home/members --dry-run preview to stdout, no writes @@ -72,6 +73,7 @@ extract_summary_callout, ) from stack.vault import correspondents_dir, slug, slugify_person # noqa: E402 +from stack.people import Person, find_people, members, resolve # noqa: E402 from stack.ai.client import LLM, LLMUnavailableError # noqa: E402 @@ -114,11 +116,11 @@ # Within-bucket reserved subdirectories. A child folder of a known # bucket with one of these names is part of the bucket's own shape -# (capture-type folder, correspondent index, rescue folder), not a -# topic. The topic-discovery walker skips these names. +# (capture-type folder, correspondent index, person files, rescue +# folder), not a topic. The topic-discovery walker skips these names. _RESERVED_BUCKET_SUBDIRS = { "notes", "bookmarks", "documents", - "correspondents", "_unfiled", + "correspondents", "people", "_unfiled", } # Subdir names a topic folder must contain at least one of to be @@ -243,6 +245,7 @@ async def run(llm: LLM, argv: list[str]) -> int: home_sel = "--home" in argv topics_only = "--topics" in argv members_sel = _arg_values(argv, "--member") + members_all = "--members" in argv topics_sel = _arg_values(argv, "--topic") correspondents_sel = _arg_values(argv, "--correspondent") @@ -272,10 +275,20 @@ async def run(llm: LLM, argv: list[str]) -> int: _err("no documents with summary callouts found in vault") return 1 - # The roster the home page links into and the default --members loop - # walks. Computed once: bucket dirs on disk plus everyone named in - # document frontmatter. - roster = _member_slugs(vault, index, shared_bucket) + # The people the wiki carries a page for, and the members among them + # the home page lists: the person files once the family has them, the + # old guess (bucket dirs plus names in frontmatter) until + # `stack up memory` has written them. A person file that does not + # read is reported, and while any is, no page is retired: a typo in + # a file must not delete a page. + problems: list[str] = [] + people = _people(shared_bucket, problems) + if people is not None: + _resolve_persons(index, people) + roster = _roster(vault, index, shared_bucket, people) + member_ids = [p.id for p in members(people or [])] or roster + persons = {p.id: p for p in people or []} + retire = people is not None and not problems # Topic locations are discovered against the roster: a topic under # a bucket whose owner the wiki doesn't generate a page for would @@ -295,20 +308,21 @@ async def run(llm: LLM, argv: list[str]) -> int: # and slugs both land on the bucket. Unknown selections warn and skip — # an auto-run fed a name with no match must not sink the batch — and the # run fails only when NOTHING in the selection matched. - if home_sel or topics_only or members_sel or topics_sel or correspondents_sel: + if (home_sel or topics_only or members_sel or members_all + or topics_sel or correspondents_sel): generated = 0 if home_sel: rc = await _generate_home( - llm, index, roster=roster, + llm, index, roster=member_ids, people=persons, shared_bucket=shared_bucket, lang=lang, write=not dry_run, ) if rc == 0: generated += 1 - member_slugs: list[str] = [] + member_slugs: list[str] = list(roster) if members_all else [] for raw in members_sel: - slug = slugify_person(raw) + slug = _member_id(raw, people) if slug and slug not in member_slugs: member_slugs.append(slug) for slug in member_slugs: @@ -316,11 +330,13 @@ async def run(llm: LLM, argv: list[str]) -> int: _err(f"no member bucket for '{slug}' — skipping") continue rc = await _generate_member( - llm, slug, index, vault, + llm, slug, index, vault, person=persons.get(slug), shared_bucket=shared_bucket, lang=lang, write=not dry_run, ) if rc == 0: generated += 1 + if members_all and retire: + _retire_undeclared(roster, shared_bucket, write=not dry_run) # Correspondents match by canonical name or slug against the roster # discovered from document frontmatter. @@ -366,11 +382,13 @@ async def run(llm: LLM, argv: list[str]) -> int: # ── Default loop ────────────────────────────────────────────────── # Bare invocation: home, every member, every correspondent, every topic. rc = await _generate_home( - llm, index, roster=roster, + llm, index, roster=member_ids, people=persons, shared_bucket=shared_bucket, lang=lang, write=not dry_run, ) if rc != 0: return rc + if retire: + _retire_undeclared(roster, shared_bucket, write=not dry_run) # A member with no content is skipped (not an error) so one # empty bucket doesn't sink the whole run. `_generate_member` @@ -378,7 +396,7 @@ async def run(llm: LLM, argv: list[str]) -> int: # here -- the overall run still succeeds. for member_slug in roster: await _generate_member( - llm, member_slug, index, vault, + llm, member_slug, index, vault, person=persons.get(member_slug), shared_bucket=shared_bucket, lang=lang, write=not dry_run, ) @@ -400,16 +418,19 @@ async def run(llm: LLM, argv: list[str]) -> int: async def _generate_home( llm: LLM, index: list[dict], *, - roster: list[str], shared_bucket: str, lang: str, write: bool, + roster: list[str], people: dict[str, Person], + shared_bucket: str, lang: str, write: bool, ) -> int: """Compose the household home page from every filed summary. The roster (every member that has a page) is woven into the Members - section so each name links to that person's page. Quartz turns those + section so each name links to that person's page; `people` gives + the declared full names. Quartz turns those links into backlinks and graph edges for free, so the home page and the member pages become navigable in both directions. """ - prompt = _build_home_prompt(index, roster=roster, lang=lang) + names = {slug: p.name for slug, p in people.items()} + prompt = _build_home_prompt(index, roster=roster, names=names, lang=lang) try: page = (await llm.complete("overview", prompt, temperature=_TEMPERATURE)).strip() except LLMUnavailableError as e: @@ -454,15 +475,26 @@ async def _generate_home( async def _generate_member( llm: LLM, slug: str, index: list[dict], vault: Path, *, + person: Person | None = None, shared_bucket: str, lang: str, write: bool, ) -> int: - """Compose one member's page from their slice of the vault.""" + """Compose one person's page from their slice of the vault. + + `person` is their declared file; without person files the name comes + from the slug, as before. + """ entries = _member_entries(index, slug) + if not entries and person is not None: + # A declared person always has a page: the home page and the + # People menu link to it. Nothing filed yet means nothing for the + # model to write, so the page says so without asking it. + return _publish_person( + _empty_person_body(person), slug, person, [], write=write) if not entries: _err(f"no content involving '{slug}' — skipping") return 1 - display = slug.capitalize() + display = person.name if person else slug.capitalize() facts = _load_facts(vault, slug) # Anchored regen: renumber the existing page's citations to the @@ -512,17 +544,17 @@ async def _generate_member( shared_bucket=shared_bucket, write=write, ) return 0 - rc = _publish( - page, target_path=f"{slug}/about.md", - # First-creation frontmatter seeds the person entity registry on - # the page itself: `canonical` is the longest synonym (usually - # the formal first name when the family also uses a nickname), - # `synonyms` are the other variants seen in document - # frontmatter. The splice keeps everything outside the markers - # on re-runs, so a hand edit or a future deriver pass takes - # ownership of the registry from here. - default_preamble=_member_preamble(slug, display, _member_synonyms(index, slug)), - ) + learned = _member_synonyms(index, slug) + if person is None: + # No person files: first-creation frontmatter seeds the names + # on the page itself (`canonical` the longest synonym seen in + # documents) and the splice keeps them from then on. + rc = _publish( + page, target_path=f"{slug}/about.md", + default_preamble=_member_preamble(slug, display, learned), + ) + else: + rc = _publish_person(page, slug, person, learned, write=True) _publish_capture_indexes( index, page_dir=slug, display=display, shared_bucket=shared_bucket, write=write, @@ -690,6 +722,163 @@ def _norm_str_list(value) -> list[str]: return [v.strip() for v in value if isinstance(v, str) and v.strip()] +def _publish_person(page: str, slug: str, person: Person, learned: list[str], *, + write: bool) -> int: + """Publish a declared person's page under the names their file gives. + + The file owns the names: the full name is canonical, the family's + aliases come first, the variants documents used follow. Refreshed on + every rebuild, so an alias added to the file reaches the page the + next time. + """ + names = _distinct([*person.aliases, person.first_name, *learned], exclude=person.name) + if not write: + print(f"\n<!-- {slug}/about.md -->\n{page}") + return 0 + return _publish( + page, target_path=f"{slug}/about.md", + transform=lambda prior: _with_member_names( + _splice_generated(prior, page, default_preamble=_member_preamble( + slug, person.name, [person.name, *names])), + canonical=person.name, synonyms=names, + ), + ) + + +def _empty_person_body(person: Person) -> str: + """The page of a person nothing has been filed about yet.""" + return (f"# {person.name}\n\n" + f"Nothing filed about {person.first_name} yet. Documents, notes and " + f"memories that name {person.first_name} will show up here.") + + +def _people(shared_bucket: str, problems: list[str]) -> list[Person] | None: + """The declared people, or None while there are no person files. + + Read from the vault working copy (`MEMORY_VAULT_DIR`) when this + process has one: an edit to a file is on disk there before the + curator has mirrored it into the brain. The brain is the fallback. + Every file that does not read is logged and added to `problems`. + """ + def report(problem: str) -> None: + problems.append(problem) + _err(f"person file skipped: {problem}") + + return find_people( + (os.environ.get("MEMORY_VAULT_DIR"), os.environ.get("BRAIN_REPO_DIR")), + shared_bucket, report=report, + ) + + +def _roster(vault: Path, index: list[dict], shared_bucket: str, + people: list[Person] | None) -> list[str]: + """The ids the wiki carries person pages for, in the order it shows them.""" + if people is not None: + return [p.id for p in people] + return _member_slugs(vault, index, shared_bucket) + + +def _member_id(raw: str, people: list[Person] | None) -> str: + """The person a `--member` value or a `persons:` name means. + + Through every name the person files give ("Margaret" is Maggie), + by first word otherwise, the way buckets are named. + """ + if people is not None and (person := resolve(raw, people)): + return person.id + return slugify_person(raw) + + +def _resolve_persons(index: list[dict], people: list[Person]) -> None: + """Point every `persons:` name in the index at its declared person. + + The index slugs names by first word, which files "Margaret" under a + bucket of her own. With person files the names resolve through the + family's aliases instead. Names no person answers to keep their + first-word slug, as before. + """ + for entry in index: + entry["persons"] = [ + _member_id(name, people) for name in entry.get("person_names", []) + ] + + +def _retire_undeclared(roster: list[str], shared_bucket: str, *, + write: bool) -> list[str]: + """Delete generated person pages for anyone no person file declares. + + Before the person files, a vault folder or a surname could become a + member page (`media/about.md`). Only pages the wiki generated as a + person are removed; anything a person wrote is left alone, and so is + the folder's other content. + """ + try: + brain = _brain_dir() + except RuntimeError: + return [] + skip = _NON_MEMBER_DIRS | {shared_bucket} + retired: list[str] = [] + for about in sorted(brain.glob("*/about.md")): + slug = about.parent.name + if slug in roster or slug in skip or slug.startswith("."): + continue + try: + text = about.read_text(encoding="utf-8") + except OSError: + continue + if _parse_frontmatter(text).get("type") != "person" or not _is_generated_page(text): + continue + rel = f"{slug}/about.md" + if write: + about.unlink() + _err(f"retired {rel}: no person file declares {slug}") + else: + _err(f"would retire {rel}: no person file declares {slug}") + retired.append(rel) + return retired + + +def _distinct(names: list[str], *, exclude: str = "") -> list[str]: + """Names in order, each once (case-insensitive), without `exclude`.""" + seen = {exclude.lower()} if exclude else set() + out: list[str] = [] + for name in names: + if name and name.lower() not in seen: + seen.add(name.lower()) + out.append(name) + return out + + +def _with_member_names(content: str, *, canonical: str, synonyms: list[str]) -> str: + """Set a member page's `title`, `canonical` and `synonyms` frontmatter. + + Every other frontmatter line is kept as it is. A page without + frontmatter is returned unchanged; the splice always gives a member + page one. + """ + if not content.startswith("---\n"): + return content + end = content.find("\n---\n", 4) + if end < 0: + return content + kept: list[str] = [] + skipping_list = False + for line in content[4:end].splitlines(): + if skipping_list and line.startswith((" ", "\t")): + continue + skipping_list = False + key = line.partition(":")[0].strip() + if not line.startswith((" ", "\t")) and key in ("title", "canonical", "synonyms"): + skipping_list = key == "synonyms" + continue + kept.append(line) + names = [f"title: {_yaml_str(canonical)}", f"canonical: {_yaml_str(canonical)}"] + if synonyms: + names.append("synonyms:") + names.extend(f" - {_yaml_str(s)}" for s in synonyms) + return "---\n" + "\n".join(names + kept) + content[end:] + + def _member_slugs(vault: Path, index: list[dict], shared_bucket: str) -> list[str]: """Every family member the wiki should carry a page for. @@ -1429,7 +1618,8 @@ def _arg_values(argv: list[str], flag: str) -> list[str]: # ── Prompts ────────────────────────────────────────────────────────────── -def _build_home_prompt(entries: list[dict], *, roster: list[str], lang: str) -> str: +def _build_home_prompt(entries: list[dict], *, roster: list[str], lang: str, + names: dict[str, str] | None = None) -> str: """Single-shot prompt: feed all summaries, ask for one home page. The section layout is fixed -- we want the same headings every time @@ -1439,8 +1629,10 @@ def _build_home_prompt(entries: list[dict], *, roster: list[str], lang: str) -> closed set of page paths so the Members section can link each name to a member page that actually exists -- the model resolves identity (the baby "Margaret" maps to the `maggie/` page), but it can only - pick from the paths we give it. + pick from the paths we give it. With person files, `names` carries + each member's full name too, and the model fills in only the detail. """ + names = names or {} evidence = _format_evidence(entries) if roster: # Closed enumeration: the model copies this list instead of @@ -1448,7 +1640,8 @@ def _build_home_prompt(entries: list[dict], *, roster: list[str], lang: str) -> # small models split one person into two ("Maggie" and # "Margaret") or drop the page links. skeleton = "\n".join( - f"- **[<full name of {slug.capitalize()}>]({slug}/about)** — <detail>. [N]" + f"- **[{names[slug]}]({slug}/about)** — <detail>. [N]" if slug in names + else f"- **[<full name of {slug.capitalize()}>]({slug}/about)** — <detail>. [N]" for slug in roster ) member_links = ( diff --git a/stacklets/memory/bot/curator.py b/stacklets/memory/bot/curator.py index c76e5ba8..2aa97224 100644 --- a/stacklets/memory/bot/curator.py +++ b/stacklets/memory/bot/curator.py @@ -80,6 +80,7 @@ ) from wiki import COMMIT_PREFIX # noqa: E402 +from stack.people import PEOPLE_DIR # noqa: E402 POLL_SECS = int(os.environ.get("CURATOR_POLL_SECS", "30")) QUIET_SECS = int(os.environ.get("WIKI_REBUILD_QUIET_SECS", "180")) @@ -211,11 +212,15 @@ def member_selection( folders touched by captures. Returns e.g. `["--home", "--member", "Homer Simpson", "--topic", "camping"]`, or `[]` when nothing relevant changed (only generated pages or skipped dirs). Home is - included whenever anything relevant changed at all. + included whenever anything relevant changed at all. A change to a + person file adds `--members`: who the family knows changed, so every + person page is rebuilt and pages for people no file declares are + retired. """ members: list[str] = [] topics: list[str] = [] relevant = False + people_changed = False for path in paths: parts = [p for p in path.split("/") if p] @@ -225,6 +230,9 @@ def member_selection( if _has_generated_marker(fm): continue relevant = True + if path.startswith(f"{shared_bucket}/{PEOPLE_DIR}/"): + people_changed = True + continue if parts[0] != shared_bucket and len(parts) > 1 and parts[0] not in members: members.append(parts[0]) topic = _topic_slug_from_capture_path(parts, shared_bucket=shared_bucket) @@ -241,6 +249,8 @@ def member_selection( if not relevant: return [] argv = ["--home"] + if people_changed: + argv.append("--members") for member in members: argv += ["--member", member] for topic in topics: diff --git a/stacklets/memory/cli/prompt.py b/stacklets/memory/cli/prompt.py index d41bb221..33b1483b 100644 --- a/stacklets/memory/cli/prompt.py +++ b/stacklets/memory/cli/prompt.py @@ -29,9 +29,11 @@ get_ontology, load_correspondents_from_vault, load_persons_from_vault, + people_prompt_section, persons_prompt_section, vault_path_for, ) +from stack.people import load_people # noqa: E402 HELP = "Show the vocabulary the models are fed: topics, family, correspondents" @@ -65,8 +67,22 @@ def run(args, stacklet, config): # Person pages are generated projections and live in the brain. data_dir = (config or {}).get("data_dir") brain = brain_path_for(Path(data_dir)) if data_dir else None - persons = persons_prompt_section( - load_persons_from_vault(brain, bucket) if brain and brain.exists() else []) + # The person files are what the classifier and the diary are given + # once they exist; the person pages are the fallback before. A file + # the readers skip is listed under the block, since this is where an + # admin looks when a name is not recognised. + problems: list[str] = [] + people = load_people(vault, bucket, report=problems.append) + if people is not None: + persons = people_prompt_section(people) + persons_used_by = "document classifier, captures, diary cards and transcription" + else: + persons = persons_prompt_section( + load_persons_from_vault(brain, bucket) if brain and brain.exists() else []) + persons_used_by = ("diary cards (the document classifier looks in the vault " + "and finds none)") + if problems: + persons += "\n\nSkipped, fix these files:\n" + "\n".join(f" - {p}" for p in problems) # So are correspondent pages. correspondents = correspondents_prompt_section( load_correspondents_from_vault(brain, bucket) if brain and brain.exists() else []) @@ -75,12 +91,11 @@ def run(args, stacklet, config): _block("Topics and document types", "document classifier, search query rewrite, diary cards", ontology, "(the ontology is empty)"), - # The document classifier reads both of these from the vault, + # The document classifier reads correspondents from the vault, # where generated pages no longer are, so today it receives - # neither. The labels say so until it reads the brain. - _block("Family members", - "diary cards (the document classifier looks in the vault and finds none)", - persons, "(no person pages yet)"), + # none. The label says so until it reads the brain. + _block("Family members", persons_used_by, + persons, "(no person files and no person pages yet)"), _block("Correspondents", "nothing yet (the document classifier looks in the vault and finds none)", correspondents, "(no correspondent pages yet)"), diff --git a/stacklets/memory/cli/wiki.py b/stacklets/memory/cli/wiki.py index 972d1dbb..aace73c6 100644 --- a/stacklets/memory/cli/wiki.py +++ b/stacklets/memory/cli/wiki.py @@ -5,6 +5,7 @@ stack memory wiki update --all the whole wiki, as at night: diary, sources, every page stack memory wiki update --home just the household home page stack memory wiki update --member homer just one member's page + stack memory wiki update --members every person's page, as the person files declare them stack memory wiki update --topic camping just one topic's page stack memory wiki update --topics every topic page, no home or members stack memory wiki update --dry-run preview home, member and topic pages, write nothing @@ -37,12 +38,12 @@ VERBS = { "update": "bring the latest filings into the wiki; --all rebuilds all of it, " - "--home/--member/--topic/--topics regenerate those pages", + "--home/--member/--members/--topic/--topics regenerate those pages", "clean": "delete every generated page (asks first)", } # Flags that name pages to regenerate directly, rather than asking the curator. -PAGE_FLAGS = ("--home", "--member", "--topic", "--topics", "--dry-run", "--dry") +PAGE_FLAGS = ("--home", "--member", "--members", "--topic", "--topics", "--dry-run", "--dry") def run(args, stacklet, config): diff --git a/stacklets/memory/cli/write.py b/stacklets/memory/cli/write.py index f8da7bf8..fcbde641 100644 --- a/stacklets/memory/cli/write.py +++ b/stacklets/memory/cli/write.py @@ -62,6 +62,7 @@ from stack.frontmatter import FrontmatterError, parse as parse_frontmatter # noqa: E402 from stack.links import go_page, public # noqa: E402 +from stack.people import PEOPLE_DIR, PersonError, read_person # noqa: E402 from stack.list_doc import diff # noqa: E402 from stack.page_patch import apply_edits # noqa: E402 @@ -94,6 +95,24 @@ def _check_frontmatter(before: str, after: str, page: str) -> None: "frontmatter block exactly as you read it.") from e +def _check_person(after: str, page: str) -> None: + """Reject a person file the readers would skip. + + Every reader leaves out a file that breaks the person schema, so + the write is where the family hears about it, not a log. + """ + parts = page.split("/") + if len(parts) != 3 or parts[1] != PEOPLE_DIR: + return + try: + read_person(Path(page).stem, after) + except PersonError as e: + raise ValueError( + f"{page} is a person file and {e}. It needs `type: person`, " + "a `title` with the full name and, optionally, `aliases` as a " + "list.") from e + + def run(args, stacklet, config): argv = list(args or []) actor, argv = _opt(argv, "--by") @@ -149,6 +168,7 @@ def _replace(prior: str) -> str: # seam sees every mutation, so the check lives here. ValueError # becomes the error message the caller relays to the model. _check_frontmatter(seen["before"], seen["after"], repo_path) + _check_person(seen["after"], repo_path) # A whole-page rewrite of a list is a restructure. A restructure # never legitimately reopens or removes an item (measured # 2026-09-15: rewrites drop [x] marks, and prompt rules do not diff --git a/stacklets/memory/hooks/on_start_ready.py b/stacklets/memory/hooks/on_start_ready.py index 5f432364..b9d5b7f3 100644 --- a/stacklets/memory/hooks/on_start_ready.py +++ b/stacklets/memory/hooks/on_start_ready.py @@ -25,6 +25,7 @@ authenticated_remote, brain_path_for, ensure_brain_projection_admin, + ensure_people, ensure_vault_cloned, host_code_url, point_remote_at, @@ -35,6 +36,30 @@ vault_path_for, vault_remote_url, ) +from stack.forgejo import ForgejoClient, ForgejoError # noqa: E402 +from stack.vault import DEFAULT_SHARED_BUCKET # noqa: E402 + + +def _ensure_people(ctx, code_url: str, token: str) -> None: + """Add a person file for every account that has none. + + Runs before the clone or pull, so the files are on disk when this + hook returns. A failure is reported and never blocks startup: every + reader keeps its previous behaviour until the files exist, and the + next `stack up memory` tries again. + """ + if not (code_url and token): + return + bucket = ctx.env.get("SHARED_BUCKET") or DEFAULT_SHARED_BUCKET + try: + added = ensure_people( + ForgejoClient(url=code_url, token=token), ctx.users, shared_bucket=bucket) + except (ForgejoError, OSError) as e: + # OSError: urllib's URLError when Forgejo is not reachable. + ctx.warn(f"Memory: could not write the person files: {e}") + return + if added: + ctx.step(f"Memory: added {', '.join(added)} to {bucket}/people/") def run(ctx): @@ -83,11 +108,14 @@ def remote_for(tok: str) -> str: if reason: if fresh := reissue_write_token(code_url, admin_user, admin_password): ctx.secret("MEMORY_BOT_TOKEN", fresh) + token = fresh remote = remote_for(fresh) ctx.step(f"{reason}; issued a new one") else: ctx.step(f"{reason} and it could not be replaced") + _ensure_people(ctx, code_url, token) + # If the vault never got cloned (install hook ran before code # stacklet was reachable, for example), try once more here. This # is the recovery path — same idempotent shape as install. diff --git a/stacklets/memory/lib.py b/stacklets/memory/lib.py index 02a31159..6fa38155 100644 --- a/stacklets/memory/lib.py +++ b/stacklets/memory/lib.py @@ -1290,6 +1290,25 @@ def persons_prompt_section(persons: List[Person]) -> str: return "\n".join(lines) +def people_prompt_section(people) -> str: + """The persons block from the person files, in the shape above. + + The canonical name is the person's first name, the name their + `Person:` tag carries; the full name and every alias the family + listed follow in parens, so "Margaret" on a birth certificate is + recognised as Maggie. A person without a chat account is listed like + any other. + """ + if not people: + return "" + lines = ["Family members (canonical first name; synonyms in parens):"] + for p in people: + others = [n for n in p.all_names() if n != p.first_name] + lines.append(f" - {p.first_name} ({', '.join(others)})" if others + else f" - {p.first_name}") + return "\n".join(lines) + + def get_ontology(vault_path: Optional[Path] = None) -> Ontology: """Return the live ontology from the vault, else the shipped seed. @@ -1568,6 +1587,49 @@ def install_seeds( return {"created": created, "skipped": skipped} +def ensure_people( + client: ForgejoClient, + users: list[dict], + *, + shared_bucket: str = DEFAULT_SHARED_BUCKET, +) -> list[str]: + """Add a person file for every account that has none; return their ids. + + The family's people are one file each under `<shared_bucket>/people/` + (`stack.people`). Every account holder gets one, marked with their + account, which makes them a member. An existing file is never read + or rewritten here, so a family edit (an alias, a corrected name) + survives every restart; a new account gets its file on the next + `stack up memory`. + + That makes this the whole upgrade path for an existing install: a + restart of the memory stacklet writes the files, and every reader + switches from its old guess to them as soon as they appear. + """ + from stack.forgejo import FileChange + from stack.people import PEOPLE_DIR, people_from_accounts, person_id, person_path, render_person + + accounts = people_from_accounts(users) + if not accounts: + return [] + listing = client.list_dir(REPO_OWNER, REPO_NAME, f"{shared_bucket}/{PEOPLE_DIR}") + existing = { + person_id(Path(entry["path"]).stem) + for entry in listing + if entry.get("type") == "file" and str(entry.get("path", "")).endswith(".md") + } + new = [p for p in accounts if p.id not in existing] + if not new: + return [] + client.change_files( + REPO_OWNER, REPO_NAME, + [FileChange("create", person_path(p.id, shared_bucket), render_person(p)) for p in new], + message="people: add a file for each chat account", + author_name=BOT_USERNAME, author_email=BOT_EMAIL, + ) + return [p.id for p in new] + + # Seed paths beginning with `_shared/` get retargeted to the # configured bucket slug. Everything else lands at the vault root # verbatim — keeps `ontology.toml`, `facts.toml`, `README.md` at the diff --git a/stacklets/memory/quartz/components/familyModel.ts b/stacklets/memory/quartz/components/familyModel.ts index fecf83dd..f3ad2906 100644 --- a/stacklets/memory/quartz/components/familyModel.ts +++ b/stacklets/memory/quartz/components/familyModel.ts @@ -63,6 +63,7 @@ const LABELS = { topic: "Topic", sender: "Sender", filedBy: "filed by", + alsoKnownAs: "Also known as", updated: "updated", familyWiki: "family wiki", welcome: "Everything the family has kept: documents, notes, the diary, and the people they belong to.", @@ -106,6 +107,7 @@ const LABELS = { topic: "Thema", sender: "Absender", filedBy: "abgelegt von", + alsoKnownAs: "Auch bekannt als", updated: "aktualisiert", familyWiki: "Familienwiki", welcome: "Alles, was die Familie aufbewahrt: Dokumente, Notizen, das Tagebuch und die Menschen, zu denen sie gehören.", @@ -295,6 +297,26 @@ interface Topic { const fm = (f: QuartzPluginData) => (f.frontmatter ?? {}) as Record<string, any> const asList = (v: unknown): string[] => (Array.isArray(v) ? v.map(String) : v ? [String(v)] : []) +// The other names a person or sender goes by: `synonyms` on a person page +// (the household page's aliases, then what documents called them), +// `aliases` on a correspondent. A name made only of words already in the +// title says nothing new ("Maggie" under "Maggie Simpson") and is left out. +const otherNames = (m: Record<string, any>): string[] => { + const title = String(m.title ?? m.canonical ?? "").toLowerCase() + const titleWords = new Set(title.split(/\s+/).filter(Boolean)) + const seen = new Set<string>() + return [...asList(m.synonyms), ...asList(m.aliases)].filter((name) => { + const key = name.trim().toLowerCase() + if (!key || seen.has(key)) return false + seen.add(key) + return !key.split(/\s+/).every((w) => titleWords.has(w)) + }) +} +const alsoKnownAs = (m: Record<string, any>): string[] => { + const names = otherNames(m) + return names.length ? [`${L.alsoKnownAs} ${names.join(", ")}`] : [] +} + function parseDate(v: unknown): Date | undefined { if (!v) return undefined const d = new Date(String(v)) @@ -726,8 +748,8 @@ function build(ctx: BuildCtx, allFiles: QuartzPluginData[]): Model { } } const updated = f?.dates?.modified - if (m.type === "person") return { kicker: [L.person], meta: [], date: updated, chips: [] } - if (m.type === "correspondent") return { kicker: [L.sender], meta: [], date: updated, chips: [] } + if (m.type === "person") return { kicker: [L.person], meta: alsoKnownAs(m), date: updated, chips: [] } + if (m.type === "correspondent") return { kicker: [L.sender], meta: alsoKnownAs(m), date: updated, chips: [] } if (m.type === "topic") { const t = [...topics.values()].find((x) => x.slug === slug) const area = onto.areas.find((a) => a.id === t?.area) diff --git a/stacklets/memory/quartz/quartz.config.ts b/stacklets/memory/quartz/quartz.config.ts index 98db3973..9e3f6b2a 100644 --- a/stacklets/memory/quartz/quartz.config.ts +++ b/stacklets/memory/quartz/quartz.config.ts @@ -60,9 +60,11 @@ const config: QuartzConfig = { // `**/diary/entries/**` keeps the diary's records off the site. They // are the raw source the diary pages are compiled from, and stay in // the brain for search and the agent; the family reads the pages. + // `*/people/*.md` keeps the declared person files off the site for + // the same reason: each person's page is compiled from theirs. ignorePatterns: [".git", ".obsidian", "private", "templates", "README.md", "**/README.md", "media/**/*.json", "media/**/.tmp-*", - "**/diary/entries/**"], + "**/diary/entries/**", "*/people/*.md"], defaultDateType: "modified", theme: { // Self-hosted. The comment above about analytics applies with more diff --git a/tests/unit/framework/test_people.py b/tests/unit/framework/test_people.py new file mode 100644 index 00000000..7b0a351c --- /dev/null +++ b/tests/unit/framework/test_people.py @@ -0,0 +1,175 @@ +"""The family's people: one declared file each, read strictly. + +`<shared_bucket>/people/<id>.md` is the only place the family says who +it knows (docs/design/brain/vault-format.md §5, `person` (declared)). +These tests pin what the readers depend on: the format, that a member +is a person with an account, that a broken file is reported by path +and never guessed at, and that "no people" is a different answer from +"these people". +""" + +from __future__ import annotations + +from stack.people import ( + Person, + find_people, + load_people, + members, + people_dir, + people_from_accounts, + read_person, + render_person, + resolve, +) + + +def _write(root, name: str, text: str, bucket: str = "family") -> None: + folder = people_dir(root, bucket) + folder.mkdir(parents=True, exist_ok=True) + (folder / name).write_text(text, encoding="utf-8") + + +HOMER = "---\ntype: person\ntitle: Homer Simpson\naccount: homer\n---\n\n# Homer Simpson\n" +MAGGIE = ("---\ntype: person\ntitle: Maggie Simpson\naliases:\n - Margaret\n" + " - Mags\n---\n\nSays her first word in season four.\n") + + +class TestReadingAPersonFile: + def test_the_file_name_is_the_id_and_the_frontmatter_the_names(self): + assert read_person("maggie", MAGGIE) == Person( + "maggie", "Maggie Simpson", ("Margaret", "Mags")) + + def test_a_person_with_an_account_is_a_member(self): + """The first iteration's guess at membership: an account.""" + assert read_person("homer", HOMER).member + + def test_a_person_without_an_account_is_known_but_not_a_member(self): + assert not read_person("maggie", MAGGIE).member + + def test_the_body_is_the_familys_and_never_read(self): + assert read_person("maggie", MAGGIE).aliases == ("Margaret", "Mags") + + def test_every_name_a_person_answers_to(self): + """Full name, first name, then aliases: what the classifier and + transcription listen for.""" + assert read_person("maggie", MAGGIE).all_names() == [ + "Maggie Simpson", "Maggie", "Margaret", "Mags"] + + def test_a_file_named_in_obsidian_still_names_a_person(self): + text = "---\ntype: person\ntitle: Abe Simpson\n---\n" + assert read_person("Grampa Simpson", text).id == "grampa-simpson" + + +class TestABrokenFileIsReportedNotGuessed: + """A family edits these by hand. A mistake must say where it is.""" + + def _problems(self, tmp_path, text: str) -> list[str]: + _write(tmp_path, "homer.md", HOMER) + _write(tmp_path, "maggie.md", text) + problems: list[str] = [] + people = load_people(tmp_path, report=problems.append) + assert [p.id for p in people or []] == ["homer"] + return problems + + def test_frontmatter_outside_the_format(self, tmp_path): + problems = self._problems( + tmp_path, "---\ntype: person\ntitle: Maggie\naliases: |\n Margaret\n---\n") + assert problems and problems[0].startswith("family/people/maggie.md: ") + + def test_one_alias_where_a_list_belongs(self, tmp_path): + problems = self._problems( + tmp_path, "---\ntype: person\ntitle: Maggie Simpson\naliases: Margaret\n---\n") + assert "`aliases` must be a list" in problems[0] + + def test_a_file_without_a_name(self, tmp_path): + problems = self._problems(tmp_path, "---\ntype: person\naliases:\n - Margaret\n---\n") + assert "`title`" in problems[0] + + def test_a_note_that_is_not_a_person(self, tmp_path): + problems = self._problems(tmp_path, "Ideas for Maggie's birthday\n") + assert "type: person" in problems[0] + + def test_bytes_that_are_not_text(self, tmp_path): + _write(tmp_path, "homer.md", HOMER) + (people_dir(tmp_path) / "maggie.md").write_bytes(b"---\ntype: \xff\xfe\n---\n") + problems: list[str] = [] + assert [p.id for p in load_people(tmp_path, report=problems.append)] == ["homer"] + assert "cannot be read" in problems[0] + + def test_an_id_declared_twice_counts_once(self, tmp_path): + _write(tmp_path, "homer.md", HOMER) + _write(tmp_path, "Homer!.md", HOMER) + problems: list[str] = [] + assert len(load_people(tmp_path, report=problems.append)) == 1 + assert "declared twice" in problems[0] + + +class TestNoPeopleIsNotAnEmptyHousehold: + def test_no_folder_reads_as_none(self, tmp_path): + """Readers keep their previous behaviour until the files exist.""" + assert load_people(tmp_path) is None + + def test_a_folder_of_broken_files_reads_as_none(self, tmp_path): + """Nobody readable is never acted on as "nobody".""" + _write(tmp_path, "maggie.md", "not a person\n") + assert load_people(tmp_path) is None + + def test_the_files_live_in_the_shared_bucket(self, tmp_path): + _write(tmp_path, "homer.md", HOMER, bucket="familie") + assert [p.id for p in load_people(tmp_path, "familie")] == ["homer"] + assert load_people(tmp_path) is None + + def test_the_first_root_with_people_wins(self, tmp_path): + """The working copy is fresher than the brain mirror of it.""" + vault, brain = tmp_path / "vault", tmp_path / "brain" + _write(brain, "homer.md", HOMER) + assert [p.id for p in find_people((None, vault, brain))] == ["homer"] + _write(vault, "maggie.md", MAGGIE) + assert [p.id for p in find_people((vault, brain))] == ["maggie"] + + +class TestWhoIsWho: + def test_members_come_first_then_everyone_else_by_name(self, tmp_path): + _write(tmp_path, "maggie.md", MAGGIE) + _write(tmp_path, "marge.md", "---\ntype: person\ntitle: Marge Simpson\naccount: marge\n---\n") + _write(tmp_path, "homer.md", HOMER) + people = load_people(tmp_path) + assert [p.id for p in people] == ["homer", "marge", "maggie"] + assert [p.id for p in members(people)] == ["homer", "marge"] + + def test_a_name_resolves_by_any_name_the_person_goes_by(self): + maggie = read_person("maggie", MAGGIE) + assert resolve("margaret", [maggie]) is maggie + assert resolve("Maggie Simpson", [maggie]) is maggie + assert resolve("Simpson", [maggie]) is None + + +class TestTheFilesFamstackWrites: + """`stack up memory` adds one for every account that has none.""" + + def test_every_account_holder_becomes_a_member(self): + users = [ + {"name": "Homer Simpson", "role": "admin"}, + {"name": "Marge Simpson"}, + {"name": "Bart", "id": "bart"}, + ] + assert people_from_accounts(users) == [ + Person("homer", "Homer Simpson", account="homer"), + Person("marge", "Marge Simpson", account="marge"), + Person("bart", "Bart", account="bart"), + ] + + def test_the_id_is_the_chat_user_name(self): + """An explicit `id` in users.toml is the Matrix localpart; so is the person id.""" + assert people_from_accounts([{"name": "Homer Simpson", "id": "hjs"}])[0].id == "hjs" + + def test_an_entry_without_a_name_is_skipped(self): + assert people_from_accounts([{"id": "ghost"}]) == [] + + def test_a_written_file_reads_back_as_the_same_person(self): + homer = Person("homer", "Homer Simpson", ("Homie",), account="homer") + assert read_person("homer", render_person(homer)) == homer + + def test_a_written_file_follows_the_vault_format(self): + page = render_person(Person("homer", "Homer Simpson", account="homer")) + assert page.startswith("---\ntype: person\ntitle: Homer Simpson\naccount: homer\n---\n") diff --git a/tests/unit/stacklets/test_frontmatter.py b/tests/unit/stacklets/test_frontmatter.py index 93523fbd..c948eb8e 100644 --- a/tests/unit/stacklets/test_frontmatter.py +++ b/tests/unit/stacklets/test_frontmatter.py @@ -322,16 +322,35 @@ def test_record_carrying_generated_error(self): assert any("must not carry `generated`" in e for e in errors) def test_projection_missing_generated_error(self): - """Validator rejects projection type (person) missing generated marker.""" + """Validator rejects projection type (correspondent) missing generated marker.""" fm = { - "type": "person", - "title": "Homer", - "slug": "homer", - "canonical": "Homer", + "type": "correspondent", + "title": "Duff Insurance", + "canonical": "Duff Insurance", } errors = validate(fm) assert any("must have `generated: true`" in e for e in errors) + def test_a_person_without_the_marker_is_the_familys_own_file(self): + """vault-format §5, `person` (declared): type and title, nothing generated.""" + fm = {"type": "person", "title": "Maggie Simpson", + "aliases": ["Margaret"], "account": "maggie"} + assert validate(fm) == [] + + def test_a_declared_person_needs_a_title(self): + errors = validate({"type": "person", "aliases": ["Margaret"]}) + assert errors == ["missing required field for `person`: `title`"] + + def test_a_declared_persons_aliases_are_a_list(self): + """`aliases: Margaret` is one name where the format asks for a list.""" + errors = validate({"type": "person", "title": "Maggie Simpson", + "aliases": "Margaret"}) + assert any("`aliases` must be a list" in e for e in errors) + + def test_a_declared_persons_name_is_one_value(self): + errors = validate({"type": "person", "title": ["Maggie", "Margaret"]}) + assert any("`title` must be a single value" in e for e in errors) + def test_list_field_not_list(self): """Validator rejects list field that is a scalar.""" fm = { diff --git a/tests/unit/stacklets/test_memory_diary_people.py b/tests/unit/stacklets/test_memory_diary_people.py index 39f83c03..d943477c 100644 --- a/tests/unit/stacklets/test_memory_diary_people.py +++ b/tests/unit/stacklets/test_memory_diary_people.py @@ -3,8 +3,9 @@ Whisper decodes a voice note against the family's names, and the diary cards list the people a note is about from the same list. A name missing here is misheard ("Maggie" as "Eggie") and left off the card. So the list -starts from everyone in users.toml, rendered into FAMILY_NAMES, and the -wiki's person pages add the family's own spellings and nicknames. +starts from the person files (`family/people/`), or from everyone in +users.toml (FAMILY_NAMES) until they exist, and the wiki's person pages +add the spellings documents used. """ from __future__ import annotations @@ -56,3 +57,77 @@ def test_a_member_without_a_page_is_still_someone_a_card_can_name(self): def test_before_any_page_the_household_alone(self): assert diary.card_people([], ["Homer", "Bart"]) == {"homer": "Homer", "bart": "Bart"} + + +# ── With person files ────────────────────────────────────────────── + +def _diary_command(): + """Load `bot/cli/diary.py` as part of its package, under a name of its own. + + Several stacklets ship a `cli` package; importing this one as `cli` + would shadow theirs for every later test. + """ + import importlib + import importlib.util + # In the bot-runner, `voice` and the framework sit at /app. + sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "core" / "bot-runner")) + sys.path.insert(0, str(_REPO_ROOT / "stacklets")) + name = "memory_bot_cli" + if name not in sys.modules: + pkg_dir = _REPO_ROOT / "stacklets" / "memory" / "bot" / "cli" + spec = importlib.util.spec_from_file_location( + name, pkg_dir / "__init__.py", submodule_search_locations=[str(pkg_dir)]) + package = importlib.util.module_from_spec(spec) + sys.modules[name] = package + spec.loader.exec_module(package) + return importlib.import_module(f"{name}.diary") + + +HOMER = "---\ntype: person\ntitle: Homer Simpson\naccount: homer\n---\n" +MAGGIE = "---\ntype: person\ntitle: Maggie Simpson\naliases:\n - Margaret\n---\n" + + +class TestThePersonFilesDecideWhoTheDiaryKnows: + """Once person files exist, the diary listens for everyone they declare. + + The accounts (FAMILY_NAMES) only stand in until then. Maggie has no + account; her file makes her known, and her aliases are names whisper + should expect and a card can use. + """ + + @staticmethod + def _instance(tmp_path, monkeypatch, people: dict[str, str] | None): + vault, brain = tmp_path / "vault", tmp_path / "brain" + (vault / "family").mkdir(parents=True) + (brain / "media").mkdir(parents=True) + (brain / "media" / "about.md").write_text( + "---\ntitle: Margaret\nslug: media\ntype: person\ncanonical: Margaret\n---\n") + for person_id, text in (people or {}).items(): + (vault / "family" / "people").mkdir(parents=True, exist_ok=True) + (vault / "family" / "people" / f"{person_id}.md").write_text(text) + monkeypatch.setenv("MEMORY_VAULT_DIR", str(vault)) + monkeypatch.setenv("BRAIN_REPO_DIR", str(brain)) + monkeypatch.setenv("SHARED_BUCKET", "family") + monkeypatch.setenv("FAMILY_NAMES", "Homer Simpson,Marge Simpson") + return _diary_command() + + def test_before_the_files_the_accounts_and_pages_as_before(self, tmp_path, monkeypatch): + cli = self._instance(tmp_path, monkeypatch, None) + assert cli._household_people() == ["Homer Simpson", "Marge Simpson", "Margaret"] + + def test_with_files_every_person_and_every_name_they_go_by(self, tmp_path, monkeypatch): + cli = self._instance(tmp_path, monkeypatch, {"homer": HOMER, "maggie": MAGGIE}) + assert cli._household_people() == [ + "Homer Simpson", "Homer", "Maggie Simpson", "Maggie", "Margaret"] + + def test_a_page_left_over_for_a_folder_is_not_family(self, tmp_path, monkeypatch): + """The old roster made `media/` a person page; it no longer counts.""" + cli = self._instance(tmp_path, monkeypatch, {"homer": HOMER}) + assert cli._household_people() == ["Homer Simpson", "Homer"] + + def test_a_card_about_margaret_is_about_maggie(self, tmp_path, monkeypatch): + cli = self._instance(tmp_path, monkeypatch, {"homer": HOMER, "maggie": MAGGIE}) + _ontology, people, section, _lang = cli._vocabulary("family") + assert people["margaret"] == "Maggie" + assert people["maggie simpson"] == "Maggie" + assert " - Maggie (Maggie Simpson, Margaret)" in section diff --git a/tests/unit/stacklets/test_memory_people.py b/tests/unit/stacklets/test_memory_people.py new file mode 100644 index 00000000..c7bbb44e --- /dev/null +++ b/tests/unit/stacklets/test_memory_people.py @@ -0,0 +1,348 @@ +"""The memory stacklet and the family's person files. + +Every person the family knows has a file, `family/people/<id>.md` +(`stack.people`). This file pins how the memory stacklet treats them: + +- `stack up memory` adds a file for every account that has none, on + every start, and never rewrites one: the restart of an existing + install is the upgrade, and the family's edits survive every restart. +- The wiki gives every declared person a page under the names their + file gives, and its home page lists the members (people with an + account). Before any file exists the wiki keeps its old roster, so + nothing changes until the restart. +- Generated person pages nobody declares (`media/about.md`, from the + old roster) are retired, but never while a person file is broken. +- A change to a person file rebuilds every person page. + +Forgejo is stubbed with `pytest-httpserver`; the wiki works on real +files in a temporary brain and vault. +""" + +from __future__ import annotations + +import asyncio +import base64 +import json +import sys +from pathlib import Path + +import pytest +import yaml + +_REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent +sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "memory")) +sys.path.insert(0, str(_REPO_ROOT / "stacklets")) +sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "memory" / "bot")) +sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "memory" / "bot" / "cli")) + +from lib import REPO_NAME, REPO_OWNER, ensure_people # noqa: E402 +from stack.forgejo import ForgejoClient # noqa: E402 +from stack.people import Person, read_person # noqa: E402 + +import wiki # noqa: E402 +from curator import member_selection # noqa: E402 + +CONTENTS = f"/api/v1/repos/{REPO_OWNER}/{REPO_NAME}/contents" +PEOPLE = f"{CONTENTS}/family/people" + +SIMPSONS = [ + {"name": "Homer Simpson", "role": "admin"}, + {"name": "Marge Simpson"}, + {"name": "Bart Simpson"}, + {"name": "Lisa Simpson"}, +] + +HOMER = "---\ntype: person\ntitle: Homer Simpson\naccount: homer\n---\n" +MARGE = "---\ntype: person\ntitle: Marge Simpson\naccount: marge\n---\n" +MAGGIE = "---\ntype: person\ntitle: Maggie Simpson\naliases:\n - Margaret\n---\n" + + +def _client(httpserver) -> ForgejoClient: + return ForgejoClient(url=httpserver.url_for(""), token="bottoken") + + +def _listing(*names: str) -> list[dict]: + return [{"type": "file", "path": f"family/people/{n}", "sha": "abc"} for n in names] + + +def _committed(httpserver) -> dict[str, str]: + """The files the one commit created, path to content.""" + post = next(r for r, _ in httpserver.log if r.method == "POST") + return {f["path"]: base64.b64decode(f["content"]).decode() + for f in json.loads(post.data)["files"]} + + +# ── Writing the files on start ─────────────────────────────────────────── + +class TestTheStartAddsAFilePerAccount: + def test_an_install_without_files_gets_one_per_account_in_one_commit(self, httpserver): + httpserver.expect_request(PEOPLE, method="GET").respond_with_data("", status=404) + httpserver.expect_request(CONTENTS, method="POST").respond_with_json({}, status=201) + + added = ensure_people(_client(httpserver), SIMPSONS) + + assert added == ["homer", "marge", "bart", "lisa"] + files = _committed(httpserver) + assert sorted(files) == [f"family/people/{i}.md" for i in ("bart", "homer", "lisa", "marge")] + homer = read_person("homer", files["family/people/homer.md"]) + assert homer == Person("homer", "Homer Simpson", account="homer") + assert homer.member + + def test_an_existing_file_is_never_rewritten(self, httpserver): + """The family gave Homer an alias; a restart adds only who is missing.""" + httpserver.expect_request(PEOPLE, method="GET").respond_with_json( + _listing("homer.md", "marge.md", "bart.md", "maggie.md")) + httpserver.expect_request(CONTENTS, method="POST").respond_with_json({}, status=201) + + assert ensure_people(_client(httpserver), SIMPSONS) == ["lisa"] + assert list(_committed(httpserver)) == ["family/people/lisa.md"] + + def test_nothing_missing_commits_nothing(self, httpserver): + httpserver.expect_request(PEOPLE, method="GET").respond_with_json( + _listing("homer.md", "marge.md", "bart.md", "lisa.md")) + + assert ensure_people(_client(httpserver), SIMPSONS) == [] + assert not [r for r, _ in httpserver.log if r.method == "POST"] + + +# ── The wiki's people ──────────────────────────────────────────────────── + +@pytest.fixture +def instance(tmp_path, monkeypatch): + """A vault and brain holding what the old roster got wrong. + + A `media/` folder (diary recordings), a bot's bucket, documents that + call the baby "Margaret" and name a surname, and a generated person + page for `media/` left over from the old roster. + """ + vault, brain = tmp_path / "vault", tmp_path / "brain" + for root in (vault, brain): + for folder in ("homer", "marge", "media", "mail-bot", "family/documents"): + (root / folder).mkdir(parents=True, exist_ok=True) + (brain / "media" / "about.md").write_text( + "---\ntitle: Media\nslug: media\ntype: person\ngenerated: true\n---\n\n" + "<!-- begin: generated -->\n\nMaggie's page, under the wrong folder.\n\n" + "<!-- end: generated -->\n") + monkeypatch.setenv("MEMORY_VAULT_DIR", str(vault)) + monkeypatch.setenv("BRAIN_REPO_DIR", str(brain)) + index = [ + {"rel": "family/documents/a.md", "persons": ["margaret"], + "person_names": ["Margaret"], "title": "Birth certificate"}, + {"rel": "family/documents/b.md", "persons": ["simpson", "homer"], + "person_names": ["Simpson", "Homer"], "title": "Car insurance"}, + ] + return vault, brain, index + + +def _declare(root: Path, **files: str) -> None: + folder = root / "family" / "people" + folder.mkdir(parents=True, exist_ok=True) + for person_id, text in files.items(): + (folder / f"{person_id}.md").write_text(text) + + +def _people(problems: list[str] | None = None): + return wiki._people("family", problems if problems is not None else []) + + +class TestTheWikisPeople: + def test_before_any_file_exists_the_old_roster_stands(self, instance): + """Nothing changes until `stack up memory` has written the files.""" + vault, brain, index = instance + people = _people() + assert people is None + assert wiki._roster(brain, index, "family", people) == wiki._member_slugs( + brain, index, "family") + + def test_every_declared_person_has_a_page_members_first(self, instance): + vault, brain, index = instance + _declare(vault, homer=HOMER, marge=MARGE, maggie=MAGGIE) + assert wiki._roster(brain, index, "family", _people()) == ["homer", "marge", "maggie"] + + def test_names_resolve_through_the_familys_aliases(self, instance): + """The birth certificate says "Margaret"; it belongs on Maggie's page.""" + vault, brain, index = instance + _declare(vault, homer=HOMER, maggie=MAGGIE) + wiki._resolve_persons(index, _people()) + assert index[0]["persons"] == ["maggie"] + assert wiki._member_entries(index, "maggie") == [index[0]] + + def test_a_name_nobody_answers_to_gets_no_page(self, instance): + vault, brain, index = instance + people = [Person("homer", "Homer Simpson", account="homer")] + wiki._resolve_persons(index, people) + assert "simpson" not in wiki._roster(brain, index, "family", people) + + def test_the_files_are_read_from_the_vault_before_the_brain(self, instance): + """An edit is on disk in the vault before the curator mirrors it.""" + vault, brain, index = instance + _declare(brain, homer=HOMER) + _declare(vault, homer=HOMER, maggie=MAGGIE) + assert [p.id for p in _people()] == ["homer", "maggie"] + + def test_a_broken_file_is_reported_and_left_out(self, instance): + vault, brain, index = instance + _declare(vault, homer=HOMER, maggie="---\ntype: person\naliases: Margaret\n---\n") + problems: list[str] = [] + assert [p.id for p in _people(problems)] == ["homer"] + assert problems[0].startswith("family/people/maggie.md: ") + + def test_member_option_takes_any_name_the_person_answers_to(self): + people = [Person("maggie", "Maggie Simpson", ("Margaret",))] + assert wiki._member_id("Margaret", people) == "maggie" + assert wiki._member_id("Maggie Simpson", people) == "maggie" + + +class TestEveryDeclaredPersonHasAPage: + def test_a_person_nothing_names_yet_gets_a_page_without_the_model(self, instance): + """The home page and the People menu link every person's page.""" + vault, brain, index = instance + maggie = Person("maggie", "Maggie Simpson", ("Margaret",)) + llm = None # never called: there is nothing for it to write about + + rc = asyncio.run(wiki._generate_member( + llm, "maggie", [], brain, person=maggie, + shared_bucket="family", lang="en", write=True)) + + page = (brain / "maggie" / "about.md").read_text() + assert rc == 0 + assert yaml.safe_load(page.split("---\n")[1])["canonical"] == "Maggie Simpson" + assert "Nothing filed about Maggie yet" in page + + def test_without_person_files_an_empty_bucket_is_still_skipped(self, instance): + vault, brain, index = instance + rc = asyncio.run(wiki._generate_member( + None, "media", [], brain, shared_bucket="family", lang="en", write=True)) + assert rc == 1 + + +class TestUndeclaredPersonPages: + def test_a_generated_page_nobody_declares_is_retired(self, instance): + vault, brain, index = instance + retired = wiki._retire_undeclared(["homer", "maggie"], "family", write=True) + assert retired == ["media/about.md"] + assert not (brain / "media" / "about.md").exists() + + def test_a_page_a_person_wrote_is_left_alone(self, instance): + vault, brain, index = instance + (brain / "media" / "about.md").write_text("---\ntitle: Media\n---\n\nOur photos.\n") + assert wiki._retire_undeclared(["homer"], "family", write=True) == [] + assert (brain / "media" / "about.md").exists() + + def test_a_dry_run_names_the_page_and_keeps_it(self, instance): + vault, brain, index = instance + assert wiki._retire_undeclared(["homer"], "family", write=False) == ["media/about.md"] + assert (brain / "media" / "about.md").exists() + + def test_no_page_is_retired_while_a_person_file_is_broken(self, instance, monkeypatch): + """Maggie's file has a typo: her page must survive until it is fixed.""" + vault, brain, index = instance + _declare(vault, homer=HOMER, maggie="---\ntype: person\naliases: Margaret\n---\n") + (brain / "maggie").mkdir() + (brain / "maggie" / "about.md").write_text( + "---\ntitle: Maggie Simpson\nslug: maggie\ntype: person\ngenerated: true\n---\n") + + async def _no_model(*args, **kwargs): + return 0 + monkeypatch.setattr(wiki, "_generate_home", _no_model) + monkeypatch.setattr(wiki, "_generate_member", _no_model) + monkeypatch.setattr(wiki, "_index_vault", lambda vault: index) + monkeypatch.setattr(wiki, "_brain_dir", lambda: brain) + + asyncio.run(wiki.run(None, ["--members"])) + + assert (brain / "maggie" / "about.md").exists() + assert (brain / "media" / "about.md").exists() + + +class TestPersonPageNames: + def test_the_declared_names_replace_what_the_page_had(self): + page = ("---\ntitle: Margaret\nslug: maggie\ntype: person\ncanonical: Margaret\n" + "synonyms:\n - Maggie\nbirthday: \"2025-01-12\"\n---\n\nbody\n") + updated = wiki._with_member_names( + page, canonical="Maggie Simpson", synonyms=["Margaret", "Maggie"]) + frontmatter = yaml.safe_load(updated.split("---\n")[1]) + assert frontmatter == { + "title": "Maggie Simpson", "canonical": "Maggie Simpson", + "synonyms": ["Margaret", "Maggie"], + "slug": "maggie", "type": "person", "birthday": "2025-01-12", + } + assert updated.endswith("---\n\nbody\n") + + def test_the_home_page_is_told_the_full_names(self): + prompt = wiki._build_home_prompt( + [], roster=["homer", "marge"], lang="en", + names={"homer": "Homer Simpson", "marge": "Marge Simpson"}) + assert "- **[Marge Simpson](marge/about)**" in prompt + assert "<full name of" not in prompt + + +# ── Rebuilding on a change ─────────────────────────────────────────────── + +class TestTheCuratorRebuildsOnAChange: + def test_a_changed_person_file_rebuilds_every_person_page(self): + argv = member_selection(["family/people/maggie.md"], lambda path: {}, + shared_bucket="family") + assert argv == ["--home", "--members"] + + def test_other_changes_select_as_before(self): + argv = member_selection(["homer/notes/2026/10/x.md"], lambda path: {}, + shared_bucket="family") + assert argv == ["--home", "--member", "homer"] + + +# ── What the archivist is told ─────────────────────────────────────────── + +class TestTheClassifierIsToldThePeople: + """The document classifier's persons block comes from the files once they exist.""" + + @staticmethod + def _vault_context(tmp_path, monkeypatch): + sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "docs" / "bot")) + from vault_context import VaultContext + monkeypatch.setenv("MEMORY_VAULT_DIR", str(tmp_path)) + (tmp_path / "family").mkdir(exist_ok=True) + return VaultContext(language="en", shared_bucket="family") + + def test_every_person_with_every_name(self, tmp_path, monkeypatch): + """Maggie has no account and is listed like anyone else.""" + vault = self._vault_context(tmp_path, monkeypatch) + _declare(tmp_path, homer=HOMER, maggie=MAGGIE) + assert vault.persons_section() == ( + "Family members (canonical first name; synonyms in parens):\n" + " - Homer (Homer Simpson)\n" + " - Maggie (Maggie Simpson, Margaret)") + + def test_without_the_files_the_old_block(self, tmp_path, monkeypatch): + """No files, no person pages in the vault: empty, and the prompt + falls back to the Person tags as before.""" + vault = self._vault_context(tmp_path, monkeypatch) + assert vault.people() is None + assert vault.persons_section() == "" + + +class TestThePromptCommandShowsThePeople: + """`stack memory prompt` is where an admin looks when a name is not recognised.""" + + @staticmethod + def _prompt(tmp_path, capsys) -> str: + import importlib.util + spec = importlib.util.spec_from_file_location( + "memory_cli_prompt", _REPO_ROOT / "stacklets" / "memory" / "cli" / "prompt.py") + prompt = importlib.util.module_from_spec(spec) + spec.loader.exec_module(prompt) + prompt.run([], None, {"data_dir": str(tmp_path)}) + return capsys.readouterr().out + + def test_the_block_the_classifier_receives(self, tmp_path, capsys): + _declare(tmp_path / "memory" / "vault", homer=HOMER, maggie=MAGGIE) + out = self._prompt(tmp_path, capsys) + assert " - Maggie (Maggie Simpson, Margaret)" in out + assert "Used by: document classifier" in out + + def test_a_skipped_file_is_named_with_the_reason(self, tmp_path, capsys): + _declare(tmp_path / "memory" / "vault", homer=HOMER, + maggie="---\ntype: person\ntitle: Maggie Simpson\naliases: Margaret\n---\n") + out = self._prompt(tmp_path, capsys) + assert "Skipped, fix these files:" in out + assert "family/people/maggie.md: field `aliases` must be a list" in out diff --git a/tests/unit/stacklets/test_memory_token_repair.py b/tests/unit/stacklets/test_memory_token_repair.py index 684fcfdb..49a26854 100644 --- a/tests/unit/stacklets/test_memory_token_repair.py +++ b/tests/unit/stacklets/test_memory_token_repair.py @@ -106,8 +106,9 @@ class _RecordingCtx: """The slice of hook context `on_start_ready` actually reads. A real `StackContext` wants an instance on disk, a parsed config and - a Docker runtime. The hook wants four things: the data dir, the - rendered env, the secret store, and somewhere to log progress and warnings. + a Docker runtime. The hook wants five things: the data dir, the + rendered env, the secret store, the accounts, and somewhere to log + progress and warnings. Standing up only those four is what lets the hook be driven end to end here, which matters because the bug this file guards lived in `run()` itself and not in any helper it calls. @@ -117,6 +118,7 @@ def __init__(self, data_dir: Path, env: dict, secrets: dict): self.stack = SimpleNamespace(data=data_dir) self.env = env self.secrets = secrets + self.users: list[dict] = [] self.steps: list[str] = [] def secret(self, name, value=None): diff --git a/tests/unit/stacklets/test_memory_write_cli.py b/tests/unit/stacklets/test_memory_write_cli.py index 4959b816..5637dfdb 100644 --- a/tests/unit/stacklets/test_memory_write_cli.py +++ b/tests/unit/stacklets/test_memory_write_cli.py @@ -321,3 +321,26 @@ def test_a_write_the_readers_have_not_caught_up_with_is_still_done(store, tmp_pa assert result["mirrored"] is False out = capsys.readouterr().out assert "Wrote" in out and "catch up" in out + + +# ── person files ───────────────────────────────────────────────────────── + +def test_a_person_file_the_readers_would_skip_is_refused(store, tmp_path): + """Every reader leaves out a broken person file; the write says so first. + + `aliases: Margaret` is one name where the format asks for a list + (vault-format §5, `person` (declared)). + """ + result = _run(store, "---\ntype: person\ntitle: Maggie Simpson\naliases: Margaret\n---\n", + path="family/people/maggie.md", tmp=tmp_path) + + assert "error" in result + assert "`aliases` must be a list" in result["error"] + assert store.commits == [] + + +def test_a_person_file_in_the_format_is_written(store, tmp_path): + result = _run(store, "---\ntype: person\ntitle: Maggie Simpson\naliases:\n" + " - Margaret\n---\n", path="family/people/maggie.md", tmp=tmp_path) + + assert result["committed"] is True diff --git a/tests/unit/stacklets/test_pipeline.py b/tests/unit/stacklets/test_pipeline.py index a150a72b..4cb52405 100644 --- a/tests/unit/stacklets/test_pipeline.py +++ b/tests/unit/stacklets/test_pipeline.py @@ -18,6 +18,7 @@ sys.path.insert(0, str(_REPO_ROOT / "lib")) sys.path.insert(0, str(_REPO_ROOT / "stacklets" / "docs" / "bot")) +from stack.people import Person # noqa: E402 from pipeline import ( # noqa: E402 LLMModelNotFoundError, LLMTimeoutError, @@ -635,6 +636,64 @@ async def test_unknown_person_not_created(self, seeded_paperless): # ── Fresh-reprocess semantics ───────────────────────────────────────────── +class TestEnrichWithThePersonFiles: + """With person files, people are filed by who they are, not by spelling. + + The files declare everyone the family knows, accounts or not, with + the names the family uses. A name resolves through those; a person + who has no `Person:` tag yet (no account, so none was seeded) gets + one the first time a document names them. Without the files nothing + changes. + """ + + PEOPLE = [ + Person("homer", "Homer Simpson", account="homer"), + Person("marge", "Marge Simpson", account="marge"), + Person("bart", "Bart Simpson", ("Bartholomew",), account="bart"), + Person("maggie", "Maggie Simpson", ("Margaret",)), + ] + + @pytest.mark.asyncio + async def test_an_alias_files_under_the_persons_tag(self, seeded_paperless): + classifier = StubClassifier(payload={"title": "t", "persons": ["Bartholomew"]}) + result = await enrich_document( + paperless=seeded_paperless, classifier=classifier, doc=_doc(), + people=self.PEOPLE, + ) + assert result.resolved_persons == ["Bart"] + + @pytest.mark.asyncio + async def test_a_person_without_an_account_gets_a_tag_when_first_named( + self, seeded_paperless): + """Maggie has no account and so no seeded tag; she is still family.""" + classifier = StubClassifier(payload={"title": "t", "persons": ["Margaret"]}) + result = await enrich_document( + paperless=seeded_paperless, classifier=classifier, doc=_doc(), + people=self.PEOPLE, + ) + assert result.resolved_persons == ["Maggie"] + assert ("Person: Maggie", "#2196f3") in seeded_paperless.created_tags + assert 'tag "Person: Maggie"' in result.created_new + + @pytest.mark.asyncio + async def test_a_name_nobody_answers_to_gets_no_person_tag(self, seeded_paperless): + classifier = StubClassifier(payload={"title": "t", "persons": ["Simpson"]}) + result = await enrich_document( + paperless=seeded_paperless, classifier=classifier, doc=_doc(), + people=self.PEOPLE, + ) + assert result.resolved_persons == [] + assert not seeded_paperless.created_tags + + @pytest.mark.asyncio + async def test_without_person_files_no_tag_is_created(self, seeded_paperless): + classifier = StubClassifier(payload={"title": "t", "persons": ["Margaret"]}) + await enrich_document( + paperless=seeded_paperless, classifier=classifier, doc=_doc(), + ) + assert not seeded_paperless.created_tags + + class TestEnrichFreshReprocess: """enrich_document treats each run as a full fresh classification: prior tags are dropped, prior document_type is overwritten.""" diff --git a/tools/family-memories/generate.py b/tools/family-memories/generate.py index 8e530cba..fd14e9c0 100644 --- a/tools/family-memories/generate.py +++ b/tools/family-memories/generate.py @@ -177,8 +177,14 @@ def main() -> None: ap.add_argument("--only", default=None, help="substring filter on item id") ap.add_argument("--locale", choices=["de", "en"], default=None, help="render one locale (default: all)") + ap.add_argument("--corpus", default=None, + help="render spec.<corpus>.yaml into out/<corpus>/ " + "instead of the locale specs, e.g. people.en") args = ap.parse_args() - locales = [args.locale] if args.locale else ["de", "en"] + if args.corpus: + locales = [args.corpus] + else: + locales = [args.locale] if args.locale else ["de", "en"] for locale in locales: render_locale(locale, args) diff --git a/tools/family-memories/ingest.py b/tools/family-memories/ingest.py index ba549597..4a28b642 100644 --- a/tools/family-memories/ingest.py +++ b/tools/family-memories/ingest.py @@ -157,12 +157,15 @@ def main(): ap.add_argument("--login", action="append", required=True, metavar="USER:PASSWORD") ap.add_argument("--locale", choices=["de", "en"], default="de") + ap.add_argument("--corpus", default=None, + help="replay out/<corpus>/ (from generate.py --corpus) " + "instead of out/<locale>/") ap.add_argument("--delay", type=float, default=2.0, help="seconds between non-burst items") ap.add_argument("--force-i-know", action="store_true") args = ap.parse_args() global OUT - OUT = OUT / args.locale + OUT = OUT / (args.corpus or args.locale) household = configured_household(REPO_ROOT) if not args.force_i_know and \ diff --git a/tools/family-memories/spec.people.en.yaml b/tools/family-memories/spec.people.en.yaml new file mode 100644 index 00000000..30ef8924 --- /dev/null +++ b/tools/family-memories/spec.people.en.yaml @@ -0,0 +1,637 @@ +# spec.people.en.yaml: two and a half years of a memories room, for +# measuring how the people around a family show up in it. +# +# The base corpus (spec.en.yaml) covers four weeks and exercises dating +# and threading. This one spreads ordinary memories from March 2024 to +# September 2026 and lets the people outside the household come and go +# the way they do in a real room: grandparents rarely but for years, a +# best friend all the time, a teacher for one school year, a doctor for +# one month, a bully often without being close. +# +# Render and replay: +# FAMSTACK_TTS_URL=https://<speech-host>/v1/audio/speech \ +# python tools/family-memories/generate.py --corpus people.en +# python tools/family-memories/ingest.py --corpus people.en ... +# +# Every item states its date in its text, with the year, because a +# replay lands on the day it runs (see README). Compile with the default +# burst window: items go out --delay apart and none of them is a burst. +# +# Ground truth +# ------------ +# `people` is the answer key, written before any pipeline ran. `tier` +# follows who these people are to the Simpsons in the show, not how +# often this corpus names them, so a counting rule can be scored against +# it rather than confirmed by it: +# family the household +# broader_family relatives outside the household +# close people the family has a lasting personal bond with +# other everyone else, however often they come up +# `names` lists the variants the corpus uses for each person. +# +# `mentions` on each item lists the people it refers to outside the +# household, by key, including references without a name ("my sisters"). + +voices: + marge: alloy + homer: alloy + +people: + homer: {tier: family, names: [Homer, Homie, Dad]} + marge: {tier: family, names: [Marge, Mom]} + bart: {tier: family, names: [Bart]} + lisa: {tier: family, names: [Lisa]} + maggie: {tier: family, names: [Maggie]} + abe: {tier: broader_family, names: [Grampa, Abe, Grandpa Simpson, my dad]} + jacqueline: {tier: broader_family, names: [Grandma Bouvier, Mom, my mother]} + patty: {tier: broader_family, names: [Patty, Aunt Patty]} + selma: {tier: broader_family, names: [Selma, Aunt Selma]} + herb: {tier: broader_family, names: [Herb]} + ling: {tier: broader_family, names: [Ling]} + milhouse: {tier: close, names: [Milhouse]} + ned: {tier: close, names: [Ned, Flanders, Ned Flanders]} + lenny: {tier: close, names: [Lenny]} + carl: {tier: close, names: [Carl]} + moe: {tier: close, names: [Moe]} + janey: {tier: close, names: [Janey]} + krabappel: {tier: other, names: [Mrs. Krabappel, Edna Krabappel]} + hoover: {tier: other, names: [Ms. Hoover]} + skinner: {tier: other, names: [Skinner, Principal Skinner]} + hibbert: {tier: other, names: [Dr. Hibbert, Hibbert]} + nelson: {tier: other, names: [Nelson]} + largo: {tier: other, names: [Mr. Largo]} + burns: {tier: other, names: [Mr. Burns]} + krusty: {tier: other, names: [Krusty]} + apu: {tier: other, names: [Apu]} + lovejoy: {tier: other, names: [Reverend Lovejoy]} + +items: + - id: people-2024-03-grampa-dinner + true_date: 2024-03-09 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [abe] + text: > + Saturday, March 9th 2024. Grampa came for dinner and told the + story about his war years again, the one with the hat. Bart + pretended to fall asleep in his mashed potatoes, but Lisa asked + him three questions and he lit up like a Christmas tree. + + - id: people-2024-03-ramp + true_date: 2024-03-23 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [milhouse] + text: > + Hi Bart, today is March twenty-third, 2024. You and Milhouse built + a ramp out of the old garage door. Milhouse went first, because + you said it was an honor. His glasses are fine. The garage door is + not. Your mother doesn't know yet. + + - id: people-2024-04-aunts + true_date: 2024-04-14 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [patty, selma] + text: > + April 14th 2024. Patty and Selma came over for Sunday lunch and + brought their iguana. Homer spent the whole afternoon in the + garage. Lisa played her new piece for her aunts, and Aunt Selma + said it was the first thing all year that didn't bore her. + + - id: people-2024-05-sleepover + true_date: 2024-05-05 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [janey] + text: > + May 5th 2024. Lisa's first sleepover here, with Janey. They + giggled until two in the morning and made a newspaper about our + street. Headline: Local Dad Snores. + + - id: people-2024-05-hedge + true_date: 2024-05-19 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [ned] + text: > + Hey Lisa, today is May nineteenth, 2024. Ned Flanders lent me his + hedge trimmer again. Fourth time this year. You made me bring it + back before dinner, and he gave me a lemonade for it. That man is + impossible to stay mad at. + + - id: people-2024-06-grandma + true_date: 2024-06-08 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [jacqueline] + text: > + June 8th 2024. Mom's birthday. Grandma Bouvier taught Lisa how to + play bridge and beat her four times in a row. Maggie fell asleep + on her lap and she didn't move for an hour. + + - id: people-2024-06-bowling + true_date: 2024-06-22 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [lenny, carl] + text: > + June 22nd 2024. Took Bart bowling with Lenny and Carl for the + first time. He bowled a 63 and Carl let him keep the score sheet. + Lenny says he has my form. I'm choosing to take that as a + compliment. + + - id: people-2024-07-fireworks + true_date: 2024-07-04 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [ned] + text: > + July 4th 2024. Fireworks at the Flanders' barbecue next door. Ned + let Bart light one sparkler and Bart behaved for a whole evening. + Homer ate eleven hot dogs. + + - id: people-2024-07-retirement-castle + true_date: 2024-07-20 + date_source: spoken + pattern: people-mention + kind: voice + sender: marge + mentions: [abe] + text: > + Hi Maggie, today is July twentieth, 2024. We visited Grampa at the + retirement home, and he held you the whole afternoon. You fell + asleep on him and then he fell asleep too, and the nurses tiptoed + around the two of you. + + - id: people-2024-08-game-night + true_date: 2024-08-10 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [moe] + text: > + August 10th 2024. Moe closed the bar early and came over to watch + the game on our couch. He brought pretzels and stayed until Marge + started vacuuming around him. + + - id: people-2024-08-squishee + true_date: 2024-08-24 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [apu] + text: > + August 24th 2024. Hottest day of the year. Apu gave Maggie a free + Squishee at the Kwik-E-Mart. Her whole face was blue until + bedtime. + + - id: people-2024-09-fourth-grade + true_date: 2024-09-03 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [krabappel] + text: > + September 3rd 2024. First day of fourth grade for Bart. His new + teacher is Mrs. Krabappel. She called at noon. On the first day. + + - id: people-2024-09-lunch-money + true_date: 2024-09-14 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [nelson] + text: > + September 14th 2024. Nelson took Bart's lunch money again. Bart + says it doesn't matter and that he has a plan. I am worried about + both. + + - id: people-2024-10-robot + true_date: 2024-10-05 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [milhouse] + text: > + Hi Bart, today is October fifth, 2024. Milhouse slept over and you + two tried to build a robot out of the vacuum cleaner. It works + better now than before. I don't know how, and I don't want to + know. + + - id: people-2024-10-conference + true_date: 2024-10-26 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [krabappel] + text: > + October 26th 2024. Parent-teacher conference with Edna Krabappel. + She says Bart is clever but bored, and that he corrected her + spelling on the board. She didn't sound angry. She sounded tired. + + - id: people-2024-11-saxophone + true_date: 2024-11-02 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [largo] + text: > + November 2nd 2024. Mr. Largo moved Lisa up to first saxophone. She + practiced on the porch until the neighbors applauded. + + - id: people-2024-11-thanksgiving + true_date: 2024-11-28 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [abe, patty, selma, jacqueline] + text: > + Thanksgiving, November 28th 2024. Everybody at our table: Grampa, + Patty and Selma, and Mom. Homer carved the turkey without injuring + anyone. Grampa fell asleep before dessert and Patty put a napkin + over his face. + + - id: people-2024-12-bonus + true_date: 2024-12-15 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [burns, lenny] + text: > + December 15th 2024. Mr. Burns cancelled the Christmas bonus again. + Lenny says next year for sure. We still got the tree, the kids + decorated it, and Bart put a picture of Burns at the top. + + - id: people-2024-12-christmas-eve + true_date: 2024-12-24 + date_source: spoken + pattern: people-mention + kind: voice + sender: marge + mentions: [ned, jacqueline] + text: > + Hi kids, today is Christmas Eve, 2024. Ned came over with a plate + of cookies shaped like the three wise men, and Grandma Bouvier + read you the Christmas story. Bart, you didn't interrupt once. I + want you to know I noticed. + + - id: people-2025-01-snow-fort + true_date: 2025-01-11 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [milhouse, nelson] + text: > + January 11th 2025. Snow day. Bart and Milhouse built a snow fort in + the front yard. Nelson knocked it over, and then, to everyone's + surprise, helped them build it again. + + - id: people-2025-02-arm + true_date: 2025-02-03 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [hibbert] + text: > + February 3rd 2025. Bart broke his arm falling off the garage roof. + Dr. Hibbert set it at the hospital and laughed that laugh of his + the whole time. Bart was brave. Homer fainted. + + - id: people-2025-02-xray + true_date: 2025-02-10 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [hibbert, krabappel] + text: > + February 10th 2025. Back at Dr. Hibbert's for the X-ray. The bone + is healing. Bart got the whole school to sign the cast, even Mrs. + Krabappel, who wrote "finally sitting still". + + - id: people-2025-02-cast-check + true_date: 2025-02-24 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [hibbert] + text: > + February 24th 2025. Cast check with Hibbert. Two more weeks. Bart + has been using it as a drum. + + - id: people-2025-03-cast-off + true_date: 2025-03-03 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [hibbert] + text: > + Hey Bart, today is March third, 2025. Dr. Hibbert took the cast + off this morning. Your arm looks like a noodle and smells worse, + but you can climb things again. Please don't climb things again. + + - id: people-2025-03-science-fair + true_date: 2025-03-22 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [janey] + text: > + March 22nd 2025. School science fair. Lisa and Janey built a model + of the water cycle that actually rained. Second place, behind a + volcano. Lisa says the judges were biased toward explosions. + + - id: people-2025-04-birthday-party + true_date: 2025-04-01 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [krusty, milhouse, nelson] + text: > + April 1st 2025, Bart's birthday. Homer booked Krusty for the party. + He came an hour late and left after twenty minutes, but Bart will + talk about it for years. Milhouse cried when the balloon animal + popped. Nelson came too, and brought a present. + + - id: people-2025-04-dads-birthday + true_date: 2025-04-12 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [abe] + text: > + April 12th 2025. My dad's birthday. We took Abe out of the + retirement castle for lunch at the diner. He told the waitress + three times that he was 29. She gave him a free pie. + + - id: people-2025-05-patty + true_date: 2025-05-18 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [patty, selma] + text: > + May 18th 2025. Patty came alone this time, Selma has a cold. She + and Lisa watched MacGyver all afternoon and quoted it at dinner. + + - id: people-2025-06-last-day + true_date: 2025-06-13 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [krabappel] + text: > + June 13th 2025. Last day of school. Bart gave Mrs. Krabappel a + card he made himself. She cried a little, I think. He says she + had something in her eye. + + - id: people-2025-06-summer-concert + true_date: 2025-06-28 + date_source: spoken + pattern: people-mention + kind: voice + sender: marge + mentions: [largo] + text: > + Hi Lisa, today is June twenty-eighth, 2025. Your summer concert + with Mr. Largo's band. You played your solo a little faster than + he wanted, and it was the best part of the evening. + + - id: people-2025-07-camping + true_date: 2025-07-19 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [ned] + text: > + July 19th 2025. Camping at Lake Springfield with Ned and his boys. + Ned brought a tent for each of us, labeled. Ours collapsed anyway. + We slept in his. + + - id: people-2025-08-roof + true_date: 2025-08-16 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [lenny, carl, moe] + text: > + August 16th 2025. Lenny and Carl helped me fix the roof, the part + Bart fell off. Moe sent a case of beer and a note saying he + doesn't do ladders. + + - id: people-2025-09-school + true_date: 2025-09-02 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [hoover, milhouse] + text: > + September 2nd 2025. School again. Bart has Ms. Hoover this year. + Milhouse is in his class again, so I give it a week. + + - id: people-2025-09-herb + true_date: 2025-09-20 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [herb] + text: > + September 20th 2025. Herb, Homer's half-brother, called for the + first time in years. They talked for an hour. Homer was quiet + afterwards, in a good way. + + - id: people-2025-10-halloween + true_date: 2025-10-31 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [ned] + text: > + Hi Maggie, today is Halloween, 2025. You went as a pacifier. Ned + gave out apples again and Bart egged nobody, which I count as + growth. + + - id: people-2025-11-thanksgiving + true_date: 2025-11-27 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [abe, patty, selma, jacqueline] + text: > + Thanksgiving, November 27th 2025. Grampa, my sisters and Mom all + came. Grampa and Mom argued about the gravy and both lost. Maggie + said "turkey" for the first time, or something close. + + - id: people-2025-12-carols + true_date: 2025-12-24 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [jacqueline, janey, lovejoy] + text: > + Christmas Eve 2025. Mom stayed over. Lisa and Janey sang carols at + church, and Reverend Lovejoy let them do a second verse. Homer + stayed awake for all of it. + + - id: people-2026-01-detention + true_date: 2026-01-17 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [milhouse, skinner] + text: > + January 17th 2026. Bart and Milhouse got detention together for + the thing with the school bell. Skinner called both families. I + told Bart I was disappointed and then asked him how they did it. + + - id: people-2026-02-valentine + true_date: 2026-02-14 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [moe] + text: > + Valentine's Day 2026. Moe gave me advice for Marge. I ignored all + of it and made her pancakes shaped like hearts. Mostly like + hearts. + + - id: people-2026-03-war-stories + true_date: 2026-03-01 + date_source: spoken + pattern: people-mention + kind: voice + sender: marge + mentions: [abe] + text: > + Hi Bart, today is March first, 2026. Grampa told you about the war + again, and this time you listened all the way to the end. He told + me afterwards that you are the only one who does. + + - id: people-2026-04-janeys + true_date: 2026-04-25 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [janey] + text: > + April 25th 2026. Lisa slept over at Janey's for the first time. + She called at nine to say goodnight and at nine thirty to say she + was fine. + + - id: people-2026-05-ling + true_date: 2026-05-09 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [selma, ling] + text: > + May 9th 2026. Selma visited with her daughter Ling. Ling and + Maggie played with the blocks all afternoon without a single + fight, which is more than I can say for their mothers. + + - id: people-2026-06-room + true_date: 2026-06-20 + date_source: written + pattern: people-mention + kind: text + sender: homer + mentions: [abe] + text: > + June 20th 2026. Grandpa Simpson moved to a room with a window on + the garden side. We brought his old armchair. He says it smells + like home, and that home smelled terrible. + + - id: people-2026-07-nelson-dinner + true_date: 2026-07-11 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [nelson] + text: > + July 11th 2026. Nelson came over for dinner. He said please twice + and thank you once. Bart says they are not friends, they just + have an arrangement. + + - id: people-2026-08-barbecue + true_date: 2026-08-08 + date_source: spoken + pattern: people-mention + kind: voice + sender: homer + mentions: [lenny, carl] + text: > + Hey Lisa, today is August eighth, 2026. Lenny and Carl came over + for a barbecue and you explained to them why you don't eat meat. + Carl had a veggie burger. Lenny had three regular ones, out of + respect. + + - id: people-2026-08-milhouse-moves + true_date: 2026-08-29 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [milhouse] + text: > + August 29th 2026. Milhouse's family is moving to Capital City. + Bart went to his room and didn't come down for dinner. Later he + asked if we could visit them in the fall. + + - id: people-2026-09-mom-fall + true_date: 2026-09-05 + date_source: written + pattern: people-mention + kind: text + sender: marge + mentions: [jacqueline] + text: > + September 5th 2026. My mother fell in her kitchen. She's fine, a + bruise and a lot of opinions. Lisa made her a get-well card with + a saxophone on it.