Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion docs/admin-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -567,6 +567,20 @@ Setup seeds the vault with three things: the classification ontology (the topics

The wiki maintains itself. A curator sidecar (`stack-memory-curator`) watches the vault: when new filings settle it regenerates the pages of the family members involved plus the home page (a couple of LLM calls, a few minutes after the burst), and once a night it rebuilds everything — topic pages, cross-references, the lot — while the GPU has nothing better to do. To bring the wiki up to date now instead of waiting, `./stack memory wiki update` regenerates what the latest filings touched and `./stack memory wiki update --all` runs the whole nightly rebuild; both return when the pages are written. `--member`, `--topic` and `--home` regenerate just those pages, and `[memory]` in `stack.toml` holds the knobs (see [Configuration](#configuration)). Edits happen in Forgejo (every wiki page links to its source), so the commit log doubles as the household's learning history.

**Who the family is.** Every person the family knows has a file of their own in the vault, `family/people/<id>.md`: the full name, the other names the family uses, and the chat account if they have one.

```markdown
---
type: person
title: Bart Simpson
aliases:
- Bartholomew
account: bart
---
```

Each `./stack up memory` adds a file for every account in `users.toml` that has none, so an update reaches an existing install with a restart of the memory stacklet. An existing file is never rewritten: it is the family's. Add someone without an account, such as a baby, by adding a file with `type`, `title` and any `aliases`, and add the names a person goes by to their file. A person with an account is a member of the household. The wiki gives every person a page under the names their file gives, and its home page lists the members. The archivist files a document under a person by any of their names ("Margaret" on a birth certificate is Maggie) and creates a `Person:` tag for a person without an account the first time a document names them. Transcription and the diary listen for every name in the files. A file the wiki cannot read is named in the curator's log (`docker logs stack-memory-curator`) and left out, and no page is removed until it reads again. A change to any file rebuilds the person pages within a few minutes, or right away with `./stack memory wiki update --home --members`. Edit the files in Forgejo, in Obsidian, or with `./stack memory write family/people/<id>.md --by <you> --from <file>`.

**The family diary.** Everything posted in `#Memories` becomes a diary entry. Each night the curator transcribes the new recordings, has the AI read each entry (a title, a summary, the people, topics from the ontology, facts), writes one card per entry to the vault under `family/diary/entries/`, and rebuilds the diary pages in the wiki from those cards. At 07:30 local time the archivist posts each new card as a quiet notice in the thread under its memory; the first time, one notice in the room says how to correct a card, and after two weeks without a new memory one notice invites the next, once per quiet stretch (`diary_job_at` in the archivist's `bot.toml` sets the time; it catches up after a night the Mac slept through). A family member corrects a card by replying in its thread, typed or spoken; the correction is committed to the vault under their name, and a corrected or hand-edited card is never overwritten by a later run. The archivist says nothing else in `#Memories`. The user guide has the family's side: [The Memories Room](user-guide.md#the-memories-room).

| | |
Expand Down Expand Up @@ -630,7 +644,7 @@ Key things to know:

### users.toml

Your family members. Generated by the installer. User accounts are seeded on the first `./stack up` of each stacklet.
The accounts: who can sign in, to what. Generated by the installer. User accounts are seeded on the first `./stack up` of each stacklet. The people the family knows are a separate list, the memory stacklet's `family/people/` (see [Memory](#memory-memory-optional)): a baby has no account but is family.

```toml
[[users]]
Expand Down
25 changes: 25 additions & 0 deletions docs/adr/adr-011-vault-brain-projection.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,28 @@ room) or edited by hand is theirs: the compile never overwrites it, and
the vault commit that changed it names who did. The diary is therefore
the one chat filing not rebuilt by replaying its thread (see
`docs/agent/dev.md`, "Reprocessing replays the source").

## Update 2026-10-09: compiled projections are the agent's fast path

The rule above keeps the agent off projections so that a stale page
never becomes a wrong answer. It assumed the agent could afford to
read the records for every question. With a local model it cannot:
reading dozens of records per question is slow, and the compiled pages
exist precisely to have read them once.

The vault stays the truth. The agent may answer from a compiled
projection when two things hold:

- every statement it uses cites the records it came from, as the home
page's numbered references and the person pages' `sources` do, so
the answer can name its source and a person can check it;
- it also reads the records filed since that projection was last
compiled, so nothing newer is missed.

What this does not remove is a projection that is wrong rather than
stale. The citations make that checkable, family corrections fix it,
and a nightly plausibility pass (the "dream cycle" in
`docs/design/brain/domain-model.md`) is meant to find it.

The agent already reads the brain (`MEMORY_VAULT_DIR` points at it),
so this update describes what it does and puts conditions on it.
76 changes: 67 additions & 9 deletions docs/design/brain/domain-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,40 @@ placement stays the deletion test: does deleting it lose information?

## Aggregates in the knowledge core

**Household** — the instance itself. Owns Members, the shared bucket
slug, language, and the scope rules. `shared_bucket` being
**Household** — the instance itself. Owns its People, the shared
bucket slug, language, and the scope rules. `shared_bucket` being
configurable is the tell that the real concept is a circle of people
sharing an archive; a family is the primary instantiation, a
non-family deployment is the same aggregate with different vocabulary.

**Member** — identity is the triple equality
`matrix localpart == bucket slug == git author`. That equality is
load-bearing: it is what makes attribution (`--by homer`), scoping,
and the vault chronicle line up. A Member owns a personal bucket with
the same shape as the shared one.
**Person** — someone the family knows, declared in one file each:
`family/people/<id>.md` (vault-format §7). Identity is the file's id
(`maggie`). The file holds the full name, the other names the family
uses for them, and the person's chat account if they have one. It is a
state document: the family writes and corrects it, by hand or through
the CLI, and `stack up memory` adds one for every account that has
none. The wiki's person page (`maggie/about.md`) is the compiled
projection of it: the declared names win, the compiled detail fills in
the rest. One entity, two sources, so the family's corrections to
what the stack learned ("Ed is Ned", "Grampa is Homer's father") are
edits to the same file, never a second format.

**Member** — a person who belongs to the household. Not a separate
entity and not a declared field: membership is derived, and the stack
makes an opinionated guess. In the first iteration a person is a
member when they have an account. For a member with an account the
triple equality `matrix localpart == bucket slug == git author` holds,
and it is what makes attribution (`--by homer`), scoping, and the
vault chronicle line up; such a member owns a personal bucket with the
same shape as the shared one. A person without an account (a baby, a
grandparent) is still declared, so the classifier, the transcription
vocabulary and the diary know them by every name, and the wiki gives
them a page; they are not listed among the members. The next iteration
suggests membership from the records when they make it likely (Maggie
lives in the house, the records say so), and asks the family before
it counts. Deriving members from folders is what put `media/` on the
wiki as a family member, which is why the guess starts from accounts
and nothing weaker.

**Record** — the generalization that ate "document". A capture, a
document mirror, an email thread, a voice memo. Identity is
Expand Down Expand Up @@ -89,7 +112,11 @@ from frontmatter; it wants to live in a registry (below).
Scope (the privacy rule as a type: shared bucket or one member's
bucket, derived from room membership). SourceContent (text + title +
origin URI, the ingestion contract). Classification (correspondent,
doctype, persons, topics, summary, facts, action items). Provenance.
doctype, persons, mentions, topics, summary, facts, action items).
`persons` are the members a record belongs to; `mentions` are the
people outside the household it names, extracted in the same model
call that classifies it, so a record can be replayed to recompute
them like every other part of its classification. Provenance.
Attribution. CaptureHash / PaperlessId / ThreadRoot. VaultPath (the
deterministic path builders are its factories). SummaryCallout with
Facts and ActionItems. PermaLink (`/go/docs|topic|person`, the
Expand All @@ -110,13 +137,40 @@ connects Conversation to Knowledge (this room feeds that topic).
identity authority. This is also the foundation for typed entities
inside standing topics: a Policy, an Account, a Vehicle, each linked
to a Correspondent and to source Records.
*Being built, people first.* A person entry is compiled from the
`mentions` of every record: canonical name, aliases, the kind of
relationship to the household (relative, friend, neighbor, teacher,
doctor, ...) and the stated relation, how present the person is
(records, months, first and last seen), and the records it cites.
It is a projection; the family's corrections ("Ed is Ned", "Grampa
is Homer's father") are the person's declared file (Person, above),
which wins over it and survives every rebuild. Correspondents become a second kind in the
same registry, and `pet`, `vehicle`, `place` after them, with the
same aliases, citations and corrections rather than a mechanism
each. The wiki's people tiers (family, broader family, close,
other) are a projection of the members plus the registry: tier
from the relationship kind, order within a tier from presence.
Evidence: `tools/family-memories/spec.people.en.yaml` and the
people-tiers prototype (2026-10-08). Relationship kind put 21 of 22
people in the right tier; mention counts alone, 14.
- **FactStore** — `facts.toml` holds facts (rule | fact | habit) with
no aggregate managing them. Typed facts with as-of validity are what
turn a standing topic from a folder into knowledge: policy number,
coverage, renewal date, each citing its source Record.
- **Reminder** — time-bound intent attached to a Member or Topic.
Standing-topic facts with dates (renewals, expiries) feed it almost
for free.
- **Dream cycle** — a nightly pass after the compile that checks what
was compiled for plausibility: one person with two fathers, a
relation that contradicts another source, near-duplicate entries,
dates that cannot both be true. Cheap deterministic checks select
the suspects; the model rereads only their cited records. It never
rewrites the family's own documents: a contradiction the sources
cannot settle becomes a question to the family in chat, and the
answer becomes a correction. Majority voting is not enough: in the
prototype Grampa was "Marge's father" in 3 records and "Homer's
father" in 2, because Marge writes "Grampa" from the children's
side. Earlier sketch: `ontology-design.md`, "Maintenance".

## Rules of thumb

Expand All @@ -127,4 +181,8 @@ connects Conversation to Knowledge (this room feeds that topic).
3. New containers are topics until proven otherwise; prefer a
provenance flag over a new mechanism.
4. Facts cite Records. A fact without a source link is a rumor.
5. The agent is a client of the database, never of the projections.
5. The vault is the truth; compiled projections are the agent's fast
path. The agent may answer from a projection whose statements cite
their records, and reads the records filed since that projection
was last compiled. A slow local model cannot search every record
for every question. (ADR-011, update 2026-10-09.)
11 changes: 10 additions & 1 deletion docs/design/brain/family-ontology.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
# Family ontology: declared entities priming classification

Status: design draft, 2026-06-12. Implementation slice of
Status: design draft, 2026-06-12. Partly superseded 2026-10-09: the
people pages below are built (`vault-format.md` §5, `person`
(declared)), with `type: person` and `title` instead of a folder-derived
kind and `canonical`; membership is derived (an account, in the first
iteration), not a `member:` field; people outside the household are
compiled from the `mentions` on records into the person registry
(`domain-model.md`, EntityRegistry); relations are a `relation_kind`
plus words, not a `role`. Onboarding, the write-path contract and
experiment round 6
still stand. Implementation slice of
`ontology-design.md` (2026-04-14), which designs the full living ontology
(persons, organizations, relations, growth loop, dream-cycle maintenance,
search expansion). Companion to `wiki-page-anatomy.md` and
Expand Down
71 changes: 68 additions & 3 deletions docs/design/brain/vault-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,18 @@ cheap).
- `diary` — one entry from the memories room (a recording, photo or note,
with its joined fragments and replies).

**State documents** (the family's current truth; mutable through the
CLI or by hand; never carry `generated`; see §7):
- `todos` — a topic's todo list.
- `person` (declared) — someone the family knows, one file each under
`<shared_bucket>/people/`.

**Entities & structure** (generated projections; MUST carry
`generated: true`):
- `person` — a household member or known person page.
- `person` — the wiki page for a person: the declared file compiled
with what the records say, and later anyone the records mention (the
person registry). `person` is the one type that is both: declared in
`people/`, generated everywhere else, told apart by `generated`.
- `correspondent` — an external party page.
- `topic` — a topic overview page.
- `index` — a folder navigation page.
Expand All @@ -97,6 +106,14 @@ Legend: **R** required, **O** optional (present-when-nonempty — absence
is a signal, so never emit an empty list/string). List fields are
one-deep string lists per §2.

Two person fields recur on records and mean different things:
`persons` lists the household members the record belongs to, by
display name; `mentions` lists the people outside the household it
names, each by the most specific name the record gives, or a
description through a member when it gives none ("Marge's mother").
Both come from the call that classifies the record. The person
registry is compiled from `mentions`; the record never links to it.

### `document`
| Field | | Notes |
|---|---|---|
Expand All @@ -111,6 +128,7 @@ one-deep string lists per §2.
| `document_type` | O | Paperless subtype (invoice, contract) — a different axis from `type`. |
| `category` | O | |
| `persons` | O | list. |
| `mentions` | O | list. |
| `tags` | O | list. |
| `paperless_url` | O | base URL. |
| `processing` | O | `ai_formatted` \| `ocr` \| `original`. |
Expand All @@ -124,6 +142,7 @@ one-deep string lists per §2.
| `title` | R | |
| `timestamp` | R | |
| `persons` | O | list. |
| `mentions` | O | list. |
| `filed_by` | O | Matrix localpart of the filer (mirrors the git author). |
| `tags` | O | list. |
| `resource` | O | source URL (OKF `resource`; bookmarks always have one). |
Expand Down Expand Up @@ -160,6 +179,7 @@ commit (`correct: <title>`, authored by whoever wrote it), not on the
card; its event id joins `event_ids`.
| `description` | O | one sentence. |
| `persons` | O | list. |
| `mentions` | O | list. |
| `tags` | O | list. |
| `filed_by` | O | Matrix localpart of the sender. |
| `addressee` | O | who the message is spoken to, as it names them; a correction can change it. |
Expand All @@ -177,16 +197,51 @@ and last the family's own words under `## Transcript`, `## Text` or
`## Caption`. The words come last so nothing in them can be mistaken
for a section.

### `person` (declared)
The family's file for one person, `<shared_bucket>/people/<id>.md`. The
file name is the id.

| Field | | Notes |
|---|---|---|
| `type` | R | `person` |
| `title` | R | the full name (OKF `title`). |
| `aliases` | O | list; every other name the family uses (Obsidian reads the same key). |
| `account` | O | the person's chat user name (Matrix localpart), when they have one. |

```
---
type: person
title: Maggie Simpson
aliases:
- Margaret
---
```

A member of the household is a person with an `account` (first
iteration; `domain-model.md`, Member). Membership is derived, never a
field the family sets. A file that breaks the §2 subset or this table
is reported by path and left out, and the wiki retires no member page
while any file is reported.

### `person` (generated)
| Field | | Notes |
|---|---|---|
| `type` | R | `person` |
| `generated` | R | `true` |
| `title` | R | canonical display name (OKF `title`). |
| `slug` | R | bucket slug (== Matrix localpart for members). |
| `slug` | R | the person's id: the declared file's name (== Matrix localpart and bucket slug for a member); otherwise derived from the canonical name. |
| `canonical` | R | canonical name (our H1/canonicalization logic). |
| `aliases` | O | list of known surface forms (the EntityRegistry coupling field). |
| `role`, `member`, `birthday`, `employer`, `owners`, `relation` | O | family-semantic custom fields. |
| `relation_kind` | O | `relative` \| `friend` \| `neighbor` \| `teacher` \| `doctor` \| `employer` \| `colleague` \| `service` \| `other`; absent for members. |
| `relation` | O | the relation in words, through a member ("Marge's sister"). |
| `first_seen`, `last_seen` | O | dates of the first and last record that mentions the person. |
| `sources` | O | list of the vault paths of the records it cites. |
| `birthday`, `employer`, `owners` | O | family-semantic custom fields. |

Whether someone is a member is never a field: it is derived from the
declared file's `account`. The tier the wiki shows (broader family, close, other) is
not stored either; the wiki derives it from `relation_kind` and
presence.

### `correspondent` (generated)
| Field | | Notes |
Expand Down Expand Up @@ -225,6 +280,16 @@ carries `generated`. It is a markdown checklist; frontmatter is optional
and, if present, uses `type: todos`. State documents get read-your-writes
through the CLI (ADR-011); they are not regenerated.

`<shared_bucket>/people/<id>.md` is a declared person (§5): the
family's own facts about someone, starting with their names. On every
start `stack up memory` adds one for each account that has none, as
a commit by the memory bot, and never rewrites an existing file.
A family member adds a person without an account (a baby) or an alias
by writing the file in Obsidian, Forgejo or with `stack memory write`;
a hand edit is a commit like every other. The family's corrections to
what the registry compiles ("Ed is Ned", "Grampa is Homer's father")
go into the same file when the registry is built.

`ontology.toml` / `facts.toml` are TOML config at the vault root, outside
this markdown-frontmatter spec.

Expand Down
Loading
Loading