feat(s134): the Tier 3 embedder client and its four gates

CHANGELOG.md

@@ -47,6 +47,35 @@ missing or wrong declaration, and full-text recall keeps working through every r
 The evaluation that decides the default model is built and waiting: a generated corpus, 300 queries,
 and labels that the scored run will not read until 100 of them have been audited and frozen.
 
+### `slice/134`: a stronger embedding model from a service the operator runs, held to a pin
+
+Semantic memory can now use an embedding model served by an Ollama the operator runs, without any
+neural runtime inside Trinity. The service is admitted on the operator's pin and nothing else: the
+model's digest as the service lists it, the weights' and tokenizer's SHA-256, the context length,
+the prompt templates and the service's version are all part of the embedding space. If the model
+behind the name is replaced, the version changes, the model disappears or the service stops
+answering, semantic recall turns off with that reason within one check interval, full-text recall
+keeps answering, and no other model or space is chosen.
+
+Every request asks the service to refuse rather than truncate an input that is too long, which is
+not its default. Trinity also counts every input's tokens itself, with the tokenizer read out of the
+signed weights (it gives the reference library's token ids on 5,000 natural sentences and 1,221
+hostile inputs), refuses an over-length input before sending it, and treats a service that reports
+fewer tokens than it was sent as cutting inputs short: no vector, and the embedder off until restart.
+
+Weights reach the service only through `mix trinity.tier3.import`: an OCI artifact whose cosign
+signature and OMS signature over the model files are verified offline against keys the operator
+holds, every blob's digest checked as it is unpacked, the model created through the service's API,
+and the resulting digest recorded. Trinity ships no Tier 3 weights.
+
+Four gates decide whether a model is admitted on a given image, and they were run on the upstream
+Ollama 0.40.0 image with Qwen3-Embedding-0.6B (16-bit weights): every one of 500 fixtures within a
+cosine of 0.999 of the sentence-transformers reference (lowest 0.99969); an input one token too long
+refused, never embedded; a running node off 160 ms after the model behind its tag was swapped; and
+the service answering with networking disabled. The 8-bit build of the same model missed the
+parity bar (lowest 0.99854) and is not offered. The gates have yet to be run on the hardened image
+this is meant for. `docs/tier3-embedder.md` is the operator's page and `docs/perf.md` has the figures.
+
 ## 2026-09-26
 
 ### `slice/128`: the checks that guard attribution now say what they check

README.md

@@ -226,7 +226,11 @@ say-so, never on its own) or the static floor (a 7.8 MB weights file, pure Elixi
 under the data directory; not yet distributed with Trinity while its licence is reviewed). Without one the tier
 is off and full-text search still works. Every stored vector belongs to one named embedding space, and a store
 answers only from the space it is pinned to: changing models is an explicit re-embed (`mix trinity.space.retier`),
-never a silent switch. `docs/perf.md` has what it costs.
+never a silent switch. A stronger model can come from an Ollama service you run instead
+(`docs/tier3-embedder.md`): its weights are imported only with their signatures verified offline
+(`mix trinity.tier3.import`), Trinity holds the service to the imported model's digest and version, and
+if either changes semantic recall turns off with the reason rather than answering from another model.
+`docs/perf.md` has what it costs.
 
 `mix gate` runs the full quality gate and has to pass before every commit: format check, compile
 with warnings as errors, Credo, Sobelow, dependency audits, version verification, the naming and

docs/01-architecture.md

@@ -88,7 +88,7 @@ without anything failing.
 | `Trinity.Effects` | The membrane; the runner in force (`Effects.Runner`, the executor `Tools.Runner` takes as a function); decision and query receipts; the boot receipt | **Tools**, Permissions, Authority, Receipts, Repo |
 | `Trinity.Authority` | Behaviour; `Local` implementation (the one caller of `execute/2` for effectful tools); selection at boot; `Staged` | Receipts, Repo |
 | `Trinity.Receipts` | Local chain (one supervised writer per scope, ADR-0013), the signer seam (Ed25519, P-384, ML-DSA-87), key custody and the registry, checkpoints, the verifier, the alarm | Repo (`Repo.Receipts`) |
-| `Trinity.Memory` | Always-on tiers with their budget and consolidator (030), search (031), semantic store and retrieval (032), compaction (023); at 133 the embedding spaces (`Space`, the fourteen-field identity; `Spaces`, the one active space, re-tier, drop), the static floor (`Embedders.Static`, `WordPiece`, `StaticArtifact`), the int8 `Scorer`, the embedder configuration's faults (`EmbedderConfig`) and the operator's `ReEmbedWorker` | LLM (summaries/embeddings), Repo |
+| `Trinity.Memory` | Always-on tiers with their budget and consolidator (030), search (031), semantic store and retrieval (032), compaction (023); at 133 the embedding spaces (`Space`, the fourteen-field identity; `Spaces`, the one active space, re-tier, drop), the static floor (`Embedders.Static`, `WordPiece`, `StaticArtifact`), the int8 `Scorer`, the embedder configuration's faults (`EmbedderConfig`) and the operator's `ReEmbedWorker`; at 134 the Tier 3 client (`Embedders.Ollama` with its pin watcher `Embedders.Ollama.Watch`), its token counter (`BPE`), the GGUF header reader (`GGUF`) and the signed import path (`Tier3.Import`, `Tier3.Verifier`, `mix trinity.tier3.import`) | LLM (summaries/embeddings), Repo |
 | `Trinity.Skills` | SKILL.md parsing, registry, loader, manager, scanner (as built at 040: parser, sources, registry, index, the three tools; at 041: staging, promotion, manager, scanner, diff, learn, `skill_manage` and `learn`) | Repo, Permissions, **Effects**, **Receipts**, Sandbox (as built at 041: Tools, Memory, Permissions, Receipts and LLM; Effects is not a dependency: the promotion is not a tool call, it writes its own effect receipt; Tools never depends on Skills) |
 | `Trinity.Scheduler` | Oban workers for agent tasks, delivery (as built at 050: the `tasks` and `task_runs` rows, `next_run_at` computed from a cron expression or a one-shot datetime, `Workers.Tick` (the one Cron plugin entry, every minute, enqueuing `Workers.RunTask` for what is due), `RunTask` (one turn in a fresh `origin: "cron"` session, the summary, the delivery), `Delivery` with `Delivery.Desktop`, `Parse.human/2` (a phrase to cron through the model, checked by Oban's parser); Oban itself on the app's repo, Lite on SQLite and Basic on Postgres, queues `agent_tasks`, `memory`, `maintenance`; the observer as `Trinity.Memory.ObserverWorker` and the curator as `Trinity.Memory.Curator` live in Memory) | Sessions, LLM, **Repo** (Gateways at 070) |
 | `Trinity.MCP` | Client manager, tool bridge, server (as built at 059: the boundary alone, holding the core's version, its JSON depth and its telemetry event names; at 060: `Servers` and `ServerConfig` (the rows), `Supervisor` and `Boot` (one `Client` per enabled row), `Client` with its `Wire` (the outbound request, the headers, the core's decoder and validator) and two transports (`Transport.Stdio`, a child on a Port; `Transport.HTTP`, one POST per request), `Bridge` (the one tool module every MCP tool runs through) and `Client.Auth` (062's seam, a static token at 060); at 061: `Server` (the module above the core, `@behaviour BeamMCP.Server`, handed to both transports through `:server`; `tools/call` answered through the membrane, everything else the core's), `Server.Catalog` (the exported tools, sorted), `Server.Exports` (the configured names, `:catalog` refused), `Server.Session` (the `origin: "mcp"` session with its own persona), `Server.Envelope` and `Server.Replay` (the sealed `requestState` and the nonce table), `Server.Auth.Local` (the static bearer, 062's seam on the server side), `Server.Plug` (the `POST /mcp` endpoint, mounted in `TrinityWeb.Endpoint` ahead of the parsers) and `Server.Stdio` with `mix trinity.mcp.stdio`; at 062: `Trinity.MCP.Auth`, a boundary of its own under `Trinity.MCP` with `deps: []` (the behaviour `authorize/2` answering a `Principal`, never the token; `Config`, `Token`, `JWKS`, `Discovery`, `Scopes`; the profiles `Local`, `External` and `Embedded` with `Embedded.Keys`; the client role `Client` with `Client.Store`), and `AuthHost` in `Trinity.MCP` (the configuration from the tree's paths and the authority selection, the personal profile's boot, the refusal receipts, the client role's flow for the page); `Server.Auth.Local` is gone, `Server.Plug` authorizes ahead of the transport and answers `401` with a challenge, `Server` checks scope before the gate) | Tools, **Effects**, **Permissions**, Memory (as built at 059: a top-level boundary, like `Trinity.Smoke`, with `deps: [Trinity, BeamMCP.JSON]`; the boundary compiler checks every call into the `beam_mcp` application and this boundary alone lists its modules; at 060 the deps are `[Trinity, BeamMCP.JSON, BeamMCP.Schema]`, Trinity's tools, permissions and receipts reached through `Trinity`'s exports, and `Trinity.Application` and `TrinityWeb` list `Trinity.MCP`; at 061 the core's `Server`, `Catalog`, `ToolSpec` and both transports are listed too, and `Trinity` exports `Tools.Registry` and `Effects.Runner` to it) |
@@ -115,7 +115,7 @@ through those modules only. `boundary` `exports:` lists enforce this.
 | `Trinity.Tools.Tool` | `name/0`, `description/0`, `schema/0`, `risk/0`, `execute/2` | config `:tools` list + MCP dynamic registration |
 | `Trinity.Gateways.Adapter` | `child_spec/1`, `deliver/2`, `capabilities/0` | config `:gateways` list |
 | `Trinity.Memory.VectorStore` | `upsert/3`, `search/3`, `delete/1`, `count/1` (as built at 032: the scope filter is `search/3`'s required argument; as built at 133: `search/3` and `count/1` over one named space, `{:error, {:mixed_space, n}}` rather than a ranking when a row does not fit it, writes through `Spaces.put_vector/5`) | the database adapter, at compile time (`Brute` on SQLite, `Pgvector` on Postgres; `Hnswlib` when a slice measures past 10^5 rows) |
-| `Trinity.Memory.Embedder` | `embed/1`, `dim/0`, `model_id/0`, `availability/0`; at 133 `space/0` and `thresholds/0` | `config :trinity, :memory, embedder:` (`:local` default, `:static` the floor, `:hosted` opt-in only with a declared `locality:`, `:fake` in the suite, or a module; a list names several, and the store is served by the one whose space is pinned) |
+| `Trinity.Memory.Embedder` | `embed/1`, `dim/0`, `model_id/0`, `availability/0`; at 133 `space/0` and `thresholds/0`; at 134 the optional `embed_query/1` (a query template), `endpoint/2` (where text is sent, for the locality check), `check_config/1` (the embedder's own configuration faults) and `children/0` (processes `Trinity.Memory.Supervisor` starts) | `config :trinity, :memory, embedder:` (`:local` default, `:static` the floor, `:hosted` opt-in only with a declared `locality:`, `:ollama` the Tier 3 service on the operator's pin with a declared `locality:`, `:fake` in the suite, or a module; a list names several, and the store is served by the one whose space is pinned) |
 | `Trinity.Skills.Loader` | `load/1`, `validate/1` | config `:skill_loaders` (not built at 040: the three roots are fixed and `Trinity.Skills.Sources` scans them; a loader behaviour waits for a fourth kind of source) |
 | `Trinity.Permissions.Policy` | `decide/3` → `:allow | :deny | {:ask, prompt}` | config |
 
@@ -138,6 +138,16 @@ trinity.space.drop <id> --confirm`). `Semantic.status/0` serves the store throug
 space is active and answers OFF with a named reason otherwise: a configuration fault (`EmbedderConfig`; under
 `:regulated` the boot is refused instead), a space mismatch, the embedder's availability, or the last runtime
 fault (an unreachable endpoint, a store row that does not fit its space). Full-text recall goes on in every case.
+
+**The Tier 3 embedder (Slice 134, as built):** `Embedders.Ollama` reaches an operator-run Ollama over HTTP and serves
+only the model the operator pinned: the `/api/tags` digest, the GGUF's SHA-256, the tokenizer's SHA-256, `num_ctx`,
+both prompt templates and the runtime version are fields of its space, all from configuration. Every request carries
+`truncate: false`; Trinity counts each input with `BPE` (the tokenizer read out of the signed weights) and refuses one
+over its limit before sending, and an answer whose `prompt_eval_count` is below that count turns the embedder off.
+`Embedders.Ollama.Watch`, started by `Trinity.Memory.Supervisor` through `children/0`, compares the service with the
+pin at boot and on an interval, and a changed digest or version is OFF with its reason. `Tier3.Import` is the only way
+weights reach the service: cosign and OMS verified offline by the reference programs, every blob's digest checked,
+the model created through the service's API and its digest recorded. `docs/tier3-embedder.md` is the operator's page.
 ```
 
 Every state transition is persisted before it is broadcast. A crash between persist and broadcast is safe

docs/perf.md

@@ -151,3 +151,34 @@ seconds, and rebuilding gave the same digest.
 `scripts/static_floor_release.sh` (`TRINITY_WITHOUT_ML=1`): the headless release assembled from a build that never
 had nx, exla, xla, axon, bumblebee or tokenizers holds none of them in `lib/`, boots, writes two memories and
 recalls one through the static space (slice 133, AC8). Slice 130 measured exla at 443 MB of a 522 MB release.
+
+## The Tier 3 embedder and its gates (slice 134)
+
+MEASURED on 2026-10-08 on the same machine as slice 133's figures above (AMD RYZEN AI MAX+ 395, 32 logical CPUs,
+125 GiB, Linux 6.14.0-37), with other work on it (load average 9 to 16 during the throughput run). The service:
+`ollama/ollama:0.40.0` (index digest `sha256:1bef639749741b375e9a1eb2c1346fb57ce52f5432de1f74846e44ccc18e1687`),
+the upstream image at the version the hardened government image pinned that day, on the CPU, limited to 16 GiB.
+**It is not that hardened image**, which these gates have yet to be run on. The model: `Qwen3-Embedding-0.6B-f16.gguf`
+(SHA-256 `421a27e58d165478cc7acb984a688c2aa41404968b0203e7cd743ece44c54340`), imported through
+`mix trinity.tier3.import`; the reference: sentence-transformers 6.1.0 over the same model's safetensors, float32.
+
+| gate | result |
+|---|---|
+| 1, parity, 500 fixtures at `num_ctx` 2048 | PASS: minimum cosine 0.99968977, median 0.999875; every `prompt_eval_count` equal to the reference tokenizer's count |
+| 1, red: `num_ctx` 256 with the service's default truncation | FAIL as intended: 60 of 500 under 0.999, minimum 0.95426396, all of them inputs over 255 tokens |
+| 1, the same model quantized to Q8_0 | FAIL: minimum 0.99853698, 60 of 500 under 0.999 (not offered) |
+| 2, truncation at `num_ctx` 512 and 2048 | PASS: 511 (2047) tokens answered, 512 (2048) refused with `truncate: false`; with `truncate: true` or omitted, a vector of the first 511 (2047) |
+| 3, the digest pin, interval 2 s | PASS: OFF with `:model_digest_changed` 160 ms after the blob behind the tag was replaced; with the comparison removed, still on after 7.4 s |
+| 4, offline | PASS: `--network none`, only `lo`; `/api/embed` answered from the verified directory; a pull failed for want of a network |
+
+Throughput through Trinity's client (tokenizing, the request and the checks), 301 sentences of 7 to 200 tokens
+(median 28) cycled, no pass mark:
+
+| batch | calls | p50 | p95 | texts per second at p50 |
+|---|---|---|---|---|
+| 1 | 200 | 34.8 ms | 57.7 ms | 28.7 |
+| 32 | 40 | 1227.8 ms | 1382.5 ms | 26.1 |
+
+Batching does not raise throughput on this CPU path. The service held 5.31 GiB of its 16 GiB afterwards. Trinity's
+own token count matches the reference tokenizer on 5,000 natural sentences and 1,221 hostile inputs, and reading
+the tokenizer out of the 1.2 GB GGUF's header takes 337 ms.

docs/regulated/data-flow.md

@@ -70,6 +70,28 @@ What this means for an accreditation:
    it is in the conversation it is in the prompt on the next turn, because the conversation tier is
    sent verbatim.
 
+## What leaves for an embedding service
+
+Semantic memory needs an embedding model. Two configurations send text off the Trinity process to
+get one: `:hosted` (a provider's embedding endpoint) and `:ollama` (an Ollama the operator runs,
+`docs/tier3-embedder.md`). In both, **the text of every memory the observer writes and of every
+query a turn recalls with leaves in the clear**, the latter prefixed by the configured query
+template. Nothing is hashed or redacted on the way.
+
+| What crosses | Raw or hashed | To | Path |
+|---|---|---|---|
+| Memory text and query text | Raw | The configured embedding endpoint | `lib/trinity/memory/embedders/hosted.ex`, `lib/trinity/memory/embedders/ollama.ex` |
+| The model name, `num_ctx`, `truncate: false` | Raw | The same endpoint, `:ollama` only | `lib/trinity/memory/embedders/ollama.ex` |
+| The model's weights, at import only (`mix trinity.tier3.import`) | Raw | The operator's own service | `lib/trinity/memory/tier3/import.ex` |
+| No text: the pin check is two reads, `GET /api/tags` and `GET /api/version` | Not applicable | The same endpoint, `:ollama` only | `lib/trinity/memory/embedders/ollama/watch.ex` |
+
+What constrains it: the endpoint's locality is declared by the operator (`:within_boundary` or
+`:external`), never inferred from the address; `:external` needs `external_opt_in: true`; and under
+`TRINITY_PROFILE=regulated` the endpoint must be on `TRINITY_REGULATED_LLM_ENDPOINTS` or the node does
+not boot (`lib/trinity/memory/embedder_config.ex`). Whether CUI or PHI may go to that endpoint is the
+same policy question as for the model endpoint above. An embedding service the operator runs inside
+the boundary may still reach out on its own: see the operating notes in `docs/tier3-embedder.md`.
+
 ## Where the data rests
 
 | Store | Contents | Encrypted by Trinity? | Path |