Skip to content

Commit b9cbcf2

Browse files
committed
docs(s011): G1 plan, and the slice opens
ROADMAP row 011 to in_progress. NOTES.md carries the twelve-line G1 plan with the test each line names, the manual queue (AC2, AC6) with the model ids and the probe outputs behind them, and two deviations stated before any code: five provider mappings written and two measured live; no Trinity.LLM.Supervisor until a process needs one. plan_check: PASS. Signed-off-by: Ayla Croft <aylacroft@proton.me>
1 parent 6af7ee6 commit b9cbcf2

2 files changed

Lines changed: 72 additions & 1 deletion

File tree

‎ROADMAP.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ standards register names the rows that ask for them.
3434
| 002 | Supply chain, early: SBOM, build provenance, the TLS floor | 0 Foundation | S | 000 | planned |
3535
| 003 | FIPS build leg in CI, from source | 0 Foundation | M | 000 | planned |
3636
| 010 | Core domain + persistence (Ecto/SQLite, schemas, Repo owner) | 1 Core loop | M | 000 | approved |
37-
| 011 | LLM provider layer (req_llm behind `Trinity.LLM` behaviour) | 1 Core loop | M | 010 | planned |
37+
| 011 | LLM provider layer (req_llm behind `Trinity.LLM` behaviour) | 1 Core loop | M | 010 | in_progress |
3838
| 012 | Session process + agent loop (gen_statem, DynamicSupervisor, rehydration) | 1 Core loop | L | 010, 011 | planned |
3939
| 013 | LiveView chat UI with streaming | 1 Core loop | M | 012 | planned |
4040
| 020 | Tool protocol + registry | 2 Tools | M | 012 | planned |
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Slice 011: NOTES
2+
3+
## G1 plan, 2026-09-20
4+
5+
Tree at `6af7ee6` on `main` (010 approved); branch `slice/011-llm-provider-layer`; ROADMAP row 011 set to
6+
`in_progress` in this commit. req_llm 1.24.0 was read from its Hex tarball before this plan was written (public
7+
entry points `generate_text/3`, `stream_text/3`, `generate_object/4`, `embed/3`; stream chunk types `:content`,
8+
`:thinking`, `:tool_call`, `:meta`; usage keys `input_tokens`, `output_tokens`, `total_tokens`, `cached_tokens`,
9+
`reasoning_tokens`, `total_cost`; model spec `"provider:model"`; keys from a per-request `:api_key` option, the
10+
`:req_llm` application env, or the provider's env var). Each line names its test; the order is the build order.
11+
12+
1. Dependencies: `req_llm` at 1.24.0 inside the `~> 1.22` pin, `mox` for tests. What enters `mix.lock` with
13+
req_llm is counted and its licences listed in NOTES.md, because the desktop binary carries it. VERSIONS rows
14+
flipped by `mix versions.gen`. Test: `mix hex.audit` and `mix deps.audit` green; `versions.verify` OK.
15+
2. `Trinity.LLM.Event`: the seven event shapes from SLICE.md as a typespec plus `valid?/1`, the single source of
16+
truth 012 and 013 consume. `Trinity.LLM.Request` struct (system, messages, tools, model, params) with a
17+
changeset-free `new/1` that validates roles and tool schemas. Tests: every shape accepted, a foreign shape
18+
refused.
19+
3. `Trinity.LLM.Provider` behaviour: `stream/3`, `generate/2`, `generate_object/3`, `embed/2`, `models/0`,
20+
`capabilities/1`. `Trinity.LLM.Registry` from `config :trinity, :llm` (models as `{id, provider, model, caps,
21+
price}`, `default_model`), `models/0`, `default_model/0`, `lookup/1`. Test: a registry entry resolves to its
22+
provider module; an unknown id is refused by name.
23+
4. `Trinity.LLM` public API, resolving the provider from the registry entry for the request's model; `stream/3`
24+
(function) and `stream_to/3` (pid, events as messages); `Trinity.LLM.ProviderMock` (Mox) as the test provider
25+
for the registry's `fake` provider. Boundary `Trinity.LLM` with `deps: [Trinity]`, exporting the API,
26+
`Request` and `Event`. Test (AC7): switching `default_model` in config changes the provider used, no code
27+
change.
28+
5. `Trinity.LLM.Providers.Fake` in `test/support`: streams a scripted response (deltas, one tool call in three
29+
chunks, usage, done), scriptable to raise transient or non-transient errors N times. Test (AC1): the full
30+
event sequence, in order.
31+
6. `Trinity.LLM.Providers.ReqLLM`: `Request` to req_llm messages and `ReqLLM.Tool`; `stream_text` chunks to
32+
events with tool calls assembled (start on the first chunk naming a call, deltas while arguments grow, end
33+
when the call is complete or the stream ends); `:meta` to `{:usage, _}` and `{:done, reason}`; `generate`,
34+
`generate_object` (req_llm's JSON schema path), `embed`. Provider mapping for `anthropic`, `openai`,
35+
`openrouter`, `google`, `openai_compatible` (the openai provider with `base_url` and a per-request key).
36+
Errors classified: timeouts, 429, 5xx, connection refused are transient; 4xx otherwise are not. Unit tests
37+
over recorded chunk sequences; live tests in line 9.
38+
7. `Trinity.LLM.Retry`: attempts and base backoff from config; transient errors retried, non-transient returned
39+
at once; optional fallback model list tried in order after the attempts. Test (AC4): the fake raises a
40+
transient error twice then succeeds; raises N+1 times then `{:error, _}`; a non-transient error returns
41+
immediately with one attempt.
42+
8. `usage_events` migration and `Trinity.LLM.Usage`: one row per completed call with model, provider, tokens,
43+
and cost computed from the registry price; req_llm's own `total_cost` kept in `provider_meta` and never
44+
used. Test (AC5): a row exists after a fake call with the expected cost; the Postgres job applies the
45+
migration.
46+
9. Live tests, `@tag :live`, excluded by default (NetworkGuard's opt-in path): OpenRouter with
47+
`TRINITY_LIVE_MODEL` and NVIDIA through `openai_compatible` with `NEMOTRON_BASE_URL`, `NEMOTRON_MODEL` and the
48+
key as `:api_key`; a preflight that the model id is on `GET /models` and names a 410 as end of life. Cases:
49+
a streamed completion with deltas and done; a tool call; `generate_object` against a schema (AC2);
50+
`embed` against `nvidia/nemotron-3-embed-1b` with the declared dimension (AC3). Output pasted, keys
51+
redacted (AC6).
52+
10. `Trinity.Config.secret/1`: environment now, keychain at slice 100; the adapter reads keys only through it.
53+
Test: a missing key is a named error, not a nil passed to the provider.
54+
11. Anthropic prompt-cache hint: a `cache: true` param maps to req_llm's cache-control option for the anthropic
55+
provider and is dropped for others. Unit test on the mapping; not measured live (no Anthropic key here).
56+
12. Gate, coverage row, PROOF.md, ROADMAP to `done`, through a pull request; tag `slice/011` after the merge.
57+
58+
Manual verification queue, as SLICE.md tags them: AC2 (live `generate_object`) and AC6 (`mix test --only live`,
59+
output pasted with keys redacted). Both run on this machine with the owner's keys in `.env`; the owner reads
60+
the pasted output at G4. The developer machine, the model ids and the probe outputs behind the choice:
61+
62+
- OpenRouter, `inclusionai/ling-3.0-flash-vl:free` (owner's pick): tool call `get_weather` with
63+
`{"city": "Paris"}`, 298 tokens, served by Novita; streamed completion 59 chunks.
64+
- NVIDIA endpoint, `nvidia/nemotron-3.5-lightning-30b-a3b`: tool call, 317 tokens; streamed completion 20
65+
chunks. The id first set, `nvidia/llama-3.3-nemotron-super-49b-v1.5`, answered `410 Gone` (end of life
66+
2026-08-26); that is why line 9 carries a preflight.
67+
68+
Deviations from SLICE.md, stated before building: the five provider mappings are written; only OpenRouter and
69+
the NVIDIA endpoint are measured live, and PROOF.md says so per provider. `Trinity.LLM.Supervisor` from docs/01
70+
(rate limiters) is not built here: nothing in this slice needs a process, and a supervisor with no children
71+
would be a claim; 012 adds it when the Session needs one. Recorded as a follow-up.

0 commit comments

Comments
 (0)