Skip to content

Run portable memory inference in private Core sessions #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Brian Krabach (bkrabach) merged 3 commits into from
Sep 23, 2026
Merged

Run portable memory inference in private Core sessions #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
40c346e
Use sessionless provider requests for portable memory suggestions
bkrabach Sep 23, 2026
65a6128
Run portable memory inference in private Core sessions
bkrabach Sep 23, 2026
10e5a25
Reject unavailable routing and clarify private inference evidence
bkrabach Sep 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 75 additions & 81 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,42 +24,57 @@ suggested, never cited (§2): `config.yaml`, its own configuration, and
the instance goes **inert**: nothing is injected, no tool is offered, no timer runs,
and every writer refuses in one line (§11).

## Install
## Standalone install

The portable suggestion job does not require `amplifier-app-cli`. It declares
Core >=2.0.1, Foundation and its setup installer as dependencies. Use the public
index and require the published Core wheel:

```bash
# 1. Session plane (load + save + /remember and /memory), composed into all sessions.
# Point --app at the behavior file, not at the root bundle: the root bundle includes
# this same behavior, so an --app install of it is a self-include the loader skips
# ("Circular Include Skipped"), leaving a session with no hook and no memory tool.
amplifier bundle add 'git+https://github.com/microsoft/amplifier-bundle-memory@main#subdirectory=behaviors/memory-session.yaml' --app

# 2. The CLI (`amplifier-memory`, a thin click wrapper over the `amplifier_memory`
# library: init · status · review · why · format_why · doctor · update_check ·
# update_plan · service_status · run_suggest · pending/accept/decline/skip —
# cli.py adds only parsing, printing and exit codes):
uv tool install git+https://github.com/microsoft/amplifier-bundle-memory@main

# 3. Create the store (a git repo at ~/.amplifier-memory) and install the daily
# suggestion timer. This is the only setup step; `--no-timer` skips the timer.
amplifier-memory init

# 4. Verify:
uv tool install --default-index https://pypi.org/simple --no-build-package amplifier-core \
git+https://github.com/microsoft/amplifier-bundle-memory@main
amplifier-memory init --no-timer
amplifier-memory setup --workspace /path/to/your/workspace
amplifier-memory service install
amplifier-memory doctor
```

Step 3 asks **one** question — *What kinds of things should I remember for you?*
— offering a default answer you can accept with Enter, and saves your answer as
`m-001`. It is your own words, verbatim: nothing is invented for you, and with no
terminal attached it takes the default and says so. It then prints what it
installed, how to turn it off (`amplifier-memory service uninstall --home
<instance>`) and where to steer what it costs (`config.yaml` inside the instance
— [which model the judge uses](#which-model-the-judge-uses)). Running it again
reports `store exists · timer installed` and changes nothing; once you have
uninstalled the timer, `init` leaves it uninstalled.

If you already had a store at the older `~/.amplifier/memory`, step 3 **offers**
to move it to the default and prints what it did. It never moves it silently, and
a no leaves it exactly where it is (with the `mv` you would run yourself).
`init` asks the existing memory-seeding question and preserves an existing store.
`setup` reads shared and workspace provider settings, prepares the selected source
modules under this memory instance, and installs their dependencies in the standalone
tool environment. It makes no model call. Setup is explicit: suggestion runs never
install or refresh anything. Timer installation/start/restart refuses an unprepared
or changed runtime. After changing provider sources/settings or upgrading runtime
packages, run `setup` again before re-enabling the timer.

Setup uses Foundation's settings overlay in this order: shared
`$AMPLIFIER_HOME/settings.yaml` (otherwise `~/.amplifier/settings.yaml`), workspace
`.amplifier/settings.yaml`, then `.amplifier/settings.local.yaml`. The chosen
workspace is recorded for the timer. Named accounts and their source/configuration
are preserved; source overrides come from `sources.modules`. Custom providers need
an explicit source. Only providers, the optional configured `hooks-routing`, and
basic Core lifecycle modules are mounted. User tools, other hooks, workspace
instructions and recursive memory jobs are not loaded. An explicitly selected
arbitrary bundle or unsupported module override requires host-resolved inference
rather than silently selecting a different account.

The default `fast` role requires an explicit `config.hooks` routing declaration;
routing supplied only through bundle includes is not composed and fails visibly.
Use the user's intended routing settings, explicitly choose provider/model, or
use host-resolved inference. See [configuration and model provenance](docs/SESSIONLESS_INFERENCE.md).

Generated state stays under the existing resolved memory home:
`runtime/generations/` contains prepared module copies and source receipts;
`runtime/jobs/<id>/` contains internal job metadata, safe usage events and the
private transcript. These files are ignored by Git and not read as memories or
new source conversations. Shared source histories and `sessions.jsonl` are never
rewritten. Provider-owned OAuth refresh stores remain under each provider's
configured ownership; the job does not relocate, clone or reset credentials.

Hosts such as Unified keep their existing mounted-provider inference path and
manage their own component installations. Installing the session memory behavior
in another host is a separate, optional host operation. Existing memory homes,
including the legacy fallback, keep their current location; no migration is added.

### `--home`: more than one instance

Expand All @@ -86,21 +101,13 @@ amplifier-memory service uninstall --home ~/work-memory
To see the session plane itself working, start a session and say a standing
preference: it is saved in that turn and announced with its id and its undo.

To remove all of it:
To stop the daily pass without deleting memories:

```bash
amplifier bundle remove 'git+https://github.com/microsoft/amplifier-bundle-memory@main#subdirectory=behaviors/memory-session.yaml' --app
uv tool uninstall amplifier-memory
rm -rf ~/.amplifier-memory # deletes your memories
amplifier-memory service uninstall
```

The daily suggestion inbox (Phase 2, below) needs nothing further: step 3
installed its timer. To turn it off, or to put it back afterwards:

```bash
amplifier-memory service uninstall # no more daily pass; the store is untouched
amplifier-memory service install # what `init` already did — run it to undo an uninstall
```
After setup, `amplifier-memory service install` enables it again.

Both act on the instance `--home` resolves to. `amplifier-memory service status`
lists **every** installed instance timer, not only that one, so a timer you set up
Expand Down Expand Up @@ -147,18 +154,11 @@ first) · `amplifier-memory review` (pending suggestions) ·
`amplifier-memory init` · `amplifier-memory update` (alias `upgrade`) ·
`amplifier-memory suggest` and `service` (Phase 2, below).

One `amplifier-memory update` is enough: it refreshes all three copies of this
bundle a device runs, and prints each as `<old> → <new>`: the `amplifier-memory`
uv tool (the shell verb), the bundle cache clone(s) under `~/.amplifier/cache/` and
`~/.amplifier/cache/skills/` (what a session loads the modules and skills
from), and the `amplifier_memory` library inside the amplifier CLI's own venv
(what those modules import). `doctor`'s `update` row compares all three
against `git ls-remote` and names which one is behind; a cache or a venv it
cannot find is INFO, never a failure. When the uv-tool step upgrades the CLI,
`update` re-runs itself from the freshly installed binary so the remaining
refreshes happen with the new code — that hand-off is why the second half of
the report shows step 1 skipped. Sessions started before an update keep the
old module code until they restart — nothing is hot-reloaded.
`amplifier-memory update` upgrades the standalone tool, checks inference readiness,
restarts an installed timer only when ready, and runs `doctor`. Other hosts manage
their own runtime updates. Legacy CLI/cache maintenance remains an explicitly
requested library interoperability operation; ordinary update never discovers or
invokes `amplifier`. Existing sessions retain their loaded code until they end.

Reading memory leaves no commit behind: loads and citations are appended to
`~/.amplifier-memory/usage.jsonl`, which git does not track. Every commit in
Expand All @@ -172,8 +172,8 @@ yesterday's recorded sessions, asks the model one question per session, checks
in code that every quote it gets back was really said by you, and *proposes*
the survivors. It never writes to `MEMORY.md`.

`amplifier-memory init` installed that timer (a systemd `--user` timer, launchd on
macOS) — there is no second setup step. The rest of the verbs:
After explicit setup and service installation, the timer is a systemd user timer
on Linux or a launchd agent on macOS. The remaining verbs:

```
amplifier-memory suggest # run the pass once, now
Expand Down Expand Up @@ -248,38 +248,32 @@ billed — a provider id, `role:<role>`, or `inherited`.

### Which model the judge uses

Three answers, in this order:

1. the `provider` / `model` / `bundle` you set in `config.yaml` (below);
2. else the **role** — `fast` as shipped — resolved by the amplifier CLI, when
your CLI has `amplifier run --model-role` (the job reads its `--help` to find
out; today's CLI does not, so this step is skipped);
3. else the CLI's own default, **inherited** — whatever `amplifier provider` has
starred.
Selection precedence:

The shipped default is a role, never a provider id: a provider id names one
machine's account. To pin one for this job alone, write the `llm:` block of the
store's own `config.yaml`, which `init` already wrote:
1. The explicit `provider` / `model` / host-resolved `bundle` in this instance's `config.yaml`.
2. Otherwise the configured routing module resolves the requested role (`fast` by default).
3. Without a routing resolver, the unique configured default account wins. Ambiguous or
unavailable selections fail visibly; there is no fallback to another account.

```yaml
enabled: true # store.v3 §11 - false makes this instance inert
enabled: true
llm:
judge:
role: "fast" # recorded and logged; resolved when the host can
provider: "luna" # an amplifier provider id -> `amplifier run -p`
model: "" # optional -> `-m`
bundle: "" # optional -> `-B`
role: fast
provider: "" # optional configured named account
model: "" # optional exact model
bundle: "" # requires host-resolved inference
```

It lives *inside* the store it configures, because a store is an instance and there may
be more than one (store.v3 §1–§2): move the instance and its configuration moves with
it. `config.yaml` is plumbing, not memory — never injected, never suggested, never
cited. Only the keys you set become flags; with no file, or an empty one, the job runs
exactly the command it always ran. Every run's log line names which provider it used,
and `amplifier-memory doctor`'s `llm judge` row names the judge before the night rather
than after it — the provider you pinned, the role it resolved through, or `inherited`
with what that default was **measured** to cost per call, and so what a full 30-call
night can bill you.
Provider configuration retains explicit credential placeholders such as `${OPENAI_API_KEY}`;
process environment takes precedence over shared `keys.env` when resolving placeholders.
The dedicated `amplifier-memory suggest` process also loads missing shared keys into
its own environment for providers using implicit environment credentials. Embedding
hosts use `run_suggest`/`complete_once`; those APIs never change process environment.
No credential values are written to runtime receipts or usage logs. Requests have no tools,
no automatic retry, a 4096 output-token cap, and no fixed healthy-call completion deadline.
The job records actual provider/model and measured usage; missing cost remains unknown.
Historical evaluation costs below are observations, not current account estimates.

What the measurements say (7 model variants, 210 real calls, `evaluations/model-class/`):

Expand Down
23 changes: 15 additions & 8 deletions conformance/cli/run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
from contextlib import contextmanager, redirect_stdout
from datetime import UTC, datetime, timedelta
from pathlib import Path
from unittest.mock import patch

if __package__ in (None, ""): # allow `python conformance/cli/run.py`
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
Expand All @@ -55,7 +56,7 @@
#: steward's device on 2026-09-07 while only the pre-v3 pair existed (item `…-5wc`).
UNIT_NAME = re.compile(r"amplifier-memory-suggest[A-Za-z0-9._-]*\.(?:timer|service)")

CONTRACT_VERBS = ["init", "status", "why", "review", "doctor", "service", "update", "suggest"]
CONTRACT_VERBS = ["init", "status", "why", "review", "doctor", "service", "update", "suggest", "setup"]
HUMAN_IDENTITY = ("Test Human", "human@example.invalid")
SHA_A, SHA_B = "a" * 40, "b" * 40

Expand Down Expand Up @@ -419,18 +420,17 @@ def _judge_rows(home: Path) -> dict[str, str]:
--help` and the wording does not depend on what this host happens to document.
"""
from amplifier_memory import llm_config
from amplifier_memory.doctor import APP_DEFAULT, INHERITED, llm_row
from amplifier_memory.doctor import llm_row

config = home / llm_config.CONFIG_NAME
shipped = config.read_text(encoding="utf-8")
rows: dict[str, str] = {}

# 1. INHERITED - the shipped config names a role and no provider, and this host
# documents no `--model-role`, so the pass runs on the app's own default.
# 1. The shipped config requests a role, never silently a provider default.
inherited = llm_row(llm_config.load(home), home=home, help_text="")
assert inherited.level == "OK", inherited.render()
assert INHERITED in inherited.detail, inherited.detail
assert APP_DEFAULT in inherited.detail, "the inherited default is not named"
assert "role fast requested" in inherited.detail, inherited.detail
assert "unavailable resolver fails visibly" in inherited.detail.lower()
assert llm_config.CONFIG_NAME in inherited.detail, "the file is not named"
assert "role fast" in inherited.detail, "the recorded role is not named"
assert "no run yet, so no measured cost" in inherited.detail, "the measured cost is missing"
Expand All @@ -444,7 +444,7 @@ def _judge_rows(home: Path) -> dict[str, str]:
assert configured.level == "OK", configured.render()
assert "provider luna" in configured.detail and "model gpt-5.6-luna" in configured.detail
assert llm_config.CONFIG_NAME in configured.detail, configured.detail
assert INHERITED not in configured.detail, "a named judge inherits nothing"
assert "role fast requested" not in configured.detail, "a named judge inherits nothing"
rows["configured"] = configured.detail

# 3. UNUSABLE - WARN, never FAIL: the run still happens and still inherits, but the
Expand All @@ -453,7 +453,7 @@ def _judge_rows(home: Path) -> dict[str, str]:
unusable = llm_row(llm_config.load(home), home=home, help_text="")
assert unusable.level == "WARN", "a bad config file is not a broken store"
assert "not valid YAML" in unusable.detail, "the reason is not named"
assert INHERITED in unusable.detail and "remedy" in unusable.detail
assert "role fast requested" in unusable.detail and "remedy" in unusable.detail
rows["unusable"] = unusable.detail

# 4. The row and the daily job are ONE sentence, not two renderings of one fact
Expand Down Expand Up @@ -493,6 +493,7 @@ def probe_core_5() -> Verdict:
"suggest timer",
"substrate",
"llm judge",
"inference runtime",
"update",
], names
assert report.exit_code == 0
Expand Down Expand Up @@ -569,6 +570,8 @@ def probe_core_5() -> Verdict:
)


@patch("amplifier_memory.runtime.readiness", lambda home=None: {})
@patch("amplifier_memory.service.which_platform", lambda platform=None: platform or "systemd")
def probe_core_6() -> Verdict:
"""service: render units, daemon-reload -> enable --now, and roll back on a failed step.

Expand Down Expand Up @@ -752,6 +755,8 @@ def lay_pre_v3() -> None:
)


@patch("amplifier_memory.runtime.readiness", lambda home=None: {})
@patch("amplifier_memory.service.which_platform", lambda platform=None: platform or "systemd")
def probe_core_7() -> Verdict:
"""update: upgrade the tool, refresh **all three** installed things, end in doctor.

Expand Down Expand Up @@ -969,6 +974,8 @@ def probe_core_7() -> Verdict:
)


@patch("amplifier_memory.runtime.readiness", lambda home=None: {})
@patch("amplifier_memory.service.which_platform", lambda platform=None: platform or "systemd")
def probe_core_8() -> Verdict:
"""init: the store, the timer, and the three arms that install no timer.

Expand Down
Loading