Skip to content

Commit 7e0bea4

Browse files
authored
Merge slice/034-export-import-restore
Slice 034: export, import, restore. PROOF.md at slices/034-export-import-restore/PROOF.md; gate, postgres and fips jobs green on the closing tree and on the merge with slice 033. Signed-off-by: Ayla Croft <aylacroft@proton.me>
2 parents fe87ea4 + 6ef89e2 commit 7e0bea4

21 files changed

Lines changed: 1543 additions & 5 deletions

File tree

‎ROADMAP.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ standards register names the rows that ask for them.
4848
| 031 | Session search (SQLite FTS5) | 3 Memory | S | 010 | approved |
4949
| 032 | Embeddings + semantic memory + hybrid retrieval | 3 Memory | L | 031 | planned |
5050
| 033 | Project context: AGENTS.md | 3 Memory | S | 030, 022 | done |
51-
| 034 | Export, import, restore | 3 Memory | S | 030, 031 | planned |
51+
| 034 | Export, import, restore | 3 Memory | S | 030, 031 | done |
5252
| 040 | Skills registry + agentskills.io format + progressive disclosure | 4 Skills | M | 020 | planned |
5353
| 041 | Skill self-management with staged approval + scanner | 4 Skills | M | 040, 021 | planned |
5454
| 050 | Scheduler: Oban cron agent tasks with delivery targets | 5 Automation | M | 012 | planned |

‎coverage.tsv‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@ slice_id percent sha date
1414
031 76.96 866dc3f 2026-09-20
1515
030 78.40 c657da8 2026-09-21
1616
033 78.50 3b5e89e 2026-09-21
17+
034 78.83 3a5785f 2026-09-21

‎docs/backup.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Backup, restore, and moving to another machine
2+
3+
Slice 034. One archive holds a Trinity; a fresh install takes it back. Losing the machine is a recoverable
4+
event, not a total one.
5+
6+
## What the archive holds
7+
8+
`mix trinity.export --out <file.tar.gz>`, or the Export button on `/settings`, writes a gzip tarball with:
9+
10+
| Entry | What it is | Always |
11+
|---|---|---|
12+
| `manifest.json` | the format (`trinity-archive/1`), this build's version, when it was made, the migrated schema versions of each database, whether private keys are inside, and one line per file with its size and SHA-256 | yes |
13+
| `trinity.db` | the primary database: personas, sessions, messages, the search index, memories and their change log, approvals and rules | yes |
14+
| `receipts.db` | the receipt chains and their checkpoints | yes |
15+
| `keys/registry.json` | the key registry: public keys, algorithms, statuses (a stranger verifies receipts with this) | yes |
16+
| `keys/receipts-<algorithm>.key` | the private signing key | only with `--keys` (or `?keys=1` on the page) |
17+
| `skills/…`, `personas/…` | files under those directories, when present (skills arrive at slice 040) | when present |
18+
19+
Each database is snapshotted with `VACUUM INTO`: a consistent copy including what the write-ahead log held,
20+
taken while Trinity runs. The manifest says `keys_included` either way, so an archive's holder knows what it
21+
carries.
22+
23+
## What it does not hold
24+
25+
- Model caches, the desktop shell's state, the `LOCK` file, `RESTORED` markers from earlier restores.
26+
- The private key, unless asked for. An archive without it verifies every receipt ever written (the registry
27+
carries the public keys) and cannot write a new one as you: a fresh install without the key generates its
28+
own on first boot and appends it to the registry as a new key id; the old chain stays verifiable under the
29+
old key's row. Hand an archive to someone else without `--keys`.
30+
- Anything from other machines: this is one data directory, not a sync.
31+
32+
## Restoring
33+
34+
On a fresh install, with Trinity stopped:
35+
36+
```
37+
mix trinity.import trinity-2026-09-21.tar.gz
38+
```
39+
40+
The task reads the manifest, checks the schema versions against this binary's migrations (an archive from a
41+
newer Trinity refuses by name; an older one migrates on the next boot), verifies every file's digest against
42+
the tarball's bytes, and only then writes the files. A data directory that is not empty is refused with the
43+
paths it found; `--force` replaces them and prints each one. A directory held by a live Trinity is refused
44+
until that process stops. A `RESTORED` file in the data directory names the archive and the time.
45+
46+
Then start Trinity. The receipt chains verify under the restored registry (`mix trinity.receipts.verify --scope
47+
<scope>`, or the Verify button on a session's receipts page), which is the sharpest check that the copy is
48+
faithful. The search index is inside the database on SQLite; on Postgres it is a generated column.
49+
50+
`--data-dir <dir>` points either task at a directory other than the configured one.
51+
52+
## Postgres
53+
54+
The archive is the SQLite data directory's. A deployment on Postgres (`TRINITY_DB=postgres`) keeps its rows in
55+
that database and backs it up with Postgres's own tools (`pg_dump`); the export there carries the key registry
56+
and the directories, not the databases, and says so in the manifest's file list.
57+
58+
## Size
59+
60+
On the machine this was written on (2026-09-21), 18 sessions and 8,029 messages made a 713 KB archive (3.1 MB
61+
of database before compression). Vectors (slice 032) and any model cache are not measured yet; the slice that
62+
adds them reads this line.

‎lib/mix/tasks/trinity.export.ex‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
2+
# SPDX-License-Identifier: Apache-2.0
3+
defmodule Mix.Tasks.Trinity.Export do
4+
@shortdoc "Writes the data directory to one archive (databases, registry, skills; private keys only with --keys)"
5+
6+
@moduledoc """
7+
Slice 034 (`docs/backup.md`):
8+
9+
mix trinity.export --out trinity-2026-09-21.tar.gz
10+
mix trinity.export --out with-keys.tar.gz --keys
11+
12+
The databases are snapshotted with `VACUUM INTO`, so a running Trinity is fine. Without
13+
`--keys` the archive carries the key registry (public) and not the private key files, so it
14+
can be handed to someone without the ability to sign as you; the manifest records which.
15+
"""
16+
use Boundary, classify_to: Trinity
17+
use Mix.Task
18+
19+
@impl Mix.Task
20+
def run(argv) do
21+
{opts, _, _} =
22+
OptionParser.parse(argv, strict: [out: :string, keys: :boolean, data_dir: :string])
23+
24+
opts = Map.new(opts)
25+
26+
unless Map.has_key?(opts, :out) do
27+
Mix.shell().error(
28+
"usage: mix trinity.export --out <file.tar.gz> [--keys] [--data-dir <dir>]"
29+
)
30+
31+
exit({:shutdown, 2})
32+
end
33+
34+
Mix.Task.run("app.config")
35+
36+
layout =
37+
if d = opts[:data_dir],
38+
do: Trinity.Archive.Layout.of(d),
39+
else: Trinity.Archive.Layout.current()
40+
41+
case Trinity.Archive.export(layout, opts.out, keys: opts[:keys] == true) do
42+
{:ok, %{manifest: m, bytes: bytes}} ->
43+
Mix.shell().info(
44+
"exported #{length(m.files)} files, #{bytes} bytes, keys #{if m.keys_included, do: "included", else: "not included"}: #{opts.out}"
45+
)
46+
47+
{:error, reason} ->
48+
Mix.raise("export failed: #{inspect(reason)}")
49+
end
50+
end
51+
end

‎lib/mix/tasks/trinity.import.ex‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
2+
# SPDX-License-Identifier: Apache-2.0
3+
defmodule Mix.Tasks.Trinity.Import do
4+
@shortdoc "Restores an archive into an empty data directory (or replaces one with --force)"
5+
6+
@moduledoc """
7+
Slice 034 (`docs/backup.md`):
8+
9+
mix trinity.import trinity-2026-09-21.tar.gz
10+
mix trinity.import trinity-2026-09-21.tar.gz --force
11+
mix trinity.import trinity-2026-09-21.tar.gz --data-dir /path/to/dir
12+
13+
Stop Trinity first: the task refuses while a live process holds the data directory's lock.
14+
Every digest in the manifest is verified and the schema versions are checked against this
15+
binary before anything is written; a non-empty directory is refused unless `--force`, and
16+
then the task prints what it replaced. Exit 2 on usage, 1 on refusal.
17+
"""
18+
use Boundary, classify_to: Trinity
19+
use Mix.Task
20+
21+
@impl Mix.Task
22+
def run(argv) do
23+
{opts, args, _} = OptionParser.parse(argv, strict: [force: :boolean, data_dir: :string])
24+
25+
case args do
26+
[archive] ->
27+
Mix.Task.run("app.config")
28+
29+
layout =
30+
if d = opts[:data_dir],
31+
do: Trinity.Archive.Layout.of(d),
32+
else: Trinity.Archive.Layout.current()
33+
34+
case Trinity.DataDir.Lock.live_holder(layout.data_dir) do
35+
{:ok, %{pid: pid, mode: mode}} ->
36+
Mix.shell().error(
37+
"refused: a #{mode} Trinity (OS pid #{pid}) holds #{layout.data_dir}; stop it first"
38+
)
39+
40+
exit({:shutdown, 1})
41+
42+
:none ->
43+
import_into(archive, layout, opts[:force] == true)
44+
end
45+
46+
_ ->
47+
Mix.shell().error("usage: mix trinity.import <file.tar.gz> [--force] [--data-dir <dir>]")
48+
exit({:shutdown, 2})
49+
end
50+
end
51+
52+
defp import_into(archive, layout, force?) do
53+
File.mkdir_p!(layout.data_dir)
54+
55+
case Trinity.Archive.import(archive, layout, force: force?) do
56+
{:ok, %{manifest: m, written: written, replaced: replaced}} ->
57+
Enum.each(replaced, &Mix.shell().info("replaced #{&1}"))
58+
59+
Mix.shell().info(
60+
"restored #{length(written)} files from #{archive} (created #{m.created_at}, keys #{if m.keys_included, do: "included", else: "not included"}) into #{layout.data_dir}"
61+
)
62+
63+
{:error, {:not_empty, present}} ->
64+
Mix.shell().error(
65+
"refused: #{layout.data_dir} is not empty (#{Enum.join(present, ", ")}); --force replaces them"
66+
)
67+
68+
exit({:shutdown, 1})
69+
70+
{:error, reason} ->
71+
Mix.shell().error("refused: #{inspect(reason)}")
72+
exit({:shutdown, 1})
73+
end
74+
end
75+
end

‎lib/trinity.ex‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ defmodule Trinity do
2828
Personas,
2929
Context,
3030
Context.AgentsMd,
31+
Archive,
32+
Archive.Layout,
33+
Archive.Manifest,
3134
LLM,
3235
Memory,
3336
Memory.Tokens,

0 commit comments

Comments
 (0)