From fd546f2cf29bdb66a58eb0a0a3967054674eb29c Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Sun, 20 Sep 2026 21:54:47 -0400 Subject: [PATCH 1/9] docs(s034): G1 plan with the snapshot statement and the archive size measured, and the slice opens ROADMAP row 034 to in_progress. NOTES.md records VACUUM INTO at 9 ms over 8,029 messages and a 713 KB tarball for this machine's dev data, then the six-line build plan, the empty manual queue and two deviations stated before any code. plan_check: PASS. Signed-off-by: Ayla Croft --- ROADMAP.md | 2 +- slices/034-export-import-restore/NOTES.md | 49 +++++++++++++++++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) create mode 100644 slices/034-export-import-restore/NOTES.md diff --git a/ROADMAP.md b/ROADMAP.md index 5f63c77..be4522b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -48,7 +48,7 @@ standards register names the rows that ask for them. | 031 | Session search (SQLite FTS5) | 3 Memory | S | 010 | approved | | 032 | Embeddings + semantic memory + hybrid retrieval | 3 Memory | L | 031 | planned | | 033 | Project context: AGENTS.md | 3 Memory | S | 030, 022 | planned | -| 034 | Export, import, restore | 3 Memory | S | 030, 031 | planned | +| 034 | Export, import, restore | 3 Memory | S | 030, 031 | in_progress | | 040 | Skills registry + agentskills.io format + progressive disclosure | 4 Skills | M | 020 | planned | | 041 | Skill self-management with staged approval + scanner | 4 Skills | M | 040, 021 | planned | | 050 | Scheduler: Oban cron agent tasks with delivery targets | 5 Automation | M | 012 | planned | diff --git a/slices/034-export-import-restore/NOTES.md b/slices/034-export-import-restore/NOTES.md new file mode 100644 index 0000000..1529781 --- /dev/null +++ b/slices/034-export-import-restore/NOTES.md @@ -0,0 +1,49 @@ +# Slice 034: NOTES + +## Measured 2026-09-21 before any code + +**A consistent copy of a live SQLite file is one statement.** `VACUUM INTO ''` writes a compacted copy that +includes everything the WAL holds, without a checkpoint and without stopping writers; on this machine's dev +database (18 sessions, 8,029 messages, 3,096,576 bytes after the copy) it took 9 ms. The export uses it for both +databases, so the archive is a snapshot at one instant per file rather than a copy racing the writer. + +**Size today.** The two databases above plus the receipts file (5 receipts, 53,248 bytes) make a 713,024-byte +gzip tarball (`:erl_tar` with `:compressed`): a 4.4 to 1 ratio on message text. SLICE.md's risk line (embeddings +and model caches) has nothing to measure yet; 032 adds the first vectors and reads this number. + +## G1 plan, 2026-09-21 + +Tree at `294eecb` on `main` (030 and 031 approved; 033 on its own branch, not a dependency); +branch `slice/034-export-import-restore`; ROADMAP row 034 to `in_progress` in this commit. Each line names its test. + +1. `Trinity.Archive.Layout`: the files of a data directory by role (`trinity.db`, `receipts.db`, `keys/registry.json`, + `keys/receipts-*.key`, `skills/`, `personas/`), derived from a directory or given explicitly (the tests give the + suite's own paths); `Trinity.Archive.Manifest`: `format`, `app_version`, `created_at`, `schema_versions` (the + migrated versions of each repo), `keys_included`, and one `{path, sha256, bytes}` per file. Test: a manifest + round-trips through JSON and its digests are the files'. +2. `Trinity.Archive.export(layout, out, opts)`: `VACUUM INTO` for each database into a temporary directory, the + registry and (only with `keys: true`) the private key files copied, the manifest written, one gzip tarball. + Tests: AC5 (keys absent by default, present with the flag, the manifest saying which). +3. `Trinity.Archive.import(archive, layout, opts)`: reads the manifest, refuses a non-empty target with the files + it found unless `force: true`, in which case it states what it replaced; refuses `schema_versions` newer than + the binary's migrations by name; verifies every digest against the tarball's bytes **before** writing anything; + then writes the files and a `RESTORED` marker naming the archive. Tests: AC3 (both halves), AC4 (a byte flipped + inside the tarball: refused, the target still empty), AC6 (a manifest version bumped past the binary's). +4. AC1 and AC2 end to end, outside the sandbox (`Ecto.Adapters.SQL.Sandbox.unboxed_run/2`, because `VACUUM` + cannot run inside the sandbox's transaction and the test's rows must be on disk): a persona, a session with + messages, memories and a receipt chain written through the contexts; export; import into an empty directory; + the restored files opened through dynamic repos; the same rows; the digest over the restored tree equal to the + manifest's; the restored receipt chain verified with the restored registry. +5. `mix trinity.export --out [--keys]` and `mix trinity.import [--force] [--data-dir ]`; the import + refuses while another Trinity holds the data directory's lock. A `/settings` page (the first; 024's boot + receipt link moves there too) with the export as a download (`GET /settings/export.tar.gz`). Tests: the + controller streams a tarball whose manifest parses; the page links to it. +6. `docs/backup.md`: what is in the archive, what is not, the restore procedure, and what the private key's + absence or presence means; docs/05 unchanged (no table). + +Manual verification queue: none; every criterion is `[auto]`. + +Deviations stated before any code: (a) `skills/` and `personas/` are archived as directories when present and are +empty at this slice (skills arrive at 040; personas are rows in the database, restored with it); the "reindex +after restore" of the design note has nothing to reindex yet and is a follow-up for 040; (b) the "Settings +action" is the first `/settings` page, small, since none existed. From 874ea1f44485af9dde9849101196718ed50a52e3 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Sun, 20 Sep 2026 22:00:09 -0400 Subject: [PATCH 2/9] feat(s034): Trinity.Archive: export by VACUUM INTO, import verified before any write, the manifest Layout (the data directory's files by role, or the running app's), Manifest (format, app version, schema versions per database, keys included, a digest per file), export (each database snapshotted, the registry copied, private keys only with keys: true) and import (the manifest read, the schema versions checked against this binary's migration files, every digest verified against the tarball's bytes, a non-empty target refused or forced with what it replaced, then the files written and a RESTORED marker). Six tests on real files: AC1 and AC2 through dynamic repos on the restored databases, AC3 to AC6, the manifest's JSON. The first run refused this machine's test database, which carried 033's migration from another branch: the check works. Signed-off-by: Ayla Croft --- lib/trinity.ex | 3 + lib/trinity/archive.ex | 223 ++++++++++++++++ lib/trinity/archive/layout.ex | 126 +++++++++ lib/trinity/archive/manifest.ex | 114 +++++++++ test/trinity/archive/round_trip_test.exs | 310 +++++++++++++++++++++++ 5 files changed, 776 insertions(+) create mode 100644 lib/trinity/archive.ex create mode 100644 lib/trinity/archive/layout.ex create mode 100644 lib/trinity/archive/manifest.ex create mode 100644 test/trinity/archive/round_trip_test.exs diff --git a/lib/trinity.ex b/lib/trinity.ex index ac6dcaf..0d43c34 100644 --- a/lib/trinity.ex +++ b/lib/trinity.ex @@ -26,6 +26,9 @@ defmodule Trinity do Sessions.SessionRow, Sessions.Prompt, Personas, + Archive, + Archive.Layout, + Archive.Manifest, LLM, Memory, Memory.Tokens, diff --git a/lib/trinity/archive.ex b/lib/trinity/archive.ex new file mode 100644 index 0000000..1f63898 --- /dev/null +++ b/lib/trinity/archive.ex @@ -0,0 +1,223 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Archive do + @moduledoc """ + Export, import, restore (slice 034): the data directory as one gzip tarball a person can + take to another machine, and back. `export/3` snapshots each database with `VACUUM INTO` + (consistent, including what the WAL holds, without stopping the app), copies the key + registry and, only on the explicit flag, the private key files, and writes a manifest with + a digest per file and the schema versions. `import/3` verifies every digest against the + tarball's bytes and the schema versions against this binary's migrations before it writes + anything, refuses a non-empty target unless forced (and then says what it replaced), and + leaves a `RESTORED` marker naming the archive. + + The layout (`Trinity.Archive.Layout`) is the set of paths by role, derived from a data + directory or given explicitly, so the same functions serve the mix tasks, the settings + page and the tests, which work on their own directories. + """ + use Boundary, deps: [Trinity], exports: [Layout, Manifest] + + alias Trinity.Archive.{Layout, Manifest} + + @marker "RESTORED" + + @type export_result :: %{path: Path.t(), manifest: Manifest.t(), bytes: non_neg_integer()} + @type import_result :: %{manifest: Manifest.t(), written: [Path.t()], replaced: [Path.t()]} + + @doc """ + Writes the archive at `out`. `opts`: `keys: true` includes the private key files (the + manifest records the choice either way). + """ + @spec export(Layout.t(), Path.t(), keyword()) :: {:ok, export_result()} | {:error, term()} + def export(%Layout{} = layout, out, opts \\ []) do + keys? = Keyword.get(opts, :keys, false) + tmp = Path.join(System.tmp_dir!(), "trinity-export-#{System.unique_integer([:positive])}") + File.mkdir_p!(tmp) + + try do + with {:ok, staged} <- stage(layout, tmp, keys?), + manifest = Manifest.build(staged, schema_versions(layout), keys?), + :ok <- File.write(Path.join(tmp, Manifest.file_name()), Manifest.encode(manifest)), + entries = [ + {~c"manifest.json", String.to_charlist(Path.join(tmp, Manifest.file_name()))} + | staged_entries(staged) + ], + :ok <- :erl_tar.create(String.to_charlist(out), entries, [:compressed]) do + {:ok, %{path: out, manifest: manifest, bytes: File.stat!(out).size}} + end + after + File.rm_rf!(tmp) + end + end + + @doc """ + Restores an archive into a layout. `opts`: `force: true` replaces what a non-empty target + holds (listed in the result). Nothing is written until every digest and the schema versions + have been checked. + """ + @spec import(Path.t(), Layout.t(), keyword()) :: {:ok, import_result()} | {:error, term()} + def import(archive, %Layout{} = layout, opts \\ []) do + force? = Keyword.get(opts, :force, false) + + with {:ok, entries} <- read_tarball(archive), + {:ok, manifest} <- Manifest.from_entries(entries), + :ok <- check_schema(manifest), + :ok <- Manifest.verify(manifest, entries), + {:ok, replaced} <- clear_target(layout, force?) do + written = write_files(manifest, entries, layout) + File.write!(Path.join(layout.data_dir, @marker), marker(archive, manifest)) + {:ok, %{manifest: manifest, written: written, replaced: replaced}} + end + end + + @doc "The digest the manifest uses: SHA-256, lowercase hex." + @spec digest(binary()) :: String.t() + def digest(bytes) when is_binary(bytes), + do: :crypto.hash(:sha256, bytes) |> Base.encode16(case: :lower) + + @doc "A digest over a restored layout's files, by manifest path, to compare with the manifest." + @spec digest_tree(Layout.t(), Manifest.t()) :: %{String.t() => String.t()} + def digest_tree(%Layout{} = layout, %Manifest{files: files}) do + Map.new(files, fn %{"path" => rel} -> + abs = Layout.absolute(layout, rel) + {rel, if(File.regular?(abs), do: digest(File.read!(abs)), else: nil)} + end) + end + + @doc "The migration versions this binary carries for a repo, from its migration files." + @spec available_versions(module()) :: [integer()] + def available_versions(repo) do + repo + |> Ecto.Migrator.migrations_path() + |> Path.join("*.exs") + |> Path.wildcard() + |> Enum.map(fn f -> + f |> Path.basename() |> String.split("_", parts: 2) |> hd() |> String.to_integer() + end) + |> Enum.sort() + end + + @doc "The marker file's name." + @spec marker_file() :: String.t() + def marker_file, do: @marker + + ## Export + + # Each database is snapshotted with VACUUM INTO through exqlite directly: no Ecto, no + # transaction, so it runs beside a live app and inside a test that must see disk. + defp stage(layout, tmp, keys?) do + Enum.reduce_while(Layout.sources(layout, keys?), {:ok, []}, fn {rel, abs, kind}, {:ok, acc} -> + dest = Path.join(tmp, rel) + File.mkdir_p!(Path.dirname(dest)) + + result = + case kind do + :sqlite -> snapshot(abs, dest) + :file -> File.cp(abs, dest) + end + + case result do + :ok -> + {:cont, + {:ok, + acc ++ + [ + %{ + path: rel, + staged: dest, + bytes: File.stat!(dest).size, + sha256: digest(File.read!(dest)) + } + ]}} + + {:error, reason} -> + {:halt, {:error, {:stage, rel, reason}}} + end + end) + end + + defp snapshot(src, dest) do + with {:ok, db} <- Exqlite.Sqlite3.open(src), + :ok <- Exqlite.Sqlite3.execute(db, "VACUUM INTO '#{String.replace(dest, "'", "''")}'"), + :ok <- Exqlite.Sqlite3.close(db) do + :ok + end + end + + defp staged_entries(staged), + do: Enum.map(staged, &{String.to_charlist(&1.path), String.to_charlist(&1.staged)}) + + # The migrated versions of each database, read from the source file itself. + defp schema_versions(layout) do + for {name, path, table} <- [ + {"trinity.db", layout.db, "schema_migrations"}, + {"receipts.db", layout.receipts_db, "receipts_schema_migrations"} + ], + File.regular?(path), + into: %{} do + {name, migrated(path, table)} + end + end + + defp migrated(path, table) do + with {:ok, db} <- Exqlite.Sqlite3.open(path), + {:ok, st} <- Exqlite.Sqlite3.prepare(db, "SELECT version FROM #{table} ORDER BY version"), + {:ok, rows} <- Exqlite.Sqlite3.fetch_all(db, st), + :ok <- Exqlite.Sqlite3.release(db, st), + :ok <- Exqlite.Sqlite3.close(db) do + List.flatten(rows) + else + _ -> [] + end + end + + ## Import + + defp read_tarball(archive) do + case :erl_tar.extract(String.to_charlist(archive), [:compressed, :memory]) do + {:ok, entries} -> {:ok, Map.new(entries, fn {name, bin} -> {List.to_string(name), bin} end)} + {:error, reason} -> {:error, {:archive, reason}} + end + end + + # A newer archive into an older binary refuses by name; the other way migrates on boot. + defp check_schema(%Manifest{schema_versions: versions}) do + binary = %{ + "trinity.db" => available_versions(Trinity.Repo), + "receipts.db" => available_versions(Trinity.Repo.Receipts) + } + + newer = + for {db, archived} <- versions, + have = Map.get(binary, db, []), + extra = archived -- have, + extra != [], + do: {db, extra} + + if newer == [], do: :ok, else: {:error, {:schema_newer_than_binary, newer}} + end + + defp clear_target(layout, force?) do + present = Layout.present(layout) + + cond do + present == [] -> {:ok, []} + force? -> Enum.each(present, &File.rm_rf!/1) && {:ok, present} + true -> {:error, {:not_empty, present}} + end + end + + defp write_files(%Manifest{files: files}, entries, layout) do + for %{"path" => rel} <- files do + abs = Layout.absolute(layout, rel) + File.mkdir_p!(Path.dirname(abs)) + File.write!(abs, Map.fetch!(entries, rel)) + if String.starts_with?(rel, "keys/"), do: File.chmod!(abs, 0o600) + abs + end + end + + defp marker(archive, %Manifest{} = m) do + "restored from #{Path.basename(archive)} (created #{m.created_at}, format #{m.format}, keys #{if m.keys_included, do: "included", else: "not included"}) at #{DateTime.utc_now() |> DateTime.to_iso8601()}\n" + end +end diff --git a/lib/trinity/archive/layout.ex b/lib/trinity/archive/layout.ex new file mode 100644 index 0000000..3954453 --- /dev/null +++ b/lib/trinity/archive/layout.ex @@ -0,0 +1,126 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Archive.Layout do + @moduledoc """ + The files of a data directory by role (slice 034): the two databases, the key registry + and the private key files under `keys/`, and the `skills/` and `personas/` directories + (empty at this slice; 040 fills skills). `of/1` derives it from a directory; the tests and + the tasks may give the paths themselves. + """ + + @type t :: %__MODULE__{ + data_dir: Path.t(), + db: Path.t(), + receipts_db: Path.t(), + keys_dir: Path.t(), + skills_dir: Path.t(), + personas_dir: Path.t() + } + + @enforce_keys [:data_dir, :db, :receipts_db, :keys_dir] + defstruct [:data_dir, :db, :receipts_db, :keys_dir, :skills_dir, :personas_dir] + + @doc "The layout of a data directory." + @spec of(Path.t()) :: t() + def of(data_dir) do + %__MODULE__{ + data_dir: data_dir, + db: Path.join(data_dir, "trinity.db"), + receipts_db: Path.join(data_dir, "receipts.db"), + keys_dir: Path.join(data_dir, "keys"), + skills_dir: Path.join(data_dir, "skills"), + personas_dir: Path.join(data_dir, "personas") + } + end + + @doc "The running application's layout: its configured databases and keys directory." + @spec current() :: t() + def current do + data_dir = Trinity.Paths.data_dir() + + %__MODULE__{ + data_dir: data_dir, + db: + Application.get_env(:trinity, Trinity.Repo)[:database] || + Path.join(data_dir, "trinity.db"), + receipts_db: + Application.get_env(:trinity, Trinity.Repo.Receipts)[:database] || + Path.join(data_dir, "receipts.db"), + keys_dir: + Application.get_env(:trinity, :receipts, [])[:keys_dir] || Path.join(data_dir, "keys"), + skills_dir: Path.join(data_dir, "skills"), + personas_dir: Path.join(data_dir, "personas") + } + end + + @doc """ + What an export takes, as `{archive path, absolute path, kind}`: the databases (`:sqlite`, + snapshotted), the registry and, with `keys?`, the private key files, and every file under + `skills/` and `personas/` (`:file`, copied). Only what exists. + """ + @spec sources(t(), boolean()) :: [{String.t(), Path.t(), :sqlite | :file}] + def sources(%__MODULE__{} = l, keys?) do + dbs = + for {rel, abs} <- [{"trinity.db", l.db}, {"receipts.db", l.receipts_db}], + File.regular?(abs), + do: {rel, abs, :sqlite} + + registry = Path.join(l.keys_dir, "registry.json") + reg = if File.regular?(registry), do: [{"keys/registry.json", registry, :file}], else: [] + + keys = + if keys?, + do: + for( + f <- Path.wildcard(Path.join(l.keys_dir, "receipts-*.key")), + do: {"keys/" <> Path.basename(f), f, :file} + ), + else: [] + + dirs = + for {name, dir} <- [{"skills", l.skills_dir}, {"personas", l.personas_dir}], + is_binary(dir) and File.dir?(dir), + f <- Path.wildcard(Path.join(dir, "**/*")), + File.regular?(f), + do: {Path.join(name, Path.relative_to(f, dir)), f, :file} + + dbs ++ reg ++ keys ++ dirs + end + + @doc "The absolute path a manifest path lands at in this layout." + @spec absolute(t(), String.t()) :: Path.t() + def absolute(%__MODULE__{} = l, "trinity.db"), do: l.db + def absolute(%__MODULE__{} = l, "receipts.db"), do: l.receipts_db + def absolute(%__MODULE__{} = l, "keys/" <> rest), do: Path.join(l.keys_dir, rest) + + def absolute(%__MODULE__{} = l, "skills/" <> rest), + do: Path.join(l.skills_dir || Path.join(l.data_dir, "skills"), rest) + + def absolute(%__MODULE__{} = l, "personas/" <> rest), + do: Path.join(l.personas_dir || Path.join(l.data_dir, "personas"), rest) + + def absolute(%__MODULE__{} = l, other), do: Path.join(l.data_dir, other) + + @doc "What is present of this layout (the files and directories an import would replace)." + @spec present(t()) :: [Path.t()] + def present(%__MODULE__{} = l) do + files = + for p <- [ + l.db, + l.db <> "-wal", + l.db <> "-shm", + l.receipts_db, + l.receipts_db <> "-wal", + l.receipts_db <> "-shm" + ], + File.regular?(p), + do: p + + dirs = + for d <- [l.keys_dir, l.skills_dir, l.personas_dir], + is_binary(d) and File.dir?(d) and File.ls!(d) != [], + do: d + + files ++ dirs + end +end diff --git a/lib/trinity/archive/manifest.ex b/lib/trinity/archive/manifest.ex new file mode 100644 index 0000000..fe8e5ac --- /dev/null +++ b/lib/trinity/archive/manifest.ex @@ -0,0 +1,114 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Archive.Manifest do + @moduledoc """ + The archive's manifest (slice 034): the format, this build's version, when it was made, + the migrated schema versions of each database, whether the private keys are inside, and + one entry per file with its size and SHA-256. It is the first thing an import reads and + the thing every write is checked against. + """ + + @format "trinity-archive/1" + @file_name "manifest.json" + + @type t :: %__MODULE__{ + format: String.t(), + app_version: String.t(), + created_at: String.t(), + schema_versions: %{String.t() => [integer()]}, + keys_included: boolean(), + files: [%{String.t() => term()}] + } + + defstruct format: @format, + app_version: nil, + created_at: nil, + schema_versions: %{}, + keys_included: false, + files: [] + + @doc "The manifest's file name inside the archive." + @spec file_name() :: String.t() + def file_name, do: @file_name + + @doc "The format string." + @spec format() :: String.t() + def format, do: @format + + @doc "Builds a manifest from staged files (`%{path, bytes, sha256}`)." + @spec build([map()], map(), boolean()) :: t() + def build(staged, schema_versions, keys?) do + %__MODULE__{ + app_version: to_string(Application.spec(:trinity, :vsn)), + created_at: DateTime.utc_now() |> DateTime.to_iso8601(), + schema_versions: schema_versions, + keys_included: keys?, + files: Enum.map(staged, &%{"path" => &1.path, "bytes" => &1.bytes, "sha256" => &1.sha256}) + } + end + + @doc "JSON." + @spec encode(t()) :: String.t() + def encode(%__MODULE__{} = m) do + JSON.encode!(%{ + "format" => m.format, + "app_version" => m.app_version, + "created_at" => m.created_at, + "schema_versions" => m.schema_versions, + "keys_included" => m.keys_included, + "files" => m.files + }) + end + + @doc "From JSON; refuses another format." + @spec decode(binary()) :: {:ok, t()} | {:error, term()} + def decode(bin) do + case JSON.decode(bin) do + {:ok, %{"format" => @format} = m} -> + {:ok, + %__MODULE__{ + format: @format, + app_version: m["app_version"], + created_at: m["created_at"], + schema_versions: m["schema_versions"] || %{}, + keys_included: m["keys_included"] == true, + files: m["files"] || [] + }} + + {:ok, %{"format" => other}} -> + {:error, {:unknown_format, other}} + + {:ok, _} -> + {:error, :not_a_manifest} + + {:error, reason} -> + {:error, {:manifest_json, reason}} + end + end + + @doc "The manifest out of an extracted tarball's entries." + @spec from_entries(%{String.t() => binary()}) :: {:ok, t()} | {:error, term()} + def from_entries(entries) do + case Map.fetch(entries, @file_name) do + {:ok, bin} -> decode(bin) + :error -> {:error, :no_manifest} + end + end + + @doc "Every file named in the manifest is in the entries with the manifest's digest and size; nothing else is required." + @spec verify(t(), %{String.t() => binary()}) :: :ok | {:error, term()} + def verify(%__MODULE__{files: files}, entries) do + # Not a comprehension filter: a `nil` bound in a `for` filter skips the element, and a + # missing file is exactly what must not be skipped (found by AC4's second half). + bad = + Enum.flat_map(files, fn %{"path" => path, "sha256" => sha, "bytes" => bytes} -> + case Map.get(entries, path) do + nil -> [{path, :missing}] + bin when byte_size(bin) != bytes -> [{path, :digest_mismatch}] + bin -> if Trinity.Archive.digest(bin) == sha, do: [], else: [{path, :digest_mismatch}] + end + end) + + if bad == [], do: :ok, else: {:error, {:verification_failed, bad}} + end +end diff --git a/test/trinity/archive/round_trip_test.exs b/test/trinity/archive/round_trip_test.exs new file mode 100644 index 0000000..f48658d --- /dev/null +++ b/test/trinity/archive/round_trip_test.exs @@ -0,0 +1,310 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Archive.RoundTripTest do + @moduledoc """ + Slice 034. AC1, AC2 and AC5 end to end on real files: the sandbox is switched to `:auto` + for these tests (VACUUM cannot run inside the sandbox's transaction, and the rows must be + on disk for the snapshot to carry them), the rows are made through the contexts, exported, + imported into an empty directory, and read back through dynamic repos on the restored files. + AC3, AC4 and AC6 work on the archive and the target directory alone. + """ + use ExUnit.Case, async: false + + import Ecto.Query, only: [from: 2] + + alias Ecto.Adapters.SQL.Sandbox + alias Trinity.{Archive, Receipts, Sessions} + alias Trinity.Archive.{Layout, Manifest} + alias Trinity.Memory.AlwaysOn + + setup do + Sandbox.mode(Trinity.Repo, :auto) + Sandbox.mode(Trinity.Repo.Receipts, :auto) + tmp = Path.join(System.tmp_dir!(), "trinity-archive-#{System.unique_integer([:positive])}") + File.mkdir_p!(tmp) + + on_exit(fn -> + Sandbox.mode(Trinity.Repo, :manual) + Sandbox.mode(Trinity.Repo.Receipts, :manual) + File.rm_rf!(tmp) + end) + + {:ok, tmp: tmp, source: Layout.current()} + end + + # A persona with a session, messages, memories and a receipt chain, all committed; the + # cleanup deletes the persona (cascading the rest) and the receipts. + defp seed do + {:ok, persona} = + Sessions.create_persona(%{ + name: "archive-#{System.unique_integer([:positive])}", + soul: "# Archived" + }) + + {:ok, session} = + Sessions.create_session(%{ + persona_id: persona.id, + origin: "desktop", + title: "To be archived" + }) + + for i <- 1..5, + do: + {:ok, _} = + Sessions.append_message(session.id, %{ + role: "user", + content: "message #{i} about widgets" + }) + + {:ok, _} = + AlwaysOn.add( + %{ + persona_id: persona.id, + tier: "always_on", + scope: AlwaysOn.persona_scope(persona.id), + key: "editor", + body: "neovim" + }, by: "test") + + scope = Receipts.session_scope(session.id) + + for i <- 1..6, + do: + {:ok, _} = + Receipts.append(scope, %{ + kind: if(rem(i, 2) == 0, do: "decision", else: "query"), + subject: %{"i" => i}, + decision: %{"outcome" => "allow"} + }) + + :ok = Receipts.stop_writer(scope) + %{persona: persona, session: session, scope: scope} + end + + defp unseed(%{persona: persona, session: session, scope: scope}) do + Trinity.Repo.delete_all( + from(m in Trinity.Sessions.Message, where: m.session_id == ^session.id) + ) + + Trinity.Repo.delete_all(from(e in Trinity.Memory.Entry, where: e.persona_id == ^persona.id)) + Trinity.Repo.delete_all(from(c in Trinity.Memory.Change, where: c.persona_id == ^persona.id)) + Trinity.Repo.delete_all(from(s in Trinity.Sessions.SessionRow, where: s.id == ^session.id)) + Trinity.Repo.delete_all(from(p in Trinity.Sessions.Persona, where: p.id == ^persona.id)) + Trinity.Repo.Receipts.delete_all(from(r in Receipts.Receipt, where: r.chain_scope == ^scope)) + + Trinity.Repo.Receipts.delete_all( + from(c in Receipts.Checkpoint, where: c.chain_scope == ^scope) + ) + end + + # Opens the restored databases as dynamic repos for the duration of `fun`. + defp on_restored(%Layout{} = layout, fun) do + {:ok, db} = Trinity.Repo.start_link(name: :restored_db, database: layout.db, pool_size: 1) + + {:ok, rdb} = + Trinity.Repo.Receipts.start_link( + name: :restored_receipts, + database: layout.receipts_db, + pool_size: 1 + ) + + Trinity.Repo.put_dynamic_repo(:restored_db) + Trinity.Repo.Receipts.put_dynamic_repo(:restored_receipts) + + try do + fun.() + after + Trinity.Repo.put_dynamic_repo(Trinity.Repo) + Trinity.Repo.Receipts.put_dynamic_repo(Trinity.Repo.Receipts) + GenServer.stop(db) + GenServer.stop(rdb) + end + end + + test "AC1 and AC2: export, import into an empty directory, the rows and the chain come back, the tree's digests match the manifest", + %{tmp: tmp, source: source} do + seeded = seed() + on_exit(fn -> unseed(seeded) end) + archive = Path.join(tmp, "trinity.tar.gz") + + assert {:ok, %{manifest: %Manifest{} = manifest, bytes: bytes}} = + Archive.export(source, archive) + + assert bytes > 0 + + assert Enum.map(manifest.files, & &1["path"]) |> Enum.sort() == [ + "keys/registry.json", + "receipts.db", + "trinity.db" + ] + + assert manifest.keys_included == false + assert manifest.schema_versions["trinity.db"] == Archive.available_versions(Trinity.Repo) + + assert manifest.schema_versions["receipts.db"] == + Archive.available_versions(Trinity.Repo.Receipts) + + target = Layout.of(Path.join(tmp, "restored")) + File.mkdir_p!(target.data_dir) + assert {:ok, %{replaced: [], written: written}} = Archive.import(archive, target) + assert length(written) == 3 + assert File.read!(Path.join(target.data_dir, "RESTORED")) =~ "restored from trinity.tar.gz" + + # The digest over the restored tree is the manifest's, file for file. + tree = Archive.digest_tree(target, manifest) + assert tree == Map.new(manifest.files, &{&1["path"], &1["sha256"]}) + + on_restored(target, fn -> + persona = Sessions.get_persona(seeded.persona.id) + assert persona.soul == "# Archived" + session = Sessions.get_session(seeded.session.id) + assert session.title == "To be archived" + + assert Enum.map(Sessions.history(session.id), & &1.content) == + for(i <- 1..5, do: "message #{i} about widgets") + + assert [%{key: "editor", body: "neovim"}] = AlwaysOn.all(persona.id) + + # AC2: the chain, read from the restored file, verified with the restored registry. + rows = Receipts.list(seeded.scope) + assert length(rows) == 6 + {:ok, registry} = Receipts.KeyRegistry.read(target.keys_dir) + + export = %{ + "receipts" => Enum.map(rows, &Receipts.Receipt.to_export/1), + "checkpoints" => + Enum.map(Receipts.checkpoints(seeded.scope), &Receipts.Checkpoint.to_export/1), + "registry" => registry + } + + assert {:ok, %{receipts: 6}} = Receipts.Verifier.verify(export) + end) + end + + test "AC5: private keys are absent from a default export and present with keys: true; the manifest records which", + %{tmp: tmp, source: source} do + assert {:ok, %{manifest: plain}} = Archive.export(source, Path.join(tmp, "plain.tar.gz")) + refute plain.keys_included + refute Enum.any?(plain.files, &String.match?(&1["path"], ~r/^keys\/receipts-.*\.key$/)) + assert Enum.any?(plain.files, &(&1["path"] == "keys/registry.json")) + + assert {:ok, %{manifest: with_keys}} = + Archive.export(source, Path.join(tmp, "keys.tar.gz"), keys: true) + + assert with_keys.keys_included + assert Enum.any?(with_keys.files, &String.match?(&1["path"], ~r/^keys\/receipts-.*\.key$/)) + + {:ok, entries} = + :erl_tar.extract(String.to_charlist(Path.join(tmp, "plain.tar.gz")), [:compressed, :memory]) + + names = Enum.map(entries, fn {n, _} -> List.to_string(n) end) + refute Enum.any?(names, &String.ends_with?(&1, ".key")) + assert "manifest.json" in names + + # Restored with the keys, the key file has the mode the custody expects. + target = Layout.of(Path.join(tmp, "with-keys")) + File.mkdir_p!(target.data_dir) + assert {:ok, _} = Archive.import(Path.join(tmp, "keys.tar.gz"), target) + [key] = Path.wildcard(Path.join(target.keys_dir, "receipts-*.key")) + assert File.stat!(key).mode |> Bitwise.band(0o777) == 0o600 + end + + test "AC3: a non-empty target refuses with what it found; forced, the result states what it replaced", + %{tmp: tmp, source: source} do + archive = Path.join(tmp, "a.tar.gz") + {:ok, _} = Archive.export(source, archive) + target = Layout.of(Path.join(tmp, "busy")) + File.mkdir_p!(target.keys_dir) + File.write!(target.db, "not a database") + File.write!(Path.join(target.keys_dir, "registry.json"), "[]") + + assert {:error, {:not_empty, present}} = Archive.import(archive, target) + assert Enum.sort(present) == Enum.sort([target.db, target.keys_dir]) + assert File.read!(target.db) == "not a database" + + assert {:ok, %{replaced: replaced}} = Archive.import(archive, target, force: true) + assert Enum.sort(replaced) == Enum.sort([target.db, target.keys_dir]) + assert File.read!(target.db) != "not a database" + end + + test "AC4: a tampered archive fails digest verification before anything is written", %{ + tmp: tmp, + source: source + } do + archive = Path.join(tmp, "good.tar.gz") + {:ok, _} = Archive.export(source, archive) + {:ok, entries} = :erl_tar.extract(String.to_charlist(archive), [:compressed, :memory]) + + tampered = + Enum.map(entries, fn + {~c"receipts.db", bin} -> {~c"receipts.db", flip(bin)} + other -> other + end) + + bad = Path.join(tmp, "bad.tar.gz") + + :ok = + :erl_tar.create(String.to_charlist(bad), Enum.map(tampered, fn {n, b} -> {n, b} end), [ + :compressed + ]) + + target = Layout.of(Path.join(tmp, "untouched")) + File.mkdir_p!(target.data_dir) + + assert {:error, {:verification_failed, [{"receipts.db", :digest_mismatch}]}} = + Archive.import(bad, target) + + assert File.ls!(target.data_dir) == [] + + # A manifest that names a file the tarball lacks is refused the same way, before writing. + missing = Enum.reject(entries, fn {n, _} -> n == ~c"receipts.db" end) + lacking = Path.join(tmp, "lacking.tar.gz") + :ok = :erl_tar.create(String.to_charlist(lacking), missing, [:compressed]) + + assert {:error, {:verification_failed, [{"receipts.db", :missing}]}} = + Archive.import(lacking, target) + + assert File.ls!(target.data_dir) == [] + end + + test "AC6: an archive from a newer schema refuses to import with the versions named", %{ + tmp: tmp, + source: source + } do + archive = Path.join(tmp, "newer.tar.gz") + {:ok, %{manifest: manifest}} = Archive.export(source, archive) + {:ok, entries} = :erl_tar.extract(String.to_charlist(archive), [:compressed, :memory]) + future = 20_991_231_000_000 + + bumped = %{ + manifest + | schema_versions: Map.update!(manifest.schema_versions, "trinity.db", &(&1 ++ [future])) + } + + rewritten = + Enum.map(entries, fn + {~c"manifest.json", _} -> {~c"manifest.json", Manifest.encode(bumped)} + other -> other + end) + + newer = Path.join(tmp, "newer2.tar.gz") + :ok = :erl_tar.create(String.to_charlist(newer), rewritten, [:compressed]) + target = Layout.of(Path.join(tmp, "old-binary")) + File.mkdir_p!(target.data_dir) + + assert {:error, {:schema_newer_than_binary, [{"trinity.db", [^future]}]}} = + Archive.import(newer, target) + + assert File.ls!(target.data_dir) == [] + end + + test "a manifest round-trips through JSON; another format is refused" do + m = Manifest.build([%{path: "x", bytes: 1, sha256: "ab"}], %{"trinity.db" => [1, 2]}, true) + assert {:ok, ^m} = Manifest.decode(Manifest.encode(m)) + assert {:error, {:unknown_format, "other/9"}} = Manifest.decode(~s({"format":"other/9"})) + assert {:error, :not_a_manifest} = Manifest.decode(~s({"x":1})) + end + + defp flip(<>), + do: <> +end From e7a8e387a9139aabe8896696c988a44cbbd8498b Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Sun, 20 Sep 2026 22:09:29 -0400 Subject: [PATCH 3/9] fix(s030): the planted-policy test no longer puts a trinity-* directory on the code path :code.lib_dir/1 takes a code-path entry named - as the application's directory, so the prepended /tmp/trinity-policy-N made Application.app_dir(:trinity) point at /tmp for every later test; slice 034's migration listing came back empty. The directory is planted-policy-N and the path is removed on exit. The persona picker test reads the session it was redirected to rather than the newest row on disk. Signed-off-by: Ayla Croft --- test/trinity/archive/round_trip_test.exs | 4 +++- test/trinity/effects/boot_receipt_test.exs | 8 +++++++- test/trinity_web/live/memory_pages_test.exs | 5 ++--- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/test/trinity/archive/round_trip_test.exs b/test/trinity/archive/round_trip_test.exs index f48658d..c9dd680 100644 --- a/test/trinity/archive/round_trip_test.exs +++ b/test/trinity/archive/round_trip_test.exs @@ -64,7 +64,9 @@ defmodule Trinity.Archive.RoundTripTest do scope: AlwaysOn.persona_scope(persona.id), key: "editor", body: "neovim" - }, by: "test") + }, + by: "test" + ) scope = Receipts.session_scope(session.id) diff --git a/test/trinity/effects/boot_receipt_test.exs b/test/trinity/effects/boot_receipt_test.exs index b16ed64..9b1ec7d 100644 --- a/test/trinity/effects/boot_receipt_test.exs +++ b/test/trinity/effects/boot_receipt_test.exs @@ -55,7 +55,11 @@ defmodule Trinity.Effects.BootReceiptTest do {:module, ^mod, binary, _} = Module.create(mod, body, Macro.Env.location(__ENV__)) # get_object_code/1 reads the loaded module's binary through the code path; a module # created in memory has none, so the test writes it where the code server looks. - dir = Path.join(System.tmp_dir!(), "trinity-policy-#{System.unique_integer([:positive])}") + # Not "trinity-": :code.lib_dir/1 reads a code-path entry named like + # `-` as the application's directory, and a prepended "trinity-policy-N" + # made Application.app_dir(:trinity) point at /tmp for every later test (slice 034 + # found it: the migration files vanished). The path is removed on exit as well. + dir = Path.join(System.tmp_dir!(), "planted-policy-#{System.unique_integer([:positive])}") File.mkdir_p!(dir) File.write!(Path.join(dir, "#{mod}.beam"), binary) :code.purge(mod) @@ -72,6 +76,8 @@ defmodule Trinity.Effects.BootReceiptTest do h2 = CorePolicy.hash_of([mod]) on_exit(fn -> + Code.delete_path(d1) + Code.delete_path(d2) File.rm_rf!(d1) File.rm_rf!(d2) end) diff --git a/test/trinity_web/live/memory_pages_test.exs b/test/trinity_web/live/memory_pages_test.exs index ba02f6b..fd3deab 100644 --- a/test/trinity_web/live/memory_pages_test.exs +++ b/test/trinity_web/live/memory_pages_test.exs @@ -131,8 +131,7 @@ defmodule TrinityWeb.MemoryPagesTest do {:ok, view, _html} = live(conn, ~p"/") view |> form("#persona-picker", persona_id: persona.id) |> render_change() view |> element("#new-session") |> render_click() - assert_redirect(view) - [session | _] = Trinity.Sessions.list_sessions(limit: 1) - assert session.persona_id == persona.id + {"/s/" <> id, _flash} = assert_redirect(view) + assert Trinity.Sessions.get_session(id).persona_id == persona.id end end From 59839e9878e3150a3c9a630ad433ecb84c826668 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Sun, 20 Sep 2026 22:11:57 -0400 Subject: [PATCH 4/9] feat(s034): the export and import tasks, the settings page with the download, docs/backup.md mix trinity.export (--out, --keys, --data-dir) and mix trinity.import (--force, --data-dir), the import refusing a data directory a live Trinity holds (Lock.live_holder/1) before anything else. The first /settings page: the export as a download with and without the private key (ExportController), the data directory, links. docs/backup.md: what the archive holds and does not, the restore procedure, the size measured. Five tests. Signed-off-by: Ayla Croft --- docs/backup.md | 56 +++++++++++++ lib/mix/tasks/trinity.export.ex | 51 ++++++++++++ lib/mix/tasks/trinity.import.ex | 75 +++++++++++++++++ lib/trinity/data_dir/lock.ex | 9 +++ .../controllers/export_controller.ex | 31 +++++++ lib/trinity_web/live/session_live/index.ex | 3 + lib/trinity_web/live/settings_live.ex | 81 +++++++++++++++++++ lib/trinity_web/router.ex | 3 + test/mix/trinity_import_task_test.exs | 60 ++++++++++++++ test/trinity_web/settings_export_test.exs | 55 +++++++++++++ 10 files changed, 424 insertions(+) create mode 100644 docs/backup.md create mode 100644 lib/mix/tasks/trinity.export.ex create mode 100644 lib/mix/tasks/trinity.import.ex create mode 100644 lib/trinity_web/controllers/export_controller.ex create mode 100644 lib/trinity_web/live/settings_live.ex create mode 100644 test/mix/trinity_import_task_test.exs create mode 100644 test/trinity_web/settings_export_test.exs diff --git a/docs/backup.md b/docs/backup.md new file mode 100644 index 0000000..4e72792 --- /dev/null +++ b/docs/backup.md @@ -0,0 +1,56 @@ +# Backup, restore, and moving to another machine + +Slice 034. One archive holds a Trinity; a fresh install takes it back. Losing the machine is a recoverable +event, not a total one. + +## What the archive holds + +`mix trinity.export --out `, or the Export button on `/settings`, writes a gzip tarball with: + +| Entry | What it is | Always | +|---|---|---| +| `manifest.json` | the format (`trinity-archive/1`), this build's version, when it was made, the migrated schema versions of each database, whether private keys are inside, and one line per file with its size and SHA-256 | yes | +| `trinity.db` | the primary database: personas, sessions, messages, the search index, memories and their change log, approvals and rules | yes | +| `receipts.db` | the receipt chains and their checkpoints | yes | +| `keys/registry.json` | the key registry: public keys, algorithms, statuses (a stranger verifies receipts with this) | yes | +| `keys/receipts-.key` | the private signing key | only with `--keys` (or `?keys=1` on the page) | +| `skills/…`, `personas/…` | files under those directories, when present (skills arrive at slice 040) | when present | + +Each database is snapshotted with `VACUUM INTO`: a consistent copy including what the write-ahead log held, +taken while Trinity runs. The manifest says `keys_included` either way, so an archive's holder knows what it +carries. + +## What it does not hold + +- Model caches, the desktop shell's state, the `LOCK` file, `RESTORED` markers from earlier restores. +- The private key, unless asked for. An archive without it verifies every receipt ever written (the registry + carries the public keys) and cannot write a new one as you: a fresh install without the key generates its + own on first boot and appends it to the registry as a new key id; the old chain stays verifiable under the + old key's row. Hand an archive to someone else without `--keys`. +- Anything from other machines: this is one data directory, not a sync. + +## Restoring + +On a fresh install, with Trinity stopped: + +``` +mix trinity.import trinity-2026-09-21.tar.gz +``` + +The task reads the manifest, checks the schema versions against this binary's migrations (an archive from a +newer Trinity refuses by name; an older one migrates on the next boot), verifies every file's digest against +the tarball's bytes, and only then writes the files. A data directory that is not empty is refused with the +paths it found; `--force` replaces them and prints each one. A directory held by a live Trinity is refused +until that process stops. A `RESTORED` file in the data directory names the archive and the time. + +Then start Trinity. The receipt chains verify under the restored registry (`mix trinity.receipts.verify --scope +`, or the Verify button on a session's receipts page), which is the sharpest check that the copy is +faithful. The search index is inside the database on SQLite; on Postgres it is a generated column. + +`--data-dir ` points either task at a directory other than the configured one. + +## Size + +On the machine this was written on (2026-09-21), 18 sessions and 8,029 messages made a 713 KB archive (3.1 MB +of database before compression). Vectors (slice 032) and any model cache are not measured yet; the slice that +adds them reads this line. diff --git a/lib/mix/tasks/trinity.export.ex b/lib/mix/tasks/trinity.export.ex new file mode 100644 index 0000000..dff9b13 --- /dev/null +++ b/lib/mix/tasks/trinity.export.ex @@ -0,0 +1,51 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Mix.Tasks.Trinity.Export do + @shortdoc "Writes the data directory to one archive (databases, registry, skills; private keys only with --keys)" + + @moduledoc """ + Slice 034 (`docs/backup.md`): + + mix trinity.export --out trinity-2026-09-21.tar.gz + mix trinity.export --out with-keys.tar.gz --keys + + The databases are snapshotted with `VACUUM INTO`, so a running Trinity is fine. Without + `--keys` the archive carries the key registry (public) and not the private key files, so it + can be handed to someone without the ability to sign as you; the manifest records which. + """ + use Boundary, classify_to: Trinity + use Mix.Task + + @impl Mix.Task + def run(argv) do + {opts, _, _} = + OptionParser.parse(argv, strict: [out: :string, keys: :boolean, data_dir: :string]) + + opts = Map.new(opts) + + unless Map.has_key?(opts, :out) do + Mix.shell().error( + "usage: mix trinity.export --out [--keys] [--data-dir ]" + ) + + exit({:shutdown, 2}) + end + + Mix.Task.run("app.config") + + layout = + if d = opts[:data_dir], + do: Trinity.Archive.Layout.of(d), + else: Trinity.Archive.Layout.current() + + case Trinity.Archive.export(layout, opts.out, keys: opts[:keys] == true) do + {:ok, %{manifest: m, bytes: bytes}} -> + Mix.shell().info( + "exported #{length(m.files)} files, #{bytes} bytes, keys #{if m.keys_included, do: "included", else: "not included"}: #{opts.out}" + ) + + {:error, reason} -> + Mix.raise("export failed: #{inspect(reason)}") + end + end +end diff --git a/lib/mix/tasks/trinity.import.ex b/lib/mix/tasks/trinity.import.ex new file mode 100644 index 0000000..a7a5a04 --- /dev/null +++ b/lib/mix/tasks/trinity.import.ex @@ -0,0 +1,75 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Mix.Tasks.Trinity.Import do + @shortdoc "Restores an archive into an empty data directory (or replaces one with --force)" + + @moduledoc """ + Slice 034 (`docs/backup.md`): + + mix trinity.import trinity-2026-09-21.tar.gz + mix trinity.import trinity-2026-09-21.tar.gz --force + mix trinity.import trinity-2026-09-21.tar.gz --data-dir /path/to/dir + + Stop Trinity first: the task refuses while a live process holds the data directory's lock. + Every digest in the manifest is verified and the schema versions are checked against this + binary before anything is written; a non-empty directory is refused unless `--force`, and + then the task prints what it replaced. Exit 2 on usage, 1 on refusal. + """ + use Boundary, classify_to: Trinity + use Mix.Task + + @impl Mix.Task + def run(argv) do + {opts, args, _} = OptionParser.parse(argv, strict: [force: :boolean, data_dir: :string]) + + case args do + [archive] -> + Mix.Task.run("app.config") + + layout = + if d = opts[:data_dir], + do: Trinity.Archive.Layout.of(d), + else: Trinity.Archive.Layout.current() + + case Trinity.DataDir.Lock.live_holder(layout.data_dir) do + {:ok, %{pid: pid, mode: mode}} -> + Mix.shell().error( + "refused: a #{mode} Trinity (OS pid #{pid}) holds #{layout.data_dir}; stop it first" + ) + + exit({:shutdown, 1}) + + :none -> + import_into(archive, layout, opts[:force] == true) + end + + _ -> + Mix.shell().error("usage: mix trinity.import [--force] [--data-dir ]") + exit({:shutdown, 2}) + end + end + + defp import_into(archive, layout, force?) do + File.mkdir_p!(layout.data_dir) + + case Trinity.Archive.import(archive, layout, force: force?) do + {:ok, %{manifest: m, written: written, replaced: replaced}} -> + Enum.each(replaced, &Mix.shell().info("replaced #{&1}")) + + Mix.shell().info( + "restored #{length(written)} files from #{archive} (created #{m.created_at}, keys #{if m.keys_included, do: "included", else: "not included"}) into #{layout.data_dir}" + ) + + {:error, {:not_empty, present}} -> + Mix.shell().error( + "refused: #{layout.data_dir} is not empty (#{Enum.join(present, ", ")}); --force replaces them" + ) + + exit({:shutdown, 1}) + + {:error, reason} -> + Mix.shell().error("refused: #{inspect(reason)}") + exit({:shutdown, 1}) + end + end +end diff --git a/lib/trinity/data_dir/lock.ex b/lib/trinity/data_dir/lock.ex index f977a16..896acb5 100644 --- a/lib/trinity/data_dir/lock.ex +++ b/lib/trinity/data_dir/lock.ex @@ -91,6 +91,15 @@ defmodule Trinity.DataDir.Lock do @spec holder(Path.t()) :: {:ok, holder()} | {:error, term()} def holder(dir), do: read(Path.join(dir, @file_name)) + @doc "The live holder of `dir`, if a live process holds it (slice 034: the import task asks before it writes)." + @spec live_holder(Path.t()) :: {:ok, holder()} | :none + def live_holder(dir) do + case holder(dir) do + {:ok, %{pid: pid} = h} -> if alive?(pid), do: {:ok, h}, else: :none + _ -> :none + end + end + ## Server @impl true diff --git a/lib/trinity_web/controllers/export_controller.ex b/lib/trinity_web/controllers/export_controller.ex new file mode 100644 index 0000000..4657d12 --- /dev/null +++ b/lib/trinity_web/controllers/export_controller.ex @@ -0,0 +1,31 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule TrinityWeb.ExportController do + @moduledoc """ + `GET /settings/export.tar.gz[?keys=1]` (slice 034): the running Trinity's archive as a + download, made the same way the mix task makes it. Private keys only with `keys=1`. + """ + use TrinityWeb, :controller + + def download(conn, params) do + keys? = params["keys"] in ["1", "true"] + stamp = DateTime.utc_now() |> DateTime.to_iso8601(:basic) |> String.slice(0, 15) + name = "trinity-#{stamp}#{if keys?, do: "-with-keys", else: ""}.tar.gz" + + out = + Path.join( + System.tmp_dir!(), + "trinity-download-#{System.unique_integer([:positive])}.tar.gz" + ) + + case Trinity.Archive.export(Trinity.Archive.Layout.current(), out, keys: keys?) do + {:ok, _} -> + conn = send_download(conn, {:file, out}, filename: name, content_type: "application/gzip") + File.rm(out) + conn + + {:error, reason} -> + conn |> put_status(500) |> text("export failed: #{inspect(reason)}") + end + end +end diff --git a/lib/trinity_web/live/session_live/index.ex b/lib/trinity_web/live/session_live/index.ex index 450e30c..7eb4234 100644 --- a/lib/trinity_web/live/session_live/index.ex +++ b/lib/trinity_web/live/session_live/index.ex @@ -66,6 +66,9 @@ defmodule TrinityWeb.SessionLive.Index do <.link navigate={~p"/personas"} class="underline opacity-70">{gettext("edit")} <.link navigate={~p"/memory"} class="underline opacity-70">{gettext("memory")} + <.link id="settings-link" navigate={~p"/settings"} class="underline opacity-70">{gettext( + "settings" + )}