From e223a2161a9c115578c7a329a482f4fa816d8e6a Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:31:52 -0400 Subject: [PATCH 01/12] fix(server): stop refused host names flooding the log The limiter on "Refused a request for host" lines cleared its table once it held 1,000 names, so the same names logged again and any client could write as many lines as it liked (slice 54, B-04). The new Kotiko.Log.Limiter keeps one line per name per hour; when 1,000 names have been logged in an hour it logs one line saying new names aren't logged until the hour ends, and how many went unlogged when the next hour starts. Regression test: 3 rounds of the same 1,001 names now give at most 1,001 lines and one summary (it gave 3,003 lines before). Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 4 + docs/security/assurance-case.md | 2 +- server/lib/kotiko/log/limiter.ex | 89 +++++++++++++++++++++ server/lib/kotiko/plug/host_check.ex | 55 +++++-------- server/test/kotiko/log/limiter_test.exs | 42 ++++++++++ server/test/kotiko/plug/host_check_test.exs | 23 ++++++ 6 files changed, 180 insertions(+), 35 deletions(-) create mode 100644 server/lib/kotiko/log/limiter.ex create mode 100644 server/test/kotiko/log/limiter_test.exs diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a8052c..a0ad749 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: requests for many different host names can no longer fill the log. + The server logs each refused name at most once an hour, and at most 1,000 names an hour; + past that it logs one line saying so, and how many it left out. + - Security: a web page that managed to run code inside Kotiko's page script could change Kotiko's settings, because browsers let that script write Kotiko's storage. It could turn Kotiko off, pause sites, hide languages, change the words pages show, queue a word to be diff --git a/docs/security/assurance-case.md b/docs/security/assurance-case.md index cf1e994..22793a1 100644 --- a/docs/security/assurance-case.md +++ b/docs/security/assurance-case.md @@ -134,7 +134,7 @@ a small JSON API. | CWE-352 Cross-site request forgery | The token travels in a header, never a cookie; the server sends no CORS headers, so a page can't attach the header (that needs a preflight the server doesn't answer) | In place | [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs) "a CORS preflight gets no CORS headers" | | CWE-918 Server-side request forgery | The server fetches only URLs the owner set (`LLM_URL`, `TRANSCRIBE_URL`) and fixed hosts (Telegram, Wiktionary with the word percent-encoded in the path). No request can choose a URL | In place | [`config.ex`](../../server/lib/kotiko/config.ex); [`pronounce.ex`](../../server/lib/kotiko/pronounce.ex) | | CWE-502 Unsafe deserialization | Nothing from the network becomes an Erlang term: requests and API answers are parsed as JSON, Wiktionary pages are read as text. One call reads back the lookup cache the server itself wrote to its own database, with `Plug.Crypto.non_executable_binary_to_term/2` and `:safe`, so a changed row can neither create atoms nor carry a function; Sobelow checks for any other | In place | [`llm/cache.ex`](../../server/lib/kotiko/llm/cache.ex) `decode/1`; `llm_test.exs` ("a stored entry holding a function is a miss") | -| CWE-400 Resource exhaustion | Body caps (64 KB, 1 MB batch); at most 500 words per batch and 20,000 per list request; each lookup has a deadline and an attempt cap; refused Host names are logged at most once an hour per name. There is no request rate limit for token holders | In place, partly | [`router.ex`](../../server/lib/kotiko/router.ex); [`router_v1.ex`](../../server/lib/kotiko/router_v1.ex) `@max_batch`, `@max_limit`; [`spec/models.json`](../../spec/models.json); [`host_check.ex`](../../server/lib/kotiko/plug/host_check.ex) | +| CWE-400 Resource exhaustion | Body caps (64 KB, 1 MB batch); at most 500 words per batch and 20,000 per list request; each lookup has a deadline and an attempt cap; refused Host names are logged at most once an hour per name, for at most 1,000 names an hour, then one summary line (`log/limiter.ex`). There is no request rate limit for token holders | In place, partly | [`router.ex`](../../server/lib/kotiko/router.ex); [`router_v1.ex`](../../server/lib/kotiko/router_v1.ex) `@max_batch`, `@max_limit`; [`spec/models.json`](../../spec/models.json); [`host_check.ex`](../../server/lib/kotiko/plug/host_check.ex) | | CWE-20 Improper input validation | Settings validation; the shared word spec and its rules; response validation in the extension; server address validation | In place | Section 6 | | CWE-798 Hard-coded credentials | No credentials in the code. CI's required `secrets` job runs gitleaks over the full history on every pull request and every push to `main`, and fails a pull request that adds a `.env`, key or token file; GitHub secret scanning and push protection are on | In place | [`ci.yml`](../../.github/workflows/ci.yml) job `secrets`; [`.gitleaks.toml`](../../.gitleaks.toml); [`check-forbidden-files.mjs`](../../scripts/check-forbidden-files.mjs) | | CWE-532 Secrets in logs | The redaction filter; words only with `LOG_LOOKUPS=true` at debug | In place | [`log/redact.ex`](../../server/lib/kotiko/log/redact.ex); [`llm_test.exs`](../../server/test/kotiko/llm_test.exs) | diff --git a/server/lib/kotiko/log/limiter.ex b/server/lib/kotiko/log/limiter.ex new file mode 100644 index 0000000..0faa966 --- /dev/null +++ b/server/lib/kotiko/log/limiter.ex @@ -0,0 +1,89 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.Log.Limiter do + @moduledoc """ + Keeps log lines about strangers' requests (a refused `Host` name, wrong API tokens) to + at most one per key in each window (an hour by default), so nobody can flood the log + by sending requests. + + The table holds at most `max_keys` keys per window. When it is full, new keys aren't + logged until the window ends: the caller logs one line saying so (`:overflow`), and + at the start of the next window one more with how many keys went unlogged. Clearing + the table instead would let the same keys log again, without limit (slice 54, B-04). + + The checks aren't one atomic step: two requests at the same moment can each log a + line. That only adds a line or two; it never removes the limit. + """ + + @window_row :__window__ + + @doc "Creates the named table `table` if it isn't there yet." + def init(table) do + if :ets.whereis(table) == :undefined do + :ets.new(table, [:named_table, :public, :set, write_concurrency: true]) + end + + :ok + end + + @doc "Forgets every key and the current window (for tests)." + def reset(table) do + if :ets.whereis(table) != :undefined, do: :ets.delete_all_objects(table) + :ok + end + + @doc """ + Whether to log a line for `key` now: `:log` the first time in this window, `:overflow` + for the first key that didn't fit (log one line saying new keys aren't logged until the + window ends), else `:quiet`. The second element is the number of keys that went unlogged + in the window that just ended, or nil: log it once. + + Options: `:window_ms` (default one hour) and `:max_keys` (default 1,000). Without the + table (it is made at boot), every call is `:log`. + """ + @spec check(atom(), term(), keyword()) :: {:log | :overflow | :quiet, nil | pos_integer()} + def check(table, key, opts \\ []) do + if :ets.whereis(table) == :undefined do + {:log, nil} + else + now = System.monotonic_time(:millisecond) + rolled = roll(table, now, Keyword.get(opts, :window_ms, :timer.hours(1))) + {decide(table, key, now, Keyword.get(opts, :max_keys, 1_000)), rolled} + end + end + + # Starts a new window when the current one is over; returns the old one's unlogged count. + defp roll(table, now, window_ms) do + case :ets.lookup(table, @window_row) do + [{_, started, _unlogged}] when now - started < window_ms -> + nil + + [{_, _started, unlogged}] -> + :ets.delete_all_objects(table) + :ets.insert(table, {@window_row, now, 0}) + if unlogged > 0, do: unlogged + + [] -> + :ets.insert_new(table, {@window_row, now, 0}) + nil + end + end + + defp decide(table, key, now, max_keys) do + cond do + :ets.member(table, {:key, key}) -> + :quiet + + # One row is the window's own. + :ets.info(table, :size) - 1 < max_keys -> + if :ets.insert_new(table, {{:key, key}}), do: :log, else: :quiet + + true -> + case :ets.update_counter(table, @window_row, {3, 1}, {@window_row, now, 0}) do + 1 -> :overflow + _ -> :quiet + end + end + end +end diff --git a/server/lib/kotiko/plug/host_check.ex b/server/lib/kotiko/plug/host_check.ex index 5d90c3a..317acc0 100644 --- a/server/lib/kotiko/plug/host_check.ex +++ b/server/lib/kotiko/plug/host_check.ex @@ -15,18 +15,13 @@ defmodule Kotiko.Plug.HostCheck do import Plug.Conn require Logger + alias Kotiko.Log.Limiter + @table __MODULE__ - @log_every_ms :timer.hours(1) @max_logged_names 1_000 @doc "Creates the table that keeps refusals to one log line per name per hour." - def init_table do - if :ets.whereis(@table) == :undefined do - :ets.new(@table, [:named_table, :public, :set]) - end - - :ok - end + def init_table, do: Limiter.init(@table) @impl true def init(opts), do: opts @@ -106,37 +101,29 @@ defmodule Kotiko.Plug.HostCheck do |> halt() end - # Names only, never paths; and at most once an hour per name so a scanner can't flood - # the log. + # Names only, never paths; and at most once an hour per name, and at most 1,000 names an + # hour, so a scanner can't flood the log (Kotiko.Log.Limiter). defp log_once(name) do - if first_time_this_hour?(name) do - Logger.info( - "Refused a request for host #{inspect(name)}. Add it to ALLOWED_HOSTS if it's yours." - ) - end - end - - defp first_time_this_hour?(name) do - now = System.monotonic_time(:millisecond) + {decision, unlogged} = Limiter.check(@table, name, max_keys: @max_logged_names) - cond do - :ets.whereis(@table) == :undefined -> - true + if unlogged do + Logger.info("Didn't log #{unlogged} more refused host names in the last hour (too many).") + end - recently_logged?(name, now) -> - false + case decision do + :log -> + Logger.info( + "Refused a request for host #{inspect(name)}. Add it to ALLOWED_HOSTS if it's yours." + ) - true -> - if :ets.info(@table, :size) >= @max_logged_names, do: :ets.delete_all_objects(@table) - :ets.insert(@table, {name, now}) - true - end - end + :overflow -> + Logger.warning( + "Refused requests for #{@max_logged_names} different host names this hour; " <> + "not logging new ones until the hour is over." + ) - defp recently_logged?(name, now) do - case :ets.lookup(@table, name) do - [{_, at}] -> now - at < @log_every_ms - [] -> false + :quiet -> + :ok end end end diff --git a/server/test/kotiko/log/limiter_test.exs b/server/test/kotiko/log/limiter_test.exs new file mode 100644 index 0000000..68301cb --- /dev/null +++ b/server/test/kotiko/log/limiter_test.exs @@ -0,0 +1,42 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.Log.LimiterTest do + use ExUnit.Case, async: true + alias Kotiko.Log.Limiter + + setup do + table = :"limiter_test_#{System.unique_integer([:positive])}" + :ok = Limiter.init(table) + %{table: table} + end + + test "each key once a window; when full, one overflow and then quiet", %{table: t} do + opts = [max_keys: 2] + assert Limiter.check(t, :a, opts) == {:log, nil} + assert Limiter.check(t, :a, opts) == {:quiet, nil} + assert Limiter.check(t, :b, opts) == {:log, nil} + assert Limiter.check(t, :c, opts) == {:overflow, nil} + assert Limiter.check(t, :d, opts) == {:quiet, nil} + # Keys already logged stay quiet: the table is never cleared mid-window. + assert Limiter.check(t, :a, opts) == {:quiet, nil} + end + + test "a new window logs again and reports how many keys went unlogged", %{table: t} do + opts = [max_keys: 1, window_ms: 30] + assert {:log, nil} = Limiter.check(t, :a, opts) + assert {:overflow, nil} = Limiter.check(t, :b, opts) + assert {:quiet, nil} = Limiter.check(t, :c, opts) + + Process.sleep(40) + assert Limiter.check(t, :a, opts) == {:log, 2} + assert Limiter.check(t, :b, opts) == {:overflow, nil} + end + + test "without its table every call logs, and reset forgets the keys", %{table: t} do + assert Limiter.check(:no_such_limiter_table, :a) == {:log, nil} + assert {:log, nil} = Limiter.check(t, :a) + :ok = Limiter.reset(t) + assert {:log, nil} = Limiter.check(t, :a) + end +end diff --git a/server/test/kotiko/plug/host_check_test.exs b/server/test/kotiko/plug/host_check_test.exs index 0007f90..691839d 100644 --- a/server/test/kotiko/plug/host_check_test.exs +++ b/server/test/kotiko/plug/host_check_test.exs @@ -89,4 +89,27 @@ defmodule Kotiko.Plug.HostCheckTest do assert length(String.split(log, name)) == 2 refute log =~ "/health" end + + # B-04 (slice 54): the table of logged names used to be cleared when it held 1,000, so + # the same names logged again and a client could write as many lines as it liked. + test "a flood of names logs each at most once, then one summary line" do + HostCheck.init_table() + :ets.delete_all_objects(HostCheck) + on_exit(fn -> :ets.delete_all_objects(HostCheck) end) + Logger.put_module_level(HostCheck, :info) + on_exit(fn -> Logger.delete_module_level(HostCheck) end) + names = for i <- 1..1_001, do: "flood-#{i}.example" + + log = + capture_log([level: :info], fn -> + for _round <- 1..3, name <- names, do: check(name) + end) + + lines = String.split(log, "\n") |> Enum.filter(&(&1 =~ "Refused a request for host")) + summaries = String.split(log, "\n") |> Enum.filter(&(&1 =~ "not logging new ones")) + + assert length(lines) <= 1_001 + assert length(summaries) == 1 + assert hd(summaries) =~ "1000 different host names" + end end From 269a648ecd66083e5375e2c0cdcab48282cc9614 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:35:01 -0400 Subject: [PATCH 02/12] fix(server): listen on both loopback addresses by default Browsers look up `localhost` themselves and try ::1 first, but the server listened only on 127.0.0.1. Another account on the computer could listen on [::1]:4747 and receive the token the extension sends to http://localhost:4747, even while Kotiko was running (slice 54, B-01). Kotiko.Listener now starts one listener per address: a loopback BIND (the default 127.0.0.1, localhost or ::1) also holds the other loopback address, 0.0.0.0 also holds ::1, and :: is dual-stack. BIND=localhost is always 127.0.0.1 plus ::1, whatever the hosts file says. Without IPv6 the server starts on 127.0.0.1 alone and logs a line; another program already listening on either address stops the start with status 1 and a message naming it. Regression tests: with the default settings a second listener on [::1]: now fails with eaddrinuse and the server answers there (before, the squatter's listen succeeded); a squatter already on [::1] stops the start. Tests that need ::1 are tagged :ipv6 and skipped on a machine without it. Docs tell users to enter http://127.0.0.1:4747. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 7 ++ docs/reference/configuration.md | 14 +++- docs/reference/http-api.md | 10 +-- docs/security/assurance-case.md | 2 +- docs/security/requirements.md | 6 +- server/.env.example | 5 +- server/lib/kotiko/application.ex | 3 +- server/lib/kotiko/config.ex | 18 ++++- server/lib/kotiko/listener.ex | 104 ++++++++++++++++++++++++++ server/test/kotiko/boot_test.exs | 52 +++++++++++-- server/test/kotiko/listener_test.exs | 101 +++++++++++++++++++++++++ server/test/test_helper.exs | 11 ++- site/src/content/docs/server/index.md | 5 +- 13 files changed, 311 insertions(+), 27 deletions(-) create mode 100644 server/lib/kotiko/listener.ex create mode 100644 server/test/kotiko/listener_test.exs diff --git a/CHANGELOG.md b/CHANGELOG.md index a0ad749..454597f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,13 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: another account on the same computer could receive your access key + by listening on `[::1]:4747`, the address browsers try first for `localhost`. The server + now listens on both `127.0.0.1` and `::1` by default (and with `BIND=localhost`), so + nobody else can take either; it refuses to start, saying which address, if another + program already holds one. In Kotiko, enter `http://127.0.0.1:4747` as your server's + address. + - Self-hosted server: requests for many different host names can no longer fill the log. The server logs each refused name at most once an hour, and at most 1,000 names an hour; past that it logs one line saying so, and how many it left out. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 4df295f..e302d68 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -53,8 +53,18 @@ The TCP port the server listens on. The address the server listens on. - Default: `127.0.0.1`, this computer only. -- Allowed: an IPv4 or IPv6 address (`100.101.102.103`, `0.0.0.0`, `::1`) or a host name - that resolves to one (`localhost`). +- Allowed: an IPv4 or IPv6 address (`100.101.102.103`, `0.0.0.0`, `::`, `::1`) or a host + name that resolves to one. `localhost` is always `127.0.0.1`, whatever the hosts file says. +- Browsers look up `localhost` themselves and try the IPv6 address `::1` first. So that no + other program or account on this computer can listen on the address the browser tries + and receive the token, the server listens on both loopback addresses: with `127.0.0.1` + (the default) or `localhost` it also listens on `::1`, with `::1` also on `127.0.0.1`, + and with `0.0.0.0` also on `::1`. `::` covers IPv4 too (dual-stack). On a computer + without IPv6 the server starts on `127.0.0.1` alone and logs a line saying so. If another + program already listens on one of these addresses at `PORT`, the server doesn't start + (status 1) and says which address; stop that program or choose another `PORT`. +- In the extension, use `http://127.0.0.1:4747` (not `localhost`) for a server on the same + computer. - Example: your Tailscale address, to use the extension on another computer. - At start, the server logs who can reach it. For a local network address, `0.0.0.0` or a public address, that line is a warning: the server speaks plain HTTP, so anyone on the way diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 0c962d1..024c4f5 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -2,8 +2,8 @@ Last reviewed: 2026-10-05. -The Kotiko server answers HTTP on `http://127.0.0.1:4747` by default (`BIND` and `PORT` in -[configuration.md](configuration.md)). The browser extension and `curl` use this API. This +The Kotiko server answers HTTP on `http://127.0.0.1:4747` by default, and on +`http://[::1]:4747` too (`BIND` and `PORT` in [configuration.md](configuration.md#bind)). The browser extension and `curl` use this API. This page lists every route the server has. A test (`server/test/kotiko/docs_test.exs`) fails when a route is added or removed without updating this page. @@ -117,7 +117,7 @@ Which server this is and whether its database works. No token needed. Answers no (no word counts, no settings). ```bash -curl -s http://localhost:4747/health +curl -s http://127.0.0.1:4747/health ``` `200` when the database works, `503` when it doesn't (the file is missing or not writable, @@ -154,7 +154,7 @@ lookups you haven't added) are left out. | `limit` | 1 to 20,000; default 20,000 | ```bash -curl -s -H "Authorization: Bearer $TOKEN" "http://localhost:4747/api/v1/words?lang=ru&limit=10" +curl -s -H "Authorization: Bearer $TOKEN" "http://127.0.0.1:4747/api/v1/words?lang=ru&limit=10" ``` `200`: @@ -201,7 +201,7 @@ words you mean, checks the answer, and saves the words, merging into words you h ```bash curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"text": "shukran", "base_langs": ["en"]}' http://localhost:4747/api/v1/words + -d '{"text": "shukran", "base_langs": ["en"]}' http://127.0.0.1:4747/api/v1/words ``` `200`: diff --git a/docs/security/assurance-case.md b/docs/security/assurance-case.md index 22793a1..b040007 100644 --- a/docs/security/assurance-case.md +++ b/docs/security/assurance-case.md @@ -27,7 +27,7 @@ its argument and evidence. | `/health` says nothing else | It returns name, version, API versions and database state only | [`health.ex`](../../server/lib/kotiko/health.ex) `report/1` | | A strong token, kept private | 32 bytes from `:crypto.strong_rand_bytes`, written to a file made 0600 before the token goes in, then renamed into place; a chosen `API_TOKEN` under 24 characters stops the boot | [`token.ex`](../../server/lib/kotiko/token.ex) `generate/1`, `save/2`, `check/3`; [`token_test.exs`](../../server/test/kotiko/token_test.exs) "first boot makes a 43-character token in a 0600 file", "short or whitespace-only tokens are refused"; [`config.ex`](../../server/lib/kotiko/config.ex) `api_token/1` | | Other accounts on the computer can't read the words | The BEAM can't set a mode on create, so files are created empty, made 0600, then filled: the database before SQLite opens it (SQLite gives `-wal`, `-shm` and `-journal` the database's mode, checked by a test), each backup before `VACUUM INTO`, the token and model cache through a private temporary file. A data folder the server makes is 0700. At every boot, before the Repo opens, `make_private/1` tightens any of Kotiko's files others can open and the folder when it holds only Kotiko's files, and warns otherwise. `run.sh` sets umask 077 and the unit `UMask=0077` | [`private.ex`](../../server/lib/kotiko/private.ex); [`data_dir.ex`](../../server/lib/kotiko/data_dir.ex) `make_private/1`; [`private_test.exs`](../../server/test/kotiko/private_test.exs); [`data_dir_test.exs`](../../server/test/kotiko/data_dir_test.exs); [`boot_test.exs`](../../server/test/kotiko/boot_test.exs) (boots with umask 022 and checks the modes); [`scripts_test.exs`](../../server/test/kotiko/scripts_test.exs) | -| Listens on this computer unless told otherwise; warns when it doesn't | `BIND` defaults to `127.0.0.1`; at boot the address is classified (loopback, Tailscale, LAN, all interfaces, public) and the last three log a warning; a public address repeats it daily | [`config.ex`](../../server/lib/kotiko/config.ex) `@default_bind`; [`exposure.ex`](../../server/lib/kotiko/exposure.ex) `classify/2`; [`application.ex`](../../server/lib/kotiko/application.ex) `repeat_public_warning/2`; [`exposure_test.exs`](../../server/test/kotiko/exposure_test.exs) | +| Listens on this computer unless told otherwise; warns when it doesn't | `BIND` defaults to `127.0.0.1`, and a loopback `BIND` holds the port on `::1` too, so no other account can listen where a browser sends `localhost` (B-01); at boot the address is classified (loopback, Tailscale, LAN, all interfaces, public) and the last three log a warning; a public address repeats it daily | [`config.ex`](../../server/lib/kotiko/config.ex) `@default_bind`; [`exposure.ex`](../../server/lib/kotiko/exposure.ex) `classify/2`; [`application.ex`](../../server/lib/kotiko/application.ex) `repeat_public_warning/2`; [`listener.ex`](../../server/lib/kotiko/listener.ex) `addresses/1`; [`exposure_test.exs`](../../server/test/kotiko/exposure_test.exs) | | Answers only to names it knows | A plug before routing allows `localhost`, IP literals, this machine's own name and `ALLOWED_HOSTS`; others get 421, with or without a token | [`host_check.ex`](../../server/lib/kotiko/plug/host_check.ex); [`host_check_test.exs`](../../server/test/kotiko/plug/host_check_test.exs); [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs) "an unknown name gets 421 with or without a token, on every route" | | Strangers can't make it do work | Body parsing is a plug after `authorize`; 64 KB cap, 1 MB for the batch route | [`router.ex`](../../server/lib/kotiko/router.ex) `parse_body/2`; [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs) "are parsed only after auth", "over 64 KB are refused with 413"; [`router_v1_test.exs`](../../server/test/kotiko/router_v1_test.exs) "takes at most 500 words, and bodies up to 1 MB" | | Words go only where the user sent them | The only outbound hosts are `LLM_URL`, `TRANSCRIBE_URL` (both set by the owner), `api.telegram.org` when a bot token is set, and the Wiktionary endpoint with the word percent-encoded in the path; tests can't reach the network at all | [`llm/client.ex`](../../server/lib/kotiko/llm/client.ex); [`transcriber.ex`](../../server/lib/kotiko/transcriber.ex); [`telegram.ex`](../../server/lib/kotiko/telegram.ex); [`pronounce.ex`](../../server/lib/kotiko/pronounce.ex) `fetch_page`; [`spec/wiktionary.json`](../../spec/wiktionary.json) `endpoint`; [`no_network_test.exs`](../../server/test/kotiko/no_network_test.exs) | diff --git a/docs/security/requirements.md b/docs/security/requirements.md index 8f81cf5..59f6f39 100644 --- a/docs/security/requirements.md +++ b/docs/security/requirements.md @@ -27,8 +27,10 @@ Kotiko has two parts: the browser extension, and an optional server you run your (0600), and a data folder the server makes only by you (0700). At every start it makes them private again if they aren't, and warns, with the command to run, about anything it can't or won't change (a data folder that also holds other files). -- **It listens only on your computer** unless you change `BIND`. When you do, it says at - startup who can now reach it, and warns when your token would cross a network in plain +- **It listens only on your computer** unless you change `BIND`, on both loopback + addresses (`127.0.0.1` and `::1`), so no other account on the computer can listen on the + one a browser tries for `localhost` and receive your token. When you change `BIND`, it + says at startup who can now reach it, and warns when your token would cross a network in plain HTTP (repeating that warning daily when the address is reachable from the internet). - **It answers only to names it knows**, so a web page can't reach it by pointing its own domain at your computer (DNS rebinding). Unknown names get `421`. You add names with diff --git a/server/.env.example b/server/.env.example index a854ebf..175e37c 100644 --- a/server/.env.example +++ b/server/.env.example @@ -40,8 +40,9 @@ LLM_MODEL= PORT=4747 -# 127.0.0.1 = only this machine. Use your Tailscale IP (or 0.0.0.0) to reach it -# from a laptop's browser too. Startup logs say who can reach it. +# 127.0.0.1 = only this machine (the server then holds the port on ::1 too, so no +# other program on this computer can take it there). Use your Tailscale IP (or 0.0.0.0) +# to reach it from a laptop's browser too. Startup logs say who can reach it. BIND=127.0.0.1 # Host names you reach the server by, comma-separated, if not localhost, an IP address diff --git a/server/lib/kotiko/application.ex b/server/lib/kotiko/application.ex index ccc0ab6..cbac392 100644 --- a/server/lib/kotiko/application.ex +++ b/server/lib/kotiko/application.ex @@ -158,8 +158,9 @@ defmodule Kotiko.Application do defp http_children(nil), do: [] + # BIND's address, and the other loopback address too (Kotiko.Listener). defp http_children(%{ip: ip, port: port, exposure: {_level, class, message}}) do - [{Bandit, plug: Kotiko.Router, ip: ip, port: port}] ++ repeat_public_warning(class, message) + Kotiko.Listener.child_specs(ip, port) ++ repeat_public_warning(class, message) end # Reachable from the internet: say so again every day, not just once at boot. diff --git a/server/lib/kotiko/config.ex b/server/lib/kotiko/config.ex index bcbdaba..204fbfd 100644 --- a/server/lib/kotiko/config.ex +++ b/server/lib/kotiko/config.ex @@ -365,11 +365,21 @@ defmodule Kotiko.Config do end end - @doc "An IP literal, or a name resolved to IPv4, then IPv6 (so `localhost` works)." + @doc """ + An IP literal, or a name resolved to IPv4, then IPv6. `localhost` is always 127.0.0.1, + whatever the hosts file says: `Kotiko.Listener` then listens on `::1` too, as browsers + try both for `localhost`. + """ def parse_bind(raw, resolver \\ &resolve_host/1) do - case :inet.parse_strict_address(to_charlist(raw)) do - {:ok, ip} -> {:ok, ip} - {:error, _} -> resolver.(raw) + cond do + raw |> String.downcase() |> String.trim_trailing(".") == "localhost" -> + {:ok, {127, 0, 0, 1}} + + match?({:ok, _}, :inet.parse_strict_address(to_charlist(raw))) -> + :inet.parse_strict_address(to_charlist(raw)) + + true -> + resolver.(raw) end end diff --git a/server/lib/kotiko/listener.ex b/server/lib/kotiko/listener.ex new file mode 100644 index 0000000..298d037 --- /dev/null +++ b/server/lib/kotiko/listener.ex @@ -0,0 +1,104 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.Listener do + @moduledoc """ + The HTTP listeners for the address in BIND. + + Browsers look up `localhost` themselves and try the IPv6 loopback address `::1` first. + A server listening only on `127.0.0.1` leaves `[::1]` at the same port free, so another + account on the computer could listen there and receive the token the extension sends to + `http://localhost:4747` (slice 54, B-01). So a loopback BIND (the default `127.0.0.1`, + `localhost` or `::1`) also listens on the other loopback address, and `0.0.0.0` (every + IPv4 address) also on `::1`; `::` is made dual-stack, so it covers IPv4 too. + + The extra address is optional: on a machine without IPv6 the server starts on + `127.0.0.1` alone and logs a line. Someone else already listening on either address + stops the start (status 1), with a message saying so. + """ + require Logger + + @ipv4_loopback {127, 0, 0, 1} + @ipv6_loopback {0, 0, 0, 0, 0, 0, 0, 1} + @ipv4_any {0, 0, 0, 0} + @ipv6_any {0, 0, 0, 0, 0, 0, 0, 0} + # The machine has no such address (IPv6 turned off, or no ::1 configured). + @unavailable [:eaddrnotavail, :eafnosupport, :eprotonosupport] + + @doc """ + The addresses to listen on for BIND's address, each `:required` or `:optional`. + """ + def addresses(@ipv4_loopback), do: [{@ipv4_loopback, :required}, {@ipv6_loopback, :optional}] + def addresses(@ipv6_loopback), do: [{@ipv6_loopback, :required}, {@ipv4_loopback, :optional}] + def addresses(@ipv4_any), do: [{@ipv4_any, :required}, {@ipv6_loopback, :optional}] + def addresses(ip), do: [{ip, :required}] + + @doc "One child spec per address in `addresses/1`, for `plug` on `port`." + def child_specs(ip, port, plug \\ Kotiko.Router) do + for {address, need} <- addresses(ip), + do: child_spec(plug: plug, ip: address, port: port, need: need) + end + + @doc "The child spec for `start_link/1`." + def child_spec(opts) do + %{id: {__MODULE__, opts[:ip]}, start: {__MODULE__, :start_link, [opts]}, type: :supervisor} + end + + @doc """ + Starts Bandit on one address. An optional address the machine doesn't have is skipped + (`:ignore`); an address someone else is listening on halts the VM with a message. + """ + def start_link(opts) do + {need, opts} = Keyword.pop(opts, :need, :required) + {on_conflict, opts} = Keyword.pop(opts, :on_conflict, &Kotiko.Config.halt!/2) + + case Bandit.start_link(bandit_options(opts)) do + {:ok, pid} -> + {:ok, pid} + + {:error, reason} = error -> + case {need, cause(reason)} do + {:optional, cause} when cause in @unavailable -> + Logger.info( + "Not listening on #{address(opts)} too: this computer doesn't have that " <> + "address (#{cause})." + ) + + :ignore + + {_need, :eaddrinuse} -> + on_conflict.(in_use(opts), 1) + error + + _ -> + error + end + end + end + + # `::` alone would follow the system's net.ipv6.bindv6only; dual-stack covers IPv4 too. + defp bandit_options(opts) do + if opts[:ip] == @ipv6_any, + do: opts ++ [thousand_island_options: [transport_options: [ipv6_v6only: false]]], + else: opts + end + + defp cause({:shutdown, {:failed_to_start_child, _child, reason}}), do: cause(reason) + defp cause(reason) when is_atom(reason), do: reason + defp cause(_reason), do: :other + + defp address(opts) do + case opts[:ip] do + {_, _, _, _} = ip -> "#{:inet.ntoa(ip)}:#{opts[:port]}" + ip -> "[#{:inet.ntoa(ip)}]:#{opts[:port]}" + end + end + + defp in_use(opts) do + "The server can't start: another program is already listening on #{address(opts)}.\n\n" <> + "If it isn't an older Kotiko server still running, it could receive the API token " <> + "the extension sends. " <> + "Find it (ss -ltnp, or lsof -i :#{opts[:port]}) and stop it, or choose another PORT " <> + "in .env and in the extension, then start the server again.\n" + end +end diff --git a/server/test/kotiko/boot_test.exs b/server/test/kotiko/boot_test.exs index f71d4f4..4bf71ef 100644 --- a/server/test/kotiko/boot_test.exs +++ b/server/test/kotiko/boot_test.exs @@ -66,6 +66,12 @@ defmodule Kotiko.BootTest do {elixir, Enum.join([erts, bin, System.get_env("PATH")], ":")} end + @start_and_stop """ + {:ok, _} = Application.ensure_all_started(:kotiko, :permanent) + IO.puts("STARTED") + :ok = Application.stop(:kotiko) + """ + defp free_port do {:ok, socket} = :gen_tcp.listen(0, ip: {127, 0, 0, 1}) {:ok, port} = :inet.port(socket) @@ -169,6 +175,46 @@ defmodule Kotiko.BootTest do refute output =~ "sk-or-v1-0123456789abcdef" end + # B-01 (slice 54): browsers try ::1 first for `localhost`. With only 127.0.0.1 taken, + # another account could listen on [::1] at the same port and receive the token the + # extension sends to http://localhost:4747. + @tag :ipv6 + test "by default it holds the port on both loopback addresses", ctx do + port = free_port() + vars = %{"PORT" => to_string(port), "KOTIKO_DATA_DIR" => Path.join(ctx.tmp, "data")} + + script = """ + {:ok, _} = Application.ensure_all_started(:kotiko, :permanent) + squat = :gen_tcp.listen(#{port}, [ip: {0, 0, 0, 0, 0, 0, 0, 1}]) + IO.puts("SQUAT " <> inspect(squat)) + {:ok, socket} = :gen_tcp.connect({0, 0, 0, 0, 0, 0, 0, 1}, #{port}, [:binary, active: false]) + :ok = :gen_tcp.send(socket, "GET /health HTTP/1.1\\r\\nhost: localhost\\r\\nconnection: close\\r\\n\\r\\n") + {:ok, response} = :gen_tcp.recv(socket, 0, 5_000) + IO.puts("V6 " <> response) + :ok = Application.stop(:kotiko) + """ + + {output, status} = boot(vars, script, ctx) + + assert status == 0, output + assert output =~ "SQUAT {:error, :eaddrinuse}" + assert output =~ "V6 HTTP/1.1 200" + end + + @tag :ipv6 + test "someone already listening on [::1] at the port stops the start", ctx do + port = free_port() + {:ok, squatter} = :gen_tcp.listen(port, ip: {0, 0, 0, 0, 0, 0, 0, 1}) + on_exit(fn -> :gen_tcp.close(squatter) end) + vars = %{"PORT" => to_string(port), "KOTIKO_DATA_DIR" => Path.join(ctx.tmp, "data")} + + {output, status} = boot(vars, @start_and_stop, ctx) + + assert status == 1, output + assert output =~ "another program is already listening on [::1]:#{port}." + refute output =~ "STARTED" + end + test "the VM exits non-zero when the supervision tree dies", ctx do vars = %{"KOTIKO_DATA_DIR" => Path.join(ctx.tmp, "data"), "PORT" => to_string(free_port())} @@ -204,12 +250,6 @@ defmodule Kotiko.BootTest do """) end - @start_and_stop """ - {:ok, _} = Application.ensure_all_started(:kotiko, :permanent) - IO.puts("STARTED") - :ok = Application.stop(:kotiko) - """ - test "first start after the rename moves the words from the old default folder", ctx do old_dir = Path.join(ctx.home, ".local/share/slovo") legacy = old_database(ctx, Path.join(old_dir, "slovo.db")) diff --git a/server/test/kotiko/listener_test.exs b/server/test/kotiko/listener_test.exs new file mode 100644 index 0000000..81f1755 --- /dev/null +++ b/server/test/kotiko/listener_test.exs @@ -0,0 +1,101 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.ListenerTest do + # Which addresses the server listens on for BIND (slice 54, B-01), on real sockets. The + # boot with the default settings is in boot_test.exs. + use ExUnit.Case, async: true + import ExUnit.CaptureLog + alias Kotiko.Listener + + @v4 {127, 0, 0, 1} + @v6 {0, 0, 0, 0, 0, 0, 0, 1} + + test "a loopback BIND holds the other loopback address too; others listen where told" do + assert Listener.addresses(@v4) == [{@v4, :required}, {@v6, :optional}] + assert Listener.addresses(@v6) == [{@v6, :required}, {@v4, :optional}] + assert Listener.addresses({0, 0, 0, 0}) == [{{0, 0, 0, 0}, :required}, {@v6, :optional}] + assert Listener.addresses({100, 101, 102, 103}) == [{{100, 101, 102, 103}, :required}] + assert Listener.addresses({0, 0, 0, 0, 0, 0, 0, 0}) == [{{0, 0, 0, 0, 0, 0, 0, 0}, :required}] + end + + test "BIND=localhost is 127.0.0.1, whatever the hosts file says" do + no_lookup = fn _ -> flunk("localhost isn't looked up") end + assert Kotiko.Config.parse_bind("localhost", no_lookup) == {:ok, @v4} + assert Kotiko.Config.parse_bind("LocalHost.", no_lookup) == {:ok, @v4} + assert Kotiko.Config.parse_bind("::1", no_lookup) == {:ok, @v6} + end + + # Bandit links to the caller and exits when its listener can't start, as a supervisor + # sees it; the test process traps that exit like the supervisor does. + test "an optional address the machine doesn't have is skipped, with a line" do + Process.flag(:trap_exit, true) + Logger.put_module_level(Listener, :info) + on_exit(fn -> Logger.delete_module_level(Listener) end) + # 192.0.2.0/24 is for documentation: no machine has it. + log = + capture_log([level: :info], fn -> + assert Listener.start_link( + plug: Plug.Head, + ip: {192, 0, 2, 1}, + port: 0, + need: :optional, + startup_log: false + ) == :ignore + end) + + assert log =~ "Not listening on 192.0.2.1:0 too: this computer doesn't have that address" + + capture_log(fn -> + assert {:error, _} = + Listener.start_link( + plug: Plug.Head, + ip: {192, 0, 2, 1}, + port: 0, + need: :required, + startup_log: false + ) + end) + end + + test "an address someone else holds stops the start, saying who to look for" do + Process.flag(:trap_exit, true) + {:ok, other} = :gen_tcp.listen(0, ip: @v4) + {:ok, port} = :inet.port(other) + on_exit(fn -> :gen_tcp.close(other) end) + test = self() + + capture_log(fn -> + assert {:error, _} = + Listener.start_link( + plug: Plug.Head, + ip: @v4, + port: port, + need: :optional, + startup_log: false, + on_conflict: fn message, status -> send(test, {:halt, message, status}) end + ) + end) + + assert_received {:halt, message, 1} + assert message =~ "another program is already listening on 127.0.0.1:#{port}." + assert message =~ "choose another PORT" + end + + @tag :ipv6 + test "BIND=:: answers on IPv4 too" do + pid = + start_supervised!( + {Listener, + plug: Plug.Head, + ip: {0, 0, 0, 0, 0, 0, 0, 0}, + port: 0, + need: :required, + startup_log: false} + ) + + {:ok, {_ip, port}} = ThousandIsland.listener_info(pid) + assert {:ok, socket} = :gen_tcp.connect(@v4, port, [:binary, active: false]) + :gen_tcp.close(socket) + end +end diff --git a/server/test/test_helper.exs b/server/test/test_helper.exs index 1c9cd97..b281dec 100644 --- a/server/test/test_helper.exs +++ b/server/test/test_helper.exs @@ -12,8 +12,15 @@ Req.default_options( ) # @tag :pending marks known bugs whose fix belongs to a later slice. Run them with -# `mix test --include pending`. -ExUnit.start(exclude: [:pending]) +# `mix test --include pending`. @tag :ipv6 tests need the IPv6 loopback address (::1); +# they are skipped on a machine without it (IPv6 turned off). +ipv6_loopback? = + case :gen_tcp.listen(0, ip: {0, 0, 0, 0, 0, 0, 0, 1}) do + {:ok, socket} -> :gen_tcp.close(socket) == :ok + {:error, _} -> false + end + +ExUnit.start(exclude: [:pending] ++ if(ipv6_loopback?, do: [], else: [:ipv6])) # The migration tests run the migrations again on their own databases; Ecto compiles the # migration files each time, which would warn about redefining their modules. diff --git a/site/src/content/docs/server/index.md b/site/src/content/docs/server/index.md index 7ab4446..0b21404 100644 --- a/site/src/content/docs/server/index.md +++ b/site/src/content/docs/server/index.md @@ -40,7 +40,7 @@ Then start it: The first run fetches dependencies and compiles, then prints a short summary: version, data folder, who can reach the server, model and Telegram. Check it with -`curl localhost:4747/health`, which prints `{"ok":true,...}` with the version. A mistake in +`curl http://127.0.0.1:4747/health`, which prints `{"ok":true,...}` with the version. A mistake in `.env` stops the server with a message naming each setting to fix. The first start also makes an **access key** (the API token), saves it in the data folder and @@ -61,7 +61,8 @@ keys). Replacing a key is then replacing its file and restarting. See ## Connect Kotiko to it 1. In Kotiko, open **Settings**, then **Your Kotiko server**. -2. Enter the server address, `http://localhost:4747` on the same computer. +2. Enter the server address, `http://127.0.0.1:4747` on the same computer. Use this + address rather than `http://localhost:4747`: it always means this computer, over IPv4. 3. Paste the access key and select **Save and connect**. Settings then say "Connected. Your server has 0 words." If your words were in the browser, From eda0e3e43c4c2bba4ea88da8c9e83caf5662de87 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:42:15 -0400 Subject: [PATCH 03/12] fix(server): throttle wrong tokens and add a token proof route Nothing limited wrong-token attempts, so a client could guess a weak API_TOKEN at thousands of tries a second (slice 54, B-09 and C-09). After 10 wrong Bearer tokens from one address within a minute (an IPv6 /64 counts as one address), Kotiko.AuthThrottle answers 429 rate_limited (details.reason auth_failures) with Retry-After to every request from that address that needs the token, the right one included, until the minute is over. Other addresses are never affected, requests without a token don't count, and the lockout is logged once an hour per address. Counters live in Kotiko.RateLimit, an ETS table capped at 10,000 entries. For B-01's defence in depth, POST /api/v1/proof (no token needed) answers {"proof": base64url(HMAC-SHA256(api_token, "kotiko-proof-v1:" <> nonce))}, without padding, for a nonce of 32 to 128 base64url characters. The extension can check that an address holds the token before it sends it there. At most 30 proofs a minute per address; only this exact path and method are open; the body is capped at 1,000 bytes. Regression tests: the 11th wrong token from one address now gets 429 while the right token from another address gets 200 (before, every attempt got 401); the proof route answers the documented test vector. The listener test is no longer async, so its log line can't land in another test's capture_log. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 6 ++ docs/reference/http-api.md | 53 +++++++++- docs/security/assurance-case.md | 5 +- server/config/test.exs | 4 + server/lib/kotiko/application.ex | 1 + server/lib/kotiko/auth_throttle.ex | 99 +++++++++++++++++ server/lib/kotiko/rate_limit.ex | 93 ++++++++++++++++ server/lib/kotiko/router.ex | 60 +++++++++-- server/lib/kotiko/token.ex | 20 ++++ server/priv/locales/en/messages.json | 8 ++ server/test/kotiko/listener_test.exs | 5 +- server/test/kotiko/rate_limit_test.exs | 44 ++++++++ server/test/kotiko/router_proof_test.exs | 111 ++++++++++++++++++++ server/test/kotiko/router_throttle_test.exs | 88 ++++++++++++++++ 14 files changed, 586 insertions(+), 11 deletions(-) create mode 100644 server/lib/kotiko/auth_throttle.ex create mode 100644 server/lib/kotiko/rate_limit.ex create mode 100644 server/test/kotiko/rate_limit_test.exs create mode 100644 server/test/kotiko/router_proof_test.exs create mode 100644 server/test/kotiko/router_throttle_test.exs diff --git a/CHANGELOG.md b/CHANGELOG.md index 454597f..b2063af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,12 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: after 10 wrong access keys from one address within a minute, the + server refuses that address for the rest of the minute (`429`) and logs it once. Other + addresses, including yours, keep working. New route `POST /api/v1/proof`: Kotiko can + check that an address really is your server, which proves it holds your access key + without sending it, before Kotiko sends the key there. + - Self-hosted server: another account on the same computer could receive your access key by listening on `[::1]:4747`, the address browsers try first for `localhost`. The server now listens on both `127.0.0.1` and `::1` by default (and with `BIND=localhost`), so diff --git a/docs/reference/http-api.md b/docs/reference/http-api.md index 024c4f5..2d1fee1 100644 --- a/docs/reference/http-api.md +++ b/docs/reference/http-api.md @@ -17,7 +17,8 @@ a route is added or removed without updating this page. ## Requests -**Authentication.** Every route except `GET /health` and `HEAD /health` needs the API token: +**Authentication.** Every route except `GET /health`, `HEAD /health` and +[`POST /api/v1/proof`](#post-apiv1proof) needs the API token: ``` Authorization: Bearer @@ -29,6 +30,15 @@ and path answers `401` with `www-authenticate: Bearer` and the error `server_key before the server reads the body or looks at the route. `mix kotiko.token` (in `server/`) prints the token; [configuration.md](configuration.md#api_token) says where it comes from. +**Wrong tokens are throttled.** After 10 wrong `Bearer` tokens from one address within a +minute, every request from that address that needs the token gets `429` with `Retry-After` +and the error `rate_limited` (`details.reason`: `auth_failures`) until the minute is over, +even with the right token, so guessing learns nothing while it waits. Other addresses are +not affected. A request with no token, or another scheme, doesn't count. An IPv6 address +counts by its /64 network. The server logs the lockout once an hour per address. Behind a +reverse proxy on the same machine, every request comes from the proxy's address, so a +lockout there applies to everyone for the rest of that minute. + **Host names.** Before anything else, the server checks the `Host` header, to stop web pages reaching it through DNS rebinding. It answers to `localhost`, IP addresses, this machine's own name (and `.local`), and the names in `ALLOWED_HOSTS`. Any other name gets `421` @@ -84,6 +94,7 @@ Routes under `/api/v1` answer errors in one shape: | 415 | `invalid_request` | The body isn't JSON (`details.reason`: `content_type`) | | 421 | `server_address_invalid` | The `Host` header names a host the server doesn't answer to | | 429 | `rate_limited`, `quota_exhausted` | The model provider is busy, or today's free lookups are used up; `Retry-After` (seconds) and `details.retry_at` when known; `details.reason` is `payment_required` when the provider wants credit | +| 429 | `rate_limited` | Too many wrong tokens from this address (`details.reason`: `auth_failures`), or too many proofs (`too_many_proofs`); `Retry-After` (seconds) | | 502 | `key_rejected`, `model_unavailable`, `bad_lookup_result` | The provider refused the server's key, no model could answer, or the answer wasn't usable | | 503 | `lookup_timeout`, `lookup_not_set_up` | No answer within the add's deadline, or no model key is set | | 500 | `internal` | A bug; `details.ref` is a reference to find in the server log. No stack trace is sent | @@ -141,6 +152,46 @@ Sent with `cache-control: no-store`. The same check as `GET /health`, with the same status (`200` or `503`) and no body. +### POST /api/v1/proof + +Proves that this server holds your API token, without sending it. No token needed. The +extension can call it before it sends the token to an address, so another program +listening at that address (say, on `[::1]` while the server listens on `127.0.0.1`) gets +nothing it can use. + +Send a fresh random nonce: 32 to 128 base64url characters (`A-Z`, `a-z`, `0-9`, `-`, `_`, +no `=` padding), for example 32 random bytes encoded as base64url. + +```bash +curl -s -H 'Content-Type: application/json' -d '{"nonce": "q1aP3n0ZKcB1x5mW0u7S9bJ2rV4yT6dE8gH0iL2nO4p"}' \ + http://127.0.0.1:4747/api/v1/proof +``` + +```json +{"proof": "E5CHaF60VqTTPPduGD1yAlfWMF5xf9_hmV2qCufV-ek"} +``` + +That is the answer of a server whose token is `example-token-0123456789abcdef`; use it to +check your own implementation. + +`proof` is `base64url(HMAC-SHA256(key = the API token, message = "kotiko-proof-v1:" + nonce))`, +without padding (43 characters). The key is the token's UTF-8 bytes as you'd send them +in `Authorization`; the message is the ASCII prefix followed by the nonce exactly as sent. +Compute the same value with the token you have and compare in constant time; send the +token only if they match. Use a new nonce each time. + +- `400 invalid_request` (`details.field`: `nonce`): the nonce is missing or not 32 to 128 + base64url characters. The body may be at most 1,000 bytes (`413`). +- `429 rate_limited` (`details.reason`: `too_many_proofs`): more than 30 proofs from one + address within a minute; `Retry-After` says when to try again. +- The answer never contains the token and is sent with `cache-control: no-store`. Like + `/health`, it is checked against the `Host` names the server answers to. +- Only this exact path and `POST` are open: any other method or spelling + (`/%61pi/v1/proof`) needs the token. +- Someone who gets a proof can test guesses at the token offline. A token the server made + (256 random bits) can't be guessed; a weak one you chose can, and the server warns about + those at start ([`API_TOKEN`](configuration.md#api_token)). + ### GET /api/v1/words Your words, newest first, with every field. Deleted words and pending ones (Telegram diff --git a/docs/security/assurance-case.md b/docs/security/assurance-case.md index b040007..d0ae310 100644 --- a/docs/security/assurance-case.md +++ b/docs/security/assurance-case.md @@ -130,11 +130,12 @@ a small JSON API. | CWE-89 SQL injection | Queries go through Ecto with bound parameters. The few raw statements use fixed table names or `?` parameters; the one built string (`VACUUM INTO` a backup path the server chose) is quoted | In place | [`words.ex`](../../server/lib/kotiko/words.ex); [`data_dir.ex`](../../server/lib/kotiko/data_dir.ex) `sql_string/1` | | CWE-78 OS command injection | The server runs no external commands (no `System.cmd`, `Port.open` or `:os.cmd` in `server/lib`). The shell scripts read only the owner's `.env` | In place | `server/lib/`; ShellCheck in [`ci.yml`](../../.github/workflows/ci.yml) | | CWE-22 Path traversal | No file path comes from a request. The data folder comes from the owner's settings; backup names are built from the version and the time. Sobelow fails CI on any new file call with a path it can't rule out; each accepted one says why | In place | [`config.ex`](../../server/lib/kotiko/config.ex) `data_dir/4`; [`migrations.ex`](../../server/lib/kotiko/migrations.ex); [`.sobelow-conf`](../../server/.sobelow-conf) | -| CWE-306, CWE-862 Missing authentication or authorization | Deny by default; one token for everything (no roles yet, so no authorization levels to get wrong) | In place; per-user authorization planned (slice 48) | `authorize/2`; [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs) | +| CWE-306, CWE-862 Missing authentication or authorization | Deny by default; one token for everything (no roles yet, so no authorization levels to get wrong). The only open routes are `/health` and `POST /api/v1/proof`, which proves the server holds the token (HMAC of a client nonce) without revealing it | In place; per-user authorization planned (slice 48) | `authorize/2`; [`token.ex`](../../server/lib/kotiko/token.ex) `proof/2`; [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs); [`router_proof_test.exs`](../../server/test/kotiko/router_proof_test.exs) | +| CWE-307 Excessive authentication attempts | After 10 wrong tokens from one address (an IPv6 /64) within a minute, that address gets 429 on every authenticated route until the minute ends; other addresses are unaffected; the lockout is logged once an hour per address. Proofs are limited to 30 a minute per address. A proof lets someone test guesses offline, so a weak chosen token gets a warning at start | In place | [`auth_throttle.ex`](../../server/lib/kotiko/auth_throttle.ex); [`rate_limit.ex`](../../server/lib/kotiko/rate_limit.ex); [`router_throttle_test.exs`](../../server/test/kotiko/router_throttle_test.exs) | | CWE-352 Cross-site request forgery | The token travels in a header, never a cookie; the server sends no CORS headers, so a page can't attach the header (that needs a preflight the server doesn't answer) | In place | [`router_auth_test.exs`](../../server/test/kotiko/router_auth_test.exs) "a CORS preflight gets no CORS headers" | | CWE-918 Server-side request forgery | The server fetches only URLs the owner set (`LLM_URL`, `TRANSCRIBE_URL`) and fixed hosts (Telegram, Wiktionary with the word percent-encoded in the path). No request can choose a URL | In place | [`config.ex`](../../server/lib/kotiko/config.ex); [`pronounce.ex`](../../server/lib/kotiko/pronounce.ex) | | CWE-502 Unsafe deserialization | Nothing from the network becomes an Erlang term: requests and API answers are parsed as JSON, Wiktionary pages are read as text. One call reads back the lookup cache the server itself wrote to its own database, with `Plug.Crypto.non_executable_binary_to_term/2` and `:safe`, so a changed row can neither create atoms nor carry a function; Sobelow checks for any other | In place | [`llm/cache.ex`](../../server/lib/kotiko/llm/cache.ex) `decode/1`; `llm_test.exs` ("a stored entry holding a function is a miss") | -| CWE-400 Resource exhaustion | Body caps (64 KB, 1 MB batch); at most 500 words per batch and 20,000 per list request; each lookup has a deadline and an attempt cap; refused Host names are logged at most once an hour per name, for at most 1,000 names an hour, then one summary line (`log/limiter.ex`). There is no request rate limit for token holders | In place, partly | [`router.ex`](../../server/lib/kotiko/router.ex); [`router_v1.ex`](../../server/lib/kotiko/router_v1.ex) `@max_batch`, `@max_limit`; [`spec/models.json`](../../spec/models.json); [`host_check.ex`](../../server/lib/kotiko/plug/host_check.ex) | +| CWE-400 Resource exhaustion | Body caps (64 KB, 1 MB batch); at most 500 words per batch and 20,000 per list request; each lookup has a deadline and an attempt cap; refused Host names are logged at most once an hour per name, for at most 1,000 names an hour, then one summary line (`log/limiter.ex`). Wrong tokens and proofs are limited per address (CWE-307). There is no request rate limit for token holders | In place, partly | [`router.ex`](../../server/lib/kotiko/router.ex); [`router_v1.ex`](../../server/lib/kotiko/router_v1.ex) `@max_batch`, `@max_limit`; [`spec/models.json`](../../spec/models.json); [`host_check.ex`](../../server/lib/kotiko/plug/host_check.ex) | | CWE-20 Improper input validation | Settings validation; the shared word spec and its rules; response validation in the extension; server address validation | In place | Section 6 | | CWE-798 Hard-coded credentials | No credentials in the code. CI's required `secrets` job runs gitleaks over the full history on every pull request and every push to `main`, and fails a pull request that adds a `.env`, key or token file; GitHub secret scanning and push protection are on | In place | [`ci.yml`](../../.github/workflows/ci.yml) job `secrets`; [`.gitleaks.toml`](../../.gitleaks.toml); [`check-forbidden-files.mjs`](../../scripts/check-forbidden-files.mjs) | | CWE-532 Secrets in logs | The redaction filter; words only with `LOG_LOOKUPS=true` at debug | In place | [`log/redact.ex`](../../server/lib/kotiko/log/redact.ex); [`llm_test.exs`](../../server/test/kotiko/llm_test.exs) | diff --git a/server/config/test.exs b/server/config/test.exs index 577ccff..e38492c 100644 --- a/server/config/test.exs +++ b/server/config/test.exs @@ -32,6 +32,10 @@ config :kotiko, llm_models: ["m1", "m2", "m3"], llm_model_source: :env, log_lookups: false, + # Router tests send many wrong tokens from 127.0.0.1; router_throttle_test.exs and + # router_proof_test.exs turn these limits on. + auth_failures_per_minute: nil, + proof_requests_per_minute: nil, llm_req_options: [plug: {Req.Test, Kotiko.LLM}], # Slice 10's budgets at 1/20: an add's 25 s deadline is 1.25 s, a Retry-After of 2 s # waits 100 ms. Quota refreshes run inline, so a test sees their effect at once. diff --git a/server/lib/kotiko/application.ex b/server/lib/kotiko/application.ex index cbac392..e82960e 100644 --- a/server/lib/kotiko/application.ex +++ b/server/lib/kotiko/application.ex @@ -33,6 +33,7 @@ defmodule Kotiko.Application do token_source = load_token() Redact.put_secrets(Redact.configured_secrets()) HostCheck.init_table() + Kotiko.AuthThrottle.init() words = Kotiko.Migrations.run!() http = http_settings() log_summary(token_source, words, http) diff --git a/server/lib/kotiko/auth_throttle.ex b/server/lib/kotiko/auth_throttle.ex new file mode 100644 index 0000000..01247aa --- /dev/null +++ b/server/lib/kotiko/auth_throttle.ex @@ -0,0 +1,99 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.AuthThrottle do + @moduledoc """ + Slows down guessing the API token (slice 54, B-09 and C-09). After + `auth_failures_per_minute` wrong tokens (default 10) from one address within a minute, + that address gets `429` with `Retry-After` on every route that needs the token, the + right token included, until the minute is over: a guesser can't tell a right guess + from a wrong one while it waits. Other addresses are never affected. + + Only a wrong `Bearer` token counts; a request with no token or another scheme can't be + a guess. Addresses are counted as `Kotiko.RateLimit.client/1` says (an IPv6 /64 is one). + Each lockout is logged once an hour per address, for at most 100 addresses an hour + (`Kotiko.Log.Limiter`). `auth_failures_per_minute: nil` turns the throttle off (tests). + + Behind a reverse proxy on the same machine every request comes from the proxy's + address, so a lockout applies to everyone behind it for the rest of that minute. + """ + require Logger + alias Kotiko.Log.Limiter + alias Kotiko.RateLimit + + @kind :auth_failures + @window_ms :timer.minutes(1) + @log_table Kotiko.AuthThrottle.Log + @max_logged 100 + + @doc "Creates the tables (at boot)." + def init do + RateLimit.init() + Limiter.init(@log_table) + end + + @doc "Forgets every count and logged lockout (for tests)." + def reset do + RateLimit.reset() + Limiter.reset(@log_table) + end + + @doc "`:ok`, or `{:locked, seconds_left}` when `remote_ip` sent too many wrong tokens." + def check(remote_ip) do + with limit when is_integer(limit) <- limit(), + {count, seconds} when count >= limit <- + RateLimit.peek(@kind, RateLimit.client(remote_ip)) do + {:locked, seconds} + else + _ -> :ok + end + end + + @doc "Counts a wrong token from `remote_ip`; logs the lockout when it starts." + def failed(remote_ip) do + with limit when is_integer(limit) <- limit() do + client = RateLimit.client(remote_ip) + {count, _seconds} = RateLimit.hit(@kind, client, @window_ms) + if count == limit, do: log_lockout(client, limit) + end + + :ok + end + + defp limit, do: Application.get_env(:kotiko, :auth_failures_per_minute, 10) + + defp log_lockout(client, limit) do + {decision, unlogged} = Limiter.check(@log_table, client, max_keys: @max_logged) + + if unlogged do + Logger.warning( + "Didn't log #{unlogged} more addresses that sent too many wrong API tokens in the " <> + "last hour (too many)." + ) + end + + case decision do + :log -> + Logger.warning( + "#{limit} wrong API tokens from #{format(client)} in a minute: refusing its " <> + "requests until the minute is over. If that's you, check the token in the " <> + "extension's Connection settings." + ) + + :overflow -> + Logger.warning( + "#{@max_logged} addresses sent too many wrong API tokens this hour; not logging " <> + "new ones until the hour is over." + ) + + :quiet -> + :ok + end + end + + defp format({a, b, c, d, :"/64"}), + do: Enum.map_join([a, b, c, d], ":", &String.downcase(Integer.to_string(&1, 16))) <> "::/64" + + defp format(ip) when is_tuple(ip), do: to_string(:inet.ntoa(ip)) + defp format(other), do: inspect(other) +end diff --git a/server/lib/kotiko/rate_limit.ex b/server/lib/kotiko/rate_limit.ex new file mode 100644 index 0000000..c97d85b --- /dev/null +++ b/server/lib/kotiko/rate_limit.ex @@ -0,0 +1,93 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.RateLimit do + @moduledoc """ + Counts requests per client address in fixed windows, in one ETS table: wrong API tokens + (`Kotiko.Router`, slice 54 B-09) and token proofs (`POST /api/v1/proof`). + + A client is its IPv4 address, or the /64 network of its IPv6 address (one machine + usually has a whole /64, so counting single IPv6 addresses would let it pick a new one + for every request). An IPv4-mapped IPv6 address counts as the IPv4 one. + + The table holds at most 10,000 counters, so spreading requests over many addresses + can't grow memory without bound. When it is full, counters whose window is over are + dropped; if it is still full, a new client isn't counted until there is room. Sharing + one counter between all new clients instead would let someone with many addresses + lock the owner out, which this throttle must never do. + + Without the table (it is made at boot), nothing is counted. + """ + + import Bitwise + + @table __MODULE__ + @max_entries 10_000 + + @doc "Creates the table if it isn't there yet." + def init do + if :ets.whereis(@table) == :undefined do + :ets.new(@table, [:named_table, :public, :set, write_concurrency: true]) + end + + :ok + end + + @doc "Forgets every count (for tests)." + def reset do + if :ets.whereis(@table) != :undefined, do: :ets.delete_all_objects(@table) + :ok + end + + @doc "The key `remote_ip` is counted under." + def client({_, _, _, _} = ip), do: ip + def client({0, 0, 0, 0, 0, 0xFFFF, ab, cd}), do: {ab >>> 8, ab &&& 0xFF, cd >>> 8, cd &&& 0xFF} + def client({a, b, c, d, _, _, _, _}), do: {a, b, c, d, :"/64"} + def client(other), do: other + + @doc """ + Counts one more `kind` of request from `client` in a window of `window_ms`. Returns + `{count, seconds_left}`: the count in the current window, this one included, and the + whole seconds until it ends (at least 1). + """ + def hit(kind, client, window_ms) do + now = System.monotonic_time(:millisecond) + key = {kind, client} + + with true <- :ets.whereis(@table) != :undefined, + true <- room_for?(key, now) do + case :ets.lookup(@table, key) do + [{_, ends, _count}] when ends > now -> + {:ets.update_counter(@table, key, {3, 1}), seconds(ends - now)} + + _ -> + :ets.insert(@table, {key, now + window_ms, 1}) + {1, seconds(window_ms)} + end + else + false -> {0, 0} + end + end + + @doc "Like `hit/3`, without counting: `{count, seconds_left}`, `{0, 0}` when none." + def peek(kind, client) do + now = System.monotonic_time(:millisecond) + + with true <- :ets.whereis(@table) != :undefined, + [{_, ends, count}] when ends > now <- :ets.lookup(@table, {kind, client}) do + {count, seconds(ends - now)} + else + _ -> {0, 0} + end + end + + defp room_for?(key, now) do + :ets.member(@table, key) or :ets.info(@table, :size) < @max_entries or + (drop_expired(now) > 0 and :ets.info(@table, :size) < @max_entries) + end + + defp drop_expired(now), + do: :ets.select_delete(@table, [{{:_, :"$1", :_}, [{:"=<", :"$1", now}], [true]}]) + + defp seconds(ms), do: max(1, div(ms + 999, 1000)) +end diff --git a/server/lib/kotiko/router.ex b/server/lib/kotiko/router.ex index 770e803..9c8ab27 100644 --- a/server/lib/kotiko/router.ex +++ b/server/lib/kotiko/router.ex @@ -5,7 +5,7 @@ defmodule Kotiko.Router do use Plug.Router use Plug.ErrorHandler require Logger - alias Kotiko.{AddRequests, I18n, Lookup, Word, Words} + alias Kotiko.{AddRequests, AuthThrottle, I18n, Lookup, RateLimit, Token, Word, Words} # First, on every route including /health: refuse names we don't answer to (DNS rebinding). plug Kotiko.Plug.HostCheck @@ -19,11 +19,16 @@ defmodule Kotiko.Router do @parsers [parsers: [:json], json_decoder: Jason] @small_body Plug.Parsers.init([length: 64_000] ++ @parsers) @batch_body Plug.Parsers.init([length: 1_000_000] ++ @parsers) + @proof_body Plug.Parsers.init([length: 1_000] ++ @parsers) - # 64 KB everywhere, except the batch add, which takes up to 500 words (slice 07). + # 64 KB everywhere, except the batch add, which takes up to 500 words (slice 07), and the + # proof, which anyone may send: a nonce fits in 1 KB. defp parse_body(%{path_info: ["api", "v1", "words", "batch"]} = conn, _opts), do: Plug.Parsers.call(conn, @batch_body) + defp parse_body(%{path_info: ["api", "v1", "proof"]} = conn, _opts), + do: Plug.Parsers.call(conn, @proof_body) + defp parse_body(conn, _opts), do: Plug.Parsers.call(conn, @small_body) # Open (no token): which server this is, its version and whether the database works. @@ -38,6 +43,26 @@ defmodule Kotiko.Router do conn |> put_resp_header("cache-control", "no-store") |> send_resp(status, "") end + # Open (no token): proves this server holds the API token without revealing it, so the + # extension can check an address before it sends the token there (slice 54, B-01). + post "/api/v1/proof" do + limit = Application.get_env(:kotiko, :proof_requests_per_minute, 30) + {count, seconds} = RateLimit.hit(:proof, RateLimit.client(conn.remote_ip), 60_000) + + cond do + is_integer(limit) and count > limit -> + rate_limited(conn, "too_many_proofs", seconds) + + Token.nonce?(conn.body_params["nonce"]) -> + token = Application.fetch_env!(:kotiko, :api_token) + proof = Token.proof(token, conn.body_params["nonce"]) + conn |> put_resp_header("cache-control", "no-store") |> json(200, %{proof: proof}) + + true -> + error(conn, true, 400, "invalid_request", %{field: "nonce"}) + end + end + # Slice 07's versioned API: words with ids, edits, deletes that can be undone. forward "/api/v1", to: Kotiko.RouterV1 @@ -200,24 +225,47 @@ defmodule Kotiko.Router do else: json(conn, status, %{error: message}) end - # Deny by default: only the exact GET or HEAD /health is open. path_info is not yet - # percent-decoded here, but routing decodes it, so matching on ["api" | _] let - # "/%61pi/words" through without a token. + # Deny by default: only the exact GET or HEAD /health and POST /api/v1/proof are open. + # path_info is not yet percent-decoded here, but routing decodes it, so matching on + # ["api" | _] let "/%61pi/words" through without a token. defp authorize(%{path_info: ["health"], method: m} = conn, _opts) when m in ~w(GET HEAD), do: conn + defp authorize(%{path_info: ["api", "v1", "proof"], method: "POST"} = conn, _opts), do: conn + + # An address that sent too many wrong tokens gets 429 whatever it sends now + # (Kotiko.AuthThrottle). defp authorize(conn, _opts) do + case AuthThrottle.check(conn.remote_ip) do + :ok -> check_token(conn) + {:locked, seconds} -> rate_limited(conn, "auth_failures", seconds) + end + end + + defp check_token(conn) do expected = Application.fetch_env!(:kotiko, :api_token) case get_req_header(conn, "authorization") do ["Bearer " <> token] -> - if Plug.Crypto.secure_compare(token, expected), do: conn, else: deny(conn) + if Plug.Crypto.secure_compare(token, expected) do + conn + else + AuthThrottle.failed(conn.remote_ip) + deny(conn) + end _ -> deny(conn) end end + defp rate_limited(conn, reason, seconds) do + conn + |> put_resp_header("retry-after", to_string(seconds)) + |> error(true, 429, "rate_limited", %{reason: reason}) + |> halt() + end + defp deny(conn) do body = %{ error: %{ diff --git a/server/lib/kotiko/token.ex b/server/lib/kotiko/token.ex index 26fefac..0cf52d1 100644 --- a/server/lib/kotiko/token.ex +++ b/server/lib/kotiko/token.ex @@ -73,6 +73,26 @@ defmodule Kotiko.Token do """ def save(path, contents), do: Kotiko.Private.write(path, contents, ".new") + @proof_label "kotiko-proof-v1:" + + @doc """ + Whether `nonce` is one the extension may send to `POST /api/v1/proof`: 32 to 128 + base64url characters (`A-Z a-z 0-9 - _`, no padding). + """ + def nonce?(nonce), do: is_binary(nonce) and nonce =~ ~r/\A[A-Za-z0-9_-]{32,128}\z/ + + @doc """ + The proof that this server holds `token`, for `nonce`: + `base64url(HMAC-SHA256(key: token, message: "kotiko-proof-v1:" <> nonce))`, without + padding. It reveals nothing about the token that helps someone who doesn't have it, + except a value to test guesses against offline, which only a weak token makes useful. + """ + def proof(token, nonce) do + :hmac + |> :crypto.mac(:sha256, token, @proof_label <> nonce) + |> Base.url_encode64(padding: false) + end + @doc """ One pasteable value carrying both the server address and the token, for the extension's connection screen: `kotiko-pair:1:`. diff --git a/server/priv/locales/en/messages.json b/server/priv/locales/en/messages.json index 18a45d4..d1fa113 100644 --- a/server/priv/locales/en/messages.json +++ b/server/priv/locales/en/messages.json @@ -99,6 +99,14 @@ "message": "Word lookup is busy. Try again in a minute.", "description": "429: the provider asked to slow down." }, + "error_rate_limited_auth_failures": { + "message": "Too many wrong access keys came from this address. Wait a minute, then check the key in Connection.", + "description": "429: 10 wrong tokens from this address within a minute; Retry-After says when to try again." + }, + "error_rate_limited_too_many_proofs": { + "message": "Too many requests from this address. Try again in a minute.", + "description": "429: too many POST /api/v1/proof requests from this address within a minute." + }, "error_model_unavailable": { "message": "Word lookup isn't answering right now. Try again in a little while.", "description": "502: no model could answer." diff --git a/server/test/kotiko/listener_test.exs b/server/test/kotiko/listener_test.exs index 81f1755..0f7d807 100644 --- a/server/test/kotiko/listener_test.exs +++ b/server/test/kotiko/listener_test.exs @@ -3,8 +3,9 @@ defmodule Kotiko.ListenerTest do # Which addresses the server listens on for BIND (slice 54, B-01), on real sockets. The - # boot with the default settings is in boot_test.exs. - use ExUnit.Case, async: true + # boot with the default settings is in boot_test.exs. Not async: its log lines would + # land in other tests' capture_log. + use ExUnit.Case, async: false import ExUnit.CaptureLog alias Kotiko.Listener diff --git a/server/test/kotiko/rate_limit_test.exs b/server/test/kotiko/rate_limit_test.exs new file mode 100644 index 0000000..79aeee7 --- /dev/null +++ b/server/test/kotiko/rate_limit_test.exs @@ -0,0 +1,44 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.RateLimitTest do + use ExUnit.Case, async: false + alias Kotiko.RateLimit + + setup do + RateLimit.init() + RateLimit.reset() + on_exit(&RateLimit.reset/0) + end + + test "counts per kind and client in a fixed window" do + assert RateLimit.hit(:a, {1, 2, 3, 4}, 60_000) == {1, 60} + assert {2, _} = RateLimit.hit(:a, {1, 2, 3, 4}, 60_000) + assert {1, _} = RateLimit.hit(:b, {1, 2, 3, 4}, 60_000) + assert {2, s} = RateLimit.peek(:a, {1, 2, 3, 4}) + assert s in 1..60 + assert RateLimit.peek(:a, {9, 9, 9, 9}) == {0, 0} + + assert {1, 1} = RateLimit.hit(:short, :c, 20) + Process.sleep(30) + assert RateLimit.peek(:short, :c) == {0, 0} + assert {1, _} = RateLimit.hit(:short, :c, 20) + end + + test "clients: IPv4, IPv4-mapped IPv6 as IPv4, IPv6 by its /64" do + assert RateLimit.client({192, 0, 2, 1}) == {192, 0, 2, 1} + assert RateLimit.client({0, 0, 0, 0, 0, 0xFFFF, 0xC000, 0x0201}) == {192, 0, 2, 1} + assert RateLimit.client({0x2001, 0xDB8, 0, 1, 2, 3, 4, 5}) == {0x2001, 0xDB8, 0, 1, :"/64"} + end + + test "at most 10,000 counters: when full, expired ones go, else new clients aren't counted" do + for i <- 1..9_999, do: RateLimit.hit(:fill, i, 60_000) + RateLimit.hit(:fill, :expiring, 20) + assert RateLimit.hit(:fill, :new, 60_000) == {0, 0} + # Clients already counted keep being counted. + assert {2, _} = RateLimit.hit(:fill, 1, 60_000) + + Process.sleep(30) + assert {1, _} = RateLimit.hit(:fill, :new, 60_000) + end +end diff --git a/server/test/kotiko/router_proof_test.exs b/server/test/kotiko/router_proof_test.exs new file mode 100644 index 0000000..7d6d471 --- /dev/null +++ b/server/test/kotiko/router_proof_test.exs @@ -0,0 +1,111 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.RouterProofTest do + # POST /api/v1/proof (slice 54, B-01): before it sends the token to an address, the + # extension can ask the server to prove it holds the token, without either side sending + # it. Another program listening at that address can't answer. + use Kotiko.ConnCase, async: false + + @nonce "n0nce_from-the-extension_0123456789abcdef" + + defp expected(token, nonce) do + :crypto.mac(:hmac, :sha256, token, "kotiko-proof-v1:" <> nonce) + |> Base.url_encode64(padding: false) + end + + defp prove(body, opts \\ []) do + conn = build_conn(body, opts) + Kotiko.Router.call(conn, Kotiko.Router.init([])) + end + + defp build_conn(body, opts) do + conn = + Plug.Test.conn("POST", "http://localhost/api/v1/proof", Jason.encode!(body)) + |> Plug.Conn.put_req_header("content-type", "application/json") + + %{conn | remote_ip: Keyword.get(opts, :ip, {127, 0, 0, 1})} + end + + test "answers HMAC-SHA256(token, \"kotiko-proof-v1:\" <> nonce), without a token" do + conn = prove(%{nonce: @nonce}) + + assert conn.status == 200 + assert json_body(conn) == %{"proof" => expected(token(), @nonce)} + refute conn.resp_body =~ token() + assert get_resp_header(conn, "cache-control") == ["no-store"] + end + + test "matches the example in docs/reference/http-api.md" do + assert Kotiko.Token.proof( + "example-token-0123456789abcdef", + "q1aP3n0ZKcB1x5mW0u7S9bJ2rV4yT6dE8gH0iL2nO4p" + ) == "E5CHaF60VqTTPPduGD1yAlfWMF5xf9_hmV2qCufV-ek" + end + + test "the proof depends on the token: another token gives another proof" do + other = "another-token-0123456789abcdefghijklmn" + refute json_body(prove(%{nonce: @nonce}))["proof"] == expected(other, @nonce) + put_app_env(:api_token, other) + assert json_body(prove(%{nonce: @nonce}))["proof"] == expected(other, @nonce) + end + + test "the nonce must be 32 to 128 base64url characters" do + for nonce <- [ + String.duplicate("a", 31), + String.duplicate("a", 129), + String.duplicate("a", 40) <> "=", + String.duplicate("a", 40) <> "+/", + String.duplicate("é", 40), + 42, + nil + ] do + conn = prove(%{nonce: nonce}) + assert conn.status == 400, inspect(nonce) + + assert %{"code" => "invalid_request", "details" => %{"field" => "nonce"}} = + json_body(conn)["error"] + end + + assert prove(%{}).status == 400 + assert prove(%{nonce: String.duplicate("A", 32)}).status == 200 + assert prove(%{nonce: String.duplicate("-", 128)}).status == 200 + end + + test "only POST, only this exact path, and a small body" do + # Another method, or a percent-encoded spelling, still needs the token. + assert request("GET", "/api/v1/proof").status == 401 + assert request("POST", "/%61pi/v1/proof", [], body: %{nonce: @nonce}).status == 401 + assert request("POST", "/api/v1/pro%6Ff", [], body: %{nonce: @nonce}).status == 401 + + {status, _headers, _body} = + request_raising("POST", "/api/v1/proof", [], + body: %{nonce: @nonce, pad: String.duplicate("x", 2_000)} + ) + + assert status == 413 + end + + test "an unknown Host is refused before the proof" do + conn = request("POST", "/api/v1/proof", [], body: %{nonce: @nonce}, host: "evil.example") + assert conn.status == 421 + end + + test "at most proof_requests_per_minute from one address, then 429 with Retry-After" do + put_app_env(:proof_requests_per_minute, 3) + ip = {10, 77, 0, System.unique_integer([:positive]) |> rem(250)} + + for _ <- 1..3, do: assert(prove(%{nonce: @nonce}, ip: ip).status == 200) + conn = prove(%{nonce: @nonce}, ip: ip) + + assert conn.status == 429 + + assert %{"code" => "rate_limited", "details" => %{"reason" => "too_many_proofs"}} = + json_body(conn)["error"] + + assert [retry] = get_resp_header(conn, "retry-after") + assert String.to_integer(retry) in 1..60 + # Another address is unaffected. + assert prove(%{nonce: @nonce}, ip: {10, 78, 0, 1}).status == 200 + end +end diff --git a/server/test/kotiko/router_throttle_test.exs b/server/test/kotiko/router_throttle_test.exs new file mode 100644 index 0000000..088e104 --- /dev/null +++ b/server/test/kotiko/router_throttle_test.exs @@ -0,0 +1,88 @@ +# SPDX-FileCopyrightText: 2026 ScriptKittyOS and the Kotiko contributors +# SPDX-License-Identifier: Apache-2.0 + +defmodule Kotiko.RouterThrottleTest do + # Wrong tokens are throttled per address (slice 54, B-09 and C-09): after 10 in a minute + # that address gets 429 for the rest of the minute, while the right token from any + # other address still works. config/test.exs turns the throttle off for the other + # tests, which send many wrong tokens from 127.0.0.1; these turn it on. + use Kotiko.ConnCase, async: false + import ExUnit.CaptureLog + + @wrong [{"authorization", "Bearer wrong-token-0123456789abcdefghijkl"}] + + setup do + put_app_env(:auth_failures_per_minute, 10) + Kotiko.AuthThrottle.reset() + on_exit(&Kotiko.AuthThrottle.reset/0) + :ok + end + + defp from(ip, headers, path \\ "/api/v1/words") do + conn = Plug.Test.conn("GET", "http://localhost" <> path) + + %{conn | remote_ip: ip, req_headers: conn.req_headers ++ headers} + |> Kotiko.Router.call(Kotiko.Router.init([])) + end + + test "the 11th wrong token in a minute gets 429; the right token elsewhere still works" do + attacker = {203, 0, 113, 9} + + log = + capture_log(fn -> + for _ <- 1..10, do: assert(from(attacker, @wrong).status == 401) + end) + + conn = from(attacker, @wrong) + assert conn.status == 429 + assert [retry] = get_resp_header(conn, "retry-after") + assert String.to_integer(retry) in 1..60 + + assert %{"code" => "rate_limited", "details" => %{"reason" => "auth_failures"}} = + json_body(conn)["error"] + + # Locked out: even the right token gets 429 from there, so guessing learns nothing. + assert from(attacker, auth()).status == 429 + # The owner, from another address, is unaffected. + assert from({198, 51, 100, 7}, auth()).status == 200 + assert from({127, 0, 0, 1}, auth()).status == 200 + # /health stays open to everyone. + assert from(attacker, [], "/health").status == 200 + + assert log =~ "10 wrong API tokens from 203.0.113.9 in a minute" + refute log =~ "wrong-token" + end + + test "requests without a token don't count; the lockout line is logged once" do + ip = {203, 0, 113, 10} + for _ <- 1..20, do: assert(from(ip, []).status == 401) + assert from(ip, auth()).status == 200 + + log = + capture_log(fn -> + for _ <- 1..15, do: from(ip, @wrong) + end) + + assert length(String.split(log, "wrong API tokens from 203.0.113.10")) == 2 + end + + test "an IPv6 /64 counts as one address; IPv4-mapped addresses as their IPv4" do + log = + capture_log(fn -> + for i <- 1..10, do: from({0x2001, 0xDB8, 1, 2, 0, 0, 0, i}, @wrong) + for _ <- 1..10, do: from({0, 0, 0, 0, 0, 0xFFFF, 0xCB00, 0x710B}, @wrong) + end) + + assert from({0x2001, 0xDB8, 1, 2, 9, 9, 9, 9}, auth()).status == 429 + assert from({0x2001, 0xDB8, 1, 3, 0, 0, 0, 1}, auth()).status == 200 + assert from({203, 0, 113, 11}, auth()).status == 429 + assert log =~ "from 2001:db8:1:2::/64 in a minute" + assert log =~ "from 203.0.113.11 in a minute" + end + + test "off when auth_failures_per_minute is nil" do + put_app_env(:auth_failures_per_minute, nil) + for _ <- 1..30, do: from({203, 0, 113, 12}, @wrong) + assert from({203, 0, 113, 12}, auth()).status == 200 + end +end From a2fb97377342e84d331562d25e5be7f7ee3e4245 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:43:44 -0400 Subject: [PATCH 04/12] fix(server): warn about an API token that is easy to guess Any API_TOKEN of 24 characters was accepted without a word, even 24 a's or "password" three times (slice 54, B-09). Kotiko.Token.weakness/1 flags a token with fewer than 10 different characters, one that repeats a shorter piece, or one under 64 bits by its characters' frequencies (length times the Shannon entropy of the counts). Config warns about such an API_TOKEN (or API_TOKEN_FILE), and the boot warns about such a token saved in api-token by hand. The warning never shows the token. This matters more now that POST /api/v1/proof lets someone test guesses offline. Regression tests: 24 a's and "password" x3 now start with a warning (they started with none); 200 generated tokens and openssl hex get no warning; a boot with a weak saved token warns without printing it. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 4 +++ docs/reference/configuration.md | 10 ++++++ docs/security/requirements.md | 4 ++- server/lib/kotiko/application.ex | 4 +++ server/lib/kotiko/config.ex | 24 +++++++++++++- server/lib/kotiko/token.ex | 52 ++++++++++++++++++++++++++++++ server/test/kotiko/boot_test.exs | 15 +++++++++ server/test/kotiko/config_test.exs | 21 ++++++++++++ server/test/kotiko/token_test.exs | 19 +++++++++++ 9 files changed, 151 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b2063af..84b76f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: an access key you chose yourself that looks easy to guess (few + different characters, a repeated piece, mostly one character) now gets a warning at + every start. Delete `API_TOKEN` to let the server make a random one. + - Self-hosted server: after 10 wrong access keys from one address within a minute, the server refuses that address for the rest of the minute (`429`) and logs it once. Other addresses, including yours, keep working. New route `POST /api/v1/proof`: Kotiko can diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index e302d68..3fed49e 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -104,6 +104,16 @@ The token the extension sends as `Authorization: Bearer ` (see the random source, written as 43 URL-safe characters, in a file only you can read (mode 0600). - Allowed: at least 24 characters. Example: the output of `openssl rand -hex 24`. - Wrong: a shorter token stops the server (the message gives its length, not its value). +- A token that looks easy to guess gets a warning at every start (never showing the + token): one with fewer than 10 different characters, one that repeats a shorter piece + (`passwordpasswordpassword`), or one with under 64 bits by its characters' frequencies + (its length times the Shannon entropy of the character counts, low when most characters + are the same). This is a simple check that catches typed tokens, not a guarantee. The + same check runs on a token you put in `api-token` yourself. Ten wrong tokens from one + address within a minute lock that address out for the rest of the minute + ([HTTP API](http-api.md#requests)), but [`POST /api/v1/proof`](http-api.md#post-apiv1proof) + lets anyone who can reach the server test guesses offline, so only a random token is + safe on a network. - To replace the saved token: `mix kotiko.token --rotate`, then restart. With `API_TOKEN` set, edit `.env` instead; with [`API_TOKEN_FILE`](#api_token_file), replace the file's contents. diff --git a/docs/security/requirements.md b/docs/security/requirements.md index 59f6f39..4865912 100644 --- a/docs/security/requirements.md +++ b/docs/security/requirements.md @@ -21,7 +21,9 @@ Kotiko has two parts: the browser extension, and an optional server you run your - **A strong token, kept private.** On first start the server makes a random 256-bit token and saves it in `api-token` in the data folder, readable only by you (mode 0600). If you pick your own with `API_TOKEN`, it must be at least 24 characters, or the server won't - start. + start, and one that looks easy to guess gets a warning at every start. After 10 wrong + tokens from one address within a minute, that address is refused until the minute is + over; other addresses keep working. - **Other accounts on the computer can't read your words.** The database, its `-wal` and `-shm` files, the backups, the token and the model list cache are readable only by you (0600), and a data folder the server makes only by you (0700). At every start it makes diff --git a/server/lib/kotiko/application.ex b/server/lib/kotiko/application.ex index e82960e..a3a1bcc 100644 --- a/server/lib/kotiko/application.ex +++ b/server/lib/kotiko/application.ex @@ -56,6 +56,10 @@ defmodule Kotiko.Application do {:ok, token, source} -> Application.put_env(:kotiko, :api_token, token) if match?({:generated, _}, source), do: print_new_token(token) + # API_TOKEN was checked with the other settings; a saved token is checked here. + with {:file, path} <- source, + do: Enum.each(Config.weak_token(token, "The token in #{path}"), &Logger.warning/1) + source {:error, message} -> diff --git a/server/lib/kotiko/config.ex b/server/lib/kotiko/config.ex index 204fbfd..224f0ba 100644 --- a/server/lib/kotiko/config.ex +++ b/server/lib/kotiko/config.ex @@ -615,7 +615,29 @@ defmodule Kotiko.Config do [] end - key_warning ++ renamed(vars) ++ unknown_prefixed(vars) + key_warning ++ weak_token(vars["API_TOKEN"]) ++ renamed(vars) ++ unknown_prefixed(vars) + end + + @doc """ + A warning when a token the owner chose looks easy to guess (`Kotiko.Token.weakness/1`), + else nothing. `what` names where it came from; the token itself is never in it. + """ + def weak_token(token, what \\ "API_TOKEN") + def weak_token(nil, _what), do: [] + + def weak_token(token, what) do + case Kotiko.Token.weakness(token) do + nil -> + [] + + reason -> + [ + "#{what} looks easy to guess: #{Kotiko.Token.weakness_text(reason)}. Anyone who " <> + "can reach the server can try guesses, and one answer from POST " <> + "/api/v1/proof lets them test guesses offline. Use a random token: delete it " <> + "and the server makes one, or use the output of `openssl rand -hex 24`." + ] + end end defp renamed(vars) do diff --git a/server/lib/kotiko/token.ex b/server/lib/kotiko/token.ex index 0cf52d1..327f20e 100644 --- a/server/lib/kotiko/token.ex +++ b/server/lib/kotiko/token.ex @@ -10,6 +10,10 @@ defmodule Kotiko.Token do @min_length 24 @file_name "api-token" + # weakness/1's thresholds. + @min_different 10 + @min_bits 64 + @max_repeat_check 256 def min_length, do: @min_length @@ -75,6 +79,54 @@ defmodule Kotiko.Token do @proof_label "kotiko-proof-v1:" + @doc """ + Why `token` looks easy to guess, or nil. A simple check, not a guarantee (slice 54, + B-09): it catches tokens a person typed, not ones made at random. + + * `:few_characters`: fewer than 10 different characters (`aaaa...`, `passwordpassword...`) + * `:repeated`: one shorter piece repeated (`abcdefghijklabcdefghijkl`) + * `:little_variety`: under 64 bits by the characters' frequencies (length times the + Shannon entropy of the character counts), as when most characters are the same + + A token the server makes (32 random bytes, 43 characters) has about 220 such bits, and + `openssl rand -hex 24` about 190. + """ + def weakness(token) do + chars = String.graphemes(token) + counts = Enum.frequencies(chars) + + cond do + map_size(counts) < @min_different -> :few_characters + repeated?(chars) -> :repeated + bits(counts, length(chars)) < @min_bits -> :little_variety + true -> nil + end + end + + @doc "The words for `weakness/1`'s reason, for a warning." + def weakness_text(:few_characters), + do: "it uses fewer than #{@min_different} different characters" + + def weakness_text(:repeated), do: "it repeats a shorter piece" + def weakness_text(:little_variety), do: "most of its characters are the same" + + # Only short tokens: the check is quadratic, and a long one with 10 or more different + # characters isn't what a person types. + defp repeated?(chars) when length(chars) > @max_repeat_check, do: false + + defp repeated?(chars) do + t = List.to_tuple(chars) + n = tuple_size(t) + + Enum.any?(1..div(n, 2)//1, fn period -> + Enum.all?(period..(n - 1)//1, &(elem(t, &1) == elem(t, &1 - period))) + end) + end + + defp bits(counts, n) do + n * -Enum.sum(for {_char, c} <- counts, p = c / n, do: p * :math.log2(p)) + end + @doc """ Whether `nonce` is one the extension may send to `POST /api/v1/proof`: 32 to 128 base64url characters (`A-Z a-z 0-9 - _`, no padding). diff --git a/server/test/kotiko/boot_test.exs b/server/test/kotiko/boot_test.exs index 4bf71ef..d65aad1 100644 --- a/server/test/kotiko/boot_test.exs +++ b/server/test/kotiko/boot_test.exs @@ -215,6 +215,21 @@ defmodule Kotiko.BootTest do refute output =~ "STARTED" end + test "a weak token saved in the data folder gets a warning, never shown", ctx do + data = Path.join(ctx.tmp, "data") + File.mkdir_p!(data) + File.chmod!(data, 0o700) + File.write!(Path.join(data, "api-token"), String.duplicate("password", 3) <> "\n") + File.chmod!(Path.join(data, "api-token"), 0o600) + vars = %{"PORT" => to_string(free_port()), "KOTIKO_DATA_DIR" => data} + + {output, status} = boot(vars, @start_and_stop, ctx) + + assert status == 0, output + assert output =~ "The token in #{data}/api-token looks easy to guess" + refute output =~ "passwordpassword" + end + test "the VM exits non-zero when the supervision tree dies", ctx do vars = %{"KOTIKO_DATA_DIR" => Path.join(ctx.tmp, "data"), "PORT" => to_string(free_port())} diff --git a/server/test/kotiko/config_test.exs b/server/test/kotiko/config_test.exs index 4828f38..45cc8b5 100644 --- a/server/test/kotiko/config_test.exs +++ b/server/test/kotiko/config_test.exs @@ -530,4 +530,25 @@ defmodule Kotiko.ConfigTest do refute Config.openrouter?("http://localhost:11434/v1") refute Config.openrouter?("https://notopenrouter.ai/v1") end + + # B-09 (slice 54): any 24 characters were accepted, even 24 a's. + describe "a weak API_TOKEN" do + test "starts, with a warning that never shows the token" do + for weak <- [String.duplicate("a", 24), String.duplicate("password", 3)] do + assert {:ok, config, warnings} = parse(%{"API_TOKEN" => weak, "LLM_API_KEY" => "k"}) + assert config[:api_token] == weak + assert [warning] = Enum.filter(warnings, &(&1 =~ "API_TOKEN")) + assert warning =~ "easy to guess" + assert warning =~ "openssl rand -hex 24" + refute warning =~ weak + end + end + + test "a random token gets no warning" do + for token <- [@token, Base.encode16(:crypto.strong_rand_bytes(24), case: :lower)] do + assert {:ok, _, warnings} = parse(%{"API_TOKEN" => token, "LLM_API_KEY" => "k"}) + assert Enum.filter(warnings, &(&1 =~ "API_TOKEN")) == [] + end + end + end end diff --git a/server/test/kotiko/token_test.exs b/server/test/kotiko/token_test.exs index b314ce9..01fcf63 100644 --- a/server/test/kotiko/token_test.exs +++ b/server/test/kotiko/token_test.exs @@ -78,4 +78,23 @@ defmodule Kotiko.TokenTest do assert %{"url" => "http://100.101.102.103:4747", "token" => "tok"} = encoded |> Base.url_decode64!(padding: false) |> Jason.decode!() end + + describe "weakness/1 (slice 54, B-09)" do + test "few different characters, a repeated piece, or little variety" do + assert Token.weakness(String.duplicate("a", 24)) == :few_characters + assert Token.weakness(String.duplicate("password", 3)) == :few_characters + assert Token.weakness(String.duplicate("abcdefghijkl", 2)) == :repeated + assert Token.weakness(String.duplicate("a", 15) <> "bcdefghij") == :little_variety + end + + test "generated tokens, hex from openssl and passphrases are fine", %{tmp_dir: dir} do + for _ <- 1..200 do + assert {:ok, token, _} = Token.generate(Path.join(dir, "api-token")) + assert Token.weakness(token) == nil + end + + assert Token.weakness(Base.encode16(:crypto.strong_rand_bytes(24), case: :lower)) == nil + assert Token.weakness("correct-horse-battery-staple-lamp") == nil + end + end end From 5e49cfe6be213926c976222752cc9bf74c60deb5 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:44:39 -0400 Subject: [PATCH 05/12] fix(server): refuse a password inside LLM_URL, TRANSCRIBE_URL or PUBLIC_URL A user name and password in LLM_URL (https://user:pass@host/v1) were accepted and written to the log at info level in the startup summary's "Model:" line (slice 54, B-05). Config now refuses userinfo in LLM_URL, TRANSCRIBE_URL and PUBLIC_URL with a message that hides the value and says where the key goes (LLM_API_KEY, TRANSCRIBE_API_KEY or their _FILE settings). An invalid address containing an @ isn't echoed either. As defence in depth the startup summary strips userinfo from the model URL. Regression tests: config refuses such URLs without echoing the password (before, they started with no warning); the summary's model line shows the URL without credentials (before, it showed them). Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 5 ++++ docs/reference/configuration.md | 10 +++++-- docs/security/requirements.md | 4 ++- server/lib/kotiko/application.ex | 5 +++- server/lib/kotiko/config.ex | 30 +++++++++++++++---- server/test/kotiko/config_test.exs | 33 +++++++++++++++++++++ server/test/kotiko/startup_summary_test.exs | 9 ++++++ 7 files changed, 86 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 84b76f6..2bc2342 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,11 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: a user name and password written inside `LLM_URL` or + `TRANSCRIBE_URL` (`https://user:password@host/...`) went into the log at start. The + server now refuses to start with such an address, without showing it; put the key in + `LLM_API_KEY` or `TRANSCRIBE_API_KEY` instead. The same goes for `PUBLIC_URL`. + - Self-hosted server: an access key you chose yourself that looks easy to guess (few different characters, a repeated piece, mostly one character) now gets a warning at every start. Delete `API_TOKEN` to let the server make a random one. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 3fed49e..13df82f 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -92,7 +92,8 @@ behind a reverse proxy). Used only in the pairing string from [`mix kotiko.token`](#mix-kotikotoken). - Default: none. Allowed: an `http://` or `https://` address; a trailing `/` is dropped. -- Wrong: anything else stops the server ("isn't a web address"). +- Wrong: anything else stops the server ("isn't a web address"), and so does a user name + or password in the address (`https://user:password@...`). ### `API_TOKEN` @@ -137,6 +138,9 @@ The model API: any OpenAI-compatible API's base URL. - Default: `https://openrouter.ai/api/v1`. - Example: `http://localhost:11434/v1` for a local Ollama. - Allowed: an `http://` or `https://` address. Wrong: anything else stops the server. +- A user name or password in the address (`https://user:password@host/v1`) stops the + server too, without showing it: the address is written to the log at start. Put the key + in [`LLM_API_KEY`](#llm_api_key) (or [`LLM_API_KEY_FILE`](#llm_api_key_file)) instead. - Your typed text goes to this address (README, "A different model"). ### `LLM_MODEL` @@ -178,7 +182,9 @@ A speech-to-text endpoint for Telegram voice notes: a local whisper.cpp server upload. - Default: none; voice notes aren't transcribed (typing still works). -- Allowed: an `http://` or `https://` address. Wrong: anything else stops the server. +- Allowed: an `http://` or `https://` address. Wrong: anything else stops the server, and + so does a user name or password in the address: put the key in + [`TRANSCRIBE_API_KEY`](#transcribe_api_key) instead. ### `TRANSCRIBE_MODEL` diff --git a/docs/security/requirements.md b/docs/security/requirements.md index 4865912..7ff4181 100644 --- a/docs/security/requirements.md +++ b/docs/security/requirements.md @@ -46,7 +46,9 @@ Kotiko has two parts: the browser extension, and an optional server you run your it, the bot only tells whoever writes to it their own Telegram ID. - **Keys and tokens stay out of the logs, at every level.** A log filter removes your token, your model and transcription keys, your bot token, and anything shaped like a - bearer token or an OpenRouter or OpenAI key, before a log line is written. + bearer token or an OpenRouter or OpenAI key, before a log line is written. A user name + or password inside `LLM_URL`, `TRANSCRIBE_URL` or `PUBLIC_URL` stops the server instead + of reaching the log. - **Your words stay out of the logs** at the default level. What you look up and what the model answered go to the log only if you set `LOG_LOOKUPS=true`, and then only at debug level. diff --git a/server/lib/kotiko/application.ex b/server/lib/kotiko/application.ex index a3a1bcc..b3bc663 100644 --- a/server/lib/kotiko/application.ex +++ b/server/lib/kotiko/application.ex @@ -137,7 +137,8 @@ defmodule Kotiko.Application do defp model_line do url = Application.fetch_env!(:kotiko, :llm_url) models = Application.fetch_env!(:kotiko, :llm_models) - provider = if Config.openrouter?(url), do: "OpenRouter", else: url + # Config refuses a user name or password in LLM_URL; never log one anyway (B-05). + provider = if Config.openrouter?(url), do: "OpenRouter", else: without_userinfo(url) case Application.get_env(:kotiko, :llm_model_source, :env) do :default -> "#{provider}, its current free models (checked 10 s after start)" @@ -145,6 +146,8 @@ defmodule Kotiko.Application do end end + defp without_userinfo(url), do: URI.to_string(%{URI.parse(url) | userinfo: nil}) + defp telegram_line do cond do is_nil(Application.get_env(:kotiko, :telegram_token)) -> diff --git a/server/lib/kotiko/config.ex b/server/lib/kotiko/config.ex index 224f0ba..5961c04 100644 --- a/server/lib/kotiko/config.ex +++ b/server/lib/kotiko/config.ex @@ -445,16 +445,34 @@ defmodule Kotiko.Config do defp optional_url(_name, nil), do: {:ok, nil} + # A user name or password in the address (slice 54, B-05) would reach the log with the + # address, so it's refused, and the value is never echoed. defp optional_url(name, raw) do - if http_url?(raw) do - {:ok, String.trim_trailing(raw, "/")} - else - error(name, raw, [ - "#{inspect(raw)} isn't a web address. It should start with http:// or https://." - ]) + cond do + userinfo?(raw) -> + secret_error(name, [ + "It has a user name or password in it (before the @), which would end up in " <> + "the log. Remove them from the address. A key for the model or transcription " <> + "service goes in LLM_API_KEY or TRANSCRIBE_API_KEY (or LLM_API_KEY_FILE, " <> + "TRANSCRIBE_API_KEY_FILE)." + ]) + + http_url?(raw) -> + {:ok, String.trim_trailing(raw, "/")} + + # Not an address, but it may hold a password: don't echo it. + String.contains?(raw, "@") -> + secret_error(name, ["It isn't a web address. It should start with http:// or https://."]) + + true -> + error(name, raw, [ + "#{inspect(raw)} isn't a web address. It should start with http:// or https://." + ]) end end + defp userinfo?(raw), do: match?({:ok, %URI{userinfo: info}} when is_binary(info), URI.new(raw)) + defp http_url?(raw) do case URI.new(raw) do {:ok, %URI{scheme: scheme, host: host}} when scheme in ["http", "https"] -> diff --git a/server/test/kotiko/config_test.exs b/server/test/kotiko/config_test.exs index 45cc8b5..c034a4a 100644 --- a/server/test/kotiko/config_test.exs +++ b/server/test/kotiko/config_test.exs @@ -551,4 +551,37 @@ defmodule Kotiko.ConfigTest do end end end + + # B-05 (slice 54): a user name and password in these URLs reached the log at info. + describe "a URL with a user name or password in it" do + test "is refused for LLM_URL, TRANSCRIBE_URL and PUBLIC_URL, never echoing it" do + for name <- ~w(LLM_URL TRANSCRIBE_URL PUBLIC_URL), + url <- [ + "https://proxyuser:Pr0xy-Pa55-SECRET@llm.example/v1", + "http://Pr0xy-Pa55-SECRET@127.0.0.1:8080/v1" + ] do + assert {:error, problems, _} = parse(%{name => url, "LLM_MODEL" => "m"}) + assert [{label, [line]}] = Enum.filter(problems, fn {l, _} -> l =~ name end) + assert label == "#{name} (value hidden)" + assert line =~ "user name or password" + assert line =~ "_API_KEY" + refute inspect(problems) =~ "SECRET" + end + end + + test "something that isn't an address but has an @ isn't echoed either" do + assert {:error, [{"LLM_URL (value hidden)", [line]}], _} = + parse(%{ + "LLM_URL" => "proxyuser:Pr0xy-Pa55-SECRET@llm.example", + "LLM_MODEL" => "m" + }) + + assert line =~ "isn't a web address" + end + + test "an @ elsewhere in the URL is fine" do + assert ok!(%{"LLM_URL" => "https://llm.example/v1/@team", "LLM_MODEL" => "m"})[:llm_url] == + "https://llm.example/v1/@team" + end + end end diff --git a/server/test/kotiko/startup_summary_test.exs b/server/test/kotiko/startup_summary_test.exs index 8139ef6..77426b5 100644 --- a/server/test/kotiko/startup_summary_test.exs +++ b/server/test/kotiko/startup_summary_test.exs @@ -77,4 +77,13 @@ defmodule Kotiko.StartupSummaryTest do put_app_env(:transcribe_url, "http://127.0.0.1:8080/inference") assert List.last(App.summary_lines(:env, 0, "")) == "Voice notes: on" end + + test "the model line never shows a user name or password in LLM_URL (B-05)" do + put_app_env(:llm_url, "http://proxyuser:Pr0xy-Pa55-SECRET@127.0.0.1:42100/llm/v1") + put_app_env(:llm_model_source, :env) + put_app_env(:llm_models, ["m"]) + + model = line(App.summary_lines(:env, 0, ""), "Model:") + assert model == "Model: http://127.0.0.1:42100/llm/v1, 1 model from LLM_MODEL" + end end From 0a1806eaf8c8facbd76ded2e28d0e4b71348c728 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:45:19 -0400 Subject: [PATCH 06/12] fix(server): warn when a provider key goes over plain HTTP The server warned when its own token crossed a network in the clear, but not when LLM_API_KEY or TRANSCRIBE_API_KEY was sent to a plain http:// address on another machine (slice 54, B-06). Config now warns at start when a key is set and its URL is http:// to a host that isn't this computer (localhost, *.localhost, 127.0.0.0/8, ::1) or Tailscale (100.64.0.0/10, fd7a:115c:a1e0::/48, *.ts.net). Kotiko.Exposure.network/1 classifies the URL's host; the warning names the key and the host, never the key's value. Regression tests: an LLM key to http://203.0.113.7 and a transcription key to http://whisper.lan now warn (they didn't); loopback, Tailscale, https and no key stay quiet. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 4 +++ docs/reference/configuration.md | 6 +++- server/lib/kotiko/config.ex | 30 +++++++++++++++++++- server/lib/kotiko/exposure.ex | 24 ++++++++++++++++ server/test/kotiko/config_test.exs | 44 ++++++++++++++++++++++++++++++ 5 files changed, 106 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2bc2342..6e5bab2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: when your model or transcription key would go over plain `http://` + to another machine (not this computer, not Tailscale), the server now warns at start, + because anyone on the network in between could read it. + - Self-hosted server: a user name and password written inside `LLM_URL` or `TRANSCRIBE_URL` (`https://user:password@host/...`) went into the log at start. The server now refuses to start with such an address, without showing it; put the key in diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 13df82f..85ef769 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -128,7 +128,9 @@ The key for the model API that works out which word you mean. A secret. and adds answer `503 lookup_not_set_up` until you set it. A local model such as Ollama needs no key. - Example: a key from . -- Sent only to `LLM_URL`. +- Sent only to `LLM_URL`. When `LLM_URL` is a plain `http://` address on another machine + (not this computer, not a Tailscale address or `*.ts.net` name), the server warns at + start: anyone on the network in between can read the key. - Or from a file: [`LLM_API_KEY_FILE`](#llm_api_key_file). ### `LLM_URL` @@ -197,6 +199,8 @@ The model name sent with each voice note. Sent as `Authorization: Bearer ` to `TRANSCRIBE_URL`, for hosted services. A secret. - Default: none (a local whisper.cpp server needs none). +- With a plain `http://` `TRANSCRIBE_URL` on another machine (not this computer or + Tailscale), the server warns at start, as for `LLM_API_KEY`. - Or from a file: [`TRANSCRIBE_API_KEY_FILE`](#transcribe_api_key_file). ### `KOTIKO_DATA_DIR` diff --git a/server/lib/kotiko/config.ex b/server/lib/kotiko/config.ex index 5961c04..fda9d2b 100644 --- a/server/lib/kotiko/config.ex +++ b/server/lib/kotiko/config.ex @@ -633,9 +633,37 @@ defmodule Kotiko.Config do [] end - key_warning ++ weak_token(vars["API_TOKEN"]) ++ renamed(vars) ++ unknown_prefixed(vars) + key_warning ++ + weak_token(vars["API_TOKEN"]) ++ + plain_http("LLM_API_KEY", vars["LLM_API_KEY"], "LLM_URL", llm_url) ++ + plain_http( + "TRANSCRIBE_API_KEY", + vars["TRANSCRIBE_API_KEY"], + "TRANSCRIBE_URL", + optional_url("TRANSCRIBE_URL", vars["TRANSCRIBE_URL"]) + ) ++ renamed(vars) ++ unknown_prefixed(vars) end + # A key sent in the clear to another machine (slice 54, B-06): anyone on the network in + # between can read it. This computer and Tailscale (encrypted) are fine. + defp plain_http(_key_var, nil, _url_var, _url), do: [] + + defp plain_http(key_var, _key, url_var, {:ok, "http://" <> _ = url}) do + host = URI.parse(url).host + + if Kotiko.Exposure.network(host) in [:loopback, :tailscale] do + [] + else + [ + "#{key_var} is sent over plain HTTP to #{host} (#{url_var}), so anyone on the " <> + "network in between can read it, and what is sent with it. Use an https:// " <> + "address, or reach that machine over Tailscale." + ] + end + end + + defp plain_http(_key_var, _key, _url_var, _url), do: [] + @doc """ A warning when a token the owner chose looks easy to guess (`Kotiko.Token.weakness/1`), else nothing. `what` names where it came from; the token itself is never in it. diff --git a/server/lib/kotiko/exposure.ex b/server/lib/kotiko/exposure.ex index 51d0854..a15a3a0 100644 --- a/server/lib/kotiko/exposure.ex +++ b/server/lib/kotiko/exposure.ex @@ -40,6 +40,30 @@ defmodule Kotiko.Exposure do end end + @doc """ + Who can see traffic to `host` (an IP tuple, or a host name from a URL): `:loopback` for + this machine (`localhost`, `*.localhost`, 127.0.0.0/8, ::1), `:tailscale` for Tailscale + (100.64.0.0/10, fd7a:115c:a1e0::/48, `*.ts.net`), else `:lan`, `:all` or `:public` as + for `classify/2`. A name that is neither is `:public`: Kotiko can't tell where it goes. + """ + def network(host) when is_binary(host) do + name = host |> String.downcase() |> String.trim_trailing(".") + + case :inet.parse_strict_address(to_charlist(String.trim(name, "[]"))) do + {:ok, ip} -> + class(ip) + + {:error, _} -> + cond do + name == "localhost" or String.ends_with?(name, ".localhost") -> :loopback + String.ends_with?(name, ".ts.net") -> :tailscale + true -> :public + end + end + end + + def network(ip) when is_tuple(ip), do: class(ip) + defp class({127, _, _, _}), do: :loopback defp class({0, 0, 0, 0, 0, 0, 0, 1}), do: :loopback defp class({0, 0, 0, 0}), do: :all diff --git a/server/test/kotiko/config_test.exs b/server/test/kotiko/config_test.exs index c034a4a..9c7647a 100644 --- a/server/test/kotiko/config_test.exs +++ b/server/test/kotiko/config_test.exs @@ -584,4 +584,48 @@ defmodule Kotiko.ConfigTest do "https://llm.example/v1/@team" end end + + # B-06 (slice 54): a key sent over plain HTTP to another machine gave no warning. + describe "a key sent over plain HTTP" do + defp plain_http_warnings(vars) do + assert {:ok, _, warnings} = parse(vars) + Enum.filter(warnings, &(&1 =~ "plain HTTP")) + end + + test "to another machine gets a warning naming the key and the host" do + key = "sk-secret-key-0123456789abcdef" + + assert [warning] = + plain_http_warnings(%{ + "LLM_URL" => "http://203.0.113.7/v1", + "LLM_MODEL" => "m", + "LLM_API_KEY" => key + }) + + assert warning =~ "LLM_API_KEY is sent over plain HTTP to 203.0.113.7" + refute warning =~ key + + assert [warning] = + plain_http_warnings(%{ + "TRANSCRIBE_URL" => "http://whisper.lan:8178/inference", + "TRANSCRIBE_API_KEY" => key, + "LLM_API_KEY" => "k" + }) + + assert warning =~ "TRANSCRIBE_API_KEY is sent over plain HTTP to whisper.lan" + end + + test "not to this machine, over Tailscale, over HTTPS, or without a key" do + for url <- ~w(http://127.0.0.1:11434/v1 http://localhost:11434/v1 http://[::1]:1/v1 + http://ollama.localhost/v1 http://100.101.102.103/v1 + http://gpu.tail1234.ts.net/v1 https://203.0.113.7/v1) do + assert plain_http_warnings(%{"LLM_URL" => url, "LLM_MODEL" => "m", "LLM_API_KEY" => "k"}) == + [], + url + end + + assert plain_http_warnings(%{"LLM_URL" => "http://203.0.113.7/v1", "LLM_MODEL" => "m"}) == + [] + end + end end From c6a13171a123102ef03891bc101c1fdf44cb383b Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:46:36 -0400 Subject: [PATCH 07/12] fix(server): keep refused words out of the debug log without LOG_LOOKUPS At LOG_LEVEL=debug the words the model answered and the server refused (native and gloss) were logged even with LOG_LOOKUPS=false, though the docs say words are never logged without it (slice 54, B-07). The same was true of a changeset that couldn't be saved and of the words a language-tag upgrade flags. Each of those debug lines now checks Kotiko.LLM.log_lookups?/0 (now public); without it only the reasons are logged. Regression tests (capture_log at debug): the legacy add with a refused word, a save error and the language-tag upgrade no longer log the words without LOG_LOOKUPS (they did), and still do with it. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 4 ++ docs/reference/configuration.md | 4 +- server/lib/kotiko/llm.ex | 7 +- server/lib/kotiko/migrations/language_tags.ex | 6 +- server/lib/kotiko/router.ex | 4 +- server/lib/kotiko/router_v1.ex | 15 +++-- .../kotiko/migration_language_tags_test.exs | 22 +++++++ server/test/kotiko/router_v1_test.exs | 66 +++++++++++++++++++ 8 files changed, 117 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e5bab2..89c00fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: with `LOG_LEVEL=debug`, words the model suggested and the server + refused were written to the log even with `LOG_LOOKUPS=false`. Now only the reasons are, + unless you turn `LOG_LOOKUPS` on. + - Self-hosted server: when your model or transcription key would go over plain `http://` to another machine (not this computer, not Tailscale), the server now warns at start, because anyone on the network in between could read it. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 85ef769..7cb1b7a 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -229,7 +229,9 @@ How much the server logs: `debug`, `info`, `warning` or `error` (`warn` works to ### `LOG_LOOKUPS` `true` to log what you look up and the model's answer, at debug level (so also set -`LOG_LEVEL=debug`). For troubleshooting a model. +`LOG_LEVEL=debug`). For troubleshooting a model. Without it, even `LOG_LEVEL=debug` logs +no words: not what you typed, not the words the model answered or the server refused +(only the reasons), not the words a database upgrade flags. - Default: `false`. Allowed: `true`, `yes`, `on`, `1`, `false`, `no`, `off`, `0`. - Wrong: anything else stops the server. diff --git a/server/lib/kotiko/llm.ex b/server/lib/kotiko/llm.ex index 82d266d..f7e1c3f 100644 --- a/server/lib/kotiko/llm.ex +++ b/server/lib/kotiko/llm.ex @@ -500,8 +500,11 @@ defmodule Kotiko.LLM do defp put_last_result(code), do: :persistent_term.put({__MODULE__, :last_result}, code) defp last_result, do: :persistent_term.get({__MODULE__, :last_result}, nil) - # User text only with LOG_LOOKUPS=true, and then only at debug. - defp log_lookups?, do: Application.get_env(:kotiko, :log_lookups, false) + @doc """ + Whether the learner's words may be logged: only with LOG_LOOKUPS=true, and then only at + debug level. Every debug line that carries a word, a gloss or typed text checks this. + """ + def log_lookups?, do: Application.get_env(:kotiko, :log_lookups, false) defp log_answer(content) do if log_lookups?(), do: Logger.debug("llm answer: #{String.slice(content, 0, 2000)}") diff --git a/server/lib/kotiko/migrations/language_tags.ex b/server/lib/kotiko/migrations/language_tags.ex index 2d7761a..53d6e76 100644 --- a/server/lib/kotiko/migrations/language_tags.ex +++ b/server/lib/kotiko/migrations/language_tags.ex @@ -233,8 +233,10 @@ defmodule Kotiko.Migrations.LanguageTags do "that language's script); they are unchanged. Check them in the dashboard." ) - # The words themselves are the learner's: only at debug. - Logger.debug("Words that may have the wrong language: #{inspect(stats.suspect_words)}") + # The words themselves are the learner's: only at debug, with LOG_LOOKUPS (B-07). + if Kotiko.LLM.log_lookups?(), + do: + Logger.debug("Words that may have the wrong language: #{inspect(stats.suspect_words)}") end end end diff --git a/server/lib/kotiko/router.ex b/server/lib/kotiko/router.ex index 9c8ab27..8dba582 100644 --- a/server/lib/kotiko/router.ex +++ b/server/lib/kotiko/router.ex @@ -131,8 +131,8 @@ defmodule Kotiko.Router do {200, Jason.encode!(%{words: [], reply: reply || "I couldn't find a word in that."})} {:ok, %{words: [], rejected: rejected}} -> - # The rejected words are the learner's: only at debug. - Logger.debug("Couldn't save #{inspect(rejected)}") + # The rejected words are the learner's: only at debug, with LOG_LOOKUPS (B-07). + if Kotiko.LLM.log_lookups?(), do: Logger.debug("Couldn't save #{inspect(rejected)}") reasons = rejected |> Enum.map(& &1.reason) |> Enum.uniq() Logger.warning("Couldn't save a word: #{Enum.join(reasons, ", ")}") {422, Jason.encode!(%{error: "Couldn't save that word: #{Enum.join(reasons, ", ")}"})} diff --git a/server/lib/kotiko/router_v1.ex b/server/lib/kotiko/router_v1.ex index 1f36385..63308f6 100644 --- a/server/lib/kotiko/router_v1.ex +++ b/server/lib/kotiko/router_v1.ex @@ -257,14 +257,21 @@ defmodule Kotiko.RouterV1 do [if(p, do: Map.put(base, :previous, Word.to_api(p)), else: base)] end - # The reason can hold the word itself (a changeset): only its shape at warning level - # (slice 10 section 6), the whole of it at debug. defp result_json({:error, reason}) do - Logger.warning("Couldn't save a word: #{save_error(reason)}") - Logger.debug("Couldn't save a word: #{inspect(reason)}") + log_save_error(reason) [] end + @doc false + # The reason can hold the word itself (a changeset): only its shape at warning level + # (slice 10 section 6), the whole of it at debug with LOG_LOOKUPS (slice 54, B-07). + def log_save_error(reason) do + Logger.warning("Couldn't save a word: #{save_error(reason)}") + + if Kotiko.LLM.log_lookups?(), + do: Logger.debug("Couldn't save a word: #{inspect(reason)}") + end + defp save_error(%Ecto.Changeset{errors: errors}), do: "invalid " <> Enum.map_join(errors, ", ", fn {field, _} -> to_string(field) end) diff --git a/server/test/kotiko/migration_language_tags_test.exs b/server/test/kotiko/migration_language_tags_test.exs index 45ccccf..6a685b3 100644 --- a/server/test/kotiko/migration_language_tags_test.exs +++ b/server/test/kotiko/migration_language_tags_test.exs @@ -73,6 +73,28 @@ defmodule Kotiko.MigrationLanguageTagsTest do assert Word.to_legacy_json(w[4]).language == "Hebrew" end + # B-07 (slice 54): the suspect words were logged at debug even without LOG_LOOKUPS. + test "at debug, the suspect words are logged only with LOG_LOOKUPS", ctx do + Logger.configure(level: :debug) + Application.put_env(:kotiko, :log_lookups, false) + on_exit(fn -> Application.put_env(:kotiko, :log_lookups, false) end) + + {_backup, log} = upgrade(ctx) + assert log =~ "2 word(s) may have the wrong language" + refute log =~ "spasibo" + refute log =~ "Qapla" + end + + test "with LOG_LOOKUPS the suspect words are logged at debug", ctx do + Logger.configure(level: :debug) + Application.put_env(:kotiko, :log_lookups, true) + on_exit(fn -> Application.put_env(:kotiko, :log_lookups, false) end) + + {_backup, log} = upgrade(ctx) + assert log =~ "Words that may have the wrong language" + assert log =~ "spasibo" + end + test "changed rows get a new seq and a later updated_at; unchanged rows keep theirs", ctx do upgrade(ctx) w = words(ctx.db) diff --git a/server/test/kotiko/router_v1_test.exs b/server/test/kotiko/router_v1_test.exs index afe2c08..10981d4 100644 --- a/server/test/kotiko/router_v1_test.exs +++ b/server/test/kotiko/router_v1_test.exs @@ -820,6 +820,72 @@ defmodule Kotiko.RouterV1Test do end end + # B-07 (slice 54): at LOG_LEVEL=debug the model's rejected words (native, gloss) were + # logged even with LOG_LOOKUPS=false. + describe "rejected words at debug level" do + setup do + Logger.put_module_level(Kotiko.Router, :debug) + Logger.put_module_level(Kotiko.RouterV1, :debug) + Logger.put_module_level(Kotiko.LLM, :debug) + + on_exit(fn -> + for m <- [Kotiko.Router, Kotiko.RouterV1, Kotiko.LLM], do: Logger.delete_module_level(m) + end) + + stub_answer([ + %{ + "lang" => "ru", + "native" => "PRIVATEWORDXYZ", + "base_lang" => "en", + "gloss" => "MYSECRETGLOSS", + "forms" => ["MYSECRETGLOSS"] + } + ]) + end + + test "only their reasons are logged without LOG_LOOKUPS" do + put_app_env(:log_lookups, false) + + log = + capture_log([level: :debug], fn -> + assert {422, _} = call("POST", "/api/words", %{text: "mytypedprivatetext"}) + end) + + assert log =~ "script_mismatch" + refute log =~ "PRIVATEWORDXYZ" + refute log =~ "MYSECRETGLOSS" + refute log =~ "mytypedprivatetext" + end + + test "with LOG_LOOKUPS=true they are logged at debug" do + put_app_env(:log_lookups, true) + + log = + capture_log([level: :debug], fn -> + assert {422, _} = call("POST", "/api/words", %{text: "mytypedprivatetext"}) + end) + + assert log =~ "Couldn't save [%{" + assert log =~ "PRIVATEWORDXYZ" + end + + test "a word that couldn't be saved: its shape without LOG_LOOKUPS, all of it with" do + changeset = + %Kotiko.Word{} + |> Ecto.Changeset.change(native: "PRIVATEWORDXYZ") + |> Ecto.Changeset.add_error(:native, "taken") + + put_app_env(:log_lookups, false) + log = capture_log([level: :debug], fn -> Kotiko.RouterV1.log_save_error(changeset) end) + assert log =~ "Couldn't save a word: invalid native" + refute log =~ "PRIVATEWORDXYZ" + + put_app_env(:log_lookups, true) + log = capture_log([level: :debug], fn -> Kotiko.RouterV1.log_save_error(changeset) end) + assert log =~ "PRIVATEWORDXYZ" + end + end + describe "legacy POST /api/words" do test "a failed lookup keeps the string error 0.2 reads, plus slice 25's code" do LLMStub.stub(fn _, conn -> LLMStub.rate_limited(conn) end) From b1b75a1e2d1144ab0df1219e6dd7e4019633b569 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Tue, 6 Oct 2026 21:49:59 -0400 Subject: [PATCH 08/12] fix(server): refuse a data folder someone else could have planted files in Someone who could write in the data folder could choose the API token (the server used whatever api-token it found, whoever owned it) and could put a kotiko.db link there that sent every word to a world-readable file of their choosing, with no warning, because the permission check skips links (slice 54, B-02). At every start, before anything in the folder is read or written, Kotiko.DataDir.check_safe/2 now stops the server (status 78) with a message naming each problem and how to fix it when: the folder or one of Kotiko's files belongs to another uid; kotiko.db, its -wal/-shm/-journal, api-token, the model cache, an old slovo.db, backups/ or a backup is a symbolic link (Kotiko never makes links, and judging where one points would mean trusting it, so none is accepted; KOTIKO_DATA_DIR itself may still be a link); or others can write in a folder that also holds other files. A folder others can write in that holds only Kotiko's files is made 0700 first, with a warning, so nobody can swap a file afterwards. api-token is read with Kotiko.Private.read_own/3: lstat, open, fstat and compare device and inode, which stands in for O_NOFOLLOW (the BEAM has none), and only a file of the server's own uid. The uid comes from /proc/self, or a probe file; no command runs. Regression tests: reviewer B's planted folder (0777, a planted api-token, kotiko.db linked to a 0666 file elsewhere, another file) now stops the boot with status 78 and nothing is written through the link (before, the server started with the planted token and wrote the words there); a linked api-token is refused by Token.resolve (before, it was followed). The docs no longer say others with a shared folder see only file names. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Ayla Croft --- CHANGELOG.md | 9 ++ docs/ARCHITECTURE.md | 12 ++- docs/reference/configuration.md | 21 ++++- docs/security/assurance-case.md | 8 +- docs/security/requirements.md | 12 ++- server/lib/kotiko/data_dir.ex | 133 +++++++++++++++++++++++++++ server/lib/kotiko/private.ex | 89 ++++++++++++++++++ server/lib/kotiko/token.ex | 25 ++++- server/test/kotiko/boot_test.exs | 27 ++++++ server/test/kotiko/data_dir_test.exs | 81 ++++++++++++++++ server/test/kotiko/private_test.exs | 31 +++++++ server/test/kotiko/token_test.exs | 13 +++ 12 files changed, 446 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 89c00fa..d3f6ad1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ Conventional Commits by release-please. ## Unreleased +- Self-hosted server: someone who could write in your data folder could choose your access + key (by leaving an `api-token` there) or send your words to a file of theirs (by putting + a link where `kotiko.db` goes). The server now refuses to start, saying what it found + and how to fix it, when the data folder or one of Kotiko's files belongs to another + account, when one of Kotiko's files is a link, or when others can write in a data folder + that also holds other files. A folder others can write in that holds only Kotiko's files + is made private first, with a warning. To keep your words on another disk, point + `KOTIKO_DATA_DIR` there instead of linking `kotiko.db`. + - Self-hosted server: with `LOG_LEVEL=debug`, words the model suggested and the server refused were written to the log even with `LOG_LOOKUPS=false`. Now only the reasons are, unless you turn `LOG_LOOKUPS` on. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index e41e56c..fd2adaa 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -120,10 +120,15 @@ order: the log filter that hides secrets, settings, the data folder, the API tok database migrations (with a backup first), a startup summary, then the supervision tree. - **HTTP**: Bandit serves [`Kotiko.Router`](../server/lib/kotiko/router.ex), a Plug - router. Every request passes `Kotiko.Plug.HostCheck` + router, on the address in `BIND` and, for a loopback one, on the other loopback address + too ([`Kotiko.Listener`](../server/lib/kotiko/listener.ex)), so no other account can + listen on `[::1]` where browsers send `localhost`. Every request passes + `Kotiko.Plug.HostCheck` ([`plug/host_check.ex`](../server/lib/kotiko/plug/host_check.ex), refuses unknown host names against DNS rebinding), then the token check (deny by default; only `GET` and - `HEAD /health` are open), then body parsing (64 KB, 1 MB for the batch route). + `HEAD /health` and `POST /api/v1/proof` are open; an address that sent 10 wrong tokens + in a minute gets `429`, [`Kotiko.AuthThrottle`](../server/lib/kotiko/auth_throttle.ex)), + then body parsing (64 KB, 1 MB for the batch route, 1 KB for the proof). `/api/v1` is forwarded to [`Kotiko.RouterV1`](../server/lib/kotiko/router_v1.ex). The routes are listed in [`reference/http-api.md`](reference/http-api.md). - **Settings**: [`Kotiko.Config`](../server/lib/kotiko/config.ex) parses the environment @@ -245,7 +250,8 @@ Kotiko treats these as untrusted and checks what crosses them: HTTP only if you set `BIND` to a network address, with a warning at startup); - the model's answers, which are checked against the spec before anything is saved; - Telegram updates (allowlisted user IDs only); -- the files in the data folder, protected by your operating system's file permissions. +- the files in the data folder, protected by your operating system's file permissions + (the server refuses to start when another account could have put files there). The full threat model, every input and how it is checked, and the evidence are in the [assurance case](security/assurance-case.md). diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 7cb1b7a..8eb22f3 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -328,8 +328,21 @@ check. | `models-cache.json` | OpenRouter's model list as last read, used when the server starts offline | `0600` | | `backups/kotiko-pre--