From 17a0c8ceab93b894f0a2c34610623d9515273803 Mon Sep 17 00:00:00 2001 From: rcardin Date: Thu, 22 Oct 2026 16:59:20 +0200 Subject: [PATCH 1/2] fix(db): databaseSchema alone puts the tables in that schema An application that named a schema through databaseSchema but did not also write a databaseInit hook split in two: the drift commands read the named schema while the DDL and the migration ledger landed in public, so sync applied cleanly and status reported the whole model missing forever. databaseSchema now lives in DbInit, where the Database is built, and the pool runs one hook composed of the schema step and the application's own databaseInit. The schema step reads pg_namespace, creates the schema only when it is missing, so a role without CREATE on the database still connects, and sets search_path to the quoted name. The public default runs nothing. Composition, not a default that overriders chain with super, keeps the split from coming back for an application that overrides databaseInit for something else. The examples, the Tour and the docs drop the duplicated hook and stop saying both halves are needed. Closes #315 Co-Authored-By: Claude Fable 5.1 --- docs/how-to/configure-the-database.md | 35 +++-- docs/reference/configuration.md | 15 ++- examples/blog/README.md | 4 +- examples/blog/src/main/scala/CreateUser.scala | 10 -- examples/blog/src/main/scala/Main.scala | 16 +-- .../src/test/scala/PostOwnershipSuite.scala | 18 ++- examples/reminders/README.md | 6 +- examples/reminders/src/main/scala/Main.scala | 15 +-- examples/shop/src/main/scala/CreateUser.scala | 10 -- examples/shop/src/main/scala/Main.scala | 16 +-- examples/shop/src/test/scala/ApiSuite.scala | 19 ++- examples/todo/README.md | 55 ++++---- examples/todo/src/main/scala/Main.scala | 16 +-- .../db/src/main/scala/io/eezo/db/DbApp.scala | 5 - .../db/src/main/scala/io/eezo/db/DbInit.scala | 88 ++++++++++-- .../main/scala/io/eezo/db/schema/Ddl.scala | 6 +- .../test/scala/io/eezo/db/DbAppSuite.scala | 92 ++++++++++++- .../test/scala/io/eezo/db/DbInitSuite.scala | 126 +++++++++++++++++- .../test/scala/io/eezo/db/support/Pg.scala | 5 +- .../example/src/test/scala/example/Tour.scala | 22 +-- 20 files changed, 400 insertions(+), 179 deletions(-) diff --git a/docs/how-to/configure-the-database.md b/docs/how-to/configure-the-database.md index f2d8d1cf..5b1cd838 100644 --- a/docs/how-to/configure-the-database.md +++ b/docs/how-to/configure-the-database.md @@ -46,8 +46,28 @@ The pool starts empty and grows to the size as connections are needed. ## One database, several applications Each application can keep its tables in a Postgres schema of its own, so two applications can -share the dev database without their drift bleeding into each other. Two overrides on `Main`, -told once each: +share the dev database without their drift bleeding into each other. Name the schema on `Main`: + +```scala +object Main extends EezoApp { + + override def databaseSchema: String = "bookshelf" + // ... +} +``` + +That one override is enough. Every connection the pool opens creates the schema when it is +missing and sets `search_path` to it, because `search_path` is per connection, so the tables +`sync` creates, the migration ledger and your own queries all land in `bookshelf`. The schema +commands read the catalog under the same name, so `status` compares your model against the right +tables. The examples under `examples/` each do this. + +The schema is created only when the catalog says it is missing, so a production role that owns an +existing schema but may not create one on the database still connects. `search_path` is the schema +alone: objects in `public`, such as an extension's types and functions, are not on it. + +`databaseInit` is for any other per connection setting, such as `application_name` or a statement +timeout. It runs after the schema step on every pooled connection and needs no `super` call: ```scala import java.sql.Connection @@ -58,20 +78,13 @@ object Main extends EezoApp { override def databaseInit: Connection => Unit = { c => val st = c.createStatement() - try { - st.execute("""create schema if not exists "bookshelf"""") - st.execute("""set search_path to "bookshelf"""") - } finally st.close() + try st.execute("set application_name to 'bookshelf'") + finally st.close() } // ... } ``` -`databaseInit` runs on every pooled connection, because `search_path` is per connection. -`databaseSchema` points the schema commands' catalog reads at the same name, so `status` compares -your model against the right tables. Both halves are needed; the examples under `examples/` each -do this. - ## In tests Point a second `DbApp` at a throwaway Postgres, such as one from testcontainers, by overriding diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 1441b099..95333327 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -49,19 +49,22 @@ overridden `boot` that still wants to serve calls it. |---|---|---| | `schema: Schema` | abstract | per command | | `boot(): Unit` | abstract on `DbApp`; `EezoApp` supplies the http layer's | the program | -| `databaseSchema: String` | `"public"` | by the commands that introspect | +| `databaseSchema: String` | `"public"` | on every physical connection the pool opens, and by the commands that introspect | | `databaseUrl: String` | from the environment, above | when a `Database` is built | | `databaseUser: String` | from the environment | same | | `databasePassword: String` | from the environment | same | | `databasePoolSize: Int` | from the environment, 10 | same | | `databaseAcquireTimeout: Duration` | from the environment, 5 s | same | -| `databaseInit: Connection -> Unit` | nothing | on every physical connection the pool opens | +| `databaseInit: Connection -> Unit` | nothing | on every physical connection the pool opens, after the schema step | A `Database` is built per command and per program run, installed around it, and closed after. -`databaseInit` is where `search_path`, `application_name` and statement timeouts go. An -application that keeps its tables in a schema other than `public` sets both `databaseSchema` -and a `databaseInit` that sets `search_path`, because the first is what the drift commands -read and the second is what every query runs under. +`databaseSchema` is where the application's tables live and what the drift commands read. Any +name but `public` makes every physical connection create that schema when the catalog says it is +missing and set `search_path` to it alone, so the DDL, the migration ledger and every query land +where `status` looks; `public` runs nothing and leaves Postgres's own `search_path`. The name is +quoted as an identifier, so it is used exactly as written. `databaseInit` is where +`application_name`, statement timeouts and any other per connection setting go; it runs after the +schema step and needs no `super` call to keep it. ## `LiveApp` diff --git a/examples/blog/README.md b/examples/blog/README.md index 4f25db31..77f5815c 100644 --- a/examples/blog/README.md +++ b/examples/blog/README.md @@ -18,8 +18,8 @@ where they came from and wraps only the derived ones in `Route.under("/admin")`, routes and the URLs the pages emit together; `app/Index.scala` explains the one link that has to name `/admin` by hand. -The blog keeps its table in a Postgres schema named `blog` (`databaseSchema` plus the `databaseInit` -hook), so it shares the dev database with the other examples. +The blog keeps its table in a Postgres schema named `blog` (`databaseSchema` in `Main.scala`, the +one override that does it), so it shares the dev database with the other examples. ```bash docker run -d --name eezo-pg -p 5442:5432 \ diff --git a/examples/blog/src/main/scala/CreateUser.scala b/examples/blog/src/main/scala/CreateUser.scala index 12d33052..b6dedaa1 100644 --- a/examples/blog/src/main/scala/CreateUser.scala +++ b/examples/blog/src/main/scala/CreateUser.scala @@ -1,5 +1,3 @@ -import java.sql.Connection - import io.eezo.auth.Password import io.eezo.core.Id import io.eezo.db.* @@ -34,14 +32,6 @@ object CreateUser extends DbApp { override def databaseSchema: String = "blog" - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "blog"""") - st.execute("""set search_path to "blog"""") - } finally st.close() - } - override def boot(): Unit = { val email = required("EEZO_BLOG_EMAIL") val password = required(PasswordVariable) diff --git a/examples/blog/src/main/scala/Main.scala b/examples/blog/src/main/scala/Main.scala index 2b90bcc3..9ab3e09e 100644 --- a/examples/blog/src/main/scala/Main.scala +++ b/examples/blog/src/main/scala/Main.scala @@ -1,5 +1,3 @@ -import java.sql.Connection - import io.eezo.EezoApp import io.eezo.db.Schema import io.eezo.generated.Routes @@ -45,20 +43,12 @@ object Main extends EezoApp { override def schema: Schema = AppSchema /** This app keeps its table in a Postgres schema of its own, so it can share the dev database - * with the other examples without their drift bleeding into each other. Two halves, told once - * each: the init hook runs on **every** pooled connection (search_path is per connection), and - * `databaseSchema` points the drift commands' catalog reads at the same name. + * with the other examples without their drift bleeding into each other. Naming it is enough: + * eezo creates the schema and sets search_path to it on **every** pooled connection, because + * search_path is per connection, and the drift commands read the same name. */ override def databaseSchema: String = "blog" - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "blog"""") - st.execute("""set search_path to "blog"""") - } finally st.close() - } - override def routes: RouteTable = { val table = Routes.table() val (derived, handwritten) = table.routes.partition(_.provenance == Provenance.Derived) diff --git a/examples/blog/src/test/scala/PostOwnershipSuite.scala b/examples/blog/src/test/scala/PostOwnershipSuite.scala index e5d390f1..937da796 100644 --- a/examples/blog/src/test/scala/PostOwnershipSuite.scala +++ b/examples/blog/src/test/scala/PostOwnershipSuite.scala @@ -1,6 +1,5 @@ import java.net.URLEncoder import java.nio.charset.StandardCharsets -import java.sql.Connection import java.util.UUID import io.eezo.core.Id @@ -94,17 +93,16 @@ class PostOwnershipSuite extends munit.FunSuite { * * A second `DbApp` for the same reason `CreateUser` is one: `withDatabase` is `protected`, so * installing a `Database` is something only a subclass can ask for, and a test is a front end - * like any other. The schema, the per connection hook and `AppSchema` are `Main`'s own values - * rather than copies, so a change to where the blog keeps its tables moves this with it. + * like any other. The schema and `AppSchema` are `Main`'s own values rather than copies, so a + * change to where the blog keeps its tables moves this with it. */ private object BlogDb extends DbApp { - override def schema: Schema = Main.schema - override def databaseUrl: String = postgres.getJdbcUrl - override def databaseUser: String = postgres.getUsername - override def databasePassword: String = postgres.getPassword - override def databaseSchema: String = Main.databaseSchema - override def databaseInit: Connection => Unit = Main.databaseInit - override def boot(): Unit = () + override def schema: Schema = Main.schema + override def databaseUrl: String = postgres.getJdbcUrl + override def databaseUser: String = postgres.getUsername + override def databasePassword: String = postgres.getPassword + override def databaseSchema: String = Main.databaseSchema + override def boot(): Unit = () /** The one thing this object exists to publish: a `Database` installed around a block, which is * what `sbt run` does around `boot` and what every handler below reaches through. diff --git a/examples/reminders/README.md b/examples/reminders/README.md index 2f606fb6..d5f79f3a 100644 --- a/examples/reminders/README.md +++ b/examples/reminders/README.md @@ -12,9 +12,9 @@ deliver every reminder that is due and not yet sent, and mark it sent, in one `t is a line on stdout. `DbApp` brings `main`, the `Database` it installs around `boot`, and the eight schema commands. There is no `EezoPlugin` on this project, because there are no routes to generate. -The job keeps its table in a Postgres schema named `reminders` (`databaseSchema` plus the -`databaseInit` hook; `Main.scala` says why both halves exist), so it shares the dev database with -the other examples without their drift bleeding into each other. +The job keeps its table in a Postgres schema named `reminders` (`databaseSchema` in `Main.scala`, +the one override that does it), so it shares the dev database with the other examples without +their drift bleeding into each other. ```bash docker run -d --name eezo-pg -p 5442:5432 \ diff --git a/examples/reminders/src/main/scala/Main.scala b/examples/reminders/src/main/scala/Main.scala index 027d450b..6c080ed9 100644 --- a/examples/reminders/src/main/scala/Main.scala +++ b/examples/reminders/src/main/scala/Main.scala @@ -1,4 +1,3 @@ -import java.sql.Connection import java.time.LocalDate import io.eezo.db.* @@ -25,20 +24,12 @@ object Main extends DbApp { override def schema: Schema = AppSchema /** This app keeps its table in a Postgres schema of its own, so it can share the dev database - * with the other examples without their drift bleeding into each other. Two halves, told once - * each: the init hook runs on **every** pooled connection (search_path is per connection), and - * `databaseSchema` points the drift commands' catalog reads at the same name. + * with the other examples without their drift bleeding into each other. Naming it is enough: + * eezo creates the schema and sets search_path to it on **every** pooled connection, because + * search_path is per connection, and the drift commands read the same name. */ override def databaseSchema: String = "reminders" - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "reminders"""") - st.execute("""set search_path to "reminders"""") - } finally st.close() - } - override def boot(): Unit = { val reminders = Table[Reminder] val today = LocalDate.now() diff --git a/examples/shop/src/main/scala/CreateUser.scala b/examples/shop/src/main/scala/CreateUser.scala index 5921f263..b33a329b 100644 --- a/examples/shop/src/main/scala/CreateUser.scala +++ b/examples/shop/src/main/scala/CreateUser.scala @@ -1,5 +1,3 @@ -import java.sql.Connection - import io.eezo.auth.Password import io.eezo.core.Id import io.eezo.db.* @@ -15,14 +13,6 @@ object CreateUser extends DbApp { /** Same Postgres schema as `Main`, so the account lands where the app reads it. */ override def databaseSchema: String = "shop" - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "shop"""") - st.execute("""set search_path to "shop"""") - } finally st.close() - } - override def boot(): Unit = { val email = sys.env("SHOP_EMAIL") val hashed = Field[Password] diff --git a/examples/shop/src/main/scala/Main.scala b/examples/shop/src/main/scala/Main.scala index 301561c9..88055c5c 100644 --- a/examples/shop/src/main/scala/Main.scala +++ b/examples/shop/src/main/scala/Main.scala @@ -1,5 +1,3 @@ -import java.sql.Connection - import io.eezo.EezoApp import io.eezo.db.Schema import io.eezo.generated.Routes @@ -12,17 +10,9 @@ object Main extends EezoApp { override def layout: Layout = views.Layout /** The shop keeps its tables in a Postgres schema of its own, so it shares the dev database with - * the other examples without their drift bleeding into each other. The init hook runs on every - * pooled connection, because search_path is per connection, and `databaseSchema` points the - * drift commands at the same name. + * the other examples without their drift bleeding into each other. Naming it is enough: eezo + * creates the schema and sets search_path to it on every pooled connection, and the drift + * commands read the same name. */ override def databaseSchema: String = "shop" - - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "shop"""") - st.execute("""set search_path to "shop"""") - } finally st.close() - } } diff --git a/examples/shop/src/test/scala/ApiSuite.scala b/examples/shop/src/test/scala/ApiSuite.scala index 1129f3be..1399fb18 100644 --- a/examples/shop/src/test/scala/ApiSuite.scala +++ b/examples/shop/src/test/scala/ApiSuite.scala @@ -1,6 +1,5 @@ import java.net.URLEncoder import java.nio.charset.StandardCharsets -import java.sql.Connection import io.eezo.auth.{Password, Token} import io.eezo.core.Id @@ -49,18 +48,16 @@ class ApiSuite extends munit.FunSuite { started } - /** The shop's own database settings pointed at the container. `Main`'s schema name and per - * connection hook rather than copies, so the routes read the `shop` search path they read in - * production. + /** The shop's own database settings pointed at the container. `Main`'s schema name rather than a + * copy, so the routes read the `shop` search path they read in production. */ private object ShopDb extends DbApp { - override def schema: Schema = Main.schema - override def databaseUrl: String = postgres.getJdbcUrl - override def databaseUser: String = postgres.getUsername - override def databasePassword: String = postgres.getPassword - override def databaseSchema: String = Main.databaseSchema - override def databaseInit: Connection => Unit = Main.databaseInit - override def boot(): Unit = () + override def schema: Schema = Main.schema + override def databaseUrl: String = postgres.getJdbcUrl + override def databaseUser: String = postgres.getUsername + override def databasePassword: String = postgres.getPassword + override def databaseSchema: String = Main.databaseSchema + override def boot(): Unit = () def around[A](body: => A): A = withDatabase(body) } diff --git a/examples/todo/README.md b/examples/todo/README.md index 38048528..689570d3 100644 --- a/examples/todo/README.md +++ b/examples/todo/README.md @@ -1,4 +1,4 @@ -# The todo app — a guided tour of the eezo CLI +# The todo app: a guided tour of the eezo CLI | | | |---|---| @@ -11,7 +11,7 @@ routes, and a Postgres schema of its own. Everything the CLI currently does can this directory. Every output block below was captured from a real run. The Store↔Table integration landed (`JdbcStore`). A model -deriving `Table` — like `Todo` — now gets derived CRUD backed by Postgres, so todos you create in +deriving `Table`, like `Todo`, now gets derived CRUD backed by Postgres, so todos you create in the browser survive restarts, and the migrate-before-serve order in this guide is load-bearing: browse `/todos` before the table exists and you'll meet a real database error. @@ -38,24 +38,24 @@ sbt The app serves on **8090** (`Main.scala` overrides `port` so the tour never collides with whatever your machine runs on 8080) and keeps its tables in a Postgres schema named `todo` -(`databaseSchema` + the `databaseInit` hook — see `Main.scala` for why both halves exist). +(`databaseSchema` in `Main.scala`, the one override that does it). ## 1. The code -- `src/main/scala/models/Todo.scala` — the model. `derives Table, Form, Resource` is the whole +- `src/main/scala/models/Todo.scala`: the model. `derives Table, Form, Resource` is the whole declaration: table, HTML form, seven CRUD routes. -- `src/main/scala/AppSchema.scala` — registers the table and adds what types alone cannot say: +- `src/main/scala/AppSchema.scala`: registers the table and adds what types alone cannot say: `table[Todo].index(_.done).unique(_.title)`. The selectors are typed; rename a field and the index breaks at compile time. -- `src/main/scala/app/Index.scala` — `GET /` by filename convention. A handler is a plain +- `src/main/scala/app/Index.scala`: `GET /` by filename convention. A handler is a plain `Request => Response` on a virtual thread. -- `src/main/scala/app/Health.scala` — a custom name mounts a GET at its own segment: +- `src/main/scala/app/Health.scala`: a custom name mounts a GET at its own segment: `GET /health`. -- `src/main/scala/Main.scala` — the entry point: name `schema` and `routes`, done. `main` is +- `src/main/scala/Main.scala`: the entry point; name `schema` and `routes`, done. `main` is inherited from `EezoApp` and dispatches. `EezoApp` is the umbrella's trait, both layers stacked; an application with one layer extends that layer's trait instead (`../hello`, `../reminders`). -## 2. The route table — no database needed +## 2. The route table, no database needed ``` sbt:examples> todo/run routes @@ -75,7 +75,7 @@ sbt:examples> todo/run routes ``` Handwritten routes first (most-static-first, so `/todos/new` beats `/todos/:id`), then the seven -derived ones. This command never opens a database connection — stop Postgres and it still answers. +derived ones. This command never opens a database connection: stop Postgres and it still answers. `todo/eezoRoutes` is the same thing as an sbt task. ## 3. Status → freeze → migrate: the reviewed path @@ -94,9 +94,9 @@ sbt:examples> todo/run db status + index uq_todo_title on todo ``` -(The task "fails" — drift is exit code 1 on purpose, so CI can gate on it.) +(The task "fails": drift is exit code 1 on purpose, so CI can gate on it.) -Freeze it as the first migration. Everything after `freeze` that is not a flag is the name — no +Freeze it as the first migration. Everything after `freeze` that is not a flag is the name, no quoting needed: ``` @@ -113,7 +113,7 @@ sbt:examples> todo/run db freeze initial schema wrote db/migrations/0001_initial_schema.sql ``` -Open `todo/db/migrations/0001_initial_schema.sql` — committed SQL with a fingerprint header (edit +Open `todo/db/migrations/0001_initial_schema.sql`: committed SQL with a fingerprint header (edit it by hand and `migrate` will refuse, telling you to change the model and re-freeze instead). `todo/db/schema.json` is the snapshot the *next* freeze diffs against. @@ -158,7 +158,7 @@ sbt:examples> todo/run ``` Open . Click through: the homepage (try `/?name=you`), `/health`, and -`/todos` — create a couple of todos, edit one, delete one. That whole CRUD flow is derived from +`/todos`: create a couple of todos, edit one, delete one. That whole CRUD flow is derived from the case class; the form's optional `notes` and the `done` checkbox come from `Option[String]` and `Boolean`. Ctrl+C (or `enter` in the sbt shell) stops it. @@ -172,9 +172,9 @@ sbt:todo> eezoDev First run compiles and starts the app (`eezo dev: started Main`), with the drift check and route listing on. Now, **without stopping anything**: -1. Edit `app/Index.scala` — change the `h1` text — and save. +1. Edit `app/Index.scala`, change the `h1` text, and save. 2. Watch the console: recompile, restart. -3. Refresh — your change is live. Budget is well under a second of +3. Refresh : your change is live. Budget is well under a second of compile plus ~a quarter second of JVM boot. Break the file on purpose (delete a `)`) and save: the compile fails, **the old server keeps @@ -190,7 +190,7 @@ With `eezoDev` still running, add a field to `Todo`: priority: Option[Int] ``` -Save. The restarted server prints a warning but serves — a column the database lacks breaks +Save. The restarted server prints a warning but serves: a column the database lacks breaks nothing until code touches it: ``` @@ -214,14 +214,13 @@ refuses to route: every path on answers **503** with the - two buttons: **apply to the dev database** (sync, no migration written) or **freeze as a migration and apply it** (name field + your decisions → `000N_*.sql` + applied) -Pick either. The page re-checks on every refresh, so it flips to *"resolved ✓ — save any file to -restart"* — save a file (or press `enter` and rerun `eezoDev`) and the app serves again. If an -action fails, the database's own error renders on the page — for example, adding a non-null -column over existing rows is a `[risky]` change that Postgres may refuse; that refusal is the -flag doing its job. +Pick either. The page re-checks on every refresh, so it flips to *"resolved ✓"*; save a file (or +press `enter` and rerun `eezoDev`) and the app serves again. If an action fails, the database's +own error renders on the page: for example, adding a non-null column over existing rows is a +`[risky]` change that Postgres may refuse; that refusal is the flag doing its job. The decisions are on the page rather than in the terminal because under `eezoDev` the forked app -shares stdin with sbt's watch — a console prompt would race it for every keystroke. +shares stdin with sbt's watch, and a console prompt would race it for every keystroke. ## 8. Start over @@ -253,8 +252,8 @@ drop + sync when you don't care about migrations.) `eezo new myapp` scaffolds one against the locally published eezo, and inside it `eezo routes`, `eezo db status --json`, `eezo dev` forward to the same dispatch these `todo/run ...` commands hit. -**For agents and scripts:** every command takes `--json` — same exit codes, machine-readable -body, each change carrying its `sql` alongside the `destructive`/`risky` classification. Try -`todo/run db status --json`. `freeze` swaps its prompt for a policy flag: `--accept-all` or -`--skip-destructive` (bare `--json` implies the latter, so a machine caller never hangs on a -prompt). +**For agents and scripts:** every command takes `--json`, with the same exit codes and a +machine-readable body, each change carrying its `sql` alongside the `destructive`/`risky` +classification. Try `todo/run db status --json`. `freeze` swaps its prompt for a policy flag: +`--accept-all` or `--skip-destructive` (bare `--json` implies the latter, so a machine caller +never hangs on a prompt). diff --git a/examples/todo/src/main/scala/Main.scala b/examples/todo/src/main/scala/Main.scala index 3a70967e..f6ea5b0e 100644 --- a/examples/todo/src/main/scala/Main.scala +++ b/examples/todo/src/main/scala/Main.scala @@ -1,5 +1,3 @@ -import java.sql.Connection - import io.eezo.EezoApp import io.eezo.generated.Routes import io.eezo.db.Schema @@ -21,17 +19,9 @@ object Main extends EezoApp { override def port: Int = 8090 /** This app keeps its tables in a Postgres schema of its own, so it can share the dev database - * with other examples without their drift bleeding into each other. Two halves, told once each: - * the init hook runs on **every** pooled connection (search_path is per-connection), and - * `databaseSchema` points the drift commands' catalog reads at the same name. + * with other examples without their drift bleeding into each other. Naming it is enough: eezo + * creates the schema and sets search_path to it on **every** pooled connection, because + * search_path is per connection, and the drift commands read the same name. */ override def databaseSchema: String = "todo" - - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try { - st.execute("""create schema if not exists "todo"""") - st.execute("""set search_path to "todo"""") - } finally st.close() - } } diff --git a/modules/db/src/main/scala/io/eezo/db/DbApp.scala b/modules/db/src/main/scala/io/eezo/db/DbApp.scala index a2e1f951..0d8e0657 100644 --- a/modules/db/src/main/scala/io/eezo/db/DbApp.scala +++ b/modules/db/src/main/scala/io/eezo/db/DbApp.scala @@ -48,11 +48,6 @@ trait DbApp extends Dispatch with DbInit { */ def schema: Schema - /** The Postgres schema the drift commands introspect. `search_path` is set per connection in - * [[databaseInit]]; this is the same name, told to the side that reads catalogs. - */ - def databaseSchema: String = "public" - /** The program: what `sbt run` does, once, with a `Database` installed. Abstract, because a * database only application is its program. The umbrella supplies the http layer's default, * which is to serve. diff --git a/modules/db/src/main/scala/io/eezo/db/DbInit.scala b/modules/db/src/main/scala/io/eezo/db/DbInit.scala index 617f0311..0cb46c11 100644 --- a/modules/db/src/main/scala/io/eezo/db/DbInit.scala +++ b/modules/db/src/main/scala/io/eezo/db/DbInit.scala @@ -1,20 +1,22 @@ package io.eezo.db import io.eezo.db.engine.Database +import io.eezo.db.schema.Ddl import java.net.{URI, URLDecoder} -import java.sql.Connection +import java.sql.{Connection, SQLException} import java.time.Duration -/** How an application names its database: the connection settings and the per connection hook. +/** How an application names its database: the connection settings, the Postgres schema it lives in, + * and the per connection hook. * * Split out of [[DbApp]] so that a front end which is not the application's entry point (a deploy * runner, the testkit) can build the same `Database` from the same settings without inheriting * `boot`. `DbApp` extends this, and the umbrella's `EezoApp` through it; users override members * here and never name the trait. * - * Named for [[databaseInit]], its one member that is not a string, because it is the one people - * get wrong; see its comment. + * Named for what happens on every connection the pool opens, because that is the part people get + * wrong: a setting made on one connection is missing from the next. See [[databaseSchema]]. * * **Two spellings of the environment, one mechanism.** `EEZO_DB_URL`/`EEZO_DB_USER`/ * `EEZO_DB_PASS` are eezo's own and always win. When they are absent and `DATABASE_URL` is present @@ -61,12 +63,30 @@ trait DbInit { def databaseAcquireTimeout: Duration = DbInit.acquireTimeout(sys.env.get("EEZO_DB_ACQUIRE_TIMEOUT")) - /** Run on **every** connection the pool creates, not once on a borrowed one (DESIGN §8.7). + /** The Postgres schema the application lives in: where its tables are created and where the drift + * commands read the catalog. Declared here, where the `Database` is built, because the name has + * to reach the connections as well as the catalog reads, and a front end that builds a + * `Database` through this trait must land in the same schema as the application. * - * This is where `search_path`, `application_name` and statement timeouts belong. Without it an - * application that works in a non-`public` schema silently splits in two: its own connections - * see one schema and eezo's see another, which shows up as a migration that appears not to have - * run. + * Naming it is enough. Any name but `public` makes every physical connection create the schema + * when it is missing and set `search_path` to it, before [[databaseInit]] runs, so the DDL the + * commands apply, the migration ledger and the application's own SQL all land where `status` + * looks. With the setup left to the application, a name set here without the matching hook split + * the application in two: tables created in `public`, the catalog read in the named schema, and + * `status` reporting the whole model missing after a `sync --apply` that succeeded. + * + * `public`, the default, runs nothing, so the connection keeps Postgres's own `search_path`. + */ + def databaseSchema: String = "public" + + /** Run on **every** connection the pool creates, not once on a borrowed one (DESIGN §8.7), + * because the settings it exists for are per connection: a statement run once would reach only + * the connection that ran it. + * + * This is where `application_name`, statement timeouts and any other per connection setting + * belong. The schema is not among them: [[databaseSchema]] sets `search_path` on its own, ahead + * of this hook, so an override here needs no `super` call to keep the application in its schema, + * and one that sets `search_path` on purpose still has the last word. */ def databaseInit: Connection -> Unit = _ => () @@ -75,7 +95,7 @@ trait DbInit { databaseUrl, databaseUser, databasePassword, - databaseInit, + DbInit.connectionInit(databaseSchema, databaseInit), databasePoolSize, databaseAcquireTimeout ) @@ -83,6 +103,54 @@ trait DbInit { object DbInit { + /** The one hook the pool runs: the schema step `databaseSchema` implies, then the application's + * own. Composed here rather than offered as `databaseInit`'s default, because a default would be + * lost by every override that forgot `super`, which is the split this exists to prevent. + */ + private[eezo] def connectionInit(schema: String, init: Connection -> Unit): Connection -> Unit = + if (schema == "public") init + else + c => { + enterSchema(c, schema) + init(c) + } + + /** Creates the schema only when the catalog says it is missing, because Postgres checks the + * CREATE privilege on the database before it honours `if not exists`: a least privileged role + * that owns an existing schema would otherwise fail every connection the pool opens. + * + * Two connections that both found it missing still race, on a fresh database where the pool + * fills or several processes boot at once, and `if not exists` does not settle that race: the + * loser fails on the catalog's unique index. Its failure means the schema is there, which is all + * this step needs, so it goes on to `search_path`; any other failure is a real one and fails the + * connection. + */ + private def enterSchema(c: Connection, schema: String): Unit = { + val name = Ddl.q(schema) + val st = c.createStatement() + try { + if (!schemaExists(c, schema)) + try st.execute(s"create schema if not exists $name"): Unit + catch { case e: SQLException if CreatedMeanwhile(e.getSQLState) => () } + st.execute(s"set search_path to $name"): Unit + } finally st.close() + } + + /** `unique_violation`, what Postgres answers the loser of the race, and `duplicate_schema`, which + * says the same thing in its own words. + */ + private val CreatedMeanwhile: Set[String | Null] = Set("23505", "42P06") + + private def schemaExists(c: Connection, schema: String): Boolean = { + val ps = c.prepareStatement("select 1 from pg_namespace where nspname = ?") + try { + ps.setString(1, schema) + val rs = ps.executeQuery() + try rs.next() + finally rs.close() + } finally ps.close() + } + private[eezo] val DefaultPoolSize: Int = 10 /** A value HikariCP would refuse, below one, is as malformed as one that is not a number: either diff --git a/modules/db/src/main/scala/io/eezo/db/schema/Ddl.scala b/modules/db/src/main/scala/io/eezo/db/schema/Ddl.scala index 8faad70d..6080574a 100644 --- a/modules/db/src/main/scala/io/eezo/db/schema/Ddl.scala +++ b/modules/db/src/main/scala/io/eezo/db/schema/Ddl.scala @@ -8,7 +8,11 @@ package io.eezo.db.schema */ object Ddl { - private def q(s: String): String = "\"" + s.replace("\"", "\"\"") + "\"" + /** A name as a Postgres quoted identifier, its own double quotes doubled. Shared with + * [[io.eezo.db.DbInit]], whose schema statements run on every connection, where a stray quote + * would fail them all or run more than the one statement meant. + */ + private[db] def q(s: String): String = "\"" + s.replace("\"", "\"\"") + "\"" private def checkName(table: String, column: String, expr: String): String = { val h = Integer.toHexString(expr.hashCode & 0xffffff) diff --git a/modules/db/src/test/scala/io/eezo/db/DbAppSuite.scala b/modules/db/src/test/scala/io/eezo/db/DbAppSuite.scala index ab2584dd..602b3800 100644 --- a/modules/db/src/test/scala/io/eezo/db/DbAppSuite.scala +++ b/modules/db/src/test/scala/io/eezo/db/DbAppSuite.scala @@ -29,8 +29,6 @@ class DbAppSuite extends PgSuite { override def databaseUser: String = Pg.username override def databasePassword: String = Pg.password - override def databaseInit: Connection -> Unit = Pg.searchPath(pgSchema) - override def boot(): Unit = { ran = true sawDatabase = read { Commands.status(schema, databaseSchema) }.changes.nonEmpty @@ -39,6 +37,96 @@ class DbAppSuite extends PgSuite { private def installed: Boolean = Installed.installed + /** An application that names its Postgres schema and nothing else: no `databaseInit`, so whatever + * puts its tables in that schema is the framework's doing. The schema is one that does not exist + * yet, so the framework has to create it, and the test drops it after. + */ + private class Named(name: String, user: String = Pg.username, password: String = Pg.password) + extends DbApp { + override def schema: Schema = Library + override def databaseSchema: String = name + + override def databaseUrl: String = Pg.jdbcUrl + override def databaseUser: String = user + override def databasePassword: String = password + + override def boot(): Unit = () + } + + /** The same application with a hook of its own for something unrelated to the schema, written + * without `super`, the way an application adds `application_name` and forgets the rest. + */ + private class NamedWithHook(name: String) extends Named(name) { + override def databaseInit: Connection -> Unit = c => { + val st = c.createStatement() + try st.execute("set application_name to 'eezo-dbappsuite'"): Unit + finally st.close() + } + } + + /** The schemas that hold a table called `table`, read from the catalog. */ + private def schemasHolding(table: String): Set[String] = { + val ps = + db.prepareStatement("select table_schema from information_schema.tables where table_name = ?") + try { + ps.setString(1, table) + val rs = ps.executeQuery() + try Iterator.continually(rs).takeWhile(_.next()).map(_.getString(1)).toSet + finally rs.close() + } finally ps.close() + } + + /** Runs `f` with `name` absent before and dropped after, so no later suite sees its tables. */ + private def inFreshSchema(name: String)(f: => Unit): Unit = { + val drop = s"""drop schema if exists "${name.replace("\"", "\"\"")}" cascade""" + exec(drop) + try f + finally exec(drop) + } + + test("databaseSchema alone puts the tables in that schema, creating it, and status agrees") { + val name = pgSchema + "_own" + inFreshSchema(name) { + val app = new Named(name) + assertEquals(captured(app.run(List("db", "sync", "--apply")))._1, 0) + assertEquals(schemasHolding("author"), Set(name)) + val (code, out, _) = captured(app.run(List("db", "status"))) + assertEquals(code, 0, out) + } + } + + test("a role that owns its schema but cannot create one on the database still connects") { + val name = pgSchema + "_owned" + val role = pgSchema + "_role" + inFreshSchema(name) { + exec( + s"""drop role if exists "$role"""", + s"""create role "$role" login password 'owner'""", + s"""create schema "$name" authorization "$role"""" + ) + try { + val app = new Named(name, user = role, password = "owner") + assertEquals(captured(app.run(List("db", "sync", "--apply")))._1, 0) + assertEquals(schemasHolding("author"), Set(name)) + } finally { + exec(s"""drop schema if exists "$name" cascade""", s"""drop role if exists "$role"""") + } + } + } + + test( + "a databaseInit of the application's own, without super, keeps the tables in databaseSchema" + ) { + val name = pgSchema + "_hook" + inFreshSchema(name) { + val app = new NamedWithHook(name) + assertEquals(captured(app.run(List("db", "sync", "--apply")))._1, 0) + assertEquals(schemasHolding("author"), Set(name)) + val (code, out, _) = captured(app.run(List("db", "status"))) + assertEquals(code, 0, out) + } + } + test("no arguments is the program, run with a Database installed and uninstalled after") { val app = new Job assert(!installed) diff --git a/modules/db/src/test/scala/io/eezo/db/DbInitSuite.scala b/modules/db/src/test/scala/io/eezo/db/DbInitSuite.scala index 4a25eb0e..741c0b1a 100644 --- a/modules/db/src/test/scala/io/eezo/db/DbInitSuite.scala +++ b/modules/db/src/test/scala/io/eezo/db/DbInitSuite.scala @@ -2,13 +2,137 @@ package io.eezo.db import munit.FunSuite +import java.lang.reflect.{InvocationHandler, Method, Proxy} +import java.sql.{Connection, PreparedStatement, ResultSet, SQLException, Statement} import java.time.Duration +import scala.collection.mutable.ListBuffer /** The `DATABASE_URL` fallback parser: the one secret a PaaS injects, split into the three values - * JDBC wants. Pure, so a plain suite. + * JDBC wants; and the schema step the pool runs on every connection, read off a connection that + * records what it is asked. Pure, so a plain suite. */ class DbInitSuite extends FunSuite { + /** A connection with no database behind it: it writes down every call made on it and every + * statement executed through it, and answers the catalog lookup as `schemaExists` says. With + * `createFails`, the `create schema` statement fails with that SQLState, the way Postgres + * answers the loser of two connections that both found the schema missing. + */ + private final class Recording(schemaExists: Boolean, createFails: Option[String] = None) { + val calls: ListBuffer[String] = ListBuffer.empty + val executed: ListBuffer[String] = ListBuffer.empty + + private def proxy[T]( + cls: Class[T] + )(answer: (Method, Array[AnyRef] | Null) => AnyRef | Null): T = + cls.cast( + Proxy.newProxyInstance( + getClass.getClassLoader, + Array[Class[?]](cls), + new InvocationHandler { + def invoke(p: AnyRef, m: Method, args: Array[AnyRef] | Null): AnyRef | Null = + answer(m, args) + } + ) + ) + + private val resultSet: ResultSet = proxy(classOf[ResultSet]) { (m, _) => + if (m.getName == "next") java.lang.Boolean.valueOf(schemaExists) else null + } + + private val prepared: PreparedStatement = proxy(classOf[PreparedStatement]) { (m, _) => + if (m.getName == "executeQuery") resultSet else null + } + + private val statement: Statement = proxy(classOf[Statement]) { (m, args) => + if (m.getName == "execute") { + val sql = String.valueOf(args.nn(0)) + executed += sql + createFails match { + case Some(state) if sql.startsWith("create schema") => + throw new SQLException("duplicate key value violates unique constraint", state) + case _ => java.lang.Boolean.FALSE + } + } else null + } + + val connection: Connection = proxy(classOf[Connection]) { (m, _) => + calls += m.getName + m.getName match { + case "createStatement" => statement + case "prepareStatement" => prepared + case _ => null + } + } + } + + test( + "the default public schema runs nothing on the connection, so its search_path is Postgres's" + ) { + val c = Recording(schemaExists = true) + DbInit.connectionInit("public", _ => ())(c.connection) + assertEquals(c.calls.toList, Nil) + } + + test("a named schema is created when missing, then becomes the search_path, before the hook") { + val c = Recording(schemaExists = false) + DbInit.connectionInit( + "shop", + conn => conn.createStatement().execute("set application_name to 'x'"): Unit + )( + c.connection + ) + assertEquals( + c.executed.toList, + List( + """create schema if not exists "shop"""", + """set search_path to "shop"""", + "set application_name to 'x'" + ) + ) + } + + test( + "a schema that exists is not created again, so a role without CREATE on the database connects" + ) { + val c = Recording(schemaExists = true) + DbInit.connectionInit("shop", _ => ())(c.connection) + assertEquals(c.executed.toList, List("""set search_path to "shop"""")) + } + + test( + "losing the race to create the schema to another connection still enters it, either SQLState" + ) { + for (state <- List("23505", "42P06")) { + val c = Recording(schemaExists = false, createFails = Some(state)) + DbInit.connectionInit("shop", _ => ())(c.connection) + assertEquals( + c.executed.toList, + List("""create schema if not exists "shop"""", """set search_path to "shop""""), + state + ) + } + } + + test("any other failure to create the schema fails the connection") { + val c = Recording(schemaExists = false, createFails = Some("42501")) + val e = intercept[SQLException](DbInit.connectionInit("shop", _ => ())(c.connection)) + assertEquals(e.getSQLState, "42501") + assertEquals(c.executed.toList, List("""create schema if not exists "shop"""")) + } + + test("a double quote in the schema name is doubled, so the name stays one identifier") { + val c = Recording(schemaExists = false) + DbInit.connectionInit("""a"; drop table x; --""", _ => ())(c.connection) + assertEquals( + c.executed.toList, + List( + """create schema if not exists "a""; drop table x; --"""", + """set search_path to "a""; drop table x; --"""" + ) + ) + } + private def parsed(url: String) = DbInit.parseDatabaseUrl(url).getOrElse(fail(s"expected $url to parse")) diff --git a/modules/db/src/test/scala/io/eezo/db/support/Pg.scala b/modules/db/src/test/scala/io/eezo/db/support/Pg.scala index 77a25e9f..af0ddb95 100644 --- a/modules/db/src/test/scala/io/eezo/db/support/Pg.scala +++ b/modules/db/src/test/scala/io/eezo/db/support/Pg.scala @@ -57,8 +57,9 @@ object Pg { acquireTimeout = DbInit.DefaultAcquireTimeout ) - /** The connection init hook that isolates a suite by schema: [[database]]'s, and the one a suite - * that builds its own `Database` through `DbInit` names as `databaseInit`. + /** The connection init hook that isolates a suite by schema: [[database]]'s. A suite that builds + * its own `Database` through `DbInit` names the schema as `databaseSchema` instead, which sets + * the same `search_path` on every connection. */ def searchPath(schema: String): Connection -> Unit = c => { val st = c.createStatement() diff --git a/modules/example/src/test/scala/example/Tour.scala b/modules/example/src/test/scala/example/Tour.scala index acfd6a45..1e0be3e2 100644 --- a/modules/example/src/test/scala/example/Tour.scala +++ b/modules/example/src/test/scala/example/Tour.scala @@ -48,21 +48,11 @@ object Tour extends DbApp { override def databaseUser: String = container.getUsername override def databasePassword: String = container.getPassword - /** The tour works in its own schema, so eezo's connections have to be told about it too; - * otherwise the migrator writes to `public` while the drift checks read `eezo_tour`, and a - * replay looks like it never happened. DESIGN §8.7. + /** The tour works in its own schema, and naming it is all eezo needs: every connection the pool + * opens creates it if it is missing and sets `search_path` to it, so the migrator writes where + * the drift checks read. DESIGN §8.7. */ - // `Connection => Unit`, not `->`: this module does not enable capture checking, so eezo's pure - // arrow presents here as an ordinary function type (research/capture-checking.md §6.3). - override def databaseInit: Connection => Unit = { c => - val st = c.createStatement() - try st.execute(s"""create schema if not exists "$Schema0"""") - catch { case NonFatal(_) => () } - finally st.close() - val st2 = c.createStatement() - try st2.execute(s"""set search_path to "$Schema0"""") - finally st2.close() - } + override def databaseSchema: String = Schema0 private var paused = true @@ -405,8 +395,8 @@ object Tour extends DbApp { private def prepare(c: Connection): Unit = { header() exec(c, s"""drop schema if exists "$Schema0" cascade""") - // `if not exists`: the pool fills in the background while this runs, and each new connection's - // `databaseInit` may create the schema between the drop and this line. + // `if not exists`: the pool fills in the background while this runs, and each new connection + // creates the schema `databaseSchema` names when it is missing, maybe between the drop and here. exec(c, s"""create schema if not exists "$Schema0"""") exec(c, s"""set search_path to "$Schema0"""") note(s"Working in schema `$Schema0` on ${Db.url}, so nothing else is touched.") From ad5acd9a42e4889b7cfed0dfb29bfe39d6928717 Mon Sep 17 00:00:00 2001 From: rcardin Date: Thu, 22 Oct 2026 17:28:53 +0200 Subject: [PATCH 2/2] docs(db): the how-to says how to put public back on search_path The page warned that the schema alone is on the path and that objects in public are not, but never said what to do about it. One sentence now points at databaseInit, which runs after the schema step. Co-Authored-By: Claude Fable 5.1 --- docs/how-to/configure-the-database.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/how-to/configure-the-database.md b/docs/how-to/configure-the-database.md index 5b1cd838..0f206a8d 100644 --- a/docs/how-to/configure-the-database.md +++ b/docs/how-to/configure-the-database.md @@ -64,7 +64,9 @@ tables. The examples under `examples/` each do this. The schema is created only when the catalog says it is missing, so a production role that owns an existing schema but may not create one on the database still connects. `search_path` is the schema -alone: objects in `public`, such as an extension's types and functions, are not on it. +alone: objects in `public`, such as an extension's types and functions, are not on it. An +application that needs those too sets the path itself in `databaseInit`, which runs after the +schema step: `set search_path to bookshelf, public`. `databaseInit` is for any other per connection setting, such as `application_name` or a statement timeout. It runs after the schema step on every pooled connection and needs no `super` call: