Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/23794-file-url-filename-rule.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'@objectstack/runtime': minor
'@objectstack/cli': patch
---

fix(runtime,cli): an upper-case `FILE:` database URL opens the file its lower-case spelling opens, `file://NAME` opens `NAME` on both hosts, and the URL-to-filename rule is exported as `sqliteFilenameFromUrl`

Clause-②: yes (widening)

- **`FILE:` is stripped case-insensitively.** The standalone host classified `FILE:./app.db` as SQLite case-insensitively but stripped the prefix case-sensitively, so the prefix stayed in the file name: `OS_DATABASE_URL=FILE:./app.db` created a directory literally named `FILE:.` and put the database inside it, and `File:///tmp/app.db` was refused as an unsupported `File://` scheme. Both now open the file the lower-case spelling opens (`./app.db`, `/tmp/app.db`), as the `sqlite-wasm` path already did. Lower-case `file:` URLs and bare paths are unchanged.
- **New export: `sqliteFilenameFromUrl(url, kind)`** from `@objectstack/runtime`, `kind` being `'sqlite'` or `'sqlite-wasm'`. It is the one rule that turns a SQLite database URL (after `normalizeDatabaseUrl`) into a file name: `file:` / `file://` stripped case-insensitively, plus `wasm-sqlite:` for the WASM kind. For `'sqlite'` it throws a plain `Error` on a URL that is not a file path, naming the URL's scheme and never the URL. Nothing is removed.
- **`os serve`'s legacy storage path (`OS_MODE=off`, `bootMode: 'off'`, or a host config) turns a SQLite URL into a file by that same rule**, so both hosts open one file per URL. It used to strip only a lower-case `file:`: `OS_DATABASE_URL=file://NAME` opened `//NAME` at the filesystem root, where the standalone host opens `./NAME`, and `FILE:./app.db` made a `FILE:.` directory there too. `file:///abs/app.db` opens `/abs/app.db`; `sqlite:` / `sqlite://` URLs are unchanged.
- **What that rule refuses, the legacy path now refuses too, and fatally**, with the standalone host's own sentence (it names the URL's scheme, never the URL): an explicit `OS_DATABASE_DRIVER=sqlite` over a server URL such as `postgres://user:password@host/db`, which used to create directories named after the URL, credentials included, and a `file:` URL that names no file, which used to declare a database with an empty file name. Fix: point `OS_DATABASE_URL` at a file (`file:./app.db`), or select the driver the URL is for. The undocumented `sql://` prefix that path also stripped under an explicit `sqlite` selection is refused the same way; write `file:` instead. A blank `OS_DATABASE_URL` under an explicit `sqlite` selection is read as absent (`:memory:`), as an absent one always was.
66 changes: 62 additions & 4 deletions packages/cli/src/utils/driver-vocabulary-parity.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -533,15 +533,27 @@ describe('the OS_DATABASE_URL door: both hosts, one verdict per URL, dev and pro
}
});

// `sqlite:` / `sqlite://` are aliases of `file:` on both hosts, by ONE rule —
// the runtime host's `normalizeDatabaseUrl`, which the CLI host calls. Both
// hosts must name the same FILE: `sqlite://x.db` used to open `//x.db` (the
// filesystem root) on the CLI host while the runtime host opened `x.db`.
// `sqlite:` / `sqlite://` are aliases of `file:` on both hosts, and a `file:`
// URL becomes a file name by ONE rule — the runtime host's
// `normalizeDatabaseUrl` and `sqliteFilenameFromUrl`, both of which the CLI
// host calls. Both hosts must name the same FILE: `sqlite://x.db` and
// `file://x.db` used to open `//x.db` (the filesystem root) on the CLI host
// while the runtime host opened `x.db`, and `FILE:./x.db` kept its upper-case
// prefix as part of the file name on both, making a `FILE:.` directory. The
// lower-case `file:` and bare-path rows are the CONTROL.
it.each([
['sqlite://x.db', 'x.db', 'sqlite'],
['sqlite:x.db', 'x.db', 'sqlite'],
['sqlite:///var/lib/objectstack/x.db', '/var/lib/objectstack/x.db', 'sqlite'],
['sqlite://x.wasm.db', 'x.wasm.db', 'sqlite-wasm'],
['file://x.db', 'x.db', 'sqlite'],
['file:///var/lib/objectstack/x.db', '/var/lib/objectstack/x.db', 'sqlite'],
['file://x.wasm.db', 'x.wasm.db', 'sqlite-wasm'],
['FILE:./x.db', './x.db', 'sqlite'],
['File:///var/lib/objectstack/x.db', '/var/lib/objectstack/x.db', 'sqlite'],
['FILE:./x.wasm.db', './x.wasm.db', 'sqlite-wasm'],
['file:./x.db', './x.db', 'sqlite'],
['./x.db', './x.db', 'sqlite'],
])('both hosts open %s as the file %s (%s), in dev and prod', (url, file, kind) => {
process.env.OS_DATABASE_URL = url;
const runtime = resolveStandaloneDatabase({ artifactPath: '/nonexistent/objectstack.json' });
Expand All @@ -553,4 +565,50 @@ describe('the OS_DATABASE_URL door: both hosts, one verdict per URL, dev and pro
expect(definition!.config.filename, `the CLI host's file for ${url} (isDev=${isDev})`).toBe(runtime.sqliteFile);
}
});

// What that rule refuses, both hosts refuse, with ONE sentence that names the
// scheme and never the URL — the CLI host's is the runtime host's minus its
// `[StandaloneStack]` tag, as for the retired engine above — and the CLI
// host's refusal is typed, the class serve.ts makes fatal (it swallows every
// other construction error). Two
// inputs reach it: an explicit sqlite selection over a server URL, which the
// CLI host used to open as a file at a path made of the URL, credentials
// included, and a `file:` that names no file.
it.each([
['sqlite', 'postgres://PLACEHOLDER-USER:PLACEHOLDER-SECRET@127.0.0.1:5432/db', 'its scheme is "postgres://"'],
[undefined, 'file:', 'it names no file after "file:"'],
[undefined, 'FILE:', 'it names no file after "file:"'],
])('both hosts REFUSE driver %s over %s, in dev AND in prod, with one sentence', (driver, url, why) => {
if (driver) process.env.OS_DATABASE_DRIVER = driver;
const runtime = runtimeUrlVerdict(url);
expect(runtime.accepted, `the runtime host accepted ${url}`).toBe(false);
const sentence = (runtime as { message: string }).message;
expect(sentence).toContain(why);
for (const isDev of [false, true]) {
let error: unknown;
try {
resolveStorageDefinition(resolveDriverType(driver, url), { databaseUrl: url, isDev });
} catch (err) {
error = err;
}
expect(error, `the CLI host (isDev=${isDev}) did not refuse ${url} by type`).toBeInstanceOf(UnsupportedDriverError);
expect((error as UnsupportedDriverError).recognized).toBe(true);
expect((error as UnsupportedDriverError).driverType).toBe('sqlite');
expect((error as Error).message).toBe(sentence.replace(/^\[StandaloneStack\] /, ''));
expect((error as Error).message).not.toBe(sentence);
expect((error as Error).message).not.toContain('PLACEHOLDER');
}
});

// CLI host only: under an explicit sqlite selection a blank URL is the absent
// URL, `:memory:`, never a file named by whitespace — blank is absent, as this
// host's URL inference reads it. The runtime host's absent URL is its default
// file, a different answer for absence that this pin does not judge.
it.each(['', ' '])('the CLI host reads the blank URL %j under an explicit sqlite selection as :memory:', (url) => {
for (const isDev of [false, true]) {
const definition = resolveStorageDefinition(resolveDriverType('sqlite', url), { databaseUrl: url, isDev });
expect(definition!.driverId).toBe('sqlite');
expect(definition!.config.filename).toBe(':memory:');
}
});
});
54 changes: 43 additions & 11 deletions packages/cli/src/utils/storage-driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ import {
namesRetiredMemoryEngine,
normalizeDatabaseUrl,
retiredMemoryEngineMessage,
sqliteFilenameFromUrl,
type LoadTursoDriverFactoryOptions,
} from '@objectstack/runtime';
import {
Expand Down Expand Up @@ -527,13 +528,11 @@ export function resolveStorageDefinition(
}

if (kind === 'sqlite') {
// `sqlite:` and `sqlite://` are aliases of `file:`, translated by the runtime
// host's own rule (`normalizeDatabaseUrl`) rather than a second copy of it,
// so both hosts open one file: `sqlite://x.db` is `x.db`, never `//x.db`
// (the filesystem root), and `sqlite:///abs/x.db` is `/abs/x.db`.
const filePath = normalizeDatabaseUrl(databaseUrl ?? ':memory:')
.replace(/^file:/, '')
.replace(/^sql:\/\//, '');
// The runtime host's file for this URL ({@link sqliteFileFor}), never a
// second copy of its rule: `sqlite://x.db` and `file://x.db` are `x.db`,
// never `//x.db` (the filesystem root), `file:///abs/x.db` is `/abs/x.db`,
// and `FILE:./x.db` is `./x.db`.
const filePath = sqliteFileFor(databaseUrl, 'sqlite');
return {
driverId: 'sqlite',
config: { filename: filePath, ...devAutoMigrateConfig('sqlite', isDev) },
Expand All @@ -548,10 +547,9 @@ export function resolveStorageDefinition(
}

if (kind === 'sqlite-wasm') {
// The same alias rule as the sqlite arm: `sqlite://x.wasm.db` is `x.wasm.db`.
const filePath = normalizeDatabaseUrl(databaseUrl ?? ':memory:')
.replace(/^file:/, '')
.replace(/^wasm-sqlite:\/\//, '');
// The same rule as the sqlite arm, `wasm-sqlite://` included:
// `sqlite://x.wasm.db` and `file://x.wasm.db` are `x.wasm.db`.
const filePath = sqliteFileFor(databaseUrl, 'sqlite-wasm');
return {
driverId: 'sqlite-wasm',
// `persist` passthrough: the CLI kept `on-disconnect` semantics here
Expand Down Expand Up @@ -629,6 +627,40 @@ export function resolveStorageDefinition(
return null;
}

/**
* The SQLite file a `default` definition opens for this URL — the runtime
* host's answer, by the runtime host's rule: `normalizeDatabaseUrl` (`sqlite:` /
* `sqlite://` are `file:`), then `sqliteFilenameFromUrl` (`file:` / `file://`,
* and `wasm-sqlite:` for the WASM kind, stripped case-insensitively). This file
* used to keep its own copy, a case-sensitive `/^file:/` strip, so
* `file://NAME` opened `//NAME` at the filesystem root, where the runtime host
* opens `NAME`, and `FILE:./x.db` made a directory named `FILE:.`.
*
* An absent or blank URL is `:memory:`: blank is absent, as
* {@link inferDriverTypeFromUrl} reads it, never a file named by whitespace.
*
* A URL the rule refuses is re-raised as {@link UnsupportedDriverError} with
* the runtime host's own sentence, which names the scheme and never the URL,
* minus its `[StandaloneStack]` tag — the shape this host's retired-engine
* refusal already has (`driver-vocabulary-parity.test.ts` holds both).
* Two inputs reach that: an explicit `sqlite` selection over a server URL
* (`OS_DATABASE_DRIVER=sqlite` beside `postgres://user:password@…`), and a
* `file:` that names no file. Both used to build a definition: the first opened
* a file at a path made of the URL, credentials included, and the second
* declared an empty file name. The type is
* what makes the refusal fatal: `serve.ts` re-throws this class and swallows
* every other construction error, so a plain `Error` would boot with no
* storage at all.
*/
function sqliteFileFor(databaseUrl: string | undefined, kind: 'sqlite' | 'sqlite-wasm'): string {
const url = normalizeDatabaseUrl(databaseUrl?.trim() ? databaseUrl : ':memory:');
try {
return sqliteFilenameFromUrl(url, kind);
} catch (err) {
throw new UnsupportedDriverError(kind, (err as Error).message.replace(/^\[StandaloneStack\] /, ''));
}
}

/**
* Load the OPTIONAL libSQL/Turso driver package and wrap it as the host driver
* factory `DefaultDatasourcePlugin` accepts (#5602).
Expand Down
2 changes: 1 addition & 1 deletion packages/runtime/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ export { Runtime } from './runtime.js';
export type { RuntimeConfig } from './runtime.js';

// Export Standalone Stack
export { createStandaloneStack, resolveObjectStackHome, resolveStandaloneDatabase } from './standalone-stack.js';
export { createStandaloneStack, resolveObjectStackHome, resolveStandaloneDatabase, sqliteFilenameFromUrl } from './standalone-stack.js';
export type { StandaloneStackConfig, StandaloneStackResult, ResolvedStandaloneDatabase } from './standalone-stack.js';

// The dev schema self-heal decision (#21733) — public because `@objectstack/cli`
Expand Down
30 changes: 30 additions & 0 deletions packages/runtime/src/resolve-project-database.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,36 @@ describe('resolveStandaloneDatabase — #6469 behaviours through the migrate sea
expect(r.sqliteFile).toBe(join(root, 'a.db'));
});

// `file:` is classified case-insensitively, so it is stripped case-insensitively
// too: an upper-case prefix the strip kept became part of the file name, and
// `FILE:./app.db` made a directory literally named `FILE:.`. Every row names
// the file its lower-case spelling names; the lower-case and bare rows are the
// CONTROL, unchanged.
it.each([
['FILE:./x.db', './x.db', 'sqlite'],
['File:///tmp/x.db', '/tmp/x.db', 'sqlite'],
['FILE://x.db', 'x.db', 'sqlite'],
['file:./x.db', './x.db', 'sqlite'],
['file:///tmp/x.db', '/tmp/x.db', 'sqlite'],
['file://x.db', 'x.db', 'sqlite'],
['./x.db', './x.db', 'sqlite'],
['/tmp/x.db', '/tmp/x.db', 'sqlite'],
['FILE:./x.wasm.db', './x.wasm.db', 'sqlite-wasm'],
])('%s opens the sqlite file %s (%s)', (url, file, driver) => {
const r = resolveStandaloneDatabase({ databaseUrl: url });
expect(r.driver).toBe(driver);
expect(r.sqliteFile).toBe(file);
});

it('an upper-case FILE: that names no file is refused like file:, never opened as a file named "FILE:"', () => {
let err: unknown;
try { resolveStandaloneDatabase({ databaseUrl: 'FILE:' }); } catch (e) { err = e; }
expect(err).toBeInstanceOf(Error);
expect((err as Error).constructor).toBe(Error);
expect((err as Error).message).toContain('sqlite driver was selected but the URL does not look like a file path');
expect((err as Error).message).toContain('it names no file after "file:"');
});

it('still refuses a genuinely unsupported scheme, message intact', () => {
expect(() => resolveStandaloneDatabase({ databaseUrl: 'redis://localhost:6379' }))
.toThrow(/Unsupported database URL scheme/);
Expand Down
23 changes: 22 additions & 1 deletion packages/runtime/src/standalone-stack.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
// owner-only `member_default`.

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { mkdtempSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { createStandaloneStack } from './standalone-stack.js';
Expand Down Expand Up @@ -397,6 +397,27 @@ describe('createStandaloneStack — default datasource declared, built via the s
expect(r.titles).toContain('hello-driver');
}, BOOT_TIMEOUT);

// An upper-case `FILE:` is the same file as `file:`, and the boot's directory
// preparation creates no `FILE:` directory on the way. Run from an empty cwd:
// the prefix used to survive into the relative filename, so the directory it
// made landed beside the process.
it('FILE: → declares the same file as file: and creates no FILE: directory', async () => {
const cwd = process.cwd();
const work = mkdtempSync(join(tmpdir(), 'os-file-url-case-'));
process.chdir(work);
try {
const upper = defaultDefOf(await createStandaloneStack({ databaseUrl: 'FILE:./upper/x.db' })).def;
const lower = defaultDefOf(await createStandaloneStack({ databaseUrl: 'file:./upper/x.db' })).def;
expect(upper.driver).toBe('sqlite');
expect(upper.config?.filename).toBe('./upper/x.db');
expect(upper.config?.filename).toBe(lower.config?.filename);
expect(readdirSync(work).filter((name) => /^file:/i.test(name))).toEqual([]);
} finally {
process.chdir(cwd);
rmSync(work, { recursive: true, force: true });
}
}, BOOT_TIMEOUT);

it('databaseDriver:sqlite-wasm → declares driver "sqlite-wasm"; factory builds SqliteWasmDriver that round-trips', async () => {
const r = await definitionRoundTrip({ databaseDriver: 'sqlite-wasm', databaseUrl: `file:${join(dir, 'wasm.db')}` });
expect(r.driverId).toBe('sqlite-wasm');
Expand Down
23 changes: 20 additions & 3 deletions packages/runtime/src/standalone-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -566,14 +566,31 @@ function refuseRetiredMemoryEngine(
throw new Error(`[StandaloneStack] ${retiredMemoryEngineMessage(selection)}`);
}

/** URL→filename for the two sqlite kinds. Throws on a URL that isn't a path. */
function sqliteFilenameFromUrl(dbUrl: string, kind: 'sqlite' | 'sqlite-wasm'): string {
/**
* URL→filename for the two sqlite kinds: the ONE rule both hosts turn a SQLite
* database URL into a file with. Throws on a URL that isn't a path.
*
* Takes the URL as {@link resolveStandaloneDatabase} has it, i.e. after
* `normalizeDatabaseUrl` (`sqlite:` / `sqlite://` are already `file:`).
* `file:` and `file://` are stripped case-insensitively on BOTH kinds, as
* {@link detectDriverFromUrl} classifies them (`/^file:/i`): a prefix the
* classifier accepts and this strip kept became part of the filename, so
* `FILE:./app.db` opened `./FILE:./app.db` and the boot's directory creation
* made a `FILE:.` directory — the "bogus directory from a URL" this module's
* header rules out.
*
* Exported because the CLI host (`resolveStorageDefinition` in
* `@objectstack/cli`'s `utils/storage-driver.ts`) calls it rather than keeping
* a second copy; its copy stripped only `/^file:/`, so `file://NAME` opened
* `//NAME` at the filesystem root there while this host opened `NAME`.
*/
export function sqliteFilenameFromUrl(dbUrl: string, kind: 'sqlite' | 'sqlite-wasm'): string {
if (kind === 'sqlite-wasm') {
return dbUrl
.replace(/^wasm-sqlite:(\/\/)?/i, '')
.replace(/^file:(\/\/)?/i, '') || ':memory:';
}
const filename = dbUrl.replace(/^file:(\/\/)?/, '');
const filename = dbUrl.replace(/^file:(\/\/)?/i, '');
const scheme = urlSchemeOf(filename);
if (!filename || scheme) {
// Names the scheme, never the URL ({@link urlSchemeOf}). This is the
Expand Down
Loading