Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .changeset/23726-read-only-arms-remainder.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
'@objectstack/cli': patch
'@objectstack/service-storage': patch
'@objectstack/objectql': patch
---

Clause-②: no

fix(cli): `os storage orphans`, the `os secret rewrap` dry run and the `os migrate summary-nulls` dry run say they write no row or table, and that opening a SQLite file can still switch its journal mode, instead of saying nothing was written

**What was wrong.** Each of these runs boots through the SQL driver before it prints, and the driver's connect switches a SQLite file to the journal mode it asks for (WAL, unless `OS_DATABASE_SQLITE_JOURNAL_MODE=delete` is set in the command's own environment or the project's `.env`). Measured on a file still on a rollback journal, each run moved the header's journal-mode bytes (18 and 19, from 1,1 to 2,2) and its change counter, and changed no row or table, while printing "nothing was written". Nothing was lost.

**What changed.** Each sentence below now claims no row or table. Where the run opened a SQLite file, the report's line is followed by the note the other read-only arms already carry: "Opening a SQLite file can still switch its journal mode (to WAL by default), which changes its header only." On PostgreSQL or MySQL the line ends without it.

- `os storage orphans` (`formatStrandedOrphanInventory`, `@objectstack/service-storage`). FROM: "This pass is READ-ONLY: nothing was written, tombstoned or deleted." TO: "This pass is READ-ONLY: no row or table was written, tombstoned or deleted." and the note.
- `os secret rewrap` (the dry run's note, `REWRAP_NOTES.dryRun`, also in `--json` `report.notes`). FROM: "Dry run: every attributed row was opened, re-sealed and verified in memory, and nothing was written. Re-run with --apply to write." TO: "… and no row or table was written. Re-run with --apply to write." The note already follows the dry run's closing line on a SQLite file, so this sentence carries none.
- `os migrate summary-nulls` (the dry run that found NULL roll-ups, `formatSummaryBackfillReport`, `@objectstack/objectql`). FROM: "Dry run — nothing was written. Re-run with --apply to backfill." TO: "Dry run — no row or table was written. Re-run with --apply to backfill." and the note.

**Unchanged.** No exit code, flag, prompt, refusal or export signature changes, and no `--json` document gains a field; the note is printed in the text output only. What the driver does on connect is unchanged: once a file is in the mode the driver asks for, these runs leave it byte-identical.
2 changes: 1 addition & 1 deletion packages/cli/src/commands/migrate/recorded-by.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ async function confirm(question: string): Promise<boolean> {
*
* Dry run by default, like every other `os migrate` subcommand (#2186): a
* bare invocation reports how many rows still carry the sentinel and writes
* nothing. `--apply` runs the conversion through the ADR-0119 D2 migration
* no row or table. `--apply` runs the conversion through the ADR-0119 D2 migration
* journal, so each chunk is one transaction and an interrupted run is
* recoverable via `os migrate resume`.
*
Expand Down
70 changes: 66 additions & 4 deletions packages/cli/src/commands/migrate/summary-nulls.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@
* (the unscoped run the ruling keeps byte-for-byte). The seams that would boot
* a database or walk a real engine are replaced; the command's own parse and
* control flow run for real.
*
* And what the human face says about the database: the dry run's read-only
* boot can still switch a SQLite file's journal mode, which rewrites its
* header and no row, so the report's dry-run line claims no row or table and,
* on a SQLite target only, is followed by the note naming the switch. The
* report's formatter is the real one from `@objectstack/objectql`.
*/

import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
Expand All @@ -21,6 +27,7 @@ import MigrateSummaryNulls from './summary-nulls.js';
import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { probeMigrationTarget } from '../../utils/migrate-occupancy-gate.js';
import { isExitSignal } from '../../utils/format.js';
import { JOURNAL_MODE_SWITCH_NOTE } from '../../utils/sqlite-occupancy.js';
import { backfillSummaryNulls } from '@objectstack/objectql';

vi.mock('../../utils/schema-migrate.js', () => ({ bootSchemaStack: vi.fn() }));
Expand All @@ -29,10 +36,15 @@ vi.mock('../../utils/migrate-occupancy-gate.js', () => ({
probeMigrationTarget: vi.fn(),
}));
vi.mock('../../utils/data-migration-plugins.js', () => ({ buildDataMigrationPlugins: vi.fn(async () => []) }));
vi.mock('@objectstack/objectql', () => ({
backfillSummaryNulls: vi.fn(),
formatSummaryBackfillReport: vi.fn(() => []),
}));
// The walk is replaced; the formatter is the real one, so the human-face pins
// below read the report's own dry-run line.
vi.mock('@objectstack/objectql', async (importOriginal) => {
const real = await importOriginal<typeof import('@objectstack/objectql')>();
return {
backfillSummaryNulls: vi.fn(),
formatSummaryBackfillReport: real.formatSummaryBackfillReport,
};
});

const HERE = dirname(fileURLToPath(import.meta.url));
const CLI_ROOT = resolve(HERE, '..', '..', '..');
Expand Down Expand Up @@ -154,3 +166,53 @@ describe('os migrate summary-nulls', () => {
expect(payload.error).toContain('customer.nope');
}, RUN_TIMEOUT);
});

// ───────────────────────────────────────────────────────────────────────────
// The human face: what the dry run says about the database
// ───────────────────────────────────────────────────────────────────────────

const SQLITE_CONFIG = { client: 'better-sqlite3', connection: { filename: '/srv/app/.objectstack/data/objectstack.db' } };
const POSTGRES_CONFIG = { client: 'pg', connection: 'postgres://db.example.com:5432/app' };
const DRY_RUN_LINE = 'Dry run — no row or table was written. Re-run with --apply to backfill.';

/** A dry run that found one NULL roll-up, so the report closes with its dry-run line. */
const DRY_REPORT_WITH_NULLS = {
...EMPTY_REPORT,
scannedObjects: ['customer'],
scannedRecords: 1,
fields: [{
object: 'customer', field: 'open_count', fn: 'count', childObject: 'case',
nullRows: 1, nonEmpty: 0, filled: 0, sampleRecordIds: ['c_1'],
}],
nullRows: 1,
};

describe('os migrate summary-nulls — the dry run names no write it cannot vouch for', () => {
/** Boot onto a primary driver with `config`; every report line the run printed. */
async function runDry(config: Record<string, unknown>): Promise<string[]> {
vi.mocked(bootSchemaStack).mockResolvedValue({
// The primary datasource's driver, as `SchemaStack.driver` carries it.
driver: { config },
kernel: { getService: () => engine },
dbLabel: 'test database',
shutdown: vi.fn(async () => {}),
} as any);
vi.mocked(backfillSummaryNulls).mockResolvedValue(DRY_REPORT_WITH_NULLS as any);
await MigrateSummaryNulls.run([], { root: CLI_ROOT });
return log.mock.calls.map((c: unknown[]) => c.map(String).join(' ')).join('\n').split('\n');
}

it('on a SQLite file the dry-run line claims no row or table and names the journal-mode switch', async () => {
const lines = await runDry(SQLITE_CONFIG);
const dryRun = lines.find((line) => line.startsWith('Dry run —'));
expect(dryRun, lines.join('\n')).toBe(`${DRY_RUN_LINE} ${JOURNAL_MODE_SWITCH_NOTE}`);
expect(lines.join('\n')).not.toContain('nothing was written');
}, RUN_TIMEOUT);

it('on a target with no file header (PostgreSQL) the dry-run line claims no row or table and names no switch', async () => {
const lines = await runDry(POSTGRES_CONFIG);
const dryRun = lines.find((line) => line.startsWith('Dry run —'));
expect(dryRun, lines.join('\n')).toBe(DRY_RUN_LINE);
expect(lines.join('\n')).not.toContain(JOURNAL_MODE_SWITCH_NOTE);
}, RUN_TIMEOUT);
});
15 changes: 14 additions & 1 deletion packages/cli/src/commands/migrate/summary-nulls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { OCCUPANCY_HINT, probeMigrationTarget } from '../../utils/migrate-occupancy-gate.js';
import { describeOccupancy, JOURNAL_MODE_SWITCH_NOTE } from '../../utils/sqlite-occupancy.js';
import { describeDriverSqliteFile } from '../../utils/connection-display.js';
import { buildDataMigrationPlugins } from '../../utils/data-migration-plugins.js';
import { refuseUndeclaredObjects } from '../../utils/migrate-object-scope.js';
// Type-only, so the heavy engine package is still loaded lazily below: this is
Expand Down Expand Up @@ -254,7 +255,19 @@ export default class MigrateSummaryNulls extends Command {

printInfo(`Database: ${chalk.white(stack.dbLabel)}`);
console.log('');
console.log(formatSummaryBackfillReport(report).join('\n'));
// [#23726] A dry run that found NULL roll-ups ends the report with "Dry
// run — no row or table was written." (its last line). The read-only
// boot's connect may still have switched a SQLite file's journal mode
// (`SqlDriver.applySqliteJournalMode`, its rule for every connect), which
// rewrites the file header and no row, so on a SQLite file that line is
// followed by the note naming the switch. A target that is no SQLite file
// (PostgreSQL, MySQL) has no header to switch. The `--json` document
// carries no sentence and is unchanged.
const journalNote = !report.applied && report.nullRows > 0
&& describeDriverSqliteFile(stack.driver?.config) !== undefined
? ` ${JOURNAL_MODE_SWITCH_NOTE}`
: '';
console.log(`${formatSummaryBackfillReport(report).join('\n')}${journalNote}`);
console.log('');

if (report.failures.length > 0) {
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/secret/orphans.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ export default class SecretOrphans extends Command {
// [#21471] Composed through the one-shot helper, never with the
// service's default provider: in a development posture with no key,
// that default mints a key file in the key home, and this report
// promises to write nothing.
// promises to write no row or table, and writes no key file either.
extraPlugins: [new PlatformObjectsPlugin(), await oneShotSettingsPlugin()],
// [#21391] The report boots READ-ONLY, the boot `os migrate plan`
// takes: `deferSchemaDdl` holds schema DDL back on every SQL
Expand Down
15 changes: 15 additions & 0 deletions packages/cli/src/commands/secret/rewrap.guards.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,21 @@ describe('os secret rewrap — the human face names no write it cannot vouch for
expect(closing, lines.join('\n')).toContain(`Dry run — no row or table was written. ${JOURNAL_MODE_SWITCH_NOTE}`);
}, 60_000);

it('the dry run\'s own note claims no row or table, in the human face and in --json report.notes', async () => {
const DRY_RUN_NOTE = 'Dry run: every attributed row was opened, re-sealed and verified in memory, and no row or '
+ 'table was written. Re-run with --apply to write.';
wireBoot(freshRows(), { driverConfig: SQLITE_CONFIG });
const lines = await runHuman(['--no-declared-datasources']);
const note = lines.find((line) => line.includes('Dry run: every attributed row'));
expect(note, lines.join('\n')).toContain(DRY_RUN_NOTE);
expect(lines.join('\n')).not.toContain('nothing was written');

wireBoot(freshRows(), { driverConfig: SQLITE_CONFIG });
const { payload } = await run(['--no-declared-datasources']);
expect(payload.report.notes).toContain(DRY_RUN_NOTE);
expect(JSON.stringify(payload.report.notes)).not.toContain('nothing was written');
}, 60_000);

it('on a target with no file header (PostgreSQL) the dry run claims no row or table and names no switch', async () => {
wireBoot(freshRows(), { driverConfig: POSTGRES_CONFIG });
const lines = await runHuman(['--no-declared-datasources']);
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/src/commands/secret/rewrap.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ import { readDeclaredDatasources } from './orphans.js';
*
* - **A dry run by default.** It boots read-only (the `os migrate plan` boot),
* opens, re-seals and verifies every attributed row in memory, prints
* classes and counts, and writes nothing. `--apply` writes.
* classes and counts, and writes no row or table. `--apply` writes.
* - **Never run for you.** Nothing on any boot or upgrade path invokes it, and
* it must not grow such a caller.
* - **Operator-only.** ⛔ No HTTP surface, so no refusal here answers a
Expand Down
81 changes: 76 additions & 5 deletions packages/cli/src/commands/storage/orphans.test.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,94 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* `os storage orphans` — what its help says the inventory writes.
* `os storage orphans` — what its help and its report say the inventory writes.
*
* The inventory boots read-only (deferred DDL, no seed, no file brought into
* existence), but the boot's connect can switch a SQLite file's journal mode,
* which rewrites the file's header and no row (measured on a rollback-journal
* file: header bytes 18 and 19 from 1,1 to 2,2, the change counter up by one,
* schema and rows byte-identical). So the help claims no row or table, never
* that the command writes nothing. The help names no target, so it carries no
* SQLite note.
* schema and rows byte-identical). So the help and the report claim no row or
* table, never that the command writes nothing. The help names no target, so
* it carries no SQLite note; the report's READ-ONLY line carries it on a
* SQLite target, and only there.
*
* The report case replaces only the boot. The inventory and its formatter
* (`@objectstack/service-storage`) run for real over an engine with no files.
*/

import { describe, expect, it } from 'vitest';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import StorageOrphans from './orphans.js';
import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { JOURNAL_MODE_SWITCH_NOTE } from '../../utils/sqlite-occupancy.js';

vi.mock('../../utils/schema-migrate.js', () => ({ bootSchemaStack: vi.fn() }));
vi.mock('../../utils/data-migration-plugins.js', () => ({ buildDataMigrationPlugins: vi.fn(async () => []) }));

const HERE = dirname(fileURLToPath(import.meta.url));
const CLI_ROOT = resolve(HERE, '..', '..', '..');
/** oclif builds its whole command table on the first `run()` in a process. */
const RUN_TIMEOUT = 60_000;

const SQLITE_CONFIG = { client: 'better-sqlite3', connection: { filename: '/srv/app/.objectstack/data/objectstack.db' } };
const POSTGRES_CONFIG = { client: 'pg', connection: 'postgres://db.example.com:5432/app' };
const READ_ONLY_LINE = 'This pass is READ-ONLY: no row or table was written, tombstoned or deleted.';

/** Wire the mocked boot to an engine that registers both objects and stores no file. */
function wireBoot(driverConfig: Record<string, unknown>): void {
const engine = {
getObject: (name: string) => (name === 'sys_file' || name === 'sys_attachment' ? { name } : undefined),
find: async () => [],
};
vi.mocked(bootSchemaStack).mockResolvedValue({
// The primary datasource's driver, as `SchemaStack.driver` carries it.
driver: { config: driverConfig },
dbLabel: 'test database',
kernel: { getService: (name: string) => (name === 'objectql' ? engine : undefined) },
tableAbsent: () => false,
shutdown: async () => {},
} as never);
}

/** One human-mode run; every line `console.log` received. */
async function runHuman(): Promise<string[]> {
const log = vi.spyOn(console, 'log').mockImplementation(() => {});
const savedExitCode = process.exitCode;
try {
await StorageOrphans.run([], { root: CLI_ROOT });
return log.mock.calls.map((c) => c.map(String).join(' ')).join('\n').split('\n');
} finally {
process.exitCode = savedExitCode;
log.mockRestore();
}
}

afterEach(() => {
vi.mocked(bootSchemaStack).mockReset();
});

describe('os storage orphans --help', () => {
it('says the inventory writes no row or table, tombstones nothing and deletes nothing', () => {
expect(StorageOrphans.description).toContain('Writes no row or table, tombstones nothing, deletes nothing.');
expect(StorageOrphans.description).not.toContain('Writes nothing');
});
});

describe('os storage orphans — the report names no write it cannot vouch for', () => {
it('on a SQLite file the READ-ONLY line claims no row or table and names the journal-mode switch', async () => {
wireBoot(SQLITE_CONFIG);
const lines = await runHuman();
const readOnly = lines.find((line) => line.includes('READ-ONLY'));
expect(readOnly, lines.join('\n')).toBe(`${READ_ONLY_LINE} ${JOURNAL_MODE_SWITCH_NOTE}`);
expect(lines.join('\n')).not.toContain('nothing was written');
}, RUN_TIMEOUT);

it('on a target with no file header (PostgreSQL) the READ-ONLY line claims no row or table and names no switch', async () => {
wireBoot(POSTGRES_CONFIG);
const lines = await runHuman();
const readOnly = lines.find((line) => line.includes('READ-ONLY'));
expect(readOnly, lines.join('\n')).toBe(READ_ONLY_LINE);
expect(lines.join('\n')).not.toContain(JOURNAL_MODE_SWITCH_NOTE);
}, RUN_TIMEOUT);
});
14 changes: 13 additions & 1 deletion packages/cli/src/commands/storage/orphans.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ import {
import { bootSchemaStack } from '../../utils/schema-migrate.js';
import { buildDataMigrationPlugins } from '../../utils/data-migration-plugins.js';
import { absentTableReads } from '../../utils/absent-table-reads.js';
import { describeDriverSqliteFile } from '../../utils/connection-display.js';
import { JOURNAL_MODE_SWITCH_NOTE } from '../../utils/sqlite-occupancy.js';
import type { IDataEngine } from '@objectstack/spec/contracts';
import type { StrandedOrphanInventoryEngine } from '@objectstack/service-storage';

Expand Down Expand Up @@ -181,7 +183,17 @@ export default class StorageOrphans extends Command {
printInfo(`Database: ${chalk.white(stack.dbLabel)}`);
reads.notice(false);
console.log('');
console.log(formatStrandedOrphanInventory(report));
// [#23726] The report's last line says the pass is READ-ONLY and wrote
// no row or table. The boot's connect may still have switched a SQLite
// file's journal mode (`SqlDriver.applySqliteJournalMode`, its rule for
// every connect), which rewrites the file header and no row, so on a
// SQLite file that line is followed by the note naming the switch. A
// target that is no SQLite file (PostgreSQL, MySQL) has no header to
// switch. The `--json` document carries no sentence and is unchanged.
const journalNote = describeDriverSqliteFile(stack.driver?.config) !== undefined
? ` ${JOURNAL_MODE_SWITCH_NOTE}`
: '';
console.log(`${formatStrandedOrphanInventory(report)}${journalNote}`);
console.log('');
if (report.truncated) {
printWarning(
Expand Down
7 changes: 5 additions & 2 deletions packages/cli/src/utils/sys-secret-rewrap.ts
Original file line number Diff line number Diff line change
Expand Up @@ -500,9 +500,12 @@ export const REWRAP_NOTES = {
resumable:
'Re-running is safe. A row already sealed under the current derivation is skipped as done, so a '
+ 'run that stopped part-way finishes the rest, and a finished run writes nothing.',
// [#23726] No row or table, never "nothing": the dry run's connect can switch
// a SQLite file's journal mode, which rewrites its header. The note naming
// that switch follows the human face's closing line, on a SQLite file only.
dryRun:
'Dry run: every attributed row was opened, re-sealed and verified in memory, and nothing was '
+ 'written. Re-run with --apply to write.',
'Dry run: every attributed row was opened, re-sealed and verified in memory, and no row or table '
+ 'was written. Re-run with --apply to write.',
unfinished:
'Some rows were not re-wrapped (see the refused and not-written counts). Each was left exactly as '
+ 'it was. A row that changed during the run is picked up by a re-run.',
Expand Down
Loading
Loading