From fc2c5a83c7d310a03a6455d7e911259ee11ee0f2 Mon Sep 17 00:00:00 2001 From: DavidBabinec Date: Tue, 13 Oct 2026 16:30:09 +0200 Subject: [PATCH] fix(import): replace import removes the previous site media A replace import wiped rows, folders and redirects but kept the old media_assets rows, so a second site shipping a file at the same storage path (two sites that embed the same font file) failed on media_assets.public_path after the data had already committed. Replace now deletes the library inside the import transaction and removes the bytes after commit, before the bundle's media is written. It only does so when the import carries media and the shell on main with no other branch, and it spares avatars and plugin-managed media, since those survive a replace. The archive endpoint now shares the JSON endpoint's apply path instead of a synthetic request, and the custom-table delete binds its boolean so replace runs on Postgres. --- docs/features/site-transfer.md | 34 +- server/handlers/cms/import.ts | 723 +++++++++++------- server/handlers/cms/importArchive.ts | 65 +- server/handlers/cms/media.ts | 12 +- server/handlers/cms/mediaVariants.ts | 16 + server/repositories/media.ts | 58 +- .../server/importReplaceMedia.test.ts | 374 +++++++++ 7 files changed, 932 insertions(+), 350 deletions(-) create mode 100644 src/__tests__/server/importReplaceMedia.test.ts diff --git a/docs/features/site-transfer.md b/docs/features/site-transfer.md index 9121ea485..ea6a6f9b7 100644 --- a/docs/features/site-transfer.md +++ b/docs/features/site-transfer.md @@ -26,6 +26,7 @@ The **Export site** dialog (`src/admin/pages/data/components/ExportDialog`) is a - Three import strategies: `replace` (destructive — the full-restore path), `merge-add` (insert if new), `merge-overwrite` (upsert). - Media bytes are stored as raw files under `media/`. Export and archive import stream media bytes; the browser and server do not assemble a media-heavy archive as a JSON/base64 payload. - Media folders + redirects are restored by the **`replace`** strategy (the full-restore path); merge strategies leave the local folder tree and redirects untouched. +- A `replace` that carries media and the shell on main (with no other branch) removes the previous media library, rows and files, before the bundle's media lands. See "Replace and the media library" below. --- @@ -229,20 +230,31 @@ type ImportStrategy = 'replace' | 'merge-add' | 'merge-overwrite' | Strategy | Tables | Rows | Media | Folders + redirects | |--------------------|-----------------------------------|-----------------------------------|------------------------------------|------------------------------| -| `replace` | Wipe + recreate from bundle | Wipe + recreate | Wipe + write all bytes | Wipe + recreate from bundle | +| `replace` | Wipe + recreate from bundle | Wipe + recreate | Wipe + write all bytes (conditions below) | Wipe + recreate from bundle | | `merge-add` | Skip if exists; add if new | Skip if id or active slug exists; add if new | Skip if exists; add if new | Left untouched | | `merge-overwrite` | Upsert (incoming wins) | Upsert (incoming wins) | Upsert (incoming bytes win) | Left untouched | Folder membership (`media_asset_folders`) is restored **after** the media bytes land, and only into folders the bundle actually carried. Redirect rows are inserted after their target rows exist (in `replace`, the rows' cascade-delete clears the old redirects first). Folders + redirects ride only the `replace` (full-restore) path because merging a folder tree or redirect set into a populated instance risks unique-key collisions — the same reason the site shell is `replace`/`merge-overwrite`-only. +### Replace and the media library + +Media is not branch-scoped, and records outside an import can reference it, so `replace` wipes the library only when nothing that survives the replace can (`replacesMediaLibrary` in `server/handlers/cms/import.ts`): + +- the import carries the media category (`media` present in the bundle or the selected archive manifest, even empty). A bundle exported without media leaves the library alone, like folders and redirects; +- it carries the site shell. A kept shell can reference media (favicon, `@font-face` sources, style-rule backgrounds); +- it runs on main and no other branch exists. Branches share main's library and their rows and shells reference it. + +When it applies, `deleteSiteMediaAssets` (`server/repositories/media.ts`) deletes every `media_assets` row, trashed ones included, inside the import transaction; folder memberships and usage refs cascade. It spares assets owned by records that survive a replace: user avatars (`users.avatar_media_id`) and plugin-managed media (`plugin_media_sources`). After the transaction commits, and before the bundle's media is written, `removeDeletedMediaFiles` sweeps each deleted asset's original and variants through its pinned storage adapter (`removeMediaAssetFiles`, the same sweep a single purge uses). A rolled-back import therefore never loses a file, and a bundle file at a deleted asset's storage path is written after the old bytes are gone. When the conditions do not hold, a bundle asset whose path is already taken by a kept asset still fails on `media_assets.public_path`. + The JSON handler: -1. Validates the bundle against `SiteBundleSchema`. -2. Runs the same preview logic to compute the change set. -3. Wraps the apply in a **single DB transaction**: +1. Checks the strategy gate (`authorizeImportStrategy`), validates the bundle against `SiteBundleSchema`, then checks the shell gate (`authorizeImportedShell`); see the capability gates below. +2. Calls `applySiteBundle`, which wraps the apply in a **single DB transaction** on the collab-aware write lane: - Tables: insert / update / delete per strategy. - Rows: insert / update / soft-delete per strategy. - Site shell: if the bundle carries `site`, overwrite the `site` row. + - `replace` only: the media library (see above), folders, and redirects. +3. Removes the bytes of media rows a `replace` deleted. 4. **Outside the transaction**, writes media bytes to `uploads/`. Before writing, the handler calls `assertPathWithin(uploadsDir, target)` (from `server/util/pathWithin.ts`) as a defense-in-depth check — the schema already forbids traversal, but the sink re-asserts containment after `path.join()` resolves symlinks. Media writes are best-effort — if a disk write fails, the row import has already committed and the asset is skipped with a log entry. 5. Returns `ImportResult` with counts. @@ -251,10 +263,11 @@ The archive handler: 1. Capability-gates on `data.import`. 2. Reads and validates the first ZIP entry as `.instatic/site-bundle.json`. 3. Applies the optional selection filter and row slug overrides to the manifest. -4. Streams selected media entries into temporary files, validating declared size, CRC descriptors, missing selected entries, and unexpected archive entries before data is mutated. -5. Delegates the manifest's site/data/folder/redirect content to the JSON import handler without media bytes. -6. Moves staged media into `uploads/` and upserts the matching media row. -7. Returns the same `ImportResult` shape with `mediaImported` from the staged entries. +4. Runs the same two capability gates (`authorizeImportStrategy`, then `authorizeImportedShell` for whether the selection carries the shell). +5. Streams selected media entries into temporary files, validating declared size, CRC descriptors, missing selected entries, and unexpected archive entries before data is mutated. +6. Applies the manifest's site/data/folder/redirect content through `applySiteBundle` (the JSON handler's transaction), stating whether the selection carries media, then removes the bytes of media rows a `replace` deleted. +7. Moves staged media into `uploads/` and upserts the matching media row. +8. Returns the same `ImportResult` shape with `mediaImported` from the staged entries. The transaction is **all-or-nothing** for the DB side — if any insert fails, the DB rolls back. Malformed archive media is rejected before the DB transaction starts. A disk failure while moving already-validated staged media can still leave media partially written after the data import, matching the JSON import handler's best-effort filesystem semantics. @@ -262,7 +275,7 @@ The transaction is **all-or-nothing** for the DB side — if any insert fails, t Per strategy: -- **`replace`**: incoming bundle wins for everything. +- **`replace`**: incoming bundle wins for everything (media under the conditions in "Replace and the media library"). - **`merge-add`**: existing rows / tables / media are untouched. Incoming-only items added. - **`merge-overwrite`**: matching ids upsert. Items not in the bundle are **kept** (vs. `replace` which deletes them). @@ -375,7 +388,7 @@ A nightly cron can hit `/admin/api/cms/export` with an admin session cookie and - Source-of-truth files: - `src/core/data/bundleSchema.ts` — `SiteBundleSchema`, `ImportStrategySchema`, `BundlePreviewSchema`, `ImportResultSchema` - `server/handlers/cms/export.ts` — `GET+POST /export` - - `server/handlers/cms/import.ts` — `POST /import` + - `server/handlers/cms/import.ts` — `POST /import`, plus `authorizeImportStrategy`, `authorizeImportedShell`, and `applySiteBundle` shared with the archive endpoint - `server/handlers/cms/importArchive.ts` — `POST /import/archive` - `server/handlers/cms/importPreview.ts` — `POST /import/preview` - `src/core/persistence/cmsTransfer.ts` — client-side transfer helpers, including native export form submission @@ -385,3 +398,4 @@ A nightly cron can hit `/admin/api/cms/export` with an admin session cookie and - `src/__tests__/architecture/cmsTransferPreview.test.ts` - `src/__tests__/architecture/cmsTransferImport.test.ts` - `src/__tests__/architecture/import-export-roundtrip.test.ts` + - `src/__tests__/server/importReplaceMedia.test.ts`: `replace` media wipe, same-path re-import, rollback keeps files, spared avatars, conditions, merge strategies untouched diff --git a/server/handlers/cms/import.ts b/server/handlers/cms/import.ts index 9ff2275d6..4895670b0 100644 --- a/server/handlers/cms/import.ts +++ b/server/handlers/cms/import.ts @@ -16,6 +16,16 @@ * the transaction (filesystem); individual media failures log and continue * without aborting the import. * + * `replace` also wipes the media library when the import replaces everything + * that can reference it (`replacesMediaLibrary`). The rows go inside the + * transaction; their stored bytes are removed after it commits and BEFORE the + * bundle's media is written, so a bundle file at a deleted asset's storage + * path lands after the old one is gone. + * + * `applySiteBundle` is the DB half both import endpoints share: this JSON + * endpoint and `/import/archive` (`./importArchive.ts`), which streams media + * from the ZIP instead of Base64. + * * Capability matrix (G6 fix — was a single `site.structure.edit` for * everything, which let a Designer with structure-edit but no * content rights wipe every row via the import endpoint): @@ -35,6 +45,7 @@ import { resolveMediaWriteTarget, } from './importMediaValidation' import { requireCapability, requireStepUp, userHasCapability } from '../../auth/authz' +import type { AuthUser } from '../../repositories/users' import { saveDraftSite } from '../../repositories/site' import { listDataTables, @@ -49,7 +60,14 @@ import { replaceDataRow, type DataRowImportInput, } from '../../repositories/data/rows' -import { importMediaAsset, assignAssetToFolders } from '../../repositories/media' +import { + importMediaAsset, + assignAssetToFolders, + deleteSiteMediaAssets, + type DeletedMediaAsset, +} from '../../repositories/media' +import { listBranches } from '../../repositories/branches' +import { removeMediaAssetFiles } from './mediaVariants' import { deleteAllDataRowRedirects, importDataRowRedirect, @@ -62,6 +80,7 @@ import { ImportStrategySchema, ImportResultSchema, type ImportStrategy, + type SiteBundle, } from '@core/data/bundleSchema' import { CMS_API_PREFIX, type CmsHandlerOptions } from './shared' import { @@ -70,7 +89,8 @@ import { serializeCollabAwareWrite, type RowWriteKind, } from '../../repositories/rowWriteEvents' -import type { BranchScope } from '../../branches/scope' +import { isMainScope, type BranchScope } from '../../branches/scope' +import { MAIN_BRANCH_ID } from '@core/branches' // The four system table ids that are always seeded and never deleted. const SYSTEM_TABLE_IDS = new Set(['posts', 'pages', 'components', 'layouts']) @@ -133,17 +153,8 @@ export async function handleImportRoute( ) } - // `replace` strategy = wipe every data row and reinsert. Highest-blast - // radius operation in the CMS. Require `content.manage` (so a caller - // with `data.import` but no content rights can still merge-add but not - // wipe) AND step-up (mirrors users.ts delete / publish.ts publish). - if (strategy === 'replace') { - if (!userHasCapability(user, 'content.manage')) { - return jsonResponse({ error: 'Forbidden' }, { status: 403 }) - } - const stepUp = await requireStepUp(req, db, user) - if (stepUp) return stepUp - } + const strategyDenied = await authorizeImportStrategy(req, db, user, strategy) + if (strategyDenied) return strategyDenied // Parse and validate the bundle body. This endpoint takes the JSON bundle; // the ZIP archive the export UI downloads belongs to /import/archive. Sent @@ -166,35 +177,219 @@ export async function handleImportRoute( return jsonResponse({ error: 'Invalid bundle: body does not conform to SiteBundleSchema' }, { status: 400 }) } - // Bundles that carry a site shell additionally require - // `site.structure.edit` — replacing the shell is a structural site edit - // even when the import strategy is merge-overwrite. - if (bundle.site) { - if (!userHasCapability(user, 'site.structure.edit')) { - return jsonResponse({ error: 'Forbidden' }, { status: 403 }) - } - } + const shellDenied = authorizeImportedShell(user, bundle.site !== undefined) + if (shellDenied) return shellDenied + + const applied = await applySiteBundle(db, scope, { + bundle, + strategy, + carriesMedia: bundle.media !== undefined, + }) + // After commit, before the bundle's media lands: a bundle file at a deleted + // asset's storage path must be written after the old bytes are removed. + await removeDeletedMediaFiles(applied.deletedMedia) + + let mediaImported = 0 // --------------------------------------------------------------------------- - // Counters + // Media — outside the DB transaction (filesystem writes) // --------------------------------------------------------------------------- + if (bundle.media && bundle.media.length > 0 && options.uploadsDir) { + const uploadsDir = options.uploadsDir + await mkdir(uploadsDir, { recursive: true }) + + for (const asset of bundle.media) { + try { + // Same security gate the archive import and upload pipeline apply: + // - destination: no traversal, never a reserved served subtree + // (published/plugins/fonts) — `resolveMediaWriteTarget`; + // - content: real MIME must match the declared type + extension and + // SVG is sanitised — `validateAndSanitizeMediaBytes`. + // Without this, a `data.import` caller could write arbitrary HTML/JS + // (e.g. `published/current/index.html`) served same-origin as /admin. + const target = resolveMediaWriteTarget(uploadsDir, asset.storagePath) + const safeBytes = validateAndSanitizeMediaBytes( + Buffer.from(asset.bytesBase64, 'base64'), + asset, + ) + await mkdir(dirname(target), { recursive: true }) + await writeFile(target, safeBytes) + + // Upsert the media_assets row + await importMediaAsset(db, { + id: asset.id, + filename: asset.filename, + mimeType: asset.mimeType, + sizeBytes: safeBytes.length, + storagePath: asset.storagePath, + publicPath: `/uploads/${asset.storagePath}`, + altText: asset.altText, + caption: asset.caption, + title: asset.title, + tags: asset.tags, + width: asset.width, + height: asset.height, + durationMs: asset.durationMs, + dominantColor: asset.dominantColor, + blurHash: asset.blurHash, + posterPath: asset.posterPath, + }) + + // Restore folder membership — but only into folders we actually + // imported, so a stale folderId can't violate the membership FK. + const targetFolders = asset.folderIds.filter((id) => applied.importedFolderIds.has(id)) + if (targetFolders.length > 0) { + await assignAssetToFolders(db, asset.id, { add: targetFolders }) + } + + mediaImported++ + } catch (err) { + console.error('[import] Failed to import media asset:', asset.id, err) + // Continue with remaining assets — a single failed asset should not + // abort the whole import (data is already committed). + } + } + } + + const result = { + ok: true as const, + strategy, + tablesAffected: applied.tablesAffected, + rowsInserted: applied.rowsInserted, + rowsReplaced: applied.rowsReplaced, + rowsSkipped: applied.rowsSkipped, + mediaImported, + mediaFoldersImported: applied.mediaFoldersImported, + redirectsImported: applied.redirectsImported, + } + + // Paranoia: validate result shape before returning + parseValue(ImportResultSchema, result) + + return jsonResponse(result) +} + +/** + * Strategy gate every bundle import passes on top of `data.import` (G6 fix — + * was a single `site.structure.edit` for everything, which let a Designer + * with structure-edit but no content rights wipe every row via the import + * endpoint). Checked before the body is read. Shared with the archive + * endpoint. Returns the error response, or `null` to proceed. + * + * `replace` = wipe every data row and reinsert, the highest-blast-radius + * operation in the CMS. It requires `content.manage` (so a caller with + * `data.import` but no content rights can still merge-add but not wipe) AND + * step-up (mirrors users.ts delete / publish.ts publish). + */ +export async function authorizeImportStrategy( + req: Request, + db: DbClient, + user: AuthUser, + strategy: ImportStrategy, +): Promise { + if (strategy !== 'replace') return null + if (!userHasCapability(user, 'content.manage')) { + return jsonResponse({ error: 'Forbidden' }, { status: 403 }) + } + return requireStepUp(req, db, user) +} + +/** + * Bundles that carry a site shell additionally require `site.structure.edit`: + * replacing the shell is a structural site edit even when the strategy is + * merge-overwrite. Shared with the archive endpoint. + */ +export function authorizeImportedShell(user: AuthUser, carriesSite: boolean): Response | null { + if (carriesSite && !userHasCapability(user, 'site.structure.edit')) { + return jsonResponse({ error: 'Forbidden' }, { status: 403 }) + } + return null +} + +export interface ApplySiteBundleInput { + /** The bundle's data. Media bytes, when present, are not read here. */ + bundle: SiteBundle + strategy: ImportStrategy + /** + * The import carries the media category (the bundle lists media, possibly + * none). The archive endpoint strips media from `bundle` and streams the + * bytes itself, so it states this separately. + */ + carriesMedia: boolean +} + +export interface AppliedSiteBundle { + tablesAffected: number + rowsInserted: number + rowsReplaced: number + rowsSkipped: number + mediaFoldersImported: number + redirectsImported: number + /** + * Folder ids that actually landed — asset memberships are restored only for + * these, so an asset's `folderIds` pointing at a folder we didn't import is + * skipped rather than violating the membership foreign key. + */ + importedFolderIds: Set + /** Media rows a `replace` deleted; their bytes are the caller's to remove. */ + deletedMedia: DeletedMediaAsset[] +} + +/** + * Whether a `replace` import also replaces the media library. Media is not + * branch-scoped, and records outside the import can reference it, so the + * library is wiped only when nothing that survives the replace can: + * + * - the import carries media at all (a bundle exported without media says + * nothing about the library, like folders and redirects); + * - it carries the site shell (a kept shell can reference media: favicon, + * `@font-face` sources, background images in style rules); + * - it runs on main and no other branch exists (branches share main's + * library and their rows and shells reference it). + * + * Assets owned by surviving records (avatars, plugin-managed media) are + * spared by `deleteSiteMediaAssets` itself. + */ +async function replacesMediaLibrary( + db: DbClient, + scope: BranchScope, + input: ApplySiteBundleInput, +): Promise { + if (!input.carriesMedia || input.bundle.site === undefined) return false + if (!isMainScope(scope)) return false + const branches = await listBranches(db) + return branches.every((branch) => branch.id === MAIN_BRANCH_ID) +} + +/** + * Remove the bytes of media rows a committed `replace` deleted. Call it after + * `applySiteBundle` returns and before writing the bundle's media. + */ +export async function removeDeletedMediaFiles(deleted: readonly DeletedMediaAsset[]): Promise { + for (const asset of deleted) await removeMediaAssetFiles(asset) +} + +/** + * Apply a bundle's site, tables, rows, folders, and redirects (and, for + * `replace`, delete the media library) per `strategy`, in one transaction on + * the collab-aware write lane, then notify open editors. Media bytes and + * media rows are written by the caller afterwards. + */ +export async function applySiteBundle( + db: DbClient, + scope: BranchScope, + input: ApplySiteBundleInput, +): Promise { + const { bundle, strategy } = input let tablesAffected = 0 let rowsInserted = 0 let rowsReplaced = 0 let rowsSkipped = 0 - let mediaImported = 0 let mediaFoldersImported = 0 let redirectsImported = 0 - - // Folder ids that actually landed — asset memberships are restored only for - // these, so an asset's `folderIds` pointing at a folder we didn't import is - // silently skipped rather than violating the membership foreign key. + let deletedMedia: DeletedMediaAsset[] = [] const importedFolderIds = new Set() - // --------------------------------------------------------------------------- - // DB transaction - // --------------------------------------------------------------------------- - await serializeCollabAwareWrite(async () => { const affectedCollabRows = new Map( ['pages', 'components', 'layouts'].map((tableId) => [tableId, new Set()]), @@ -214,44 +409,123 @@ export async function handleImportRoute( } } - if (strategy === 'replace') { - // Wipe-and-replace: delete all rows + custom tables, then reimport. - await db.transaction(async (tx) => { - // 1. Delete ALL of this branch's data rows (covers all tables) - await tx`delete from data_rows where branch_id = ${scope.branchId}` - - // 2. Delete this branch's non-system data tables - await tx` - delete from data_tables - where branch_id = ${scope.branchId} - and (system = 0 or system = false) - ` - - // 3. Load remaining system tables so we know which bundle tables to - // update vs insert. - const existingTables = await listDataTables(tx, scope) - const existingTableIds = new Set(existingTables.map((t) => t.id)) - - // 4. Upsert tables from the bundle - for (const table of bundle.tables) { - if (existingTableIds.has(table.id)) { - // System table already present — update its fields - await updateDataTable(tx, scope, table.id, { - name: table.name, - slug: table.slug, - routeBase: table.routeBase, - singularLabel: table.singularLabel, - pluralLabel: table.pluralLabel, - primaryFieldId: table.primaryFieldId, - fields: table.fields, - }) - tablesAffected++ - } else if (!SYSTEM_TABLE_IDS.has(table.id)) { - // Custom table — insert with original id. `createdByPluginId` is - // access-control state (the `@own-created` contentAccess marker - // resolves against it), so a restore keeps plugin-created tables - // owned by their plugin. - await createDataTable(tx, scope, { + if (strategy === 'replace') { + // Wipe-and-replace: delete all rows + custom tables, then reimport. + await db.transaction(async (tx) => { + // 1. Delete ALL of this branch's data rows (covers all tables) + await tx`delete from data_rows where branch_id = ${scope.branchId}` + + // 2. Delete this branch's non-system data tables. The flag is bound, + // not written as a literal: `system = 0` is `boolean = integer` on + // Postgres (an error), and the SQLite adapter binds `false` as 0. + await tx` + delete from data_tables + where branch_id = ${scope.branchId} + and system = ${false} + ` + + // 3. Load remaining system tables so we know which bundle tables to + // update vs insert. + const existingTables = await listDataTables(tx, scope) + const existingTableIds = new Set(existingTables.map((t) => t.id)) + + // 4. Upsert tables from the bundle + for (const table of bundle.tables) { + if (existingTableIds.has(table.id)) { + // System table already present — update its fields + await updateDataTable(tx, scope, table.id, { + name: table.name, + slug: table.slug, + routeBase: table.routeBase, + singularLabel: table.singularLabel, + pluralLabel: table.pluralLabel, + primaryFieldId: table.primaryFieldId, + fields: table.fields, + }) + tablesAffected++ + } else if (!SYSTEM_TABLE_IDS.has(table.id)) { + // Custom table — insert with original id. `createdByPluginId` is + // access-control state (the `@own-created` contentAccess marker + // resolves against it), so a restore keeps plugin-created tables + // owned by their plugin. + await createDataTable(tx, scope, { + id: table.id, + name: table.name, + slug: table.slug, + kind: table.kind, + routeBase: table.routeBase, + singularLabel: table.singularLabel, + pluralLabel: table.pluralLabel, + primaryFieldId: table.primaryFieldId, + fields: table.fields, + createdByPluginId: table.createdByPluginId ?? null, + }) + tablesAffected++ + } + // Bundle tables whose id is a known SYSTEM_TABLE_ID but wasn't seeded + // locally are silently skipped — they should never occur in practice. + } + + // 5. Insert all rows (plain insert — table was just wiped) + for (const row of bundle.rows) { + const input: DataRowImportInput = { + id: row.id, + tableId: row.tableId, + cells: row.cells, + slug: row.slug, + status: row.status, + publishedAt: row.publishedAt, + createdAt: row.createdAt, + updatedAt: row.updatedAt, + } + await replaceDataRow(tx, scope, input) + affectedCollabRows.get(row.tableId)?.add(row.id) + rowsInserted++ + } + + // 6. Replace the site shell (only when the bundle carries one) + if (bundle.site) { + await saveDraftSite(tx, scope, bundle.site, null, { collabInternal: true }) + shellWasWritten = true + } + + // 7. Media library. Wiped only when this import replaces everything + // that can reference it (see `replacesMediaLibrary`). Rows go + // here, inside the transaction; their bytes are removed by the + // caller after commit, so a rolled-back import never loses a file. + if (await replacesMediaLibrary(tx, scope, input)) { + deletedMedia = await deleteSiteMediaAssets(tx) + } + + // 8. Media folder tree. `delete from data_rows` above does NOT touch + // media_folders (unrelated FK), so wipe explicitly, then insert + // parent-first to satisfy the self-referencing parent_id FK. + if (bundle.mediaFolders) { + await deleteAllMediaFolders(tx) + for (const folder of orderFoldersParentFirst(bundle.mediaFolders)) { + await importMediaFolder(tx, folder) + importedFolderIds.add(folder.id) + mediaFoldersImported++ + } + } + + // 9. Redirects. Old ones already cascade-deleted with their target rows + // in step 1; wipe explicitly for clarity, then reinsert from the + // bundle now that the target rows exist. + if (bundle.redirects) { + await deleteAllDataRowRedirects(tx) + for (const redirect of bundle.redirects) { + await importDataRowRedirect(tx, redirect) + redirectsImported++ + } + } + }) + } else if (strategy === 'merge-add') { + // Add what's missing; never overwrite existing content. + await db.transaction(async (tx) => { + // Tables: insert if absent, skip if the id already exists + for (const table of bundle.tables) { + const inserted = await insertDataTableIfAbsent(tx, scope, { id: table.id, name: table.name, slug: table.slug, @@ -263,178 +537,109 @@ export async function handleImportRoute( fields: table.fields, createdByPluginId: table.createdByPluginId ?? null, }) - tablesAffected++ + if (inserted) tablesAffected++ } - // Bundle tables whose id is a known SYSTEM_TABLE_ID but wasn't seeded - // locally are silently skipped — they should never occur in practice. - } - // 5. Insert all rows (plain insert — table was just wiped) - for (const row of bundle.rows) { - const input: DataRowImportInput = { - id: row.id, - tableId: row.tableId, - cells: row.cells, - slug: row.slug, - status: row.status, - publishedAt: row.publishedAt, - createdAt: row.createdAt, - updatedAt: row.updatedAt, + // Rows: insert if absent, skip if the id already exists + for (const row of bundle.rows) { + const input: DataRowImportInput = { + id: row.id, + tableId: row.tableId, + cells: row.cells, + slug: row.slug, + status: row.status, + publishedAt: row.publishedAt, + createdAt: row.createdAt, + updatedAt: row.updatedAt, + } + const inserted = await insertDataRowIfAbsent(tx, scope, input) + if (inserted) { + affectedCollabRows.get(row.tableId)?.add(row.id) + rowsInserted++ + } else { + rowsSkipped++ + } } - await replaceDataRow(tx, scope, input) - affectedCollabRows.get(row.tableId)?.add(row.id) - rowsInserted++ - } - - // 6. Replace the site shell (only when the bundle carries one) - if (bundle.site) { - await saveDraftSite(tx, scope, bundle.site, null, { collabInternal: true }) - shellWasWritten = true - } - - // 7. Media folder tree. `delete from data_rows` above does NOT touch - // media_folders (unrelated FK), so wipe explicitly, then insert - // parent-first to satisfy the self-referencing parent_id FK. - if (bundle.mediaFolders) { - await deleteAllMediaFolders(tx) - for (const folder of orderFoldersParentFirst(bundle.mediaFolders)) { - await importMediaFolder(tx, folder) - importedFolderIds.add(folder.id) - mediaFoldersImported++ - } - } - - // 8. Redirects. Old ones already cascade-deleted with their target rows - // in step 1; wipe explicitly for clarity, then reinsert from the - // bundle now that the target rows exist. - if (bundle.redirects) { - await deleteAllDataRowRedirects(tx) - for (const redirect of bundle.redirects) { - await importDataRowRedirect(tx, redirect) - redirectsImported++ - } - } - }) - } else if (strategy === 'merge-add') { - // Add what's missing; never overwrite existing content. - await db.transaction(async (tx) => { - // Tables: insert if absent, skip if the id already exists - for (const table of bundle.tables) { - const inserted = await insertDataTableIfAbsent(tx, scope, { - id: table.id, - name: table.name, - slug: table.slug, - kind: table.kind, - routeBase: table.routeBase, - singularLabel: table.singularLabel, - pluralLabel: table.pluralLabel, - primaryFieldId: table.primaryFieldId, - fields: table.fields, - createdByPluginId: table.createdByPluginId ?? null, - }) - if (inserted) tablesAffected++ - } - // Rows: insert if absent, skip if the id already exists - for (const row of bundle.rows) { - const input: DataRowImportInput = { - id: row.id, - tableId: row.tableId, - cells: row.cells, - slug: row.slug, - status: row.status, - publishedAt: row.publishedAt, - createdAt: row.createdAt, - updatedAt: row.updatedAt, - } - const inserted = await insertDataRowIfAbsent(tx, scope, input) - if (inserted) { - affectedCollabRows.get(row.tableId)?.add(row.id) - rowsInserted++ - } else { - rowsSkipped++ + // Site shell: merge-add never overwrites the site shell — the local site + // has existing content that must not be destroyed by an additive import. + }) + } else { + // merge-overwrite: upsert rows/tables; update existing with bundle values. + await db.transaction(async (tx) => { + // Pre-fetch all existing rows. An upsert can move an id from a table the + // bundle does not mention, and both its old and new collab docs/rosters + // must be invalidated after commit. + const existingRowTables = new Map() + for (const table of await listDataTables(tx, scope)) { + const existing = await listDataRows(tx, scope, table.id) + for (const row of existing) existingRowTables.set(row.id, row.tableId) } - } - - // Site shell: merge-add never overwrites the site shell — the local site - // has existing content that must not be destroyed by an additive import. - }) - } else { - // merge-overwrite: upsert rows/tables; update existing with bundle values. - await db.transaction(async (tx) => { - // Pre-fetch all existing rows. An upsert can move an id from a table the - // bundle does not mention, and both its old and new collab docs/rosters - // must be invalidated after commit. - const existingRowTables = new Map() - for (const table of await listDataTables(tx, scope)) { - const existing = await listDataRows(tx, scope, table.id) - for (const row of existing) existingRowTables.set(row.id, row.tableId) - } - // Tables: insert if absent, update if already present - for (const table of bundle.tables) { - const inserted = await insertDataTableIfAbsent(tx, scope, { - id: table.id, - name: table.name, - slug: table.slug, - kind: table.kind, - routeBase: table.routeBase, - singularLabel: table.singularLabel, - pluralLabel: table.pluralLabel, - primaryFieldId: table.primaryFieldId, - fields: table.fields, - createdByPluginId: table.createdByPluginId ?? null, - }) - if (!inserted) { - await updateDataTable(tx, scope, table.id, { + // Tables: insert if absent, update if already present + for (const table of bundle.tables) { + const inserted = await insertDataTableIfAbsent(tx, scope, { + id: table.id, name: table.name, slug: table.slug, + kind: table.kind, routeBase: table.routeBase, singularLabel: table.singularLabel, pluralLabel: table.pluralLabel, primaryFieldId: table.primaryFieldId, fields: table.fields, + createdByPluginId: table.createdByPluginId ?? null, }) + if (!inserted) { + await updateDataTable(tx, scope, table.id, { + name: table.name, + slug: table.slug, + routeBase: table.routeBase, + singularLabel: table.singularLabel, + pluralLabel: table.pluralLabel, + primaryFieldId: table.primaryFieldId, + fields: table.fields, + }) + } + tablesAffected++ } - tablesAffected++ - } - // Rows: upsert all; track inserted vs replaced via the pre-fetched set - for (const row of bundle.rows) { - const input: DataRowImportInput = { - id: row.id, - tableId: row.tableId, - cells: row.cells, - slug: row.slug, - status: row.status, - publishedAt: row.publishedAt, - createdAt: row.createdAt, - updatedAt: row.updatedAt, + // Rows: upsert all; track inserted vs replaced via the pre-fetched set + for (const row of bundle.rows) { + const input: DataRowImportInput = { + id: row.id, + tableId: row.tableId, + cells: row.cells, + slug: row.slug, + status: row.status, + publishedAt: row.publishedAt, + createdAt: row.createdAt, + updatedAt: row.updatedAt, + } + await upsertDataRow(tx, scope, input) + const previousTableId = existingRowTables.get(row.id) + if (previousTableId === undefined) { + createdCollabRows.get(row.tableId)?.add(row.id) + } else if (previousTableId !== row.tableId) { + removedCollabRows.get(previousTableId)?.add(row.id) + createdCollabRows.get(row.tableId)?.add(row.id) + } else { + affectedCollabRows.get(row.tableId)?.add(row.id) + } + if (previousTableId !== undefined) { + rowsReplaced++ + } else { + rowsInserted++ + } } - await upsertDataRow(tx, scope, input) - const previousTableId = existingRowTables.get(row.id) - if (previousTableId === undefined) { - createdCollabRows.get(row.tableId)?.add(row.id) - } else if (previousTableId !== row.tableId) { - removedCollabRows.get(previousTableId)?.add(row.id) - createdCollabRows.get(row.tableId)?.add(row.id) - } else { - affectedCollabRows.get(row.tableId)?.add(row.id) - } - if (previousTableId !== undefined) { - rowsReplaced++ - } else { - rowsInserted++ - } - } - // Site shell: overwrite if the bundle carries one - if (bundle.site) { - await saveDraftSite(tx, scope, bundle.site, null, { collabInternal: true }) - shellWasWritten = true - } - }) - } + // Site shell: overwrite if the bundle carries one + if (bundle.site) { + await saveDraftSite(tx, scope, bundle.site, null, { collabInternal: true }) + shellWasWritten = true + } + }) + } const branchId = scope.branchId if (shellWasWritten) notifyShellWrite(branchId) @@ -456,80 +661,14 @@ export async function handleImportRoute( } }) - // --------------------------------------------------------------------------- - // Media — outside the DB transaction (filesystem writes) - // --------------------------------------------------------------------------- - if (bundle.media && bundle.media.length > 0 && options.uploadsDir) { - const uploadsDir = options.uploadsDir - await mkdir(uploadsDir, { recursive: true }) - - for (const asset of bundle.media) { - try { - // Same security gate the archive import and upload pipeline apply: - // - destination: no traversal, never a reserved served subtree - // (published/plugins/fonts) — `resolveMediaWriteTarget`; - // - content: real MIME must match the declared type + extension and - // SVG is sanitised — `validateAndSanitizeMediaBytes`. - // Without this, a `data.import` caller could write arbitrary HTML/JS - // (e.g. `published/current/index.html`) served same-origin as /admin. - const target = resolveMediaWriteTarget(uploadsDir, asset.storagePath) - const safeBytes = validateAndSanitizeMediaBytes( - Buffer.from(asset.bytesBase64, 'base64'), - asset, - ) - await mkdir(dirname(target), { recursive: true }) - await writeFile(target, safeBytes) - - // Upsert the media_assets row - await importMediaAsset(db, { - id: asset.id, - filename: asset.filename, - mimeType: asset.mimeType, - sizeBytes: safeBytes.length, - storagePath: asset.storagePath, - publicPath: `/uploads/${asset.storagePath}`, - altText: asset.altText, - caption: asset.caption, - title: asset.title, - tags: asset.tags, - width: asset.width, - height: asset.height, - durationMs: asset.durationMs, - dominantColor: asset.dominantColor, - blurHash: asset.blurHash, - posterPath: asset.posterPath, - }) - - // Restore folder membership — but only into folders we actually - // imported, so a stale folderId can't violate the membership FK. - const targetFolders = asset.folderIds.filter((id) => importedFolderIds.has(id)) - if (targetFolders.length > 0) { - await assignAssetToFolders(db, asset.id, { add: targetFolders }) - } - - mediaImported++ - } catch (err) { - console.error('[import] Failed to import media asset:', asset.id, err) - // Continue with remaining assets — a single failed asset should not - // abort the whole import (data is already committed). - } - } - } - - const result = { - ok: true as const, - strategy, + return { tablesAffected, rowsInserted, rowsReplaced, rowsSkipped, - mediaImported, mediaFoldersImported, redirectsImported, + importedFolderIds, + deletedMedia, } - - // Paranoia: validate result shape before returning - parseValue(ImportResultSchema, result) - - return jsonResponse(result) } diff --git a/server/handlers/cms/importArchive.ts b/server/handlers/cms/importArchive.ts index 7a01cb843..b996d0efe 100644 --- a/server/handlers/cms/importArchive.ts +++ b/server/handlers/cms/importArchive.ts @@ -7,7 +7,10 @@ * The manifest is the first stored entry, so the handler can validate the * selected media stream before applying site data. Selected media bytes are * staged in temporary files first, then moved into uploads after the data - * import succeeds. + * import succeeds. The data half (capability gates, the transaction, a + * `replace` media wipe) is the JSON endpoint's, shared through + * `authorizeImportStrategy`, `authorizeImportedShell`, and `applySiteBundle` + * in `./import.ts`. */ import { createWriteStream } from 'node:fs' @@ -40,7 +43,12 @@ import { import { createCrc32 } from '../../archive/storedZip' import { CMS_API_PREFIX, type CmsHandlerOptions } from './shared' import type { BranchScope } from '../../branches/scope' -import { handleImportRoute } from './import' +import { + applySiteBundle, + authorizeImportedShell, + authorizeImportStrategy, + removeDeletedMediaFiles, +} from './import' const IMPORT_ARCHIVE_PATH = `${CMS_API_PREFIX}/import/archive` const ZIP_LOCAL_FILE_HEADER = 0x04034b50 @@ -99,6 +107,8 @@ export async function handleImportArchiveRoute( const strategy = parseImportStrategy(url) if (strategy instanceof Response) return strategy + const strategyDenied = await authorizeImportStrategy(req, db, user, strategy) + if (strategyDenied) return strategyDenied const selection = parseImportSelection(url) if (selection instanceof Response) return selection @@ -106,6 +116,9 @@ export async function handleImportArchiveRoute( ? filterArchiveManifestForSelection(manifest, selection) : manifest + const shellDenied = authorizeImportedShell(user, selectedManifest.site !== undefined) + if (shellDenied) return shellDenied + let stagedMedia: StagedArchiveMedia | null = null if (selectedManifest.media && selectedManifest.media.length > 0 && options.uploadsDir) { try { @@ -120,19 +133,14 @@ export async function handleImportArchiveRoute( } try { - const dataBundle = siteBundleWithoutMediaBytes(selectedManifest) - const dataImportReq = makeInternalImportRequest(req, strategy, dataBundle) - const dataImportRes = await handleImportRoute(dataImportReq, db, scope, options) - if (!dataImportRes || !dataImportRes.ok) { - return dataImportRes ?? jsonResponse({ error: 'Import route did not handle archive manifest' }, { status: 500 }) - } - - const baseResult = parseValue(ImportResultSchema, await dataImportRes.json()) - const importedFolderIds = new Set( - strategy === 'replace' - ? selectedManifest.mediaFolders?.map((folder) => folder.id) ?? [] - : [], - ) + const applied = await applySiteBundle(db, scope, { + bundle: siteBundleWithoutMediaBytes(selectedManifest), + strategy, + carriesMedia: selectedManifest.media !== undefined, + }) + // After commit, before staged media moves in: a bundle file at a deleted + // asset's storage path must land after the old bytes are removed. + await removeDeletedMediaFiles(applied.deletedMedia) let mediaImported = 0 if (stagedMedia && options.uploadsDir) { @@ -141,7 +149,7 @@ export async function handleImportArchiveRoute( stagedMedia, db, uploadsDir: options.uploadsDir, - importedFolderIds, + importedFolderIds: applied.importedFolderIds, }) } catch (err) { return badRequest(err instanceof Error ? err.message : 'Invalid CMS bundle archive') @@ -149,8 +157,15 @@ export async function handleImportArchiveRoute( } const result: ImportResult = { - ...baseResult, + ok: true, + strategy, + tablesAffected: applied.tablesAffected, + rowsInserted: applied.rowsInserted, + rowsReplaced: applied.rowsReplaced, + rowsSkipped: applied.rowsSkipped, mediaImported, + mediaFoldersImported: applied.mediaFoldersImported, + redirectsImported: applied.redirectsImported, } parseValue(ImportResultSchema, result) return jsonResponse(result) @@ -382,22 +397,6 @@ function parseImportSelection(url: URL): BundleImportSelection | null | Response } } -function makeInternalImportRequest(req: Request, strategy: ImportStrategy, bundle: SiteBundle): Request { - const url = new URL(`${CMS_API_PREFIX}/import`, req.url) - url.searchParams.set('strategy', strategy) - const headers = new Headers(req.headers) - headers.set('content-type', 'application/json') - headers.delete('content-length') - const internalReq = new Request(url, { - method: 'POST', - headers, - body: JSON.stringify(bundle), - }) - const cookie = req.headers.get('cookie') - if (cookie) internalReq.headers.set('cookie', cookie) - return internalReq -} - function siteBundleWithoutMediaBytes(manifest: SiteBundleArchiveManifest): SiteBundle { const bundle: Partial = { ...manifest } delete bundle.media diff --git a/server/handlers/cms/media.ts b/server/handlers/cms/media.ts index 00c122be2..949162c50 100644 --- a/server/handlers/cms/media.ts +++ b/server/handlers/cms/media.ts @@ -59,8 +59,7 @@ import { acceptUploadedMedia, readUploadForm, } from './mediaUpload' -import { removeVariantFiles } from './mediaVariants' -import { dispatchDelete } from './mediaUploadDispatch' +import { removeMediaAssetFiles } from './mediaVariants' import { materializeAssetListForClient } from '../../publish/mediaPresentation' const MEDIA_LIBRARY_MIMES = Object.keys(EXTENSION_FOR_MIME) as Array< @@ -273,17 +272,10 @@ async function handleDeleteMedia( if (!existing) return notFound() if (!existing.deletedAt) return badRequest('Asset must be soft-deleted before purge') - // Snapshot the variant list BEFORE the row delete so we know which - // extra bytes to sweep from each variant's adapter alongside the original. - const variants = existing.variants - const adapterId = existing.storageAdapterId const deleted = await deleteMediaAsset(db, params.id) if (!deleted) return notFound() - await dispatchDelete(adapterId, deleted.storagePath).catch((err) => { - console.error('[media] hard-delete original byte sweep failed (orphaned bytes):', err) - }) - await removeVariantFiles(variants) + await removeMediaAssetFiles(deleted) return jsonResponse({ ok: true }) } diff --git a/server/handlers/cms/mediaVariants.ts b/server/handlers/cms/mediaVariants.ts index 225c5f6ba..00830b57e 100644 --- a/server/handlers/cms/mediaVariants.ts +++ b/server/handlers/cms/mediaVariants.ts @@ -27,6 +27,7 @@ * adapter-agnostic; this module never touches the filesystem directly. */ import type { DbClient } from '../../db/client' +import type { DeletedMediaAsset } from '../../repositories/media' import { dispatchDelete, dispatchUpload } from './mediaUploadDispatch' import { getElectedVariantDelegate, type ElectedVariantDelegate } from '../../repositories/mediaStorageAdapters' import { runImageVariantJob, isImageVariantOk } from './imageVariantWorkerHost' @@ -320,3 +321,18 @@ export async function removeVariantFiles( } } } + +/** + * Sweep a hard-deleted asset's bytes: the original through the adapter pinned + * on its row, then every variant. Call it only after the row delete has + * committed. Best-effort like `removeVariantFiles`: a failed delete logs and + * leaves orphaned bytes, never throws. + */ +export async function removeMediaAssetFiles(asset: DeletedMediaAsset): Promise { + try { + await dispatchDelete(asset.storageAdapterId, asset.storagePath) + } catch (err) { + console.error('[mediaVariants] original delete failed (orphaned bytes):', err) + } + await removeVariantFiles(asset.variants) +} diff --git a/server/repositories/media.ts b/server/repositories/media.ts index 5878f6702..8079cd138 100644 --- a/server/repositories/media.ts +++ b/server/repositories/media.ts @@ -46,6 +46,27 @@ export interface UpdateMediaAssetMetadataInput { interface DeletedMediaAssetRow { storage_path: string + storage_adapter_id: string + variants_json: unknown +} + +/** + * What a hard delete leaves for the caller to sweep from storage once the row + * delete has committed: the original through the adapter pinned on the row, + * plus every responsive variant (each pins its own adapter). + */ +export interface DeletedMediaAsset { + storagePath: string + storageAdapterId: string + variants: MediaVariant[] +} + +function mapDeletedMediaAssetRow(row: DeletedMediaAssetRow): DeletedMediaAsset { + return { + storagePath: row.storage_path, + storageAdapterId: row.storage_adapter_id, + variants: parseVariants(row.variants_json), + } } /** @@ -297,20 +318,47 @@ export async function restoreMediaAsset( } /** - * Hard delete — removes the row. Caller is responsible for removing the - * on-disk file using the returned `storagePath`. + * Hard delete — removes the row (folder memberships cascade). Caller is + * responsible for removing the stored bytes the returned value names, after + * the delete commits. */ export async function deleteMediaAsset( db: DbClient, id: string, -): Promise<{ storagePath: string } | null> { +): Promise { const { rows } = await db` delete from media_assets where id = ${id} - returning storage_path + returning storage_path, storage_adapter_id, variants_json ` const row = rows[0] - return row ? { storagePath: row.storage_path } : null + return row ? mapDeletedMediaAssetRow(row) : null +} + +/** + * Hard-delete the site's media library (every asset, trashed ones included) + * for a `replace` import. Spares the assets that records surviving a replace + * own: a user's avatar (`users.avatar_media_id`) and plugin-managed media + * (`plugin_media_sources`, whose provenance row would cascade away with it). + * Folder memberships and usage refs cascade with the deleted rows. + * + * Runs inside the import transaction; the caller removes the returned bytes + * only after it commits, so a rolled-back import never loses a file. + * `not exists` rather than `not in`: `avatar_media_id` is nullable, and + * `not in` over a set holding NULL matches nothing on either dialect. + */ +export async function deleteSiteMediaAssets(db: DbClient): Promise { + const { rows } = await db` + delete from media_assets + where not exists ( + select 1 from users where users.avatar_media_id = media_assets.id + ) + and not exists ( + select 1 from plugin_media_sources where plugin_media_sources.asset_id = media_assets.id + ) + returning storage_path, storage_adapter_id, variants_json + ` + return rows.map(mapDeletedMediaAssetRow) } /** diff --git a/src/__tests__/server/importReplaceMedia.test.ts b/src/__tests__/server/importReplaceMedia.test.ts new file mode 100644 index 000000000..7a2126c34 --- /dev/null +++ b/src/__tests__/server/importReplaceMedia.test.ts @@ -0,0 +1,374 @@ +/** + * `replace` import and the media library. + * + * `replace` is wipe-and-recreate: when the import carries the media category, + * the previous site's media rows AND bytes go, so a second site that ships a + * file at the same storage path (two sites that embed the same font file) + * imports cleanly instead of failing on `media_assets.public_path`. Rows are + * deleted inside the import transaction; bytes only after it commits. Media + * owned by records that survive a replace (user avatars, plugin-managed + * media) and the library shared with other branches are left alone. + * + * Runs on SQLite by default and on Postgres with + * `DB=postgres TEST_POSTGRES_URL=...` (see `createTestDb`). + */ +import { afterEach, beforeEach, describe, expect, test } from 'bun:test' +import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { strToU8, zipSync } from 'fflate' +import { createTestDb, type TestDb } from '../helpers/createTestDb' +import type { DbClient } from '../../../server/db/client' +import { createUser, setUserAvatarMediaId } from '../../../server/repositories/users' +import { createSession } from '../../../server/auth/sessions' +import { + createSessionToken, + hashSessionToken, + SESSION_COOKIE_NAME, + sessionExpiry, +} from '../../../server/auth/tokens' +import { getDraftSite, saveDraftSite } from '../../../server/repositories/site' +import { createMediaAsset, getMediaAsset, listMediaAssets } from '../../../server/repositories/media' +import { insertBranch } from '../../../server/repositories/branches' +import { handleImportArchiveRoute } from '../../../server/handlers/cms/importArchive' +import { handleImportRoute } from '../../../server/handlers/cms/import' +import { mediaStorageRegistry } from '@core/plugins/mediaStorageRegistry' +import { BUNDLE_ARCHIVE_MANIFEST_PATH } from '@core/data/bundleArchive' +import { ImportResultSchema } from '@core/data/bundleSchema' +import { parseValue } from '@core/utils/typeboxHelpers' +import type { SiteShell } from '@core/page-tree' +import { MAIN_SCOPE } from '../../../server/branches/scope' + +const OWNER_ID = 'owner-replace-media' + +const SEED_SHELL: SiteShell = { + id: 'default', + name: 'Replace Media Site', + breakpoints: [], + settings: { shortcuts: {} }, + styleRules: {}, + files: [], + packageJson: { dependencies: {}, devDependencies: {} }, + runtime: { + dependencyLock: { version: 1, packages: {}, updatedAt: 0 }, + scripts: {}, + }, + createdAt: 0, + updatedAt: 0, +} + +/** The seeded shell as the server normalizes it, which is what an export carries. */ +let SHELL: SiteShell + +const PNG_MAGIC = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] +const WOFF2_MAGIC = [0x77, 0x4f, 0x46, 0x32] + +/** Magic bytes followed by a one-byte marker, so two files of one type differ. */ +function bytesOf(magic: number[], marker: number): Uint8Array { + return new Uint8Array([...magic, marker]) +} + +interface ArchiveMedia { + id: string + storagePath: string + mimeType: string + bytes: Uint8Array + folderIds?: string[] +} + +function mediaEntry(asset: ArchiveMedia) { + return { + id: asset.id, + filename: asset.storagePath, + mimeType: asset.mimeType, + sizeBytes: asset.bytes.byteLength, + altText: '', + caption: '', + title: '', + tags: [], + width: null, + height: null, + durationMs: null, + dominantColor: null, + blurHash: null, + storagePath: asset.storagePath, + posterPath: null, + folderIds: asset.folderIds ?? [], + } +} + +function buildArchive(input: { + media: ArchiveMedia[] + includeSite?: boolean + mediaFolders?: Array<{ id: string; parentId: string | null; name: string; slug: string; sortOrder: number }> + redirects?: Array<{ id: string; tableId: string; fromRouteBase: string; fromSlug: string; targetRowId: string }> +}): Uint8Array { + const manifest = { + schemaVersion: 1, + exportedAt: new Date().toISOString(), + ...(input.includeSite === false ? {} : { site: SHELL }), + tables: [], + rows: [], + media: input.media.map(mediaEntry), + ...(input.mediaFolders ? { mediaFolders: input.mediaFolders } : {}), + ...(input.redirects ? { redirects: input.redirects } : {}), + } + const entries: Record = { + [BUNDLE_ARCHIVE_MANIFEST_PATH]: strToU8(JSON.stringify(manifest)), + } + for (const asset of input.media) entries[`media/${asset.storagePath}`] = asset.bytes + return zipSync(entries, { level: 0 }) +} + +async function seedOwner(db: DbClient): Promise { + await saveDraftSite(db, MAIN_SCOPE, SEED_SHELL) + const normalized = await getDraftSite(db, MAIN_SCOPE) + if (!normalized) throw new Error('seeded site shell did not load') + SHELL = normalized + await createUser(db, { + id: OWNER_ID, + email: 'owner@replace-media.test', + displayName: 'Owner', + passwordHash: 'placeholder-hash', + roleId: 'owner', + allowOwnerRole: true, + }) + const token = createSessionToken() + await createSession(db, { + idHash: await hashSessionToken(token), + userId: OWNER_ID, + expiresAt: sessionExpiry(), + ipAddress: null, + userAgent: null, + // `replace` requires a fresh step-up; open the window directly. + stepUpExpiresAt: new Date(Date.now() + 60 * 60 * 1000), + }) + return `${SESSION_COOKIE_NAME}=${token}` +} + +async function fileExists(path: string): Promise { + try { + await stat(path) + return true + } catch { + return false + } +} + +const FONT_PATH = 'inter-400.woff2' +const fontA = bytesOf(WOFF2_MAGIC, 0xa1) +const fontB = bytesOf(WOFF2_MAGIC, 0xb2) + +/** Site A: a font plus a logo only site A ships. */ +const SITE_A: ArchiveMedia[] = [ + { id: 'a-font', storagePath: FONT_PATH, mimeType: 'font/woff2', bytes: fontA, folderIds: ['folder-a'] }, + { id: 'a-logo', storagePath: 'logo-a.png', mimeType: 'image/png', bytes: bytesOf(PNG_MAGIC, 0xa2) }, +] +const SITE_A_FOLDERS = [{ id: 'folder-a', parentId: null, name: 'Fonts', slug: 'fonts-a', sortOrder: 0 }] + +/** Site B: a different asset id at the same storage path as site A's font. */ +const SITE_B: ArchiveMedia[] = [ + { id: 'b-font', storagePath: FONT_PATH, mimeType: 'font/woff2', bytes: fontB }, +] + +describe('replace import removes the previous site media', () => { + let testDb: TestDb + let db: DbClient + let uploadsDir: string + let cookie: string + + async function importArchive( + archive: Uint8Array, + strategy: 'replace' | 'merge-add' | 'merge-overwrite' = 'replace', + ): Promise { + const req = new Request(`http://localhost/admin/api/cms/import/archive?strategy=${strategy}`, { + method: 'POST', + headers: { 'content-type': 'application/zip' }, + body: archive, + }) + req.headers.set('cookie', cookie) + const res = await handleImportArchiveRoute(req, db, MAIN_SCOPE, { uploadsDir }) + if (!res) throw new Error('archive import route did not handle the request') + return res + } + + async function mediaIds(): Promise { + const active = await listMediaAssets(db) + const trashed = await listMediaAssets(db, { includeDeleted: true }) + return [...active, ...trashed].map((asset) => asset.id).sort() + } + + async function membershipCount(): Promise { + const { rows } = await db<{ n: number | string }>`select count(*) as n from media_asset_folders` + return Number(rows[0]?.n ?? 0) + } + + beforeEach(async () => { + testDb = await createTestDb() + db = testDb.db + uploadsDir = await mkdtemp(join(tmpdir(), 'instatic-replace-media-')) + // Old files are removed through the asset's pinned storage adapter, + // exactly like a single hard delete; local disk is adapter ''. + mediaStorageRegistry.__reset() + mediaStorageRegistry.configureLocalDisk({ uploadsDir }) + cookie = await seedOwner(db) + }) + + afterEach(async () => { + mediaStorageRegistry.__reset() + await testDb.cleanup() + await rm(uploadsDir, { recursive: true, force: true }) + }) + + test('a second site sharing a storage path imports and leaves only its own media', async () => { + const first = await importArchive(buildArchive({ media: SITE_A, mediaFolders: SITE_A_FOLDERS })) + expect(first.status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo']) + expect(await membershipCount()).toBe(1) + + const second = await importArchive(buildArchive({ media: SITE_B })) + expect(second.status).toBe(200) + const result = parseValue(ImportResultSchema, await second.json()) + expect(result.mediaImported).toBe(1) + + expect(await mediaIds()).toEqual(['b-font']) + expect((await getMediaAsset(db, 'b-font'))?.publicPath).toBe(`/uploads/${FONT_PATH}`) + // Same storage path: the new bytes are on disk, not deleted after writing. + expect(new Uint8Array(await readFile(join(uploadsDir, FONT_PATH)))).toEqual(fontB) + // Site A's other file is gone from disk with its row. + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(false) + // The folder join rows went with the asset rows. + expect(await membershipCount()).toBe(0) + }) + + test('trashed media of the previous site is removed too', async () => { + await importArchive(buildArchive({ media: SITE_A })) + await db`update media_assets set deleted_at = ${new Date().toISOString()} where id = 'a-logo'` + + expect((await importArchive(buildArchive({ media: SITE_B }))).status).toBe(200) + expect(await mediaIds()).toEqual(['b-font']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(false) + }) + + test('a replace that fails inside the transaction keeps the old media rows and files', async () => { + await importArchive(buildArchive({ media: SITE_A })) + + // The redirect targets a row the bundle does not carry, so its insert + // violates the foreign key after the media rows were deleted. + const failing = buildArchive({ + media: SITE_B, + redirects: [{ id: 'r1', tableId: 'posts', fromRouteBase: '/posts', fromSlug: 'gone', targetRowId: 'missing-row' }], + }) + await expect(importArchive(failing)).rejects.toThrow() + + expect(await mediaIds()).toEqual(['a-font', 'a-logo']) + expect(new Uint8Array(await readFile(join(uploadsDir, FONT_PATH)))).toEqual(fontA) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) + + test('the JSON import endpoint applies the same replace semantics', async () => { + await importArchive(buildArchive({ media: SITE_A })) + + const bundle = { + schemaVersion: 1, + exportedAt: new Date().toISOString(), + site: SHELL, + tables: [], + rows: [], + media: SITE_B.map((asset) => ({ + ...mediaEntry(asset), + bytesBase64: Buffer.from(asset.bytes).toString('base64'), + })), + } + const req = new Request('http://localhost/admin/api/cms/import?strategy=replace', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(bundle), + }) + req.headers.set('cookie', cookie) + const res = await handleImportRoute(req, db, MAIN_SCOPE, { uploadsDir }) + expect(res?.status).toBe(200) + expect(parseValue(ImportResultSchema, await res!.json()).mediaImported).toBe(1) + + expect(await mediaIds()).toEqual(['b-font']) + expect(new Uint8Array(await readFile(join(uploadsDir, FONT_PATH)))).toEqual(fontB) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(false) + }) + + test('a user avatar survives a replace with its file', async () => { + await writeFile(join(uploadsDir, 'avatar.png'), bytesOf(PNG_MAGIC, 0x01)) + await createMediaAsset(db, { + id: 'avatar', + filename: 'avatar.png', + mimeType: 'image/png', + sizeBytes: 9, + storagePath: 'avatar.png', + publicPath: '/uploads/avatar.png', + uploadedByUserId: OWNER_ID, + storageAdapterId: '', + externallyHosted: false, + }) + await setUserAvatarMediaId(db, OWNER_ID, 'avatar') + await importArchive(buildArchive({ media: SITE_A })) + + expect((await importArchive(buildArchive({ media: SITE_B }))).status).toBe(200) + expect(await mediaIds()).toEqual(['avatar', 'b-font']) + expect(await fileExists(join(uploadsDir, 'avatar.png'))).toBe(true) + }) + + test('a replace without the media category keeps the library', async () => { + await importArchive(buildArchive({ media: SITE_A })) + const archive = zipSync({ + [BUNDLE_ARCHIVE_MANIFEST_PATH]: strToU8(JSON.stringify({ + schemaVersion: 1, + exportedAt: new Date().toISOString(), + site: SHELL, + tables: [], + rows: [], + })), + }, { level: 0 }) + + expect((await importArchive(archive)).status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) + + test('a replace that keeps the local site shell keeps the media it may reference', async () => { + await importArchive(buildArchive({ media: SITE_A })) + + expect((await importArchive(buildArchive({ media: [], includeSite: false }))).status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) + + test('a replace on main keeps the library while other branches share it', async () => { + await importArchive(buildArchive({ media: SITE_A })) + await insertBranch(db, { id: 'redesign', name: 'Redesign', baseBranchId: 'main', createdByUserId: OWNER_ID }) + + expect((await importArchive(buildArchive({ media: [] }))).status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) + + test('merge-overwrite keeps local media the bundle does not carry', async () => { + await importArchive(buildArchive({ media: SITE_A })) + const res = await importArchive( + buildArchive({ media: [{ id: 'b-logo', storagePath: 'logo-b.png', mimeType: 'image/png', bytes: bytesOf(PNG_MAGIC, 0xb3) }] }), + 'merge-overwrite', + ) + expect(res.status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo', 'b-logo']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) + + test('merge-add keeps local media the bundle does not carry', async () => { + await importArchive(buildArchive({ media: SITE_A })) + const res = await importArchive( + buildArchive({ media: [{ id: 'b-logo', storagePath: 'logo-b.png', mimeType: 'image/png', bytes: bytesOf(PNG_MAGIC, 0xb3) }] }), + 'merge-add', + ) + expect(res.status).toBe(200) + expect(await mediaIds()).toEqual(['a-font', 'a-logo', 'b-logo']) + expect(await fileExists(join(uploadsDir, 'logo-a.png'))).toBe(true) + }) +})