diff --git a/documentation/integrations/audible.md b/documentation/integrations/audible.md index ef89ba31..8d4993f6 100644 --- a/documentation/integrations/audible.md +++ b/documentation/integrations/audible.md @@ -286,7 +286,8 @@ interface AudibleAudiobook { durationMinutes?: number; releaseDate?: string; rating?: number; - genres?: string[]; + genres?: string[]; // display: genres + sub-genre tags, max 5 + primaryGenres?: string[]; // top-level genres only (Audnexus type 'genre' / top of each catalog ladder); metadata tags series?: string; seriesPart?: string; seriesAsin?: string; diff --git a/documentation/phase3/file-organization.md b/documentation/phase3/file-organization.md index 2e6221f8..f1a90046 100644 --- a/documentation/phase3/file-organization.md +++ b/documentation/phase3/file-organization.md @@ -41,7 +41,7 @@ Result: Douglas Adams/Stephen Fry/The Hitchhiker's Guide to the Galaxy/ 2. Identify audiobook files (.m4b, .m4a, .mp3, .mp4, .aa, .aax, .flac, .ogg) - supports both directories and single files 3. Read media directory and path template from database config (`media_dir`, `audiobook_path_template`) 4. Apply template to create target path: `[media_dir]/[template result]/` -5. **Tag metadata** (if enabled, before the move) - writes correct title, author, narrator, ASIN to tagged copies of the audio files +5. **Tag metadata** (if enabled, before the move) - writes correct title, author, narrator, ASIN, series, description, genre to tagged copies of the audio files 6. **ASIN step** - adds the MP4 freeform `ASIN` to the file about to be placed, when RMAB wrote it (see MP4 Freeform Tags), then **copy** files (not move - originals stay for seeding) 7. Copy cover art if found, else download from Audible. `coverArtUrl` must be a cached thumbnail `/api/cache/thumbnails/` (plain file name, resolved in `/app/cache/thumbnails`) or an https URL on `COVER_ART_HOSTS` (`*.media-amazon.com`, `*.ssl-images-amazon.com`, `*.images-amazon.com`), up to `MAX_COVER_ART_REDIRECTS` (3) redirects with every hop checked; anything else is skipped with a warning (`src/lib/utils/cover-art-url.ts`). An existing `cover.jpg` is never replaced and not fetched for (kept, debug log; a re-import with the same cover lands here too); an empty regular-file `cover.jpg` (older interrupted write) counts as missing and is replaced (`replace` publish, size re-checked under the lock). New covers are published like audio (`src/lib/utils/cover-placement.ts` → `publishCover`): hidden partial (`copyFile` for a cached thumbnail, `writeFileAtomic` for downloaded bytes), fsync + size check, `create` publish (`EEXIST` → kept), under the per-target lock; an interrupted write leaves no `cover.jpg`. Downloaded and copied covers get the library file mode (`applyFileMode`: a refused chmod is a permission warning, a missing file a failure). At request creation, covers are stored for display if they are any http(s) URL or a safe cached path (`sanitizeCoverArtUrl`; shelf-sync Goodreads/Hardcover fallbacks stay visible); a stored cover the library can't use is replaced by a later usable one (`isLibraryCoverSource`) 8. **Coerce file formats** (if enabled) - rename .mp4 → .m4b and single-file .m4a → .m4b for Plex compatibility (see: Plex Format Coercion below) @@ -109,6 +109,22 @@ Result: Douglas Adams/Stephen Fry/The Hitchhiker's Guide to the Galaxy/ - M4B/M4A/MP4: iTunes freeform item `----` (mean `com.apple.iTunes`, name `ASIN`), written after ffmpeg (see MP4 Freeform Tags) - MP3: ID3v2 `TXXX:ASIN`; FLAC: Vorbis comment `ASIN` - Read back as ffprobe format tag `ASIN`; Audiobookshelf maps it (`asin`, case-insensitive) to `tagASIN` → book `asin` +- Description - publisher summary; raw input first cut to `MAX_RAW_DESCRIPTION_LENGTH` (4x the cap, UTF-16 units, `truncateRawText`, surrogate-safe); non-whitespace C0 controls dropped, each whitespace run (space/tab/CR/LF/VT/FF/U+2028/U+2029) → one space (linear); capped at `MAX_DESCRIPTION_BYTES` (16 KiB UTF-8, `capUtf8Bytes`: last word boundary + `…`; one argv string each) + - M4B/M4A/MP4: `description` (`desc`) + `synopsis` (`ldes`), each capped + - MP3: `description` (TXXX); FLAC: `comment` (= Vorbis `DESCRIPTION`; replaces the source's DESCRIPTION/COMMENT, which ffmpeg reads as one `comment`; a `description` key would add a second one, read as "old;new") +- `genre` - top-level genres (`primaryGenres`, no sub-genre tags) joined with `; `; `/` or `;` in a name → ` & ` (split/trim/join; Audiobookshelf splits on `/`, `//`, `;`) +- M4B/M4A/MP4 only: `media_type=2` (`stik` Audiobook), `gapless_playback=1` (`pgap`) - always written, before `-movflags +faststart` +- Fixed set, no per-field settings; all under `metadata_tagging_enabled` + +**Tag details lookup** (`src/lib/processors/organize-tag-details.ts`): +- Processor reads `metadata_tagging_enabled` once (`readMetadataTaggingEnabled`), passes it to the lookup and `organize()` (`placement.metadataTaggingEnabled`); unreadable → undefined: no lookup, `organize()` reads it itself +- Only when tagging is on; one `getAudiobookDetails(asin)` per import (`createAudibleDetailsLoader`), shared with the missing-series backfill +- Series known (lookup serves tags only): raced against `TAG_DETAILS_TIMEOUT_MS` (15 s); series backfill awaits it in full (unchanged) +- Description: Audible summary (Audnexus `summary`) → stored `Audiobook.description` fallback (client text: bounded by `truncateRawText`, then `stripHtml` like the lookup) +- Started right after the switch read, awaited just before `organize()` (setup reads run meanwhile; a no-op handler keeps a rejection handled if setup throws first) +- A failed ffmpeg run reports `describeFfmpegFailure` (stderr tail, never the argv, so no description in job logs) +- Lookup failure or timeout is non-fatal (warning): stored description, no genres +- Genres/description are tags only (not path template variables); ebooks unchanged **Note:** ASIN is a custom metadata tag and may not appear in standard file properties viewers (Windows/macOS/Linux). Use specialized tools to verify: ```bash @@ -139,7 +155,7 @@ exiftool "audiobook.m4b" | grep -i asin **Verification:** before anything is written, the new moov's chunk offsets = offsets computed from the input, chunk sizes unchanged, each tag present exactly once; after the write and before rename: output size and top-level layout as planned, written moov bytes = the verified moov, chunk bytes equal input's (every chunk of non-audio tracks up to 1 MiB each, 64 spread audio chunks × 4 KiB), ffprobe reads every tag. All handles closed before rename (Windows). -**Call site (one):** `file-organizer.ts`, per audio file right before placement, on the file about to be placed when RMAB wrote it: the tagged copy (`${file}.tmp`), else the merged file (tagging off, or the tagging pass failed on it). Never on a download's own file (tagging off, no merge → no ASIN in MP4s). It runs after the merge's own checks. The merger and tagger never write the ASIN; `metadata_tagging_enabled` is read once per `organize()` (unreadable → tagging off, warning + `result.errors` "Metadata tagging skipped: could not read setting ()"). A written ASIN logs "Embedded ASIN in " (info); a refused one logs a warning. +**Call site (one):** `file-organizer.ts`, per audio file right before placement, on the file about to be placed when RMAB wrote it: the tagged copy (`${file}.tmp`), else the merged file (tagging off, or the tagging pass failed on it). Never on a download's own file (tagging off, no merge → no ASIN in MP4s). It runs after the merge's own checks. The merger and tagger never write the ASIN; `metadata_tagging_enabled` is read once per `organize()`, or taken from `placement.metadataTaggingEnabled` (unreadable → tagging off, warning + `result.errors` "Metadata tagging skipped: could not read setting ()"). A written ASIN logs "Embedded ASIN in " (info); a refused one logs a warning. **Resources:** - Disk: the merge check (`TEMP_DIR`) counts the estimated merged size × (1 merged file + 1 post-step copy when the book has an ASIN + 1 tagged copy when tagging is on): 1x-3x. A tagged single file with an ASIN needs 2x its size next to the download (tagged copy + post-step copy) @@ -172,7 +188,9 @@ exiftool "audiobook.m4b" | grep -i asin **Tech Stack:** - ffmpeg (system dependency - included in Docker image) - `src/lib/utils/metadata-tagger.ts` - Tagging utility +- `src/lib/utils/metadata-text.ts` - text helpers: `sanitizeMetadataValue`, `sanitizeMultilineValue`, `sanitizeGenreName`, `truncateRawText`, `capUtf8Bytes`, `describeFfmpegFailure`, `MAX_DESCRIPTION_BYTES`, `MAX_RAW_DESCRIPTION_LENGTH` - Integrated into `src/lib/utils/file-organizer.ts` +- `src/lib/processors/organize-tag-details.ts` - description/genres for the tags **Requirements:** - ffmpeg + ffprobe must be on PATH (`/usr/local/bin`) diff --git a/e2e/specs/regressions/208-asin-tag.spec.ts b/e2e/specs/regressions/208-asin-tag.spec.ts index f2fa271b..a0721b52 100644 --- a/e2e/specs/regressions/208-asin-tag.spec.ts +++ b/e2e/specs/regressions/208-asin-tag.spec.ts @@ -11,6 +11,9 @@ * 2. multi-file mp3, chapter merging on, tagging off (on the merged file) * 3. multi-file mp3, chapter merging on, tagging on (on the tagged copy of the merge) * Each also checks the placed file is moov first (faststart) through Audiobookshelf's container. + * With tagging on (1 and 3) it also checks the #397 tags as Audiobookshelf reads them: the book's + * genres array is the genre tag split on "; " (names like "Science Fiction & Fantasy" kept whole), + * and the book has one description, the tag's. * Requests all three fixture books (phm too, like smoke): run with --isolate-specs next to other specs. */ @@ -87,6 +90,8 @@ async function importAndScan(scenario: string, bookKey: string, label: string) { numAudioFiles: audio.length, metaTags: audio[0]?.metaTags, bookAsin: item.media.metadata.asin, + bookGenres: item.media.metadata.genres as string[], + bookDescription: item.media.metadata.description as string | null, chapters: item.media.chapters?.length, embeddedCoverArt: audio[0]?.embeddedCoverArt ?? null, layout: audio[0] ? topLevelTypes(audio[0].metadata.path) : [], @@ -104,10 +109,25 @@ function expectAsinInFile(book: Book, e: Awaited>['evidence']) { + const tagGenre = e.metaTags?.tagGenre as string | undefined; + expect(tagGenre, 'genre tag written').toBeTruthy(); + const parts = tagGenre!.split('; '); + expect(parts.length, 'more than one top-level genre').toBeGreaterThan(1); + expect(e.bookGenres, 'Audiobookshelf genres = the tag split on "; "').toEqual(parts); + for (const genre of e.bookGenres) expect(genre).toMatch(/^[^;/]+$/); + expect(e.bookGenres.some((g) => g.includes(' & ')), 'an "A & B" genre stays one genre').toBe(true); + const tagDescription = e.metaTags?.tagDescription as string | undefined; + expect(tagDescription, 'description tag written').toBeTruthy(); + expect(e.bookDescription, 'one description, the tag\'s').toBe(tagDescription); +} + test('#208: single m4b, tagging on: the tagged m4b carries the ASIN', async () => { await withSettings({ chapter_merging_enabled: 'false', metadata_tagging_enabled: 'true' }, async () => { const { book, evidence } = await importAndScan('asin-m4b', 'martian', 'single m4b'); expectAsinInFile(book, evidence, 2); + expectTagDetails(evidence); }); }); @@ -122,5 +142,6 @@ test('#208: chapter merge, tagging on: the merged and tagged m4b carries the ASI await withSettings({ chapter_merging_enabled: 'true', metadata_tagging_enabled: 'true' }, async () => { const { book, evidence } = await importAndScan('asin-merge-tagged', 'phm', 'merge, tagging on'); expectAsinInFile(book, evidence, 3); + expectTagDetails(evidence); }); }); diff --git a/src/lib/integrations/audible.service.ts b/src/lib/integrations/audible.service.ts index 4ff73874..32b5cff0 100644 --- a/src/lib/integrations/audible.service.ts +++ b/src/lib/integrations/audible.service.ts @@ -53,6 +53,8 @@ export interface AudibleAudiobook { releaseDate?: string; rating?: number; genres?: string[]; + /** Top-level genres only (no sub-genre tags), as written into the audio file tags */ + primaryGenres?: string[]; series?: string; seriesPart?: string; seriesAsin?: string; @@ -156,6 +158,14 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook { ? [...new Set(genreNames)].slice(0, 5) : undefined; + // Each ladder runs from its top-level genre down to sub-genres + const primaryGenreNames = + product.category_ladders + ?.map((ladder) => ladder.ladder?.[0]?.name as unknown) + .filter((name): name is string => typeof name === 'string' && name.trim() !== '') ?? []; + const primaryGenres = + primaryGenreNames.length > 0 ? [...new Set(primaryGenreNames)] : undefined; + let series: string | undefined; let seriesPart: string | undefined; let seriesAsin: string | undefined; @@ -186,6 +196,7 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook { releaseDate: product.release_date ?? undefined, rating: product.rating?.overall_distribution?.display_stars ?? undefined, genres, + primaryGenres, series, seriesPart, seriesAsin, @@ -195,6 +206,18 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook { }; } +/** + * Top-level genres from an Audnexus genre list. Audnexus marks the top of each + * category ladder `type: 'genre'` and the sub-genres below it `type: 'tag'`. + */ +function audnexusPrimaryGenres(genres: unknown): string[] | undefined { + if (!Array.isArray(genres)) return undefined; + const names = (genres as Array) + .map((g) => (typeof g === 'string' ? g : g?.type !== 'tag' ? g?.name : undefined)) + .filter((name): name is string => typeof name === 'string' && name.trim() !== ''); + return names.length > 0 ? [...new Set(names)] : undefined; +} + export class AudibleService { private htmlClient!: AxiosInstance; private apiClient!: AxiosInstance; @@ -686,6 +709,7 @@ export class AudibleService { releaseDate: data.releaseDate || undefined, rating: data.rating ? parseFloat(data.rating) : undefined, genres: data.genres?.map((g: any) => (typeof g === 'string' ? g : g.name)).slice(0, 5) || undefined, + primaryGenres: audnexusPrimaryGenres(data.genres), series: data.seriesPrimary?.name || undefined, seriesPart: data.seriesPrimary?.position || undefined, seriesAsin: data.seriesPrimary?.asin || undefined, diff --git a/src/lib/processors/organize-files.processor.ts b/src/lib/processors/organize-files.processor.ts index 9a6c964f..87b1af0b 100644 --- a/src/lib/processors/organize-files.processor.ts +++ b/src/lib/processors/organize-files.processor.ts @@ -27,6 +27,7 @@ import { } from './organize-evidence'; import { getAudibleService } from '../integrations/audible.service'; import { addAutoBlock } from '../services/blocklist.service'; +import { createAudibleDetailsLoader, readMetadataTaggingEnabled, resolveTagDetails } from './organize-tag-details'; /** * Process organize files job @@ -134,11 +135,13 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi let series = audiobook.series || undefined; let seriesPart = audiobook.seriesPart || undefined; + // At most one Audible details lookup per import, shared with the metadata tags below + const loadAudibleDetails = createAudibleDetailsLoader(audiobook.audibleAsin); + if (audiobook.audibleAsin && !series) { try { logger.info(`Missing series data, fetching from Audnexus for ASIN: ${audiobook.audibleAsin}`); - const audibleService = getAudibleService(); - const audnexusData = await audibleService.getAudiobookDetails(audiobook.audibleAsin); + const audnexusData = await loadAudibleDetails(); if (audnexusData) { const updates: Record = {}; @@ -187,6 +190,13 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi logger.info(`Final metadata for path organization: year=${year || 'null'}, narrator=${narrator || 'null'}, series=${series || 'null'}, seriesPart=${seriesPart || 'null'}`); + // One read of the tagging switch: the tag details lookup and the organizer both use it + const metadataTaggingEnabled = await readMetadataTaggingEnabled(logger); + // Started now, awaited just before organize(): the setup below does not wait on Audible. + // The no-op handler keeps a rejection from going unhandled if that setup throws first. + const tagDetailsPending = resolveTagDetails(audiobook.description, loadAudibleDetails, metadataTaggingEnabled, logger); + tagDetailsPending.catch(() => undefined); + // Get file organizer (reads media_dir from database config) const organizer = await getFileOrganizer(); @@ -211,6 +221,8 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi template: fileRenameTemplateConfig?.value || '{title}', }; + const tagDetails = await tagDetailsPending; + // Organize files (pass template, logger, and rename config to file organizer) const result = await organizer.organize( downloadPath, @@ -223,12 +235,13 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi year, series, seriesPart, + ...tagDetails, }, template, jobId ? { jobId, context: 'FileOrganizer' } : undefined, renameConfig, selectedFiles, - { compareExisting } + { compareExisting, metadataTaggingEnabled } ); if (!result.success) { diff --git a/src/lib/processors/organize-tag-details.ts b/src/lib/processors/organize-tag-details.ts new file mode 100644 index 00000000..3a36294f --- /dev/null +++ b/src/lib/processors/organize-tag-details.ts @@ -0,0 +1,92 @@ +/** + * Component: Organize Tag Details (Audible details for series backfill and metadata tags) + * Documentation: documentation/phase3/file-organization.md + */ + +import { prisma } from '../db'; +import type { RMABLogger } from '../utils/logger'; +import { getAudibleService, type AudibleAudiobook } from '../integrations/audible.service'; +import { stripHtml } from '../utils/strip-html'; +import { truncateRawText } from '../utils/metadata-text'; + +export type AudibleDetailsLoader = () => Promise; + +/** Description and genres written into the audio file tags. */ +export interface TagDetails { + description?: string; + genres?: string[]; +} + +/** Longest wait for a lookup that only serves the tags (the series backfill awaits it in full). */ +export const TAG_DETAILS_TIMEOUT_MS = 15_000; + +/** + * The import's one read of `metadata_tagging_enabled`, passed to the details lookup + * and the organizer. A failed read is undefined: no lookup, and the organizer reads + * (and reports) the setting itself. + */ +export async function readMetadataTaggingEnabled(logger: RMABLogger): Promise { + try { + const config = await prisma.configuration.findUnique({ where: { key: 'metadata_tagging_enabled' } }); + return config?.value === 'true'; + } catch (error) { + logger.warn(`Could not read metadata_tagging_enabled for tag details: ${error instanceof Error ? error.message : String(error)}`); + return undefined; + } +} + +/** + * At most one Audible details lookup per import, shared by the series backfill and + * the metadata tags. Resolves to null without an ASIN; never called until needed. + */ +export function createAudibleDetailsLoader(asin: string | null | undefined): AudibleDetailsLoader { + let pending: Promise | undefined; + return () => { + if (!asin) return Promise.resolve(null); + pending ??= getAudibleService().getAudiobookDetails(asin); + return pending; + }; +} + +/** + * Description and genres for the metadata tags, only when tagging is on. Prefers the + * full Audible summary, falls back to the stored description. The lookup is raced + * against `timeoutMs` (a lookup the series backfill already awaited is settled, so + * that path is unchanged); a failed or slow lookup never blocks the import: the + * book is tagged with the stored description and no genres. + */ +export async function resolveTagDetails( + storedDescription: string | null | undefined, + loadDetails: AudibleDetailsLoader, + taggingEnabled: boolean | undefined, + logger: RMABLogger, + timeoutMs: number = TAG_DETAILS_TIMEOUT_MS +): Promise { + if (!taggingEnabled) return {}; + + let details: AudibleAudiobook | null = null; + let timer: ReturnType | undefined; + try { + const timedOut = new Promise<'timeout'>((resolve) => { + timer = setTimeout(() => resolve('timeout'), timeoutMs); + }); + const outcome = await Promise.race([loadDetails(), timedOut]); + if (outcome === 'timeout') { + logger.warn(`Audible details lookup for metadata tags took over ${timeoutMs / 1000} s; tagging with the stored description and no genres`); + } else { + details = outcome; + } + } catch (error) { + logger.warn(`Failed to fetch Audible details for metadata tags: ${error instanceof Error ? error.message : String(error)}`); + } finally { + clearTimeout(timer); + } + + // The stored description is client text (older rows, or any request body): bounded, then HTML stripped like the lookup's + const description = details?.description || (storedDescription ? stripHtml(truncateRawText(storedDescription)) : '') || undefined; + const genres = details?.primaryGenres?.length ? details.primaryGenres : undefined; + + logger.info(`Metadata tag details: description=${description ? `${description.length} chars` : 'none'}, genres=${genres ? genres.join(', ') : 'none'}`); + + return { description, genres }; +} diff --git a/src/lib/utils/file-organizer.ts b/src/lib/utils/file-organizer.ts index 2f7fcedd..d3bab342 100644 --- a/src/lib/utils/file-organizer.ts +++ b/src/lib/utils/file-organizer.ts @@ -38,6 +38,8 @@ export interface AudiobookMetadata { asin?: string; series?: string; seriesPart?: string; + description?: string; // Metadata tags only (not used in path templates) + genres?: string[]; // Metadata tags only (not used in path templates) } export interface OrganizationResult { @@ -59,6 +61,8 @@ export interface OrganizationResult { export interface PlacementOptions { /** Compare same-size existing targets byte-for-byte; only worth it when a delete depends on it. */ compareExisting?: boolean; + /** The caller's read of `metadata_tagging_enabled`; read here when omitted. */ + metadataTaggingEnabled?: boolean; } export interface EbookOrganizationResult { @@ -158,7 +162,7 @@ export class FileOrganizer { // Determine base path for source files const baseSourcePath = isFile ? path.dirname(downloadPath) : downloadPath; - const metadataTaggingEnabled = await prisma.configuration.findUnique({ where: { key: 'metadata_tagging_enabled' } }).then((c) => c?.value === 'true', async (error: unknown) => { + const metadataTaggingEnabled = placement?.metadataTaggingEnabled ?? await prisma.configuration.findUnique({ where: { key: 'metadata_tagging_enabled' } }).then((c) => c?.value === 'true', async (error: unknown) => { const skipped = `Metadata tagging skipped: could not read setting (${error instanceof Error ? error.message : 'Unknown error'})`; await logger?.warn(skipped); result.errors.push(skipped); return false; }); // read once: merge disk-space check + tagging pass @@ -297,6 +301,8 @@ export class FileOrganizer { asin: audiobook.asin, series: audiobook.series, seriesPart: audiobook.seriesPart, + description: audiobook.description, + genres: audiobook.genres, }); const successCount = taggingResults.filter((r) => r.success).length; diff --git a/src/lib/utils/metadata-tagger.ts b/src/lib/utils/metadata-tagger.ts index d809318b..de8a02ba 100644 --- a/src/lib/utils/metadata-tagger.ts +++ b/src/lib/utils/metadata-tagger.ts @@ -9,6 +9,15 @@ import path from 'path'; import fs from 'fs/promises'; import { METADATA_TAG_FORMATS, MP4_CONTAINER_FORMATS } from '../constants/audio-formats'; import { addFreeformTags, type FreeformTag, type FreeformTagResult } from './mp4-freeform-tags'; +import { + capUtf8Bytes, + describeFfmpegFailure, + MAX_DESCRIPTION_BYTES, + sanitizeGenreName, + sanitizeMetadataValue, + sanitizeMultilineValue, + truncateRawText, +} from './metadata-text'; const execFilePromise = promisify(execFile); @@ -20,8 +29,16 @@ export interface MetadataTaggingOptions { asin?: string; series?: string; seriesPart?: string; + description?: string; // Publisher summary (plain text, may contain paragraphs) + genres?: string[]; // Top-level genres, joined into one tag } +/** + * Joins genres into one tag. Audiobookshelf splits genre tags on '/', '//' or ';' + * (first match wins), so a '/' inside a genre name becomes ' & '. + */ +const GENRE_SEPARATOR = '; '; + export interface TaggingResult { success: boolean; filePath: string; // Original file path @@ -40,21 +57,6 @@ export function ffmpegRemuxTimeoutMs(sizeBytes: number): number { return 120_000 + Math.ceil((sizeBytes / 1024 ** 3) * 120_000); } -/** - * Sanitize a metadata value before writing. - * Conservative — only removes characters that are clearly not intent: - * - Leading BOM, NBSP, zero-width space, zero-width non-joiner, zero-width joiner - * - Null bytes, CR, LF anywhere (ffmpeg metadata semantics break on these) - * - Leading/trailing whitespace - * Does NOT normalize Unicode forms or otherwise transform content. - */ -function sanitizeMetadataValue(value: string): string { - return value - .replace(/^[ ​‌‍]+/, '') // leading BOM / NBSP / ZWSP / ZWNJ / ZWJ - .replace(/[\x00\r\n]/g, '') // null bytes and line breaks anywhere - .trim(); -} - /** * iTunes freeform ('----', com.apple.iTunes) tags for MP4 containers: ffmpeg's * mp4 muxer drops these keys, so they are added after ffmpeg by @@ -115,6 +117,13 @@ export async function tagAudioFileMetadata( const asin = metadata.asin ? sanitizeMetadataValue(metadata.asin) : undefined; const series = metadata.series ? sanitizeMetadataValue(metadata.series) : undefined; const seriesPart = metadata.seriesPart ? sanitizeMetadataValue(metadata.seriesPart) : undefined; + const description = metadata.description + ? capUtf8Bytes(sanitizeMultilineValue(truncateRawText(metadata.description)), MAX_DESCRIPTION_BYTES) || undefined + : undefined; + const genre = metadata.genres + ?.map(sanitizeGenreName) + .filter((name) => name.length > 0) + .join(GENRE_SEPARATOR); // Create temporary file path const tempFile = `${filePath}.tmp`; @@ -154,6 +163,23 @@ export async function tagAudioFileMetadata( args.push('-metadata', `episode_id=${seriesPart}`); } + if (description) { + args.push( + '-metadata', `description=${description}`, // desc atom (read by Audiobookshelf) + '-metadata', `synopsis=${description}` // ldes atom (long description) + ); + } + + if (genre) { + args.push('-metadata', `genre=${genre}`); + } + + // Mark the file as an audiobook (stik=2) with gapless playback (pgap=1) + args.push( + '-metadata', 'media_type=2', + '-metadata', 'gapless_playback=1' + ); + // moov first (faststart), also when the freeform post-step later refuses the file args.push('-movflags', '+faststart'); @@ -190,6 +216,16 @@ export async function tagAudioFileMetadata( args.push('-metadata', `SERIES-PART=${seriesPart}`); } + if (description) { + // ffmpeg's `comment` is the Vorbis DESCRIPTION field; it replaces the source's + // (a `description` key would be a second DESCRIPTION, read as "old;new") + args.push('-metadata', `comment=${description}`); + } + + if (genre) { + args.push('-metadata', `genre=${genre}`); + } + // Explicitly specify output format args.push('-f', 'flac'); } @@ -224,6 +260,15 @@ export async function tagAudioFileMetadata( args.push('-metadata', `SERIES-PART=${seriesPart}`); } + if (description) { + // TXXX frame, read back as `description` (Audiobookshelf reads this) + args.push('-metadata', `description=${description}`); + } + + if (genre) { + args.push('-metadata', `genre=${genre}`); // TCON frame + } + // Explicitly specify output format (fixes .tmp extension issue) args.push('-f', 'mp3'); } @@ -241,7 +286,7 @@ export async function tagAudioFileMetadata( // Ignore cleanup errors } - throw new Error(`ffmpeg failed: ${error instanceof Error ? error.message : 'Unknown error'}`); + throw new Error(`ffmpeg failed: ${describeFfmpegFailure(error)}`); } // DO NOT replace original file - return temp file path instead diff --git a/src/lib/utils/metadata-text.ts b/src/lib/utils/metadata-text.ts new file mode 100644 index 00000000..bd641c26 --- /dev/null +++ b/src/lib/utils/metadata-text.ts @@ -0,0 +1,100 @@ +/** + * Component: Metadata Text Helpers (sanitize, bound and cap tag values; ffmpeg failure text) + * Documentation: documentation/phase3/file-organization.md + */ + +/** + * Longest description written, in UTF-8 bytes. Each `-metadata` value is one argv + * string and the OS caps those (~128 KiB on Linux, "Argument list too long"); + * MP4 passes the description twice (description + synopsis). + */ +export const MAX_DESCRIPTION_BYTES = 16 * 1024; + +/** + * Sanitize a metadata value before writing. + * Conservative — only removes characters that are clearly not intent: + * - Leading BOM, NBSP, zero-width space, zero-width non-joiner, zero-width joiner + * - Null bytes, CR, LF anywhere (ffmpeg metadata semantics break on these) + * - Leading/trailing whitespace + * Does NOT normalize Unicode forms or otherwise transform content. + */ +export function sanitizeMetadataValue(value: string): string { + return value + .replace(/^[ ​‌‍]+/, '') // leading BOM / NBSP / ZWSP / ZWNJ / ZWJ + .replace(/[\x00\r\n]/g, '') // null bytes and line breaks anywhere + .trim(); +} + +/** + * Longest raw description read, in UTF-16 code units (each at least one UTF-8 byte, + * so 4x the cap): client text is unbounded, and this bounds the work before any + * sanitizing or HTML stripping. + */ +export const MAX_RAW_DESCRIPTION_LENGTH = 4 * MAX_DESCRIPTION_BYTES; + +/** The first `maxLength` UTF-16 code units of `value`, never ending inside a surrogate pair. */ +export function truncateRawText(value: string, maxLength: number = MAX_RAW_DESCRIPTION_LENGTH): string { + if (value.length <= maxLength) return value; + let end = Math.max(0, maxLength); + const last = value.charCodeAt(end - 1); + if (end > 0 && last >= 0xd800 && last <= 0xdbff) end--; // a high surrogate whose pair is cut off + return value.slice(0, end); +} + +/** + * Sanitize a multi-paragraph value: C0 control characters other than whitespace are + * dropped, then every run of whitespace (space, tab, CR, LF, VT, FF, U+2028, U+2029) + * becomes one space, so paragraphs never run together. Linear in the input. + */ +export function sanitizeMultilineValue(value: string): string { + return sanitizeMetadataValue( + value + .replace(/[\x00-\x08\x0e-\x1f]/g, '') + .replace(/[ \t\r\n\v\f\u2028\u2029]+/g, ' ') + ); +} + +/** + * A genre name as one part of the joined tag: '/' and ';' (Audiobookshelf's + * separators) inside it become ' & '. Split and trim, no regex backtracking. + */ +export function sanitizeGenreName(name: string): string { + return name + .split(/[/;]/) + .map((part) => sanitizeMetadataValue(part)) + .filter((part) => part.length > 0) + .join(' & '); +} + +/** + * Cap `value` at `maxBytes` UTF-8 bytes. A longer value is cut at its last word + * boundary that fits (never inside a code point) and ends with an ellipsis (left + * out when `maxBytes` is smaller than the ellipsis). + */ +export function capUtf8Bytes(value: string, maxBytes: number): string { + if (Buffer.byteLength(value, 'utf8') <= maxBytes) return value; + const ellipsisBytes = Buffer.byteLength('\u2026', 'utf8'); + const ellipsis = maxBytes >= ellipsisBytes ? '\u2026' : ''; + const bytes = Buffer.from(value, 'utf8'); + let end = Math.max(0, maxBytes - Buffer.byteLength(ellipsis, 'utf8')); + while (end > 0 && (bytes[end] & 0xc0) === 0x80) end--; // back to the start of a code point + let head = bytes.subarray(0, end).toString('utf8'); + const lastSpace = head.lastIndexOf(' '); + if (lastSpace >= head.length / 2) head = head.slice(0, lastSpace); // a word boundary, unless it drops half + return `${head.trimEnd()}${ellipsis}`; +} + +/** + * ffmpeg's failure without its argv (execFile's message starts with the whole + * command line, description included): the last lines of stderr, else the + * message without its "Command failed:" line, else the exit or kill reason. + */ +export function describeFfmpegFailure(error: unknown): string { + if (!(error instanceof Error)) return 'Unknown error'; + const e = error as Error & { stderr?: unknown; killed?: boolean; signal?: unknown; code?: unknown }; + if (e.killed) return `ffmpeg was stopped (${String(e.signal ?? 'timeout')})`; + const stderr = typeof e.stderr === 'string' ? e.stderr.trim() : ''; + if (stderr) return stderr.split(/\r?\n/).slice(-3).join(' | ').slice(-500); + const rest = e.message.startsWith('Command failed:') ? e.message.split('\n').slice(1).join(' ').trim() : e.message.trim(); + return rest || (e.code !== undefined ? `exit code ${String(e.code)}` : 'Unknown error'); +} diff --git a/tests/integrations/audible.service.test.ts b/tests/integrations/audible.service.test.ts index f8d177b5..ecbd8322 100644 --- a/tests/integrations/audible.service.test.ts +++ b/tests/integrations/audible.service.test.ts @@ -732,6 +732,53 @@ describe('AudibleService', () => { expect(genreSet.size).toBe(5); }); + it('maps primaryGenres to the deduplicated top of each category ladder', async () => { + const products = [ + makeProduct({ + category_ladders: [ + { ladder: [{ name: 'Fiction' }, { name: 'Fantasy' }, { name: 'Epic Fantasy' }] }, + { ladder: [{ name: 'Fiction' }, { name: 'Adventure' }] }, + { ladder: [{ name: 'Young Adult' }, { name: 'Coming of Age' }] }, + { ladder: [] }, + ], + }), + ]; + apiClientMock.get.mockResolvedValue(apiResponse(makeProductsResponse(products))); + + const service = new AudibleService(); + const { results } = await service.search('test', 1); + + expect(results[0].primaryGenres).toEqual(['Fiction', 'Young Adult']); + }); + + it('drops blank and non-string ladder names from primaryGenres', async () => { + const products = [ + makeProduct({ + category_ladders: [ + { ladder: [{ name: ' ' }, { name: 'Sub' }] }, + { ladder: [{ name: 42 as unknown as string }] }, + { ladder: [{ name: 'History' }] }, + ], + }), + ]; + apiClientMock.get.mockResolvedValue(apiResponse(makeProductsResponse(products))); + + const service = new AudibleService(); + const { results } = await service.search('test', 1); + + expect(results[0].primaryGenres).toEqual(['History']); + }); + + it('leaves primaryGenres undefined without category ladders', async () => { + const products = [makeProduct({ category_ladders: [] })]; + apiClientMock.get.mockResolvedValue(apiResponse(makeProductsResponse(products))); + + const service = new AudibleService(); + const { results } = await service.search('test', 1); + + expect(results[0].primaryGenres).toBeUndefined(); + }); + it('maps language from catalog product', async () => { const products = [makeProduct({ language: 'english' })]; apiClientMock.get.mockResolvedValue(apiResponse(makeProductsResponse(products))); @@ -1438,6 +1485,45 @@ describe('AudibleService', () => { expect(apiClientMock.get).not.toHaveBeenCalled(); }); + it('maps primaryGenres to the Audnexus genres of type genre, without the sub-genre tags', async () => { + axiosMock.get.mockResolvedValueOnce({ + data: { + title: 'Audnexus Book', + genres: [ + { asin: '1', name: 'Literature & Fiction', type: 'genre' }, + { asin: '2', name: 'Science Fiction & Fantasy', type: 'genre' }, + { asin: '3', name: 'Hard Science Fiction', type: 'tag' }, + { asin: '4', name: 'Space Opera', type: 'tag' }, + ], + }, + }); + + const service = new AudibleService(); + const details = await service.getAudiobookDetails('B000GENRES'); + + expect(details?.primaryGenres).toEqual(['Literature & Fiction', 'Science Fiction & Fantasy']); + // The display list keeps its existing shape (genres and tags, capped at 5) + expect(details?.genres).toEqual([ + 'Literature & Fiction', + 'Science Fiction & Fantasy', + 'Hard Science Fiction', + 'Space Opera', + ]); + }); + + it.each([ + ['plain string genres', { genres: ['Fiction', 'Fiction', 'History'] }, ['Fiction', 'History']], + ['only tags', { genres: [{ name: 'Space Opera', type: 'tag' }] }, undefined], + ['no genres', {}, undefined], + ])('maps Audnexus primaryGenres for %s', async (_label, extra, expected) => { + axiosMock.get.mockResolvedValueOnce({ data: { title: 'Audnexus Book', ...extra } }); + + const service = new AudibleService(); + const details = await service.getAudiobookDetails('B000GENRE2'); + + expect(details?.primaryGenres).toEqual(expected); + }); + it('prefers the full Audnexus summary over the short description, as plain text with paragraphs', async () => { axiosMock.get.mockResolvedValueOnce({ data: { diff --git a/tests/processors/organize-files.processor.test.ts b/tests/processors/organize-files.processor.test.ts index c32aef4a..fc79dda2 100644 --- a/tests/processors/organize-files.processor.test.ts +++ b/tests/processors/organize-files.processor.test.ts @@ -54,6 +54,17 @@ vi.mock('@/lib/integrations/audible.service', () => ({ getAudibleService: () => audibleServiceMock, })); +// resolveTagDetails is real unless a test sets an override (it never rejects on its own). Plain holder: survives mock resets. +const tagDetailsOverride = vi.hoisted(() => ({ resolve: undefined as undefined | (() => Promise) })); +vi.mock('@/lib/processors/organize-tag-details', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + resolveTagDetails: (...args: Parameters) => + tagDetailsOverride.resolve ? tagDetailsOverride.resolve() : actual.resolveTagDetails(...args), + }; +}); + describe('processOrganizeFiles', () => { // The processor's first import is slow under a loaded parallel run; pay it once here beforeAll(async () => { @@ -783,6 +794,174 @@ describe('processOrganizeFiles', () => { }) ); }); + + describe('metadata tag details', () => { + function setupBook(overrides: Record = {}) { + prismaMock.request.update.mockResolvedValue({}); + prismaMock.audiobook.findUnique.mockResolvedValue({ + id: 'a-tags', + title: 'Book', + author: 'Author', + narrator: 'Narrator', + coverArtUrl: null, + audibleAsin: 'ASIN-TAGS', + year: 2020, + description: 'Stored short blurb.', + series: null, + seriesPart: null, + ...overrides, + }); + organizerMock.organize.mockResolvedValue({ + success: true, + targetPath: '/media/Author/Book', + filesMovedCount: 1, + errors: [], + audioFiles: ['/media/Author/Book/Book.m4b'], + }); + prismaMock.audiobook.update.mockResolvedValue({}); + configMock.getBackendMode.mockResolvedValue('audiobookshelf'); + configMock.get.mockResolvedValue(null); + } + + function taggingEnabled(enabled: boolean) { + prismaMock.configuration.findUnique.mockImplementation(async (args: { where?: { key?: string } }) => + args?.where?.key === 'metadata_tagging_enabled' ? { value: String(enabled) } : null + ); + } + + async function runOrganize() { + const { processOrganizeFiles } = await import('@/lib/processors/organize-files.processor'); + const result = await processOrganizeFiles({ + requestId: 'req-tags', + audiobookId: 'a-tags', + downloadPath: '/downloads/book', + jobId: 'job-tags', + }); + expect(result.success).toBe(true); + const call = organizerMock.organize.mock.calls[0]; + return { metadata: call[1], placement: call[6] }; + } + + const taggingReads = () => + prismaMock.configuration.findUnique.mock.calls.filter((c: unknown[]) => (c[0] as { where?: { key?: string } } | undefined)?.where?.key === 'metadata_tagging_enabled'); + + it('passes the Audible summary and top-level genres, looked up once for series and tags', async () => { + setupBook(); + taggingEnabled(true); + audibleServiceMock.getAudiobookDetails.mockResolvedValue({ + asin: 'ASIN-TAGS', + title: 'Book', + author: 'Author', + description: 'Full Audible summary.', + primaryGenres: ['Science Fiction & Fantasy'], + series: 'The Saga', + seriesPart: '1', + }); + + const { metadata } = await runOrganize(); + + expect(audibleServiceMock.getAudiobookDetails).toHaveBeenCalledTimes(1); + expect(metadata).toMatchObject({ + series: 'The Saga', + seriesPart: '1', + description: 'Full Audible summary.', + genres: ['Science Fiction & Fantasy'], + }); + }); + + it('series already known: the lookup serves only the tags (one call)', async () => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + taggingEnabled(true); + audibleServiceMock.getAudiobookDetails.mockResolvedValue({ description: 'Full Audible summary.', primaryGenres: ['History'] }); + + const { metadata } = await runOrganize(); + + expect(audibleServiceMock.getAudiobookDetails).toHaveBeenCalledTimes(1); + expect(metadata).toMatchObject({ description: 'Full Audible summary.', genres: ['History'] }); + }); + + it('falls back to the stored description when the lookup finds nothing', async () => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + taggingEnabled(true); + audibleServiceMock.getAudiobookDetails.mockResolvedValue(null); + + const { metadata } = await runOrganize(); + + expect(audibleServiceMock.getAudiobookDetails).toHaveBeenCalledWith('ASIN-TAGS'); + expect(metadata.description).toBe('Stored short blurb.'); + expect(metadata.genres).toBeUndefined(); + }); + + it('does not look anything up for tags when metadata tagging is off', async () => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + taggingEnabled(false); + + const { metadata, placement } = await runOrganize(); + + expect(audibleServiceMock.getAudiobookDetails).not.toHaveBeenCalled(); + expect(metadata.description).toBeUndefined(); + expect(metadata.genres).toBeUndefined(); + expect(placement).toMatchObject({ metadataTaggingEnabled: false }); + }); + + it('runs the setup reads while the lookup is pending, and awaits it before organize()', async () => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + taggingEnabled(true); + let answer!: (value: unknown) => void; + audibleServiceMock.getAudiobookDetails.mockReturnValue(new Promise((resolve) => (answer = resolve))); + const templateRead = () => + prismaMock.configuration.findUnique.mock.calls.some( + (c: unknown[]) => (c[0] as { where?: { key?: string } } | undefined)?.where?.key === 'audiobook_path_template' + ); + + const { processOrganizeFiles } = await import('@/lib/processors/organize-files.processor'); + const run = processOrganizeFiles({ requestId: 'req-tags', audiobookId: 'a-tags', downloadPath: '/downloads/book', jobId: 'job-tags' }); + await vi.waitFor(() => expect(templateRead()).toBe(true)); + expect(organizerMock.organize).not.toHaveBeenCalled(); + + answer({ description: 'Full Audible summary.', primaryGenres: ['History'] }); + await expect(run).resolves.toMatchObject({ success: true }); + expect(organizerMock.organize.mock.calls[0][1]).toMatchObject({ description: 'Full Audible summary.', genres: ['History'] }); + }); + + it('a setup failure while the tag details are pending leaves no unhandled rejection when they then reject', async () => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + let fail!: (error: Error) => void; + tagDetailsOverride.resolve = () => new Promise((_resolve, reject) => (fail = reject)); + prismaMock.configuration.findUnique.mockImplementation(async (args: { where?: { key?: string } }) => { + if (args?.where?.key === 'metadata_tagging_enabled') return { value: 'true' }; + if (args?.where?.key === 'audiobook_path_template') throw new Error('db gone during setup'); + return null; + }); + const unhandled: unknown[] = []; + const onUnhandled = (reason: unknown) => unhandled.push(reason); + process.on('unhandledRejection', onUnhandled); + try { + const { processOrganizeFiles } = await import('@/lib/processors/organize-files.processor'); + await processOrganizeFiles({ requestId: 'req-tags', audiobookId: 'a-tags', downloadPath: '/downloads/book', jobId: 'job-tags' }).catch( + () => undefined + ); + expect(organizerMock.organize).not.toHaveBeenCalled(); + fail(new Error('tag details rejected after the setup failed')); + await new Promise((resolve) => setTimeout(resolve, 20)); + expect(unhandled).toEqual([]); + } finally { + process.off('unhandledRejection', onUnhandled); + tagDetailsOverride.resolve = undefined; + } + }); + + it.each([true, false])('reads the tagging switch once (tagging %s) and hands that read to the organizer', async (enabled) => { + setupBook({ series: 'The Saga', seriesPart: '1' }); + taggingEnabled(enabled); + + const { placement } = await runOrganize(); + + expect(taggingReads()).toHaveLength(1); + expect(configMock.get).not.toHaveBeenCalledWith('metadata_tagging_enabled'); + expect(placement).toMatchObject({ metadataTaggingEnabled: enabled }); + }); + }); }); diff --git a/tests/processors/organize-tag-details.test.ts b/tests/processors/organize-tag-details.test.ts new file mode 100644 index 00000000..45f0fa7f --- /dev/null +++ b/tests/processors/organize-tag-details.test.ts @@ -0,0 +1,165 @@ +/** + * Component: Organize Tag Details Tests + * Documentation: documentation/phase3/file-organization.md + */ + +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { + createAudibleDetailsLoader, + readMetadataTaggingEnabled, + resolveTagDetails, + TAG_DETAILS_TIMEOUT_MS, +} from '@/lib/processors/organize-tag-details'; +import type { RMABLogger } from '@/lib/utils/logger'; +import { MAX_RAW_DESCRIPTION_LENGTH } from '@/lib/utils/metadata-text'; + +const prismaMock = vi.hoisted(() => ({ configuration: { findUnique: vi.fn() } })); +const audibleServiceMock = vi.hoisted(() => ({ getAudiobookDetails: vi.fn() })); + +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); + +vi.mock('@/lib/integrations/audible.service', () => ({ + getAudibleService: () => audibleServiceMock, +})); + +const logger = { info: vi.fn(), warn: vi.fn() } as unknown as RMABLogger; + +describe('organize tag details', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + afterEach(() => { + vi.useRealTimers(); + }); + + describe('createAudibleDetailsLoader', () => { + it('looks the ASIN up once, however often it is asked', async () => { + const details = { asin: 'B01', title: 'Book', author: 'Author' }; + audibleServiceMock.getAudiobookDetails.mockResolvedValue(details); + const load = createAudibleDetailsLoader('B01'); + + await expect(load()).resolves.toBe(details); + await expect(load()).resolves.toBe(details); + expect(audibleServiceMock.getAudiobookDetails).toHaveBeenCalledTimes(1); + }); + + it('resolves to null without an ASIN and never calls Audible', async () => { + await expect(createAudibleDetailsLoader(null)()).resolves.toBeNull(); + expect(audibleServiceMock.getAudiobookDetails).not.toHaveBeenCalled(); + }); + + it('does not call Audible until it is asked', () => { + createAudibleDetailsLoader('B01'); + expect(audibleServiceMock.getAudiobookDetails).not.toHaveBeenCalled(); + }); + }); + + describe('readMetadataTaggingEnabled', () => { + it.each([ + ['true', true], + ['false', false], + [undefined, false], + ])('reads %s as %s', async (value, expected) => { + prismaMock.configuration.findUnique.mockResolvedValue(value === undefined ? null : { value }); + + await expect(readMetadataTaggingEnabled(logger)).resolves.toBe(expected); + expect(prismaMock.configuration.findUnique).toHaveBeenCalledWith({ where: { key: 'metadata_tagging_enabled' } }); + }); + + it('is undefined when the setting cannot be read (the organizer then reads and reports it), and warns', async () => { + prismaMock.configuration.findUnique.mockRejectedValue(new Error('db down')); + + await expect(readMetadataTaggingEnabled(logger)).resolves.toBeUndefined(); + expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('db down')); + }); + }); + + describe('resolveTagDetails', () => { + it('prefers the Audible summary and takes the top-level genres', async () => { + const load = vi.fn().mockResolvedValue({ description: 'Full summary.', primaryGenres: ['Romance'] }); + + await expect(resolveTagDetails('Stored blurb.', load, true, logger)).resolves.toEqual({ + description: 'Full summary.', + genres: ['Romance'], + }); + }); + + it('keeps the stored description when the lookup fails, and warns', async () => { + const load = vi.fn().mockRejectedValue(new Error('Audnexus down')); + + await expect(resolveTagDetails('Stored blurb.', load, true, logger)).resolves.toEqual({ + description: 'Stored blurb.', + genres: undefined, + }); + expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('Audnexus down')); + }); + + it('strips HTML from the stored description (client text, older rows) like the lookup path', async () => { + const load = vi.fn().mockResolvedValue(null); + + await expect( + resolveTagDetails('

Stored blurb.

Two & three.

', load, true, logger) + ).resolves.toEqual({ description: 'Stored blurb.\n\nTwo & three.', genres: undefined }); + }); + + it('bounds a huge stored description before stripping it', async () => { + const load = vi.fn().mockResolvedValue(null); + const started = performance.now(); + + const { description } = await resolveTagDetails('x '.repeat(200_000), load, true, logger); + + expect(performance.now() - started).toBeLessThan(1000); + expect(description!.length).toBeLessThanOrEqual(MAX_RAW_DESCRIPTION_LENGTH); + expect(description!.startsWith('x x x')).toBe(true); + }); + + it('treats an empty summary and an empty genre list as missing', async () => { + const load = vi.fn().mockResolvedValue({ description: '', primaryGenres: [] }); + + await expect(resolveTagDetails(null, load, true, logger)).resolves.toEqual({ + description: undefined, + genres: undefined, + }); + }); + + it.each([[false], [undefined]])('returns nothing and skips the lookup when tagging is %s', async (enabled) => { + const load = vi.fn(); + + await expect(resolveTagDetails('Stored blurb.', load, enabled, logger)).resolves.toEqual({}); + expect(load).not.toHaveBeenCalled(); + }); + + it('gives up on a lookup slower than 15 s: stored description, no genres, a warning', async () => { + vi.useFakeTimers(); + const load = vi.fn(() => new Promise(() => {})); // Audible never answers + + const pending = resolveTagDetails('Stored blurb.', load, true, logger); + await vi.advanceTimersByTimeAsync(TAG_DETAILS_TIMEOUT_MS - 1); + let settled = false; + void pending.then(() => (settled = true)); + await Promise.resolve(); + expect(settled).toBe(false); + + await vi.advanceTimersByTimeAsync(1); + await expect(pending).resolves.toEqual({ description: 'Stored blurb.', genres: undefined }); + expect(TAG_DETAILS_TIMEOUT_MS).toBe(15_000); + expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('took over 15 s')); + expect(vi.getTimerCount()).toBe(0); + }); + + it('uses a lookup the series backfill already settled without waiting, and clears its timer', async () => { + vi.useFakeTimers(); + audibleServiceMock.getAudiobookDetails.mockResolvedValue({ description: 'Full summary.', primaryGenres: ['History'] }); + const load = createAudibleDetailsLoader('B01'); + await load(); // the series backfill's await + + await expect(resolveTagDetails('Stored blurb.', load, true, logger)).resolves.toEqual({ + description: 'Full summary.', + genres: ['History'], + }); + expect(audibleServiceMock.getAudiobookDetails).toHaveBeenCalledTimes(1); + expect(vi.getTimerCount()).toBe(0); + }); + }); +}); diff --git a/tests/processors/organize-verified-cleanup.test.ts b/tests/processors/organize-verified-cleanup.test.ts index 686be030..5b7bccaa 100644 --- a/tests/processors/organize-verified-cleanup.test.ts +++ b/tests/processors/organize-verified-cleanup.test.ts @@ -205,11 +205,11 @@ describe('verified-import cleanup', () => { it('compares existing library files only when a delete depends on it', async () => { organizeWith(target(), { [parts[0]]: 'copied', [parts[1]]: 'copied', [parts[2]]: 'copied' }); await runOrganize({ downloadPath: download, downloadHistoryId: 'dh-1' }); - expect(organizerMock.organize.mock.calls[0][6]).toEqual({ compareExisting: true }); + expect(organizerMock.organize.mock.calls[0][6]).toEqual({ compareExisting: true, metadataTaggingEnabled: false }); indexers = [{ id: 7, protocol: 'torrent', removeAfterProcessing: true }]; await runOrganize({ downloadPath: download, downloadHistoryId: 'dh-1' }); - expect(organizerMock.organize.mock.calls[1][6]).toEqual({ compareExisting: false }); + expect(organizerMock.organize.mock.calls[1][6]).toEqual({ compareExisting: false, metadataTaggingEnabled: false }); }); }); diff --git a/tests/utils/file-organizer.test.ts b/tests/utils/file-organizer.test.ts index 248c76e1..cd12d132 100644 --- a/tests/utils/file-organizer.test.ts +++ b/tests/utils/file-organizer.test.ts @@ -433,15 +433,49 @@ describe('file organizer', () => { const result = await organizer.organize('/downloads/book', { title: 'Book', author: 'Author', + description: 'Summary.', + genres: ['Romance'], }, '{author}/{title}'); const expectedDir = path.join('/media', 'Author', 'Book'); expect(result.success).toBe(true); expect(result.targetPath).toBe(expectedDir); + // Description and genres reach the tagger (they are tags only, never path parts) + expect(metadataMock.tagMultipleFiles).toHaveBeenCalledWith( + [sourcePath], + expect.objectContaining({ description: 'Summary.', genres: ['Romance'] }) + ); expect(copyFileMock.copyFile).toHaveBeenCalledWith('/tmp/tagged.m4b', path.join(expectedDir, 'book.m4b')); expect(fsMock.unlink).toHaveBeenCalledWith('/tmp/tagged.m4b'); }); + it.each([ + [true, 'false'], + [false, 'true'], + ])('uses the tagging read passed in (%s) over the stored setting (%s), without reading it again', async (passed, stored) => { + configState.values.set('metadata_tagging_enabled', stored); + configState.values.set('ebook_sidecar_enabled', 'false'); + metadataMock.checkFfmpegAvailable.mockResolvedValue(true); + const sourcePath = path.join('/downloads', 'book', 'book.m4b'); + metadataMock.tagMultipleFiles.mockResolvedValue([{ success: true, filePath: sourcePath, taggedFilePath: '/tmp/tagged.m4b' }]); + fsMock.access.mockImplementation(async (filePath: string) => { + if (path.normalize(filePath).startsWith(path.normalize('/media'))) throw new Error('missing'); + }); + fsMock.mkdir.mockResolvedValue(undefined); + copyFileMock.copyFile.mockResolvedValue(undefined); + fsMock.chmod.mockResolvedValue(undefined); + fsMock.unlink.mockResolvedValue(undefined); + const organizer = new FileOrganizer('/media', '/tmp'); + (organizer as any).findAudiobookFiles = vi.fn().mockResolvedValue({ audioFiles: ['book.m4b'], coverFile: undefined, isFile: false }); + + const result = await organizer.organize('/downloads/book', { title: 'Book', author: 'Author' }, '{author}/{title}', + undefined, undefined, undefined, { metadataTaggingEnabled: passed }); + + expect(result.success).toBe(true); + expect(metadataMock.tagMultipleFiles).toHaveBeenCalledTimes(passed ? 1 : 0); + expect(prismaMock.configuration.findUnique).not.toHaveBeenCalledWith({ where: { key: 'metadata_tagging_enabled' } }); + }); + it('skips metadata tagging when ffmpeg is unavailable', async () => { configState.values.set('metadata_tagging_enabled', 'true'); configState.values.set('ebook_sidecar_enabled', 'false'); diff --git a/tests/utils/metadata-tagger.integration.test.ts b/tests/utils/metadata-tagger.integration.test.ts index 5ff66a0b..fd83b334 100644 --- a/tests/utils/metadata-tagger.integration.test.ts +++ b/tests/utils/metadata-tagger.integration.test.ts @@ -60,7 +60,7 @@ afterAll(async () => { }); /** Create a small silent fixture audio file in the requested format. */ -async function makeFixture(ext: '.m4a' | '.mp3' | '.flac'): Promise { +async function makeFixture(ext: '.m4a' | '.mp3' | '.flac', extra: string[] = []): Promise { const out = path.join(workDir, `fixture-${Date.now()}-${Math.random().toString(36).slice(2)}${ext}`); const codecArgs = ext === '.mp3' @@ -74,6 +74,7 @@ async function makeFixture(ext: '.m4a' | '.mp3' | '.flac'): Promise { '-i', 'anullsrc=channel_layout=mono:sample_rate=22050', '-t', '1', ...codecArgs, + ...extra, out, ]); return out; @@ -134,6 +135,75 @@ describe('metadata-tagger integration (real ffmpeg)', () => { 60_000 ); + describe('description, genre and audiobook flags', () => { + const DESCRIPTION = 'A heist on Luthadel.\n\nThe second paragraph \u2014 with "quotes".'; + const FLAT_DESCRIPTION = 'A heist on Luthadel. The second paragraph \u2014 with "quotes".'; + const GENRES = ['Science Fiction & Fantasy', 'Mystery, Thriller & Suspense']; + const GENRE_TAG = 'Science Fiction & Fantasy; Mystery, Thriller & Suspense'; + const details = { title: 'Book', author: 'Author', description: DESCRIPTION, genres: GENRES }; + + it.skipIf(skipReal)( + 'm4a: desc, ldes, genre, stik and pgap are written and read back', + async () => { + const result = await tagAudioFileMetadata(await makeFixture('.m4a'), details); + expect(result.success).toBe(true); + const tagged = result.taggedFilePath!; + expect(await readTag(tagged, 'description')).toBe(FLAT_DESCRIPTION); + expect(await readTag(tagged, 'synopsis')).toBe(FLAT_DESCRIPTION); + expect(await readTag(tagged, 'genre')).toBe(GENRE_TAG); + expect(await readTag(tagged, 'media_type')).toBe('2'); + expect(await readTag(tagged, 'gapless_playback')).toBe('1'); + }, + 60_000 + ); + + it.skipIf(skipReal)( + 'mp3: description and genre are written and read back', + async () => { + const result = await tagAudioFileMetadata(await makeFixture('.mp3'), details); + expect(result.success).toBe(true); + expect(await readTag(result.taggedFilePath!, 'description')).toBe(FLAT_DESCRIPTION); + expect(await readTag(result.taggedFilePath!, 'genre')).toBe(GENRE_TAG); + }, + 60_000 + ); + + it.skipIf(skipReal)( + 'flac with a DESCRIPTION already in the source: exactly one DESCRIPTION (the new one), genre read back', + async () => { + const src = await makeFixture('.flac', ['-metadata', 'DESCRIPTION=old']); + expect(await readTag(src, 'comment')).toBe('old'); // the fixture really carries one + const result = await tagAudioFileMetadata(src, details); + expect(result.success).toBe(true); + const tagged = result.taggedFilePath!; + // ffprobe (what Audiobookshelf reads) joins repeated fields: "old;new" if both were kept + expect(await readTag(tagged, 'comment')).toBe(FLAT_DESCRIPTION); + expect(await readTag(tagged, 'genre')).toBe(GENRE_TAG); + // Byte scan of the Vorbis comments, independent of ffprobe + const raw = (await fs.readFile(tagged)).toString('utf8'); + expect(raw.match(/DESCRIPTION=/gi)).toHaveLength(1); + expect(raw).toContain(`DESCRIPTION=${FLAT_DESCRIPTION}`); + expect(raw).not.toContain('DESCRIPTION=old'); + }, + 60_000 + ); + + it.skipIf(skipReal)( + 'mp3: a 200 KB description is written capped at 16 KiB (ffmpeg still runs)', + async () => { + const result = await tagAudioFileMetadata(await makeFixture('.mp3'), { + ...details, + description: 'word '.repeat(40_000), + }); + expect(result.success).toBe(true); + const readBack = (await readTag(result.taggedFilePath!, 'description'))!; + expect(Buffer.byteLength(readBack, 'utf8')).toBeLessThanOrEqual(16 * 1024); + expect(readBack.endsWith('word\u2026')).toBe(true); + }, + 60_000 + ); + }); + it.skipIf(skipReal)( 'm4a: author does NOT inherit a stray leading `"` when title contains `"`', async () => { @@ -220,4 +290,55 @@ describe.skipIf(skipReal)('m4b ASIN and layout (real ffmpeg)', () => { expect((await probe(again)).tags.ASIN).toBe('B0TESTASIN'); expect(topLevelTypes(again).slice(0, 2)).toEqual(['ftyp', 'moov']); }, 60_000); + + it('round trip: description, synopsis, genre, media type, gapless and the ASIN freeform tag all read back', async () => { + const src = await makeM4b(workDir, 'details-src.m4b', { faststart: false }); + const result = await tagAudioFileMetadata(src, { + title: 'Book', + author: 'Author', + asin: 'B0TESTASIN', + description: 'First paragraph.\n\nSecond paragraph.', + genres: ['Science Fiction & Fantasy', 'Sci-Fi/Horror'], + }); + expect(result.success).toBe(true); + const tagged = result.taggedFilePath!; + expect(await addMp4FreeformTags(tagged, '.m4b', { asin: 'B0TESTASIN' })).toEqual({ written: true }); + + const { tags } = await probe(tagged); + expect(tags).toMatchObject({ + description: 'First paragraph. Second paragraph.', + synopsis: 'First paragraph. Second paragraph.', + genre: 'Science Fiction & Fantasy; Sci-Fi & Horror', + media_type: '2', + gapless_playback: '1', + ASIN: 'B0TESTASIN', + title: 'Book', + }); + expect(countFreeform(tagged, 'ASIN')).toBe(1); + expect(topLevelTypes(tagged).slice(0, 2)).toEqual(['ftyp', 'moov']); + }, 60_000); + + it('the ASIN rewrite keeps every other tag the tagging pass wrote', async () => { + const src = await makeM4b(workDir, 'keep-src.m4b', { faststart: true }); + const result = await tagAudioFileMetadata(src, { + title: 'Book', + author: 'Author', + narrator: 'Narrator', + year: 2001, + series: 'Saga', + seriesPart: '2', + description: 'A summary.', + genres: ['History'], + }); + expect(result.success).toBe(true); + const tagged = result.taggedFilePath!; + const before = await probe(tagged); + expect(before.tags.ASIN).toBeUndefined(); + + expect(await addMp4FreeformTags(tagged, '.m4b', { asin: 'B0TESTASIN' })).toEqual({ written: true }); + + const after = await probe(tagged); + expect(after.tags).toEqual({ ...before.tags, ASIN: 'B0TESTASIN' }); + expect(after.streams).toEqual(before.streams); + }, 60_000); }); diff --git a/tests/utils/metadata-tagger.test.ts b/tests/utils/metadata-tagger.test.ts index 81df6627..cb9dcfc4 100644 --- a/tests/utils/metadata-tagger.test.ts +++ b/tests/utils/metadata-tagger.test.ts @@ -12,6 +12,7 @@ import { tagAudioFileMetadata, tagMultipleFiles, } from '@/lib/utils/metadata-tagger'; +import { MAX_DESCRIPTION_BYTES } from '@/lib/utils/metadata-text'; const execFileMock = vi.hoisted(() => vi.fn()); const fsMock = vi.hoisted(() => ({ @@ -359,6 +360,238 @@ describe('metadata tagger', () => { }); }); + describe('description, genre and audiobook flags', () => { + const DESCRIPTION = 'A heist on Luthadel.\n\nThe second paragraph.'; + const FLAT_DESCRIPTION = 'A heist on Luthadel. The second paragraph.'; + const GENRES = ['Science Fiction & Fantasy', 'Mystery, Thriller & Suspense']; + const GENRE_TAG = 'Science Fiction & Fantasy; Mystery, Thriller & Suspense'; + + function metadataPayloads(argv: string[]): string[] { + return argv.filter((_, i) => i > 0 && argv[i - 1] === '-metadata'); + } + + const payloadFor = (argv: string[], key: string) => + metadataPayloads(argv).filter((p) => p.startsWith(`${key}=`)).map((p) => p.slice(key.length + 1)); + + it('m4b: writes desc + ldes, joined genres, media type Audiobook and gapless, before -movflags and -f mp4', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4b', { + title: 'Book', + author: 'Author', + description: DESCRIPTION, + genres: GENRES, + }); + + const argv = lastCallArgv(); + expectMetadataArg(argv, 'description', FLAT_DESCRIPTION); + expectMetadataArg(argv, 'synopsis', FLAT_DESCRIPTION); + expectMetadataArg(argv, 'genre', GENRE_TAG); + expectMetadataArg(argv, 'media_type', '2'); + expectMetadataArg(argv, 'gapless_playback', '1'); + expect(argv.slice(-5)).toEqual(['-movflags', '+faststart', '-f', 'mp4', '/tmp/book.m4b.tmp']); + }); + + it('m4b: always marks the file as an audiobook, even without description or genres', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4a', { title: 'Book', author: 'Author' }); + + const argv = lastCallArgv(); + expectMetadataArg(argv, 'media_type', '2'); + expectMetadataArg(argv, 'gapless_playback', '1'); + const payloads = metadataPayloads(argv); + expect(payloads.some((p) => p.startsWith('description='))).toBe(false); + expect(payloads.some((p) => p.startsWith('synopsis='))).toBe(false); + expect(payloads.some((p) => p.startsWith('genre='))).toBe(false); + }); + + it('mp3: writes description (TXXX) and joined genres, no MP4-only fields', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.mp3', { title: 'Book', author: 'Author', description: DESCRIPTION, genres: GENRES }); + + const argv = lastCallArgv(); + expectMetadataArg(argv, 'description', FLAT_DESCRIPTION); + expectMetadataArg(argv, 'genre', GENRE_TAG); + for (const key of ['comment', 'synopsis', 'media_type', 'gapless_playback']) { + expect(payloadFor(argv, key)).toEqual([]); + } + }); + + it('flac: writes the description as comment (the Vorbis DESCRIPTION field), never a second description key', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.flac', { title: 'Book', author: 'Author', description: DESCRIPTION, genres: GENRES }); + + const argv = lastCallArgv(); + expectMetadataArg(argv, 'comment', FLAT_DESCRIPTION); + expectMetadataArg(argv, 'genre', GENRE_TAG); + for (const key of ['description', 'synopsis', 'media_type', 'gapless_playback']) { + expect(payloadFor(argv, key)).toEqual([]); + } + }); + + it('turns CRLF and indented line breaks into single spaces (words never run together)', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4b', { + title: 'Book', + author: 'Author', + description: ' First line.\r\n\r\n Second line.\nThird\u0000 line. ', + }); + + expectMetadataArg(lastCallArgv(), 'description', 'First line. Second line. Third line.'); + }); + + it('turns U+2028/U+2029, tabs, VT and FF into spaces and drops other C0 controls', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.mp3', { + title: 'Book', + author: 'Author', + description: 'One.\u2028Two.\u2029Three.\tFour\u0007\u001b five\u000bsix\u000cseven.', + }); + + expectMetadataArg(lastCallArgv(), 'description', 'One. Two. Three. Four five six seven.'); + }); + + it('sanitizes each genre and drops empty ones', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4b', { + title: 'Book', + author: 'Author', + genres: ['\ufeff Romance ', '', ' ', 'History'], + }); + + expectMetadataArg(lastCallArgv(), 'genre', 'Romance; History'); + }); + + it("replaces '/' inside a genre name with ' & ' (Audiobookshelf would split on it)", async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4b', { + title: 'Book', + author: 'Author', + genres: ['Science Fiction/Fantasy', 'Mystery / Thriller'], + }); + + expectMetadataArg(lastCallArgv(), 'genre', 'Science Fiction & Fantasy; Mystery & Thriller'); + }); + + it('omits description and genre when they sanitize to empty', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.mp3', { + title: 'Book', + author: 'Author', + description: ' \r\n ', + genres: [' ', ''], + }); + + const payloads = metadataPayloads(lastCallArgv()); + expect(payloads.some((p) => p.startsWith('description='))).toBe(false); + expect(payloads.some((p) => p.startsWith('genre='))).toBe(false); + }); + + it('caps a 200 KB description at 16 KiB in both MP4 fields, cut at a word boundary with an ellipsis', async () => { + mockExecFileSuccess('done'); + const huge = 'word '.repeat(40_000); // 200,000 bytes + + await tagAudioFileMetadata('/tmp/book.m4b', { title: 'Book', author: 'Author', description: huge }); + + const argv = lastCallArgv(); + for (const key of ['description', 'synopsis']) { + const [value] = payloadFor(argv, key); + expect(Buffer.byteLength(value, 'utf8')).toBeLessThanOrEqual(MAX_DESCRIPTION_BYTES); + expect(Buffer.byteLength(value, 'utf8')).toBeGreaterThan(MAX_DESCRIPTION_BYTES - 16); + expect(value.endsWith('word\u2026')).toBe(true); + const head = value.slice(0, -1); + expect(huge.startsWith(head)).toBe(true); + expect(huge[head.length]).toBe(' '); // the cut falls between words + } + }); + + it('collapses a whitespace-only line or a run of tabs into one space', async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.mp3', { title: 'Book', author: 'Author', description: 'One.\n \n\t\tTwo. \t Three.' }); + + expectMetadataArg(lastCallArgv(), 'description', 'One. Two. Three.'); + }); + + it.each([ + // past the raw bound: cut to spaces only, so no description at all + [200_000, []], + // inside the raw bound: the whitespace regex alone (the old one took ~30 s here) + [60_000, ['x']], + ])('sanitizes %i spaces before a word in linear time (was quadratic: ~1 min at 200k)', async (spaces, expected) => { + mockExecFileSuccess('done'); + const started = performance.now(); + + await tagAudioFileMetadata('/tmp/book.m4b', { title: 'Book', author: 'Author', description: ' '.repeat(spaces) + 'x' }); + + expect(performance.now() - started).toBeLessThan(1000); + expect(payloadFor(lastCallArgv(), 'description')).toEqual(expected); + }); + + it('bounds a 1 MB mixed-whitespace description: fast, single spaces, within the cap', async () => { + mockExecFileSuccess('done'); + const chunk = 'word \t\r\n\u000b\u000c\u2028\u2029 '; + const huge = chunk.repeat(Math.ceil(1_000_000 / chunk.length)); + const started = performance.now(); + + await tagAudioFileMetadata('/tmp/book.m4b', { title: 'Book', author: 'Author', description: huge }); + + expect(performance.now() - started).toBeLessThan(1000); + const [value] = payloadFor(lastCallArgv(), 'description'); + expect(Buffer.byteLength(value, 'utf8')).toBeLessThanOrEqual(MAX_DESCRIPTION_BYTES); + expect(value).toMatch(/^word( word)+\u2026$/); + }); + + it("replaces ';' inside a genre name with ' & ' too", async () => { + mockExecFileSuccess('done'); + + await tagAudioFileMetadata('/tmp/book.m4b', { title: 'Book', author: 'Author', genres: ['Sci-Fi; Horror', ' ; History / '] }); + + expectMetadataArg(lastCallArgv(), 'genre', 'Sci-Fi & Horror; History'); + }); + + it('sanitizes a genre with 200,000 spaces (and one with spaces around a slash) in linear time', async () => { + mockExecFileSuccess('done'); + const started = performance.now(); + + await tagAudioFileMetadata('/tmp/book.m4b', { + title: 'Book', + author: 'Author', + genres: [' '.repeat(200_000) + 'x', ' '.repeat(1_000) + '/' + ' '.repeat(1_000) + 'y'], + }); + + expect(performance.now() - started).toBeLessThan(1000); + expectMetadataArg(lastCallArgv(), 'genre', 'x; y'); + }); + + it('a failed run reports ffmpeg stderr, never the argv (so no description in job logs)', async () => { + execFileMock.mockImplementation((file: string, args: string[], options: unknown, callback?: unknown) => { + const cb = (typeof options === 'function' ? options : callback) as (e: Error, out: string, err: string) => void; + const error = Object.assign(new Error(`Command failed: ffmpeg ${args.join(' ')}\nInvalid data found`), { + code: 1, + stderr: 'Input #0 ...\n[mp3 @ 0x1] Header missing\nInvalid data found when processing input\n', + }); + cb(error, '', ''); + }); + + const result = await tagAudioFileMetadata('/tmp/book.mp3', { title: 'Book', author: 'Author', description: 'SECRET summary text' }); + + expect(result.success).toBe(false); + expect(result.error).toContain('Invalid data found when processing input'); + expect(result.error).not.toContain('SECRET'); + expect(result.error).not.toContain('-metadata'); + }); + + }); + describe('quote regression (#171) — embedded `"` flows through untouched', () => { const tricky = 'Test "Quoted" Book'; diff --git a/tests/utils/metadata-text.test.ts b/tests/utils/metadata-text.test.ts new file mode 100644 index 00000000..c1b6f55d --- /dev/null +++ b/tests/utils/metadata-text.test.ts @@ -0,0 +1,63 @@ +/** + * Component: Metadata Text Helpers Tests + * Documentation: documentation/phase3/file-organization.md + */ + +import { describe, expect, it } from 'vitest'; +import { + capUtf8Bytes, + describeFfmpegFailure, + MAX_DESCRIPTION_BYTES, + MAX_RAW_DESCRIPTION_LENGTH, + truncateRawText, +} from '@/lib/utils/metadata-text'; + +describe('metadata text helpers', () => { + it.each([ + ['2-byte', '\u00e9'], + ['4-byte', '\u{1f600}'], + ])('caps %s multibyte text without splitting a code point', (_label, char) => { + const text = char.repeat(20_000); // no spaces: cut at the byte limit + const capped = capUtf8Bytes(text, MAX_DESCRIPTION_BYTES); + + expect(Buffer.byteLength(capped, 'utf8')).toBeLessThanOrEqual(MAX_DESCRIPTION_BYTES); + expect(capped.endsWith('\u2026')).toBe(true); + expect(capped).not.toContain('\ufffd'); + expect(Buffer.from(capped, 'utf8').toString('utf8')).toBe(capped); // no lone surrogate + expect(text.startsWith(capped.slice(0, -1))).toBe(true); + }); + + it('leaves a description that fits unchanged', () => { + const exact = 'a'.repeat(MAX_DESCRIPTION_BYTES); + expect(capUtf8Bytes(exact, MAX_DESCRIPTION_BYTES)).toBe(exact); + }); + + it('cuts raw text to its bound without splitting a surrogate pair', () => { + expect(truncateRawText('aaa\u{1f600}b', 4)).toBe('aaa'); + expect(truncateRawText('aaa\u{1f600}b', 5)).toBe('aaa\u{1f600}'); + expect(truncateRawText('short', 10)).toBe('short'); + expect(truncateRawText('x'.repeat(MAX_RAW_DESCRIPTION_LENGTH + 5))).toHaveLength(MAX_RAW_DESCRIPTION_LENGTH); + expect(MAX_RAW_DESCRIPTION_LENGTH).toBe(4 * MAX_DESCRIPTION_BYTES); + }); + + it.each([ + [0, 'abcdef', ''], + [1, 'abcdef', 'a'], + [2, 'abcdef', 'ab'], + [1, '\u00e9\u00e9', ''], + [3, 'abcdef', '\u2026'], + ])('caps at maxBytes %i without going over (no ellipsis below 3 bytes)', (maxBytes, text, expected) => { + const capped = capUtf8Bytes(text, maxBytes); + expect(capped).toBe(expected); + expect(Buffer.byteLength(capped, 'utf8')).toBeLessThanOrEqual(maxBytes); + }); + + it.each([ + ['no stderr', Object.assign(new Error('Command failed: ffmpeg -metadata description=SECRET\nboom'), { code: 1 }), 'boom'], + ['killed', Object.assign(new Error('Command failed: ffmpeg -metadata description=SECRET'), { killed: true, signal: 'SIGTERM' }), 'ffmpeg was stopped (SIGTERM)'], + ['only the command line', Object.assign(new Error('Command failed: ffmpeg -metadata description=SECRET'), { code: 2 }), 'exit code 2'], + ['a plain error', new Error('spawn ffmpeg ENOENT'), 'spawn ffmpeg ENOENT'], + ])('describeFfmpegFailure: %s', (_label, error, expected) => { + expect(describeFfmpegFailure(error)).toBe(expected); + }); +});