Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion documentation/integrations/audible.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,8 @@ interface AudibleAudiobook {
durationMinutes?: number;
releaseDate?: string;
rating?: number;
genres?: string[];
genres?: string[]; // display: genres + sub-genre tags, max 5
primaryGenres?: string[]; // top-level genres only (Audnexus type 'genre' / top of each catalog ladder); metadata tags
series?: string;
seriesPart?: string;
seriesAsin?: string;
Expand Down
15 changes: 14 additions & 1 deletion documentation/phase3/file-organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Result: Douglas Adams/Stephen Fry/The Hitchhiker's Guide to the Galaxy/
3. Read media directory and path template from database config (`media_dir`, `audiobook_path_template`)
4. Apply template to create target path: `[media_dir]/[template result]/`
5. **Copy** files (not move - originals stay for seeding)
6. **Tag metadata** (if enabled) - writes correct title, author, narrator, ASIN to audio files
6. **Tag metadata** (if enabled) - writes correct title, author, narrator, ASIN, series, description, genre to audio files
7. Copy cover art if found, else download from Audible. `coverArtUrl` must be a cached thumbnail `/api/cache/thumbnails/<file>` (plain file name, resolved in `/app/cache/thumbnails`) or an https URL on `COVER_ART_HOSTS` (`*.media-amazon.com`, `*.ssl-images-amazon.com`, `*.images-amazon.com`), up to `MAX_COVER_ART_REDIRECTS` (3) redirects with every hop checked; anything else is skipped with a warning (`src/lib/utils/cover-art-url.ts`). An existing `cover.jpg` is never replaced and not fetched for (kept, debug log; a re-import with the same cover lands here too); an empty regular-file `cover.jpg` (older interrupted write) counts as missing and is replaced (`replace` publish, size re-checked under the lock). New covers are published like audio (`src/lib/utils/cover-placement.ts` → `publishCover`): hidden partial (`copyFile` for a cached thumbnail, `writeFileAtomic` for downloaded bytes), fsync + size check, `create` publish (`EEXIST` → kept), under the per-target lock; an interrupted write leaves no `cover.jpg`. Downloaded and copied covers get the library file mode (`applyFileMode`: a refused chmod is a permission warning, a missing file a failure). At request creation, covers are stored for display if they are any http(s) URL or a safe cached path (`sanitizeCoverArtUrl`; shelf-sync Goodreads/Hardcover fallbacks stay visible); a stored cover the library can't use is replaced by a later usable one (`isLibraryCoverSource`)
8. **Coerce file formats** (if enabled) - rename .mp4 → .m4b and single-file .m4a → .m4b for Plex compatibility (see: Plex Format Coercion below)
9. **Generate file hash** - SHA256 of sorted audio filenames for library matching (see: [fixes/file-hash-matching.md](../fixes/file-hash-matching.md))
Expand Down Expand Up @@ -108,6 +108,18 @@ Result: Douglas Adams/Stephen Fry/The Hitchhiker's Guide to the Galaxy/
- `ASIN` - Audible ASIN (custom tag)
- M4B/M4A/MP4: `----:com.apple.iTunes:ASIN`
- MP3: Custom ID3v2 tag
- Description - publisher summary, line breaks → single spaces
- M4B/M4A/MP4: `description` (`desc`) + `synopsis` (`ldes`)
- MP3: `description` (TXXX); FLAC: `description` (Vorbis DESCRIPTION, ffprobe shows it as `comment`)
- `genre` - top-level genres (`primaryGenres`, no sub-genre tags) joined with `; ` (Audiobookshelf splits on `;`)
- M4B/M4A/MP4 only: `media_type=2` (`stik` Audiobook), `gapless_playback=1` (`pgap`) - always written
- Fixed set, no per-field settings; all under `metadata_tagging_enabled`

**Tag details lookup** (`src/lib/processors/organize-tag-details.ts`):
- Only when tagging is on; one `getAudiobookDetails(asin)` per import, shared with the missing-series backfill
- Description: Audible summary (Audnexus `summary`) → stored `Audiobook.description` fallback
- Lookup failure is non-fatal (logged); the book is tagged with what is stored
- Genres/description are tags only (not path template variables)

**Note:** ASIN is a custom metadata tag and may not appear in standard file properties viewers (Windows/macOS/Linux). Use specialized tools to verify:
```bash
Expand Down Expand Up @@ -142,6 +154,7 @@ exiftool "audiobook.m4b" | grep -i asin
- ffmpeg (system dependency - included in Docker image)
- `src/lib/utils/metadata-tagger.ts` - Tagging utility
- Integrated into `src/lib/utils/file-organizer.ts`
- `src/lib/processors/organize-tag-details.ts` - description/genres for the tags

**Requirements:**
- ffmpeg + ffprobe must be on PATH (`/usr/local/bin`)
Expand Down
24 changes: 24 additions & 0 deletions src/lib/integrations/audible.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ export interface AudibleAudiobook {
releaseDate?: string;
rating?: number;
genres?: string[];
/** Top-level genres only (no sub-genre tags), as written into the audio file tags */
primaryGenres?: string[];
series?: string;
seriesPart?: string;
seriesAsin?: string;
Expand Down Expand Up @@ -156,6 +158,14 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook {
? [...new Set(genreNames)].slice(0, 5)
: undefined;

// Each ladder runs from its top-level genre down to sub-genres
const primaryGenreNames =
product.category_ladders
?.map((ladder) => ladder.ladder[0]?.name)
.filter((name): name is string => !!name) ?? [];
const primaryGenres =
primaryGenreNames.length > 0 ? [...new Set(primaryGenreNames)] : undefined;

let series: string | undefined;
let seriesPart: string | undefined;
let seriesAsin: string | undefined;
Expand Down Expand Up @@ -186,6 +196,7 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook {
releaseDate: product.release_date ?? undefined,
rating: product.rating?.overall_distribution?.display_stars ?? undefined,
genres,
primaryGenres,
series,
seriesPart,
seriesAsin,
Expand All @@ -195,6 +206,18 @@ function mapCatalogProduct(product: CatalogProduct): AudibleAudiobook {
};
}

/**
* Top-level genres from an Audnexus genre list. Audnexus marks the top of each
* category ladder `type: 'genre'` and the sub-genres below it `type: 'tag'`.
*/
function audnexusPrimaryGenres(genres: unknown): string[] | undefined {
if (!Array.isArray(genres)) return undefined;
const names = (genres as Array<string | { name?: unknown; type?: unknown } | null>)
.map((g) => (typeof g === 'string' ? g : g?.type !== 'tag' ? g?.name : undefined))
.filter((name): name is string => typeof name === 'string' && name.trim() !== '');
return names.length > 0 ? [...new Set(names)] : undefined;
}

export class AudibleService {
private htmlClient!: AxiosInstance;
private apiClient!: AxiosInstance;
Expand Down Expand Up @@ -686,6 +709,7 @@ export class AudibleService {
releaseDate: data.releaseDate || undefined,
rating: data.rating ? parseFloat(data.rating) : undefined,
genres: data.genres?.map((g: any) => (typeof g === 'string' ? g : g.name)).slice(0, 5) || undefined,
primaryGenres: audnexusPrimaryGenres(data.genres),
series: data.seriesPrimary?.name || undefined,
seriesPart: data.seriesPrimary?.position || undefined,
seriesAsin: data.seriesPrimary?.asin || undefined,
Expand Down
11 changes: 9 additions & 2 deletions src/lib/processors/organize-files.processor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import {
} from './organize-evidence';
import { getAudibleService } from '../integrations/audible.service';
import { addAutoBlock } from '../services/blocklist.service';
import { createAudibleDetailsLoader, resolveTagDetails } from './organize-tag-details';

/**
* Process organize files job
Expand Down Expand Up @@ -134,11 +135,13 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi
let series = audiobook.series || undefined;
let seriesPart = audiobook.seriesPart || undefined;

// At most one Audible details lookup per import, shared with the metadata tags below
const loadAudibleDetails = createAudibleDetailsLoader(audiobook.audibleAsin);

if (audiobook.audibleAsin && !series) {
try {
logger.info(`Missing series data, fetching from Audnexus for ASIN: ${audiobook.audibleAsin}`);
const audibleService = getAudibleService();
const audnexusData = await audibleService.getAudiobookDetails(audiobook.audibleAsin);
const audnexusData = await loadAudibleDetails();

if (audnexusData) {
const updates: Record<string, any> = {};
Expand Down Expand Up @@ -187,6 +190,9 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi

logger.info(`Final metadata for path organization: year=${year || 'null'}, narrator=${narrator || 'null'}, series=${series || 'null'}, seriesPart=${seriesPart || 'null'}`);

// Description and genres for the metadata tags (looked up only when tagging is on)
const tagDetails = await resolveTagDetails(audiobook.description, loadAudibleDetails, logger);

// Get file organizer (reads media_dir from database config)
const organizer = await getFileOrganizer();

Expand Down Expand Up @@ -223,6 +229,7 @@ export async function processOrganizeFiles(payload: OrganizeFilesPayload): Promi
year,
series,
seriesPart,
...tagDetails,
},
template,
jobId ? { jobId, context: 'FileOrganizer' } : undefined,
Expand Down
57 changes: 57 additions & 0 deletions src/lib/processors/organize-tag-details.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/**
* Component: Organize Tag Details (Audible details for series backfill and metadata tags)
* Documentation: documentation/phase3/file-organization.md
*/

import type { RMABLogger } from '../utils/logger';
import { getConfigService } from '../services/config.service';
import { getAudibleService, type AudibleAudiobook } from '../integrations/audible.service';

export type AudibleDetailsLoader = () => Promise<AudibleAudiobook | null>;

/** Description and genres written into the audio file tags. */
export interface TagDetails {
description?: string;
genres?: string[];
}

/**
* At most one Audible details lookup per import, shared by the series backfill and
* the metadata tags. Resolves to null without an ASIN; never called until needed.
*/
export function createAudibleDetailsLoader(asin: string | null | undefined): AudibleDetailsLoader {
let pending: Promise<AudibleAudiobook | null> | undefined;
return () => {
if (!asin) return Promise.resolve(null);
pending ??= getAudibleService().getAudiobookDetails(asin);
return pending;
};
}

/**
* Description and genres for the metadata tags. Looked up only when metadata tagging
* is on. Prefers the full Audible summary, falls back to the stored description.
* A failed lookup never blocks the import: the book is tagged with what is stored.
*/
export async function resolveTagDetails(
storedDescription: string | null | undefined,
loadDetails: AudibleDetailsLoader,
logger: RMABLogger
): Promise<TagDetails> {
const taggingEnabled = (await getConfigService().get('metadata_tagging_enabled')) === 'true';
if (!taggingEnabled) return {};

let details: AudibleAudiobook | null = null;
try {
details = await loadDetails();
} catch (error) {
logger.warn(`Failed to fetch Audible details for metadata tags: ${error instanceof Error ? error.message : String(error)}`);
}

const description = details?.description || storedDescription || undefined;
const genres = details?.primaryGenres?.length ? details.primaryGenres : undefined;

logger.info(`Metadata tag details: description=${description ? `${description.length} chars` : 'none'}, genres=${genres ? genres.join(', ') : 'none'}`);

return { description, genres };
}
4 changes: 4 additions & 0 deletions src/lib/utils/file-organizer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ export interface AudiobookMetadata {
asin?: string;
series?: string;
seriesPart?: string;
description?: string; // Metadata tags only (not used in path templates)
genres?: string[]; // Metadata tags only (not used in path templates)
}

export interface OrganizationResult {
Expand Down Expand Up @@ -299,6 +301,8 @@ export class FileOrganizer {
asin: audiobook.asin,
series: audiobook.series,
seriesPart: audiobook.seriesPart,
description: audiobook.description,
genres: audiobook.genres,
});

const successCount = taggingResults.filter((r) => r.success).length;
Expand Down
55 changes: 55 additions & 0 deletions src/lib/utils/metadata-tagger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,16 @@ export interface MetadataTaggingOptions {
asin?: string;
series?: string;
seriesPart?: string;
description?: string; // Publisher summary (plain text, may contain paragraphs)
genres?: string[]; // Top-level genres, joined into one tag
}

/**
* Joins genres into one tag. Audiobookshelf splits genre tags on '/', '//' or ';'
* (first match wins); Audible genre names contain '&' and ',' but never ';'.
*/
const GENRE_SEPARATOR = '; ';

export interface TaggingResult {
success: boolean;
filePath: string; // Original file path
Expand All @@ -43,6 +51,14 @@ function sanitizeMetadataValue(value: string): string {
.trim();
}

/**
* Sanitize a multi-paragraph value: line breaks become single spaces first, so
* paragraphs never run together when sanitizeMetadataValue drops the breaks.
*/
function sanitizeMultilineValue(value: string): string {
return sanitizeMetadataValue(value.replace(/[ \t]*[\r\n]+[ \t]*/g, ' '));
}

/**
* Tag audio file metadata using ffmpeg
* Supports m4b/m4a/mp4, mp3, and flac files
Expand Down Expand Up @@ -77,6 +93,11 @@ export async function tagAudioFileMetadata(
const asin = metadata.asin ? sanitizeMetadataValue(metadata.asin) : undefined;
const series = metadata.series ? sanitizeMetadataValue(metadata.series) : undefined;
const seriesPart = metadata.seriesPart ? sanitizeMetadataValue(metadata.seriesPart) : undefined;
const description = metadata.description ? sanitizeMultilineValue(metadata.description) : undefined;
const genre = metadata.genres
?.map(sanitizeMetadataValue)
.filter((name) => name.length > 0)
.join(GENRE_SEPARATOR);

// Create temporary file path
const tempFile = `${filePath}.tmp`;
Expand Down Expand Up @@ -119,6 +140,23 @@ export async function tagAudioFileMetadata(
args.push('-metadata', `episode_id=${seriesPart}`);
}

if (description) {
args.push(
'-metadata', `description=${description}`, // desc atom (read by Audiobookshelf)
'-metadata', `synopsis=${description}` // ldes atom (long description)
);
}

if (genre) {
args.push('-metadata', `genre=${genre}`);
}

// Mark the file as an audiobook (stik=2) with gapless playback (pgap=1)
args.push(
'-metadata', 'media_type=2',
'-metadata', 'gapless_playback=1'
);

// Explicitly specify output format (fixes .tmp extension issue)
args.push('-f', 'mp4');
}
Expand Down Expand Up @@ -152,6 +190,14 @@ export async function tagAudioFileMetadata(
args.push('-metadata', `SERIES-PART=${seriesPart}`);
}

if (description) {
args.push('-metadata', `description=${description}`);
}

if (genre) {
args.push('-metadata', `genre=${genre}`);
}

// Explicitly specify output format
args.push('-f', 'flac');
}
Expand Down Expand Up @@ -186,6 +232,15 @@ export async function tagAudioFileMetadata(
args.push('-metadata', `SERIES-PART=${seriesPart}`);
}

if (description) {
// TXXX frame, read back as `description` (Audiobookshelf reads this)
args.push('-metadata', `description=${description}`);
}

if (genre) {
args.push('-metadata', `genre=${genre}`); // TCON frame
}

// Explicitly specify output format (fixes .tmp extension issue)
args.push('-f', 'mp3');
}
Expand Down
Loading