feat: Add Whisper speech-to-text captioning for video and audio (.srt) (Continues #774)

THIRD-PARTY-NOTICES.md

@@ -81,6 +81,8 @@ Downloaded from their official release pages when the user chooses to install th
 | calibre (ebook-convert) | GPL-3.0 | https://calibre-ebook.com |
 | Tesseract OCR 5.4 (UB Mannheim Windows build, with Leptonica and the MinGW libraries it ships; their notices are in its `doc` folder) | Apache-2.0 (Tesseract); Leptonica BSD-2-Clause; others as listed in the build | https://github.com/tesseract-ocr/tesseract, build: https://github.com/UB-Mannheim/tesseract |
 | Tesseract language data (tessdata_fast 4.1.0), downloaded per language when used | Apache-2.0 | https://github.com/tesseract-ocr/tessdata_fast |
+| whisper.cpp (whisper-cli.exe Windows build) | MIT | https://github.com/ggml-org/whisper.cpp |
+| ggml Whisper base model (`ggml-base.bin`) | MIT | https://huggingface.co/ggerganov/whisper.cpp |
 
 These programs run as separate processes. Their source code is available from the linked projects.
 

docs/ENGINES.md

@@ -37,6 +37,7 @@ Filee chooses engines automatically. Settings → *Engines* shows their status,
 | Apple Vision | part of macOS (Mac only) | **text recognition (OCR) on a Mac**, the same step as Tesseract without a download; Tesseract reads the languages Vision doesn't (see "Apple Vision on macOS") |
 | FFmpeg | **optional download** (~100 MB, 272 MB on disk) | **all video and audio**: video ↔ video (H.264, HEVC, AV1), video → animated GIF / APNG, a still frame or every frame, audio and subtitle extraction, audio ↔ audio (also APE, WavPack, TTA, DSD, DTS, ALAC, …), GIF, APNG and photos → MP4/WEBM/MOV, Live / Motion Photos → their video, a clip of a video → an iPhone Live Photo or Android Motion Photo (experimental), images → HEIC, clips or audio files joined into one, pictures → a slideshow MP4, copies without metadata (see "Media tools") |
 | Calibre | **optional download** (~216 MB, 660 MB on disk) | rare e-book formats: LIT, LRF, CHM, PDB, PML, RB, SNB, TCR, OEB → EPUB; EPUB → MOBI, AZW3, LIT, LRF, PDB, PML, RB, SNB, TCR; e-books → PDF (ranked above the HWPX writer, see below) |
+| Whisper | **optional download** (Windows only; ~157 MB with model, ~160 MB on disk) | **Speech-to-text transcription**: video and audio → SRT subtitles in ~100 languages using OpenAI Whisper (whisper.cpp); runs locally and offline; needs FFmpeg |
 
 DOCX, XLSX, XLS, ODS and PPTX → PDF need neither Microsoft Office nor LibreOffice: they are read in-process, written as HWPX
 and rendered by rhwp. Anything the built-in readers understand (including PDF) is written as DOCX by the DOCX writer.
@@ -1762,3 +1763,18 @@ Tesseract and is registered before it, so the route planner takes it; on Windows
   autorelease pool per page. A cancelled job cancels the running request.
 - Tests: `VisionLanguageTests`, `OcrTextLayerTests` and `VisionConverterTests` (a stand-in recognizer) run everywhere;
   `AppleVisionTests` read real pictures on macOS (CI's Apple silicon and Intel jobs).
+
+## Whisper
+
+Filee transcribes speech in video and audio files into SRT subtitle files using OpenAI Whisper (`whisper.cpp`).
+Transcription runs entirely locally and offline on your computer; no audio or data is sent to external servers.
+
+- **Platform**: Windows only (x64). On macOS and Linux, the Whisper package is not offered and the engine reports
+  `engine.reason.unsupported_platform` ("Not available on this platform").
+- **Download size**: ~157 MB total (~9 MB for `whisper-cli.exe` and ~148 MB for the default `ggml-base.bin` model; ~160 MB on disk).
+  Downloads are pinned by SHA-256 in `engines.json`. Users can also place larger Whisper models (such as `ggml-small.bin`
+  or `ggml-large-v3-turbo.bin`) into the engine directory or point to a custom installation in Settings → *Engines*.
+- **FFmpeg requirement**: Whisper operates on 16 kHz mono 16-bit WAV audio. Extracting audio from arbitrary video and audio
+  formats requires FFmpeg. FFmpeg must be installed (via Settings → *Engines*, bundled, or found on `PATH`).
+- **Languages**: Whisper supports auto-detection (`auto`) as well as specific language codes (`-l <code>`, e.g. `hi` for
+  Hindi, `en` for English, `es` for Spanish, `fr` for French, `ja` for Japanese, `zh` for Chinese).

src/Filee.App/Assets/i18n/ar.json

@@ -206,6 +206,8 @@
   "engine.subtitle.name": "Subtitles (built-in)",
   "engine.subtitle.description": "التحويل بين ملفات الترجمة: SRT وASS/SSA وWebVTT وSAMI ‏(SMI) وLRC وTTML وMicroDVD وSCC، وحفظها بترميز UTF-8.\nتُقرأ ملفات SMI وSRT الكورية بترميز CP949 / EUC-KR بصورة صحيحة.",
   "engine.ffmpeg.description": "الفيديو والصوت: MP4 وMOV وMKV وWEBM وMP3 وM4A وWAV وFLAC وغيرها.\nواستخراج الصوت والترجمة، وإنشاء رسوم GIF وAPNG المتحركة، وحفظ الصور بصيغة HEIC.",
+  "engine.whisper.name": "Whisper",
+  "engine.whisper.description": "تحويل الكلام إلى نص مدعوم بواسطة OpenAI Whisper. يُنشئ ترجمات SRT تلقائياً من الكلام في الفيديو والصوت.",
   "engine.pandoc.description": "التحويل بين Markdown وHTML وبين ODT وRTF، ومن DOCX إلى Markdown\nالتحويل بين reStructuredText وLaTeX وOrg وAsciiDoc وTypst وMediaWiki وDocBook وJupyter وغيرها وبين Markdown وHTML وDOCX وODT\nتحويل Markdown وOrg إلى شرائح PPTX",
   "engine.ghostscript.description": "تحويل EPS وPostScript وملفات Illustrator القديمة إلى PDF وصور؛ وتحويل PDF إلى EPS وPS.",
   "engine.tesseract.description": "يتعرّف على نص الصور والمستندات الممسوحة (OCR): PDF قابل للبحث وDOCX وTXT.",
@@ -409,6 +411,8 @@
   "preset.extract": "استخراج",
   "preset.zip_all": "ZIP واحد",
   "preset.extract_subtitles": "استخراج الترجمة",
+  "preset.caption_video": "تسميات توضيحية",
+  "preset.caption_video.donut": "تسميات",
   "preset.trace_svg": "تتبّع إلى SVG",
   "preset.trace_svg.donut": "تتبّع إلى SVG",
   "preset.trace_line_art": "رسم خطي إلى SVG",
@@ -840,6 +844,10 @@
   "presets.video_codec.av1": "AV1 (الأصغر، بطيء)",
   "presets.hardware_encoding": "استخدام تسريع العتاد (معالج الرسوميات)",
   "presets.hardware_encoding_hint": "ترميز H.264 وHEVC وAV1 باستخدام معالج الرسوميات.\nأسرع بكثير وأقل استهلاكًا للطاقة.\nالجودة أقل قليلًا عند حجم الملف نفسه.\nعند عدم توفر معالج مناسب (يتطلب AV1 معالجًا حديثًا)، يُرمَّز الفيديو بالطريقة المعتادة.",
+  "presets.caption_video": "إنشاء ترجمات (Whisper)",
+  "presets.caption_video_hint": "تحويل الكلام في الفيديو والصوت إلى ملف ترجمة SRT باستخدام Whisper.",
+  "presets.spoken_language": "اللغة المنطوقة",
+  "presets.spoken_language.auto": "تلقائي",
   "presets.audio_codec": "ترميز الصوت",
   "presets.audio_codec.aac": "AAC",
   "presets.audio_codec.alac": "ALAC (دون فقدان)",
@@ -962,6 +970,8 @@
   "engines.package.tesseract.description": "يقرأ نص ملفات PDF الممسوحة والصور: PDF قابل للبحث وDOCX وTXT.\nتُنزَّل كل لغة عند استخدامها أول مرة.",
   "engines.package.ffmpeg.name": "FFmpeg (الفيديو والصوت)",
   "engines.package.ffmpeg.description": "يضيف تحويل الفيديو والصوت، وتحويل الفيديو إلى GIF أو صورة ثابتة، واستخراج الصوت من الفيديو، وتحويل GIF إلى MP4.\nمطلوب للفيديو والصوت فقط.",
+  "engines.package.whisper.name": "Whisper (ترجمة بالذكاء الاصطناعي)",
+  "engines.package.whisper.description": "تحويل الكلام في الفيديو والصوت إلى ملفات ترجمة SRT. يعمل بدون اتصال بالإنترنت.",
   "engines.package.calibre.name": "Calibre (Kindle وصيغ الكتب النادرة)",
   "engines.package.calibre.description": "يحفظ MOBI وAZW3 لأجهزة Kindle، ويحوّل صيغ الكتب الإلكترونية النادرة (LIT وLRF وPDB وCHM…).\nتُقرأ EPUB وMOBI وAZW3 وFB2 والقصص المصورة بدونه.",
   "engines.package.size": "التنزيل {0} · على القرص {1}",

src/Filee.App/Assets/i18n/de.json

@@ -206,6 +206,8 @@
   "engine.subtitle.name": "Untertitel (integriert)",
   "engine.subtitle.description": "Untertitel: Konvertiert SRT, ASS/SSA, WebVTT, SAMI (SMI), LRC, TTML, MicroDVD und SCC untereinander und speichert sie als UTF-8.\nKoreanische SMI- und SRT-Dateien in CP949 / EUC-KR werden korrekt gelesen.",
   "engine.ffmpeg.description": "Video und Audio: MP4, MOV, MKV, WEBM, MP3, M4A, WAV, FLAC und mehr.\nExtrahiert auch Ton und Untertitel, erstellt GIF- und APNG-Animationen und speichert Bilder als HEIC.",
+  "engine.whisper.name": "Whisper",
+  "engine.whisper.description": "Sprache-zu-Text-Transkription powered by OpenAI Whisper. Erstellt automatisch SRT-Untertitel aus Sprache in Video und Audio.",
   "engine.pandoc.description": "Markdown und HTML ↔ ODT und RTF, DOCX → Markdown\nreStructuredText, LaTeX, Org, AsciiDoc, Typst, MediaWiki, DocBook, Jupyter und mehr ↔ Markdown, HTML, DOCX und ODT\nMarkdown und Org → PPTX-Folien",
   "engine.ghostscript.description": "EPS, PostScript und ältere Illustrator-Dateien in PDF und Bilder; PDF in EPS und PS.",
   "engine.tesseract.description": "Erkennt Text in Scans und Bildern (OCR): durchsuchbare PDF, DOCX, TXT.",
@@ -409,6 +411,8 @@
   "preset.extract": "Entpacken",
   "preset.zip_all": "Eine ZIP-Datei",
   "preset.extract_subtitles": "Untertitel extrahieren",
+  "preset.caption_video": "Untertitel",
+  "preset.caption_video.donut": "Untertitel",
   "preset.trace_svg": "Vektor-SVG",
   "preset.trace_svg.donut": "Vektorisieren",
   "preset.trace_line_art": "Linien-SVG",
@@ -840,6 +844,10 @@
   "presets.video_codec.av1": "AV1 (am kleinsten, langsam)",
   "presets.hardware_encoding": "Hardwarebeschleunigung nutzen (Grafikchip)",
   "presets.hardware_encoding_hint": "Kodiert H.264, HEVC und AV1 auf dem Grafikchip.\nDeutlich schneller und stromsparender.\nEtwas geringere Qualität bei gleicher Dateigröße.\nOhne geeigneten Chip (AV1 benötigt einen neueren) wird das Video wie gewohnt kodiert.",
+  "presets.caption_video": "Untertitel erstellen (Whisper)",
+  "presets.caption_video_hint": "Transkribiert Sprache in Videos und Audio mit Whisper in eine SRT-Untertiteldatei.",
+  "presets.spoken_language": "Gesprochene Sprache",
+  "presets.spoken_language.auto": "Automatisch",
   "presets.audio_codec": "Audio-Codec",
   "presets.audio_codec.aac": "AAC",
   "presets.audio_codec.alac": "ALAC (verlustfrei)",
@@ -962,6 +970,8 @@
   "engines.package.tesseract.description": "Liest Text aus gescannten PDFs und Bildern: durchsuchbare PDF, DOCX, TXT.\nJede Sprache wird bei der ersten Verwendung heruntergeladen.",
   "engines.package.ffmpeg.name": "FFmpeg (Video und Audio)",
   "engines.package.ffmpeg.description": "Ergänzt Video- und Audiokonvertierung, Video → GIF oder Einzelbild, Ton aus Videos und GIF → MP4.\nNur für Video und Audio erforderlich.",
+  "engines.package.whisper.name": "Whisper (KI-Untertitel)",
+  "engines.package.whisper.description": "Transkribiert Sprache in Videos und Audio in SRT-Untertiteldateien. Funktioniert offline ohne Internetverbindung.",
   "engines.package.calibre.name": "Calibre (Kindle und seltene E-Book-Formate)",
   "engines.package.calibre.description": "Speichert MOBI und AZW3 für Kindle und konvertiert seltene E-Book-Formate (LIT, LRF, PDB, CHM…).\nEPUB, MOBI, AZW3, FB2 und Comics lassen sich auch ohne Calibre lesen.",
   "engines.package.size": "Download {0} · Platzbedarf {1}",

src/Filee.App/Assets/i18n/en.json

@@ -213,6 +213,8 @@
   "engine.subtitle.name": "Subtitles (built-in)",
   "engine.subtitle.description": "Subtitles: SRT, ASS/SSA, WebVTT, SAMI (SMI), LRC, TTML, MicroDVD and SCC into each other, saved as UTF-8.\nKorean SMI and SRT files in CP949 / EUC-KR read correctly.",
   "engine.ffmpeg.description": "Video and audio: MP4, MOV, MKV, WEBM, MP3, M4A, WAV, FLAC and more.\nAlso extracts sound and subtitles, makes GIF and APNG animations and saves images as HEIC.",
+  "engine.whisper.name": "Whisper",
+  "engine.whisper.description": "Speech-to-text transcription powered by OpenAI Whisper. Automatically generates SRT subtitles from speech in video and audio.",
   "engine.pandoc.description": "Markdown and HTML ↔ ODT and RTF, DOCX → Markdown\nreStructuredText, LaTeX, Org, AsciiDoc, Typst, MediaWiki, DocBook, Jupyter and more ↔ Markdown, HTML, DOCX and ODT\nMarkdown and Org → PPTX slides",
   "engine.ghostscript.description": "EPS, PostScript and older Illustrator files to PDF and images; PDF to EPS and PS.",
   "engine.tesseract.description": "Recognizes the text of scans and pictures (OCR): searchable PDF, DOCX, TXT.",
@@ -419,6 +421,8 @@
   "preset.extract": "Extract",
   "preset.zip_all": "One ZIP",
   "preset.extract_subtitles": "Extract subtitles",
+  "preset.caption_video": "Caption",
+  "preset.caption_video.donut": "Caption",
   "preset.trace_svg": "Trace to SVG",
   "preset.trace_svg.donut": "Trace",
   "preset.trace_line_art": "Line art to SVG",
@@ -855,6 +859,10 @@
   "presets.video_codec.av1": "AV1 (smallest, slow)",
   "presets.hardware_encoding": "Use hardware acceleration (graphics chip)",
   "presets.hardware_encoding_hint": "Encodes H.264, HEVC and AV1 on the graphics chip.\nMuch faster and uses less power.\nSlightly lower quality at the same file size.\nWithout a usable chip (AV1 needs a recent one), the video is encoded as usual.",
+  "presets.caption_video": "Generate subtitles (Whisper)",
+  "presets.caption_video_hint": "Transcribes speech in videos and audio to an SRT subtitle file using Whisper.",
+  "presets.spoken_language": "Spoken language",
+  "presets.spoken_language.auto": "Auto",
   "presets.audio_codec": "Audio codec",
   "presets.audio_codec.aac": "AAC",
   "presets.audio_codec.alac": "ALAC (lossless)",
@@ -979,6 +987,8 @@
   "engines.package.tesseract.description": "Reads the text of scanned PDFs and pictures: searchable PDF, DOCX, TXT.\nEach language downloads the first time it is used.",
   "engines.package.ffmpeg.name": "FFmpeg (video and audio)",
   "engines.package.ffmpeg.description": "Adds video and audio conversion, video → GIF or still image, sound from videos and GIF → MP4.\nOnly needed for video and audio.",
+  "engines.package.whisper.name": "Whisper (AI Captions)",
+  "engines.package.whisper.description": "Transcribes speech in video and audio to SRT subtitle files. Runs offline without an internet connection.",
   "engines.package.calibre.name": "Calibre (Kindle and rare e-book formats)",
   "engines.package.calibre.description": "Saves MOBI and AZW3 for Kindle and converts rare e-book formats (LIT, LRF, PDB, CHM…).\nEPUB, MOBI, AZW3, FB2 and comics are read without it.",
   "engines.package.size": "Download {0} · {1} on disk",

src/Filee.App/Assets/i18n/es.json

@@ -206,6 +206,8 @@
   "engine.subtitle.name": "Subtitles (built-in)",
   "engine.subtitle.description": "Subtítulos: SRT, ASS/SSA, WebVTT, SAMI (SMI), LRC, TTML, MicroDVD y SCC entre sí, guardados como UTF-8.\nLee correctamente archivos SMI y SRT coreanos en CP949 / EUC-KR.",
   "engine.ffmpeg.description": "Vídeo y audio: MP4, MOV, MKV, WEBM, MP3, M4A, WAV, FLAC y más.\nTambién extrae sonido y subtítulos, crea animaciones GIF y APNG y guarda imágenes como HEIC.",
+  "engine.whisper.name": "Whisper",
+  "engine.whisper.description": "Transcripción de voz a texto con OpenAI Whisper. Genera automáticamente subtítulos SRT a partir de voz en vídeo y audio.",
   "engine.pandoc.description": "Markdown y HTML ↔ ODT y RTF, DOCX → Markdown\nreStructuredText, LaTeX, Org, AsciiDoc, Typst, MediaWiki, DocBook, Jupyter y más ↔ Markdown, HTML, DOCX y ODT\nMarkdown y Org → diapositivas PPTX",
   "engine.ghostscript.description": "EPS, PostScript y archivos antiguos de Illustrator a PDF e imágenes; PDF a EPS y PS.",
   "engine.tesseract.description": "Reconoce el texto de escaneos e imágenes (OCR): PDF con búsqueda, DOCX, TXT.",
@@ -409,6 +411,8 @@
   "preset.extract": "Extraer",
   "preset.zip_all": "Un solo ZIP",
   "preset.extract_subtitles": "Extraer subtítulos",
+  "preset.caption_video": "Subtítulos",
+  "preset.caption_video.donut": "Subtítulos",
   "preset.trace_svg": "Trazar a SVG",
   "preset.trace_svg.donut": "Trazar a SVG",
   "preset.trace_line_art": "Líneas a SVG",
@@ -840,6 +844,10 @@
   "presets.video_codec.av1": "AV1 (el más pequeño, lento)",
   "presets.hardware_encoding": "Usar aceleración por hardware (chip gráfico)",
   "presets.hardware_encoding_hint": "Codifica H.264, HEVC y AV1 en el chip gráfico.\nMucho más rápido y consume menos energía.\nCalidad ligeramente inferior con el mismo tamaño de archivo.\nSin un chip compatible (AV1 necesita uno reciente), el vídeo se codifica de la forma habitual.",
+  "presets.caption_video": "Generar subtítulos (Whisper)",
+  "presets.caption_video_hint": "Transcribe la voz de vídeos y audios a un archivo de subtítulos SRT usando Whisper.",
+  "presets.spoken_language": "Idioma hablado",
+  "presets.spoken_language.auto": "Automático",
   "presets.audio_codec": "Códec de audio",
   "presets.audio_codec.aac": "AAC",
   "presets.audio_codec.alac": "ALAC (sin pérdida)",
@@ -962,6 +970,8 @@
   "engines.package.tesseract.description": "Lee el texto de PDF escaneados e imágenes: PDF con búsqueda, DOCX, TXT.\nCada idioma se descarga la primera vez que se usa.",
   "engines.package.ffmpeg.name": "FFmpeg (vídeo y audio)",
   "engines.package.ffmpeg.description": "Añade conversión de vídeo y audio, vídeo → GIF o imagen fija, sonido de vídeos y GIF → MP4.\nSolo es necesario para vídeo y audio.",
+  "engines.package.whisper.name": "Whisper (subtítulos IA)",
+  "engines.package.whisper.description": "Transcribe la voz de vídeos y audios a archivos de subtítulos SRT. Funciona sin conexión a internet.",
   "engines.package.calibre.name": "Calibre (Kindle y libros electrónicos poco habituales)",
   "engines.package.calibre.description": "Guarda MOBI y AZW3 para Kindle y convierte formatos de libro electrónico poco habituales (LIT, LRF, PDB, CHM…).\nEPUB, MOBI, AZW3, FB2 y cómics se leen sin él.",
   "engines.package.size": "Descargar {0} · {1} en disco",