Repository navigation
Add OpenAI-compatible LLM translation provider with background jobs - #5
Conversation
…jobs - Add TRANSLATION_PROVIDER=llm using any OpenAI-compatible chat API (OpenAI, OpenRouter, Anthropic, LiteLLM, Ollama). Only subtitle text is sent to the model as JSON; timestamps never leave the server. - Bundle a default system prompt, overridable by mounting /config/llm-system-prompt.txt (or LLM_PROMPT_FILE). - Validate a strict 1:1 id mapping, retry malformed batches and split them in half; retry rate limits and server errors with backoff. - Run translations as queued background jobs with batch progress, so long LLM runs no longer hit the 60s proxy timeout. Progress survives page reloads and Plex is refreshed server-side when a job finishes. - Show LLM model and token usage in the header. Closes #3
The server rejects a new job with 409 when the same subtitle file and target language is already queued or running, and the translate button is disabled in that case. Other files and languages can still be queued.
- LLM_FALLBACK_MODELS lists models tried in order when LLM_MODEL keeps failing. Models with a fallback fail over after 2 attempts and are skipped for 5 minutes; auth errors do not trigger a fallback. - Jobs keep the translations of finished batches. A failed job can be resumed with POST /jobs/:id/retry, which skips completed batches for both DeepL and LLM providers. - When one LLM batch fails, workers stop taking new batches and wait for in-flight ones, so a retry never overlaps a run still in progress. - The UI shows where a job stopped and a Resume translation button.
fe519d2 to
acb3462
Compare
|
@claude review |
|
Claude finished @garanda21's task in 32s —— View job Review of PR #5: OpenAI-compatible LLM provider and background jobsI read the server-side changes in full: The design is solid. The strict 1:1 id validation, the retry then split-in-half handling, the fallback cooldown, and the resumable Should fix
Consider
Nits
SummaryItem 1 is a real hazard from a bad env var, and I would fix it before merging. Items 2 and 6 are small robustness fixes. The rest are optional. |
- Validate numeric LLM env vars and fall back to defaults; LLM_BATCH_SIZE=0 previously looped forever and LLM_CONCURRENCY=0 produced empty output. - Reject paths that resolve outside DATA_DIR/MOVIES_DIR, invalid language codes (they end up in the output file name), non-.srt files and unknown types in translate_subtitle and extract_subtitles. - Stop returning error stacks from extract_subtitles. - Clear the previous attempt's timings, path and Plex state on retry. - Prune completed jobs before failed ones so resume progress is not lost. - Keep the job queue running if an unexpected error escapes a job. - Document the error status contract in translateBatch and the fallback retry budget and LLM_JSON_MODE in the README.
|
@claude review |
|
Claude finished @garanda21's task in 30s —— View job Review of PR #5: OpenAI-compatible LLM provider and background jobsI read Most of the earlier review's points are already fixed on this head:
The Should fix
Consider
Nits
SummaryItem 1 is a real bug. The slow failure and wrong error message are the visible effects. Item 2 is a one-line cap. The rest are optional. |
- Treat an empty or missing model response as a malformed response (retry, then split the batch) instead of a network failure that was retried with backoff and reported as "LLM API unreachable". - Cap Retry-After waits at 60 seconds. - Only put a model on cooldown for 429, 5xx and network errors; other HTTP errors fall back for that request only. - Discard saved resume progress when the source texts change, using a hash instead of comparing only the number of blocks. - Stop returning originalError from extract_subtitles and mask media directory paths in its error message.
|
@claude review |
|
Claude finished @garanda21's task in 16s —— View job Review of PR #5 at head
|
Closes #3
Summary
Adds an LLM translation engine as an alternative to DeepL, using any OpenAI-compatible chat API (OpenAI, OpenRouter, Anthropic, LiteLLM, Ollama), and moves translations to background jobs so long LLM runs no longer time out.
LLM provider
TRANSLATION_PROVIDER=llmwithLLM_BASE_URL,LLM_API_KEY,LLM_MODEL. DeepL stays the default.server/prompts/llm-system-prompt.txt, overridable by mounting/config/llm-system-prompt.txt(orLLM_PROMPT_FILE).LLM_FALLBACK_MODELS: models tried in order when the main one keeps failing; a failing model is skipped for 5 minutes.Background jobs
POST /translate_subtitlenow returns202with a job; progress is exposed viaGET /jobsandGET /jobs/:id. This avoids nginx's default 60s proxy timeout.409).POST /jobs/:id/retry), skipping batches already translated. Works for both DeepL and LLM.DeepL compatibility
translateSRTfrommainand from this branch produce byte-identical output with the same requests and batches.404.Limitations
Testing
:devimage was published and tested on a real server.google/gemma-4-26b-a4b-it:freeon OpenRouter: identical timestamps, good quality.