Skip to content

Map saved Auto-Translate languages across providers - #154

Draft
Blackspirits wants to merge 1 commit into
upl/review-base-dda1from
audit/auto-translate-cross-engine-language-dda1
Draft

Blackspirits wants to merge 1 commit into
upl/review-base-dda1from
audit/auto-translate-cross-engine-language-dda1

Conversation

@Blackspirits

@Blackspirits Blackspirits commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Purpose

Current-main follow-up to the Auto-Translate changes merged in upstream commit dda1fce8f9d4a605b1b6732acc072c7109dc6323 (SubtitleEdit#14956 follow-ups).

The upstream change improved source-language fallback by using the last selected source when language detection cannot identify a tiny/ambiguous selection. This audit found one cross-provider gap: the saved language identifier belongs to the provider that wrote it, but providers use incompatible identifier schemes.

Examples:

  • NLLB: eng_Latn, deu_Latn, zho_Hans;
  • Google-style engines: en, de, zh-CN;
  • LLM engines: language names such as English / German, with separate ISO metadata.

Current upstream main was rechecked at dda1fce8f9d4a605b1b6732acc072c7109dc6323, exactly this PR's base.

Finding fixed

When detection returned no source language, EvaluateDefaultSourceLanguageCode() returned AutoTranslateLastSource verbatim.

If the previous engine was NLLB and the current engine was Google/Gemini/ChatGPT, a saved code such as eng_Latn did not exist in the current source list.

UpdateSourceLanguages() then failed every lookup and silently selected SourceLanguages[0].

The same identifier mismatch also affected FindDefaultTargetLanguage(): a saved NLLB target such as deu_Latn could not be restored by an LLM provider, so the target fell through to UI-language/default candidates instead of the user's last choice.

Fix

The shared FindLanguage() resolver now handles provider identifier differences in fail-safe order:

  1. exact code / name / provider ISO metadata;
  2. separator-normalized script identifiers (zho_Hans == zho-Hans);
  3. existing two-letter ISO-to-English-name mapping;
  4. NLLB-style ISO-639-3 prefix conversion (eng -> en, deu -> de);
  5. prefer an exact English-name match in the current engine;
  6. otherwise accept a two-letter match only when it is unique.

Ambiguous multi-variant mappings are not guessed.

When source detection fails, the last saved source is first resolved through this current-engine mapping. The returned value is therefore the current engine's code, not a stale code copied from another provider. If no safe mapping exists, the existing English fallback remains.

Regression coverage

Tests pin three real cross-provider cases:

  • saved NLLB source eng_Latn maps to Google en;
  • saved NLLB target deu_Latn maps to Gemini German;
  • NLLB zho_Hans preserves Gemini's Chinese (Simplified) variant across underscore/hyphen conventions.

Existing detectable-source and same-provider fallback tests remain unchanged.

Scope / branch state

  • base/current upstream main: dda1fce8f9d4a605b1b6732acc072c7109dc6323
  • HEAD: 0a60681a76f9ec7061bee3163133b0374c137afd
  • 1 commit
  • 2 files
  • +125 / -5
  • audit draft only; no merge/promotion intended

Acceptance

  • full restore/build/test suite on this exact HEAD;
  • no new warning attributable to this tranche;
  • retry step unused;
  • final adversarial review of cross-provider source/target mapping and ambiguous-language fail-safe behavior;
  • upstream drift rechecked before freeze.

AI assistance: ChatGPT was used for adversarial provider-language state review, NLLB/Google/LLM identifier comparison, fail-safe mapping design and regression coverage.

Final CI

Authoritative run: SubtitleEdit#312 on 0a60681a76f9ec7061bee3163133b0374c137afd

  • SeConv: 505 passed / 2 skipped / 0 failed
  • LibUiLogic: 918 passed / 0 skipped / 0 failed
  • LibSE: 2030 passed / 0 skipped / 0 failed
  • UI: 5413 passed / 10 skipped / 0 failed
  • build: 0 errors / 8 pre-existing warnings
  • retry step skipped
  • failure-artifact upload skipped

All eight warnings are outside the two-file #154 changed set.

Final drift gate

Upstream main was rechecked at dda1fce8f9d4a605b1b6732acc072c7109dc6323, exactly the base of this PR. No post-base drift exists.

Final adversarial review was performed on this exact HEAD after CI; no blocker found.

@Blackspirits Blackspirits left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final adversarial review on 0a60681a76f9ec7061bee3163133b0374c137afd after CI SubtitleEdit#312: no blocker found. Rechecked NLLB/Google/LLM source and target restoration, underscore/hyphen script identifiers, ISO-639-3 prefix conversion, English-name preference and ambiguous multi-variant fail-closed behavior. The fallback now returns a code belonging to the current provider rather than a stale previous-provider identifier. All eight warnings are outside the changed set; upstream is still exactly the audited base dda1fce8.... No merge/promotion performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant