Skip to content

Add drag-and-drop audio transcription - #232

Open
supnim wants to merge 2 commits into
kitlangton:mainfrom
supnim:drag-drop-audio-transcription
Open

supnim wants to merge 2 commits into
kitlangton:mainfrom
supnim:drag-drop-audio-transcription

Conversation

@supnim

@supnim supnim commented May 28, 2026

Copy link
Copy Markdown

Summary

  • Add a Transcribe tab for drag-and-drop and picker-based audio/video transcription.
  • Support common audio formats plus MP4/MOV/M4V files with readable audio tracks, with clearer errors for unsupported or silent video files.
  • Save imported transcripts into History when history is enabled, preserving the imported file name and audio extension.
  • Polish the transcript result card to match the History card pattern and align the Audio Behavior settings row.

Testing

  • xcodebuild -scheme Hex -configuration Debug -skipMacroValidation CODE_SIGNING_ALLOWED=NO -quiet build
  • cd HexCore && swift test

Summary by CodeRabbit

  • New Features

    • Added "Transcribe" tab with drag-and-drop and file-import audio/video transcription
    • Real-time job list with statuses, per-job copy-to-clipboard and remove actions
    • "Clear Finished" toolbar action and multi-file import support
  • Improvements

    • Better drop/import error messages and user notices
    • Saved recordings preserve original audio extensions
    • Improved fallback when identifying a transcript's source app
  • Localization

    • New UI strings for transcription workflow and controls

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 81b880d5-4506-4926-be40-80de267e1ba8

📥 Commits

Reviewing files that changed from the base of the PR and between bfb654a and 444a934.

📒 Files selected for processing (3)
  • .changeset/d2cc062c.md
  • Hex/Features/FileTranscription/FileTranscriptionFeature.swift
  • Hex/Features/FileTranscription/FileTranscriptionView.swift
🚧 Files skipped from review as they are similar to previous changes (1)
  • .changeset/d2cc062c.md

📝 Walkthrough

Walkthrough

This PR adds a new "Transcribe" tab to Hex with a FileTranscription TCA feature: job model and status, reducer with async transcription effects (security-scoped access, duration detection, VAD chunking, optional history persistence), SwiftUI views for drag-and-drop and file import, app integration, localization, and related small UI/persistence tweaks.

Changes

File Transcription Feature

Layer / File(s) Summary
Job Model and Status Types
Hex/Features/FileTranscription/FileTranscriptionJob.swift
FileTranscriptionJob stores job identity, source URL, derived filename, and mutable status/transcript/error fields. FileTranscriptionStatus enum models lifecycle states with display metadata and terminal-state detection.
Transcription Feature Reducer and Effects
Hex/Features/FileTranscription/FileTranscriptionFeature.swift
TCA reducer handles addFiles, job lifecycle actions, copy/remove/clear actions, filters/deduplicates imports, computes audio duration (with AVAudioFile decode and track-probe fallback), runs async transcription with VAD chunking and optional language detection, trims/validates transcript text, optionally persists transcripts by copying source into a temp import dir, and removes oldest history entries when limits are exceeded.
Transcription UI Views
Hex/Features/FileTranscription/FileTranscriptionView.swift, Hex/Features/FileTranscription/FileTranscriptionJobRow.swift
FileTranscriptionView provides a drop zone, drag-and-drop/fileImporter handling (async NSItemProvider bridging), error/notice labels, job list, and toolbar actions ("Choose Audio", "Clear Finished"). FileTranscriptionJobRow renders per-job header/status, conditional copy-to-clipboard with 1.5s "Copied" animation, remove action, and content area for progress/error/transcript.
App Navigation and Routing
Hex/Features/App/AppFeature.swift
Adds .fileTranscription ActiveTab case, fileTranscription state and action, scopes the feature reducer, and adds a "Transcribe" navigation button plus detail branch to render FileTranscriptionView.
Localization Strings and Changeset
Localizable.xcstrings, .changeset/d2cc062c.md
Adds localization for import/prompt/toolbar/status/section strings and updates the changeset to mark a minor version bump with note about the new Transcribe tab.
Supporting Improvements
Hex/Features/History/HistoryFeature.swift, Hex/Features/Settings/GeneralSectionView.swift, HexCore/Sources/HexCore/TranscriptPersistenceClient/TranscriptPersistenceClient.swift
History header falls back to a "doc" icon plus source app name when bundle ID is missing. Settings UI refactors the "Audio Behavior while Recording" row. TranscriptPersistenceClient preserves the original audio file extension (defaults to wav).

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 A waveform hops through files with care,
Drop, transcribe, then save with flair,
Jobs queue up and dance in line,
Clipboard gleams — the transcript's fine! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add drag-and-drop audio transcription' directly and clearly describes the primary feature addition—a new Transcribe tab with drag-and-drop functionality for audio files, which is the main focus of the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@supnim
supnim marked this pull request as ready for review May 28, 2026 11:42

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/d2cc062c.md:
- Line 5: The changeset summary "Add a Transcribe tab for drag-and-drop audio
transcription." is missing the required GitHub PR/issue reference token; update
that summary line to include the reference in the format "Add a Transcribe tab
for drag-and-drop audio transcription. (#<PR_OR_ISSUE_NUMBER>)" so the
release-note formatting requirement is satisfied.

In `@Hex/Features/FileTranscription/FileTranscriptionFeature.swift`:
- Line 54: Remove the trailing comma after the "wav" element in the set/array
literal in FileTranscriptionFeature (the literal containing "wav") to satisfy
SwiftLint's trailing_comma rule; locate the literal in
FileTranscriptionFeature.swift and delete the comma so the collection ends with
"wav" without a trailing comma.
- Around line 188-189: The log statements in FileTranscriptionFeature use
unannotated interpolations for file/path values: update the
fileTranscriptionLogger calls to mark path-derived interpolations as private
(e.g. change fileTranscriptionLogger.notice("Transcribing imported audio
file=\(url.lastPathComponent)") to use \(url.lastPathComponent, privacy:
.private)) and similarly mark the file/path in the fileTranscriptionLogger.error
call (and any interpolated error details if sensitive) with privacy: .private;
locate these in the FileTranscriptionFeature (references:
fileTranscriptionLogger.notice and fileTranscriptionLogger.error) and apply the
privacy annotation to each path/file interpolation.

In `@Hex/Features/FileTranscription/FileTranscriptionView.swift`:
- Around line 122-156: The method loadDroppedFileURLs(from:) currently uses
NSItemProvider.loadItem with callbacks plus DispatchGroup/NSLock; replace this
with structured concurrency by creating an async wrapper around
NSItemProvider.loadItem (using
withCheckedContinuation/withCheckedThrowingContinuation) and then use a Task or
TaskGroup to await all providers, collect URLs via Self.fileURL(from:) and
capture the first error if any; finally call store.send(.addFiles(droppedURLs))
or store.send(.importFailed(firstError)) on the main actor once when complete.
Ensure the wrapper returns URL? or throws so you can handle nil/throwing
results, and update loadDroppedFileURLs to be async (or spawn a Task) so callers
remain non-blocking. Use the existing symbols: loadDroppedFileURLs(from:),
NSItemProvider.loadItem(forTypeIdentifier:...), Self.fileURL(from:),
store.send(.addFiles(...)) and store.send(.importFailed(...)).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c3e25c6f-3e09-43d0-972c-84fd1af60813

📥 Commits

Reviewing files that changed from the base of the PR and between f988cb7 and bfb654a.

📒 Files selected for processing (10)
  • .changeset/d2cc062c.md
  • Hex/Features/App/AppFeature.swift
  • Hex/Features/FileTranscription/FileTranscriptionFeature.swift
  • Hex/Features/FileTranscription/FileTranscriptionJob.swift
  • Hex/Features/FileTranscription/FileTranscriptionJobRow.swift
  • Hex/Features/FileTranscription/FileTranscriptionView.swift
  • Hex/Features/History/HistoryFeature.swift
  • Hex/Features/Settings/GeneralSectionView.swift
  • HexCore/Sources/HexCore/TranscriptPersistenceClient/TranscriptPersistenceClient.swift
  • Localizable.xcstrings

Comment thread .changeset/d2cc062c.md Outdated
Comment thread Hex/Features/FileTranscription/FileTranscriptionFeature.swift Outdated
Comment thread Hex/Features/FileTranscription/FileTranscriptionFeature.swift Outdated
Comment thread Hex/Features/FileTranscription/FileTranscriptionView.swift
@coderabbitai

coderabbitai Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Actionable comments posted: 0

@what-name

Copy link
Copy Markdown

there are lots of open source apps that do this already. imo this would just bloat hex and doesn't belong in this app.

@supnim

supnim commented Jun 11, 2026

Copy link
Copy Markdown
Author

@what-name like which ones?

sperka added a commit to sperka/Hex that referenced this pull request Jul 1, 2026
Add a "Transcribe" tab for importing audio/video files via drag-and-drop or the
file picker, transcribing them with the currently selected model. Adapted from
kitlangton#232 for this fork's dual-engine setup.

- Works with every engine. WhisperKit and Parakeet batch use the existing
  file-based transcribe; the Nemotron streaming model gains a one-shot
  transcribeFile path that feeds the whole file through the streaming manager
  (process(audioBuffer:) + finish()) in ~5s blocks. TranscriptionClient routes
  streaming models here instead of rejecting file-based transcribe.
- Per-job progress: determinate percentage + bar for the Nemotron file path
  (real framesFed/total), indeterminate bar for batch engines that don't expose
  a decode fraction, plus a live elapsed timer for every job.
- Multi-file import, per-job copy/remove, Clear Finished, and clear errors for
  unsupported or silent files. Imported transcripts save to History (preserving
  the original file extension) when history is enabled.

Part of the Nemotron streaming integration (kitlangton#236).

Verified: Debug build succeeds and HexCore swift test passes (62 tests, 5 suites);
file transcription runtime-tested across engines with live progress.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants