Skip to content

Add Parallel Mode transcription with Settings UI toggle and ~/Documents model discovery #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gauravsaini wants to merge 1 commit into from
Closed

Add Parallel Mode transcription with Settings UI toggle and ~/Documents model discovery #273

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/9b51fe61.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"hex-app": patch
---

Add Parallel Mode transcription with Settings UI toggle and ~/Documents model discovery
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,7 @@ node_modules/

# Claude
.claude/

# Test artifacts
test_audio/

257 changes: 221 additions & 36 deletions Hex/Clients/TranscriptionClient.swift
View file
Open in desktop

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions Hex/Features/Settings/GeneralSectionView.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,19 @@ struct GeneralSectionView: View {
Image(systemName: "bolt.circle")
}

Label {
Toggle(
"Parallel Mode",
isOn: Binding(
get: { store.hexSettings.transcribeMode == .parallel },
set: { store.send(.toggleParallelMode($0)) }
)
)
Text("Process Whisper audio in parallel 30s chunks across Neural Engine and GPU cores for up to 10x faster transcription.")
} icon: {
Image(systemName: "cpu")
}

Label {
HStack(alignment: .center) {
Text("Audio Behavior while Recording")
Expand Down
5 changes: 5 additions & 0 deletions Hex/Features/Settings/SettingsFeature.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ struct SettingsFeature {
case togglePreventSystemSleep(Bool)
case setRecordingAudioBehavior(RecordingAudioBehavior)
case toggleSuperFastMode(Bool)
case toggleParallelMode(Bool)
case setUseClipboardPaste(Bool)
case setCopyToClipboard(Bool)
case setDoubleTapLockEnabled(Bool)
Expand Down Expand Up @@ -486,6 +487,10 @@ struct SettingsFeature {
state.$hexSettings.withLock { $0.minimumKeyTime = value }
return .none

case let .toggleParallelMode(enabled):
state.$hexSettings.withLock { $0.transcribeMode = enabled ? .parallel : .default }
return .none

case let .setOutputLanguage(language):
state.$hexSettings.withLock { $0.outputLanguage = language }
return .none
Expand Down
8 changes: 5 additions & 3 deletions Hex/Features/Transcription/TranscriptionFeature.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ private extension TranscriptionFeature {
let minimumKeyTime = state.hexSettings.minimumKeyTime
let hotkeyHasKey = state.hexSettings.hotkey.key != nil
transcriptionFeatureLogger.notice(
"Recording stopped duration=\(String(format: "%.3f", duration))s start=\(startStamp) stop=\(stopStamp) decision=\(String(describing: decision)) minimumKeyTime=\(String(format: "%.2f", minimumKeyTime)) hotkeyHasKey=\(hotkeyHasKey)"
"Recording stopped duration=\(String(format: "%.3f", duration), privacy: .public)s start=\(startStamp, privacy: .public) stop=\(stopStamp, privacy: .public) decision=\(String(describing: decision), privacy: .public) minimumKeyTime=\(String(format: "%.2f", minimumKeyTime), privacy: .public) hotkeyHasKey=\(hotkeyHasKey, privacy: .public)"
)

guard decision == .proceedToTranscription else {
Expand Down Expand Up @@ -365,6 +365,7 @@ private extension TranscriptionFeature {
state.isTranscribing = true
state.error = nil
let language = state.hexSettings.outputLanguage
let isParallelMode = state.hexSettings.transcribeMode == .parallel

state.isPrewarming = true

Expand Down Expand Up @@ -407,12 +408,13 @@ private extension TranscriptionFeature {
let decodeOptions = DecodingOptions(
language: language,
detectLanguage: language == nil, // Only auto-detect if no language specified
chunkingStrategy: .vad,
concurrentWorkerCount: isParallelMode ? 8 : 0,
chunkingStrategy: .vad
)
Comment on lines +411 to 413

@coderabbitai coderabbitai Bot Jul 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

WhisperKit DecodingOptions concurrentWorkerCount 0 meaning default value

💡 Result:

In WhisperKit, the concurrentWorkerCount property within DecodingOptions determines the number of audio chunks processed in parallel [1][2]. A value of 0 (or in some older versions, the absence of an explicit setting) has historically been interpreted as a signal to process all available audio chunks in a single batch, or effectively without a specific cap on concurrency [3]. However, recent updates have moved toward setting a specific default value to manage memory usage [4]. As of the current implementation, the default value for concurrentWorkerCount is 16 [4]. This change was introduced to cap peak memory consumption, particularly for long audio files when using Voice Activity Detection (VAD) chunking, preventing devices from running out of memory by limiting the number of concurrent asynchronous CoreML predictions [5][4]. Users are encouraged to adjust this value based on their specific hardware: - 16 for macOS devices with >16GB of RAM [6]. - 8 for macOS devices with 8-16GB of RAM [6]. - 4 for iOS devices [6]. - 1 for sequential, memory-minimal processing [6].

Citations:

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== repo files around TranscriptionFeature.swift =="
git ls-files | rg 'TranscriptionFeature\.swift|TranscriptionClient\.swift|Package\.swift|Package\.lock' || true

echo
echo "== TranscriptionFeature relevant lines =="
sed -n '370,440p' Hex/Features/Transcription/TranscriptionFeature.swift 2>/dev/null || true

echo
echo "== TranscriptionClient occurrences =="
rg -n "concurrentWorkerCount|parallel|ParallelMode|parallelMode|Sequential|sequential" Hex -S || true

echo
echo "== WhisperKit dependency pins =="
if [ -f Package.swift ]; then
  sed -n '1,160p' Package.swift
fi
if [ -f Package.resolved ]; then
  cat Package.resolved
fi
if [ -d .build ]; then
  find .build -name Package.resolved -o -name Package.swift 2>/dev/null | head -20
fi

echo
echo "== Check if WhisperKit local source exists in sandbox =="
fd -a 'Configurations.swift|TranscriptionFeature.swift' . | head -50

Repository: kitlangton/Hex

Length of output: 4984

🌐 Web query:

WhisperKit DecodingOptions concurrentWorkerCount source 0 unbounded default 16

💡 Result:

In WhisperKit, the concurrentWorkerCount parameter within DecodingOptions controls the number of concurrent tasks (workers) used for audio processing and batch inference [1][2]. Historically, this value was set to 0, which effectively allowed for an unbounded number of concurrent tasks when using features like VAD (Voice Activity Detection) chunking [3][4]. This behavior could lead to excessive memory consumption and crashes on resource-constrained devices, particularly when processing long audio files [3]. To mitigate these memory issues, the default value was updated to 16 in September 2024 (via PR #205) to provide a balance between throughput and peak memory usage [3][4]. Current implementations typically utilize this default of 16, though it can be adjusted based on the target platform's available memory [5][2]. For instance, on iOS or memory-limited environments, it is often recommended to use a lower value (such as 4), while high-memory macOS environments may support higher values [5][2][6].

Citations:

🌐 Web query:

site:github.com/argmaxinc/WhisperKit DecodingOptions concurrentWorkerCount default 0

💡 Result:

In the WhisperKit framework, the concurrentWorkerCount parameter within DecodingOptions controls the number of concurrent tasks (specifically audio chunks) processed during transcription [1][2]. While earlier versions of the framework utilized a default value of 0—which signified that all audio arrays were processed in a single batch (effectively unlimited concurrency relative to the number of input items) [2]—this was later changed to mitigate memory issues during Voice Activity Detection (VAD) processing [3][4]. As of the current implementation, the default value for concurrentWorkerCount is 16 [3][5]. This value was selected to provide a balance between maintaining high throughput and limiting peak memory consumption, particularly on devices like those running iOS [4]. Users may still manually configure this value to optimize performance based on their specific hardware and input files [4].

Citations:

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== TranscriptionClient parallel code section =="
sed -n '270,330p' Hex/Clients/TranscriptionClient.swift 2>/dev/null || true

echo
echo "== Package dependency references =="
rg -n "WhisperKit|github.com/argmaxinc/WhisperKit|\.package(url:.*WhisperKit|name: \"WhisperKit\"" . -S || true

echo
echo "== Swift config package files =="
if [ -f HexCore/Package.swift ]; then
  sed -n '1,220p' HexCore/Package.swift
fi

Repository: kitlangton/Hex

Length of output: 4753

Make sequential transcription explicitly sequential

Current WhisperKit docs indicate concurrentWorkerCount of 0 does not force sequential work and newer versions default this to a capped concurrent value. As written, disabling Parallel Mode uses the standard WhisperKit transcribe(...) path with options that may still run concurrently, while TranscriptionClient treats options.concurrentWorkerCount > 0 as parallel. Use an explicit sequential value/config if the setting is meant to disable parallel chunking, and avoid hardcoding 8 by deriving the cap from available cores/memory.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Hex/Features/Transcription/TranscriptionFeature.swift` around lines 411 -
413, Update the transcription options construction in the sequential/parallel
mode flow so disabling parallel mode explicitly configures WhisperKit for
sequential processing rather than passing concurrentWorkerCount as 0. For
parallel mode, derive the worker cap from available system CPU and memory
resources instead of hardcoding 8, while preserving the existing .vad chunking
strategy and TranscriptionClient’s parallel-mode detection.


let result = try await transcription.transcribe(capturedURL, model, decodeOptions) { _ in }

transcriptionFeatureLogger.notice("Transcribed audio from \(capturedURL.lastPathComponent) to text length \(result.count)")
transcriptionFeatureLogger.notice("Transcribed audio from \(capturedURL.lastPathComponent, privacy: .public) to text length \(result.count, privacy: .public)")
audioURL = nil
await send(.transcriptionResult(result, capturedURL, duration))
} catch {
Expand Down
1 change: 1 addition & 0 deletions Hex/Models/AppHexSettings.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import HexCore

// Re-export types so the app target can use them without HexCore prefixes.
typealias RecordingAudioBehavior = HexCore.RecordingAudioBehavior
typealias TranscriptionMode = HexCore.TranscriptionMode
typealias HexSettings = HexCore.HexSettings

extension SharedReaderKey
Expand Down
26 changes: 24 additions & 2 deletions HexCore/Sources/HexCore/Settings/HexSettings.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ public enum RecordingAudioBehavior: String, Codable, CaseIterable, Equatable, Se
case doNothing
}

public enum TranscriptionMode: String, Codable, CaseIterable, Equatable, Sendable {
case `default`
case parallel
}

/// User-configurable settings saved to disk.
public struct HexSettings: Codable, Equatable, Sendable {
public static let defaultPasteLastTranscriptHotkey = HotKey(key: .v, modifiers: [.option, .shift])
Expand Down Expand Up @@ -47,8 +52,10 @@ public struct HexSettings: Codable, Equatable, Sendable {
public var wordRemovalsEnabled: Bool
public var wordRemovals: [WordRemoval]
public var wordRemappings: [WordRemapping]
public var transcribeMode: TranscriptionMode
public var lowercaseTranscripts: Bool
public var removePunctuation: Bool
public var customModelDirectory: String?

private mutating func normalizeDoubleTapSettings() {
if !doubleTapLockEnabled {
Expand Down Expand Up @@ -81,8 +88,10 @@ public struct HexSettings: Codable, Equatable, Sendable {
wordRemovalsEnabled: Bool = false,
wordRemovals: [WordRemoval] = HexSettings.defaultWordRemovals,
wordRemappings: [WordRemapping] = [],
transcribeMode: TranscriptionMode = .default,
lowercaseTranscripts: Bool = false,
removePunctuation: Bool = false
removePunctuation: Bool = false,
customModelDirectory: String? = nil
) {
self.soundEffectsEnabled = soundEffectsEnabled
self.soundEffectsVolume = soundEffectsVolume
Expand All @@ -108,8 +117,10 @@ public struct HexSettings: Codable, Equatable, Sendable {
self.wordRemovalsEnabled = wordRemovalsEnabled
self.wordRemovals = wordRemovals
self.wordRemappings = wordRemappings
self.transcribeMode = transcribeMode
self.lowercaseTranscripts = lowercaseTranscripts
self.removePunctuation = removePunctuation
self.customModelDirectory = customModelDirectory
normalizeDoubleTapSettings()
}

Expand Down Expand Up @@ -160,6 +171,8 @@ private enum HexSettingKey: String, CodingKey, CaseIterable {
case wordRemappings
case lowercaseTranscripts
case removePunctuation
case transcribeMode
case customModelDirectory
}

private struct SettingsField<Value: Codable & Sendable> {
Expand Down Expand Up @@ -294,6 +307,15 @@ private enum HexSettingsSchema {
default: defaults.wordRemappings
).eraseToAny(),
SettingsField(.lowercaseTranscripts, keyPath: \.lowercaseTranscripts, default: defaults.lowercaseTranscripts).eraseToAny(),
SettingsField(.removePunctuation, keyPath: \.removePunctuation, default: defaults.removePunctuation).eraseToAny()
SettingsField(.removePunctuation, keyPath: \.removePunctuation, default: defaults.removePunctuation).eraseToAny(),
SettingsField(.transcribeMode, keyPath: \.transcribeMode, default: defaults.transcribeMode).eraseToAny(),
SettingsField(
.customModelDirectory,
keyPath: \.customModelDirectory,
default: defaults.customModelDirectory,
encode: { container, key, value in
try container.encodeIfPresent(value, forKey: key)
}
).eraseToAny()
Comment thread
coderabbitai[bot] marked this conversation as resolved.
]
}
68 changes: 68 additions & 0 deletions HexCore/Tests/HexCoreTests/ParallelModeAndSettingsTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import XCTest
@testable import HexCore

final class ParallelModeAndSettingsTests: XCTestCase {
func testDefaultSettingsHaveParallelModeDisabledByDefault() {
let settings = HexSettings()
XCTAssertEqual(settings.transcribeMode, .default)
XCTAssertNil(settings.customModelDirectory)
}

func testParallelModeSettingRoundTripSerialization() throws {
var settings = HexSettings()
settings.transcribeMode = .parallel
settings.customModelDirectory = "/Users/test/Documents/models"

let encoder = JSONEncoder()
let data = try encoder.encode(settings)

let decoder = JSONDecoder()
let decoded = try decoder.decode(HexSettings.self, from: data)

XCTAssertEqual(decoded.transcribeMode, .parallel)
XCTAssertEqual(decoded.customModelDirectory, "/Users/test/Documents/models")
XCTAssertEqual(decoded, settings)
}

func testDecodingJsonPayloadWithParallelTranscribeMode() throws {
let jsonString = """
{
"transcribeMode": "parallel",
"customModelDirectory": "/Users/test/Documents"
}
"""
guard let data = jsonString.data(using: .utf8) else {
XCTFail("Failed to convert JSON string to Data")
return
}

let decoded = try JSONDecoder().decode(HexSettings.self, from: data)
XCTAssertEqual(decoded.transcribeMode, .parallel)
XCTAssertEqual(decoded.customModelDirectory, "/Users/test/Documents")
}

func testTranscriptionModeEnumCasesAndRawValues() {
XCTAssertEqual(TranscriptionMode.default.rawValue, "default")
XCTAssertEqual(TranscriptionMode.parallel.rawValue, "parallel")
XCTAssertEqual(TranscriptionMode.allCases.count, 2)
}

func testModelPatternMatcherResolvesExactMatch() {
let match = ModelPatternMatcher.matches("openai_whisper-base", "openai_whisper-base")
XCTAssertTrue(match)
}

func testModelPatternMatcherResolvesWildcardMatch() {
let match = ModelPatternMatcher.matches("openai_whisper-*", "openai_whisper-large-v3-v20240930_turbo_632MB")
XCTAssertTrue(match)
}

func testModelPatternMatcherResolvesDownloadedPreference() {
let models = [
(name: "openai_whisper-large-v3", isDownloaded: false),
(name: "openai_whisper-large-v3_turbo", isDownloaded: true)
]
let resolved = ModelPatternMatcher.resolvePattern("openai_whisper-large-v3*", from: models)
XCTAssertEqual(resolved, "openai_whisper-large-v3_turbo")
}
}