GemmaRun

A SwiftUI iOS chat app that runs Google's Gemma language model entirely on-device. The first launch downloads the model weights from HuggingFace; every session after that is fully offline — no server, no API key, no data leaves the phone.

Requirements

• Xcode 26.4 or later
• iOS 26.4 deployment target
• Apple Silicon Mac for development
• A physical iPhone recommended (simulator works but has no Metal acceleration for MLX)

Getting started

git clone <this-repo>
cd GemmaRun
open GemmaRun.xcodeproj

Xcode will resolve Swift Package Manager dependencies on first open (pinned in Package.resolved). Select a real iPhone as the run destination, build, and run.

On first launch you'll see a download progress bar while the model (~700 MB, 4-bit quantized Gemma 3 1B) is pulled from HuggingFace and cached in the app sandbox. Subsequent launches skip straight to a spinner while the weights decode into memory, then you're chatting.

Model

Default: mlx-community/gemma-3-1b-it-4bit — Gemma 3 1B instruction-tuned, 4-bit quantized.

To use a larger variant, change GemmaService.defaultModelID and add the extended-virtual-addressing and increased-memory-limit capabilities to the target. See CLAUDE.md for details.

How it works

• MLX Swift (mlx-swift-examples) runs inference via Metal on the device GPU.
• ChatSession (from MLXLMCommon) owns conversation history and handles Gemma's chat template.
• swift-markdown-ui renders assistant replies with proper lists, code blocks, and inline formatting.

The three source files that matter:

File	Role
GemmaRun/GemmaService.swift	@Observable service: loads the model off the main actor, exposes a streaming AsyncThrowingStream<String, Error>
GemmaRun/ContentView.swift	Chat UI, message state, send/stop button, "New Chat" action
GemmaRun/MarkdownTheme.swift	Theme.gemmaRun used for rendering assistant bubbles

See CLAUDE.md for architectural notes — particularly the project-wide MainActor default and the Gemma-specific <end_of_turn> EOS registration.

Features

• Fully offline inference after first launch
• Streaming token-by-token responses
• Markdown-rendered assistant output (code blocks, lists, inline code, blockquotes, links)
• Stop mid-generation, start a new chat, reset conversation context
• Interactive scroll-to-dismiss keyboard, plus inline dismiss button

Dependencies

• ml-explore/mlx-swift-examples — products MLXLLM, MLXLMCommon
• gonzalezreal/swift-markdown-ui — product MarkdownUI

Transitive deps (MLX core, swift-transformers, swift-jinja for chat templates, swift-cmark, etc.) are pulled in automatically and pinned in Package.resolved.