🎭 What Is This?

Jarvis is a conversational AI that runs entirely on your machine using LM Studio. She's not another sycophantic assistant—she has opinions, standards, and won't coddle you.

The only cloud touch is ElevenLabs for premium neural TTS (optional—free tier gives you 10K chars/month).

Why Local?

🔒 Your conversations never leave your hardware

💰 No API rate limits or surprise bills

🔄 Swap models anytime—Nemotron, Qwen, Llama, whatever

✈️ Works offline (except voice synthesis)

Why Jarvis?

🎯 Direct feedback, not corporate pleasantries

🧠 Actually helpful, not just agreeable

🎙️ Premium voice that doesn't sound like a robot

⚡ Fast—runs on your GPU, not a queue

---

🖥️ See It In Action

_{Jarvis roasting someone's Rust rewrite idea. As she does.}

---

⚔️ Jarvis vs. The Cloud

Privacy	✅ 100% Local	❌ Cloud	❌ Cloud	❌ Cloud
Monthly Cost	$0*	$20+	$20+	Free (with limits)
Rate Limits	∞ Unlimited	⚠️ Throttled	⚠️ Throttled	⚠️ Throttled
Model Choice	✅ Any GGUF	❌ GPT only	❌ Claude only	❌ Locked
Works Offline	✅ Yes	❌ No	❌ No	❌ No
Custom Voice	✅ Clone yours	❌ No	❌ No	❌ Limited
Personality	🎭 Unfiltered	🤖 Corporate	🤖 Corporate	🤖 Corporate
Data Training	✅ Never	⚠️ Maybe	⚠️ Maybe	⚠️ Likely

_{*Free except optional ElevenLabs voice (free tier: 10K chars/month)}

🏗️ Architecture

flowchart LR
    subgraph LOCAL ["  💻 YOUR MACHINE  "]
        direction LR
        A[" 👤 You "] -->|"⌨️ Input"| B[" ⚡ Jarvis\nOrchestrator "]
        B -->|"💭 Query"| C[" 🧠 LM Studio\nLocal LLM "]
        C -->|"💬 Response"| B
    end
    
    B -->|"🔊 TTS Request"| D[" ☁️ ElevenLabs\nVoice API "]
    D -->|"🎙️ Audio"| B
    B -->|"🔈 Speech"| A

    style LOCAL fill:#1e1e2e,stroke:#a855f7,stroke-width:2px,color:#ffffff
    style A fill:#6366f1,stroke:#818cf8,color:#ffffff
    style B fill:#a855f7,stroke:#c084fc,color:#ffffff
    style C fill:#00d084,stroke:#34d399,color:#1e1e2e
    style D fill:#f97316,stroke:#fb923c,color:#ffffff

Loading

Text or Voice Input

➡️

Orchestrator

➡️

Local LLM

_{🌐 Cloud TTS • Optional • Free Tier Available}

📜 View Text Diagram

┌─────────────────────────────────────────────────────────────────────────┐
│                         💻  YOUR MACHINE                            │
│  ┌──────────┐    ┌─────────────────┐    ┌─────────────────────┐  │
│  │  👤 You   │───▶│  ⚡ Jarvis    │───▶│  🧠 LM Studio      │  │
│  │ keyboard │    │  orchestrator  │    │  local LLM engine  │  │
│  └──────────┘    └────────┬────────┘    └─────────────────────┘  │
│                        │                                          │
└────────────────────────┼────────────────────────────────────────────────┘
                         │
                         ▼
                ┌─────────────────┐
                │ ☁️ ElevenLabs  │  ← Only external call
                │   voice API    │
                └─────────────────┘

✨ Features

	Feature	Description
🧠	Local LLM	Any GGUF model via LM Studio—Nemotron, Qwen, Llama, Mistral, DeepSeek
🎙️	Premium Voice	ElevenLabs neural TTS with custom voice cloning support
🎭	Real Personality	Sharp, confident, witty—inspired by Jarvis of Vengerberg
📊	Token Tracking	Visual context window with auto-trim at 85% capacity
💳	Credits Monitor	Real-time ElevenLabs character usage tracking
🧹	Thinking Stripper	Auto-removes <think> tags so reasoning isn't spoken aloud
🖥️	Cross-Platform	Windows native or Mac → Windows remote via LAN

🚀 Quick Start

Prerequisites

Requirement	Why	Get It
Python 3.10+	Runtime	python.org
LM Studio	Local LLM	lmstudio.ai
ElevenLabs	Voice	elevenlabs.io
NVIDIA GPU	Speed	Recommended

Installation

# Clone the repo
git clone https://github.com/kunalnano/jarvis.git
cd jarvis

# Windows
.\setup.bat

# Mac/Linux
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Configuration

# Create your environment file
cp .env.example .env

Edit .env with your keys:

ELEVENLABS_API_KEY=your_key_here      # From elevenlabs.io/settings/api-keys
ELEVENLABS_VOICE_ID=your_voice_id     # From your Voice Library

Launch

1️⃣ Start LM Studio Load a model Go to Local Server Click Start

2️⃣ Run Jarvis # Windows .\start_jarvis.bat # Mac/Linux ./start_jarvis.sh

That's it. She's waiting.

🎮 Commands

Command	What It Does
status	📊 Show token usage and context window health
credits	💳 Display ElevenLabs character usage
voice	🎙️ Show voice settings and session stats
clear	🧹 Wipe conversation memory
quit	👋 Exit gracefully

💡 Pro tip: On Windows, press Win+H for system-level voice dictation.

🤖 Recommended Models

Model	VRAM	Speed	Notes
NVIDIA Nemotron-Mini-4B	~4GB	⚡⚡⚡	Great for quick interactions
Nemotron-3-Nano-30B-A3B	~18GB	⚡⚡	Best reasoning-to-VRAM ratio
Qwen3-30B-A3B	~18GB	⚡⚡	Excellent all-around performer
Llama-3.1-8B-Instruct	~6GB	⚡⚡⚡	Good for lighter hardware
DeepSeek-R1-Distill-Qwen-14B	~10GB	⚡⚡	Strong reasoning model

🧠 Reasoning models that use <think>...</think> tags are automatically filtered. Jarvis thinks before speaking, but keeps her thoughts to herself.

🎙️ Voice Configuration

Model	Latency	Quality	Best For
eleven_turbo_v2_5	⚡ Fastest	Good	Daily use
eleven_flash_v2_5	Fast	Good	Balance
eleven_multilingual_v2	Slower	Best	Quality priority

Fine-tune the voice in config/jarvis.yaml:

voice_output:
  stability: 0.6          # 0-1: Higher = more consistent pitch
  similarity_boost: 0.75  # 0-1: Voice matching accuracy
  style: 0.0              # 0-1: Style exaggeration (keep low)
  speed: 1.15             # 0.25-4.0: Speech rate

🌐 Remote Setup (Mac → Windows)

Running LM Studio on a beefy Windows rig but want to talk from your Mac?

┌─────────────────┐                    ┌─────────────────┐
│  🍎 Mac         │    LAN/WiFi        │  🪟 Windows     │
│  (thin client)  │◄──────────────────▶│  (GPU server)   │
│  Jarvis CLI   │                    │  LM Studio      │
└─────────────────┘                    └─────────────────┘

1. Windows (LM Studio): Enable "Serve on Local Network" in Local Server settings
2. Windows: Run ipconfig → note your LAN IP (e.g., 192.168.1.100)
3. Mac: Update config/jarvis.yaml:

   llm:
     api_base: "http://192.168.1.100:1234/v1"

🎭 Personality

Jarvis doesn't do corporate AI pleasantries. She's helpful, but she'll call out bad ideas.

You: I'm thinking of learning three programming languages at once. Jarvis: How ambitious. You'll drown in syntax before you master any of them. Pick one. Learn it properly. Then consider the others.

You: Can you help me with my code? Jarvis: Show me what you've got. I'll tell you what's wrong with it.

You: I want to build a startup but I have no idea what problem to solve. Jarvis: Then you don't want to build a startup—you want the idea of building one. Find a problem that genuinely irritates you first.

She's an equal, not a servant. Inspired by Jarvis of Vengerberg—confident, sharp, doesn't suffer fools gladly.

🗂️ Project Structure

jarvis/
├── jarvis/                 # Core Python package
│   ├── main.py             # Entry point + ASCII banner
│   ├── orchestrator.py     # Conversation loop controller
│   ├── brain.py            # LLM integration (OpenAI-compatible API)
│   ├── voice.py            # ElevenLabs TTS + thinking tag stripper
│   ├── ears.py             # Input handler
│   └── config.py           # YAML loader with ${ENV_VAR} expansion
├── config/
│   └── jarvis.yaml         # Main configuration file
├── .env.example            # API key template
├── requirements.txt        # Python dependencies
├── start_jarvis.bat      # Windows launcher
├── start_jarvis.sh       # Mac/Linux launcher
├── CHANGELOG.md            # Version history
└── README.md               # You are here

🐛 Troubleshooting

🔴 "Cannot connect to LM Studio"

• Is LM Studio running with a model loaded?
• Check Local Server tab shows "Running"
• Verify api_base in config matches your setup (default: http://localhost:1234/v1)

🔴 "ElevenLabs 401 error"

• Verify API key in .env file
• Check key validity at https://elevenlabs.io/app/settings/api-keys
• Ensure you haven't exceeded your character limit

🔴 Voice sounds robotic or jarring

• Increase stability to 0.7-0.8 in config
• Try eleven_multilingual_v2 model for smoother output
• Reduce speed if words are clipping

🔴 Thinking tags being spoken aloud

• Update to v0.3.0+ (automatic stripping included)
• The stripper handles <think>, <thinking>, unclosed tags, and edge cases

📋 Roadmap

🔜 Coming Soon

Feature	Status
Wake word detection — "Hey Jarvis"	🔲
Streaming TTS — speak before generation completes	🔲
Interrupt handling — stop mid-sentence	🔲

🔮 Future

Feature	Status
Memory persistence across sessions	🔲
Multi-voice support — switch characters	🔲
Tool plugins — file ops, web search, etc.	🔲

📜 License

MIT — Do whatever you want. Credit appreciated but not required.

🤝 Contributing

PRs welcome. See CHANGELOG.md for what's been done.

Good First Contributions
🎤 Wake word detection integration (Porcupine, Snowboy, etc.)
🔊 Alternative TTS backends (Coqui, Bark, local options)
🎙️ Voice activity detection for natural turn-taking
🔢 Tiktoken integration for accurate token counting

Built with spite and good taste.

If you find this useful, star the repo.

Jarvis doesn't ask for validation, but the algorithm appreciates it.