A voice-controlled AI assistant for macOS that sees your screen, understands context, and helps you work smarter.
Say "Hey JARVIS" and it responds. Like Iron Man's assistant, but real - and optimized for Apple Silicon.
Portfolio: jarvis-portfolio.vercel.app
Voice Control - Hands-free operation with natural language. No memorizing commands.
Computer Vision - Monitors your screen in real-time. Detects notifications, errors, and updates across all apps.
Temporal Change Detection - Tracks visual changes over time. Ask "What changed in space 3?" or "Has the error been fixed?" and get accurate answers with 4-method image comparison.
Multi-Space Intelligence - Analyzes multiple Mission Control spaces simultaneously with 3x speedup. Compare spaces, locate errors, and search across your entire workspace instantly.
Context Awareness - Understands your workflow. Screen locked? JARVIS unlocks it before executing your command.
Display Automation - Voice-controlled display mirroring. Say "Connect to Living Room TV" and watch it happen automatically via coordinate automation and OCR fallbacks.
Proactive Intelligence - Alerts you to important information without being asked. Helps you prepare for meetings, catch errors, and stay focused.
- 95% CPU reduction - From 80-100% down to 1-5%
- 99.9% ML accuracy - Intent classification that actually works
- <50ms response - Real-time command routing
- 3x parallel speedup - Multi-space queries with async execution
- 10ms-2s change detection - 4-method temporal analysis with accuracy/speed tradeoffs
- ~3 second display connections - Voice-to-connection with OCR fallbacks
- Self-healing - Automatic recovery from failures
Custom ARM64 Assembly - Hand-optimized NEON code for Apple Silicon (M1/M2/M3). 40-50x faster than standard implementations.
ML-Powered Routing - Intelligent intent prediction. Adapts to how you speak, not hardcoded patterns.
Vision Intelligence - Uses Claude AI to analyze screen content. Works with ANY app through visual understanding, not API integrations.
Temporal Memory System - Screenshot caching with 4-method change detection (perceptual hash, OCR diff, pixel analysis, error state tracking). Enables time-based queries and visual history.
Parallel Async Architecture - Concurrent space analysis with graceful degradation. Analyzes 10 spaces in 625ms vs 1800ms sequential.
Coordinate Automation + OCR - Hybrid approach for display control. Fast coordinates with intelligent OCR fallbacks for resolution independence.
Zero Configuration - Autonomous service discovery. Self-healing with automatic error recovery.
Languages - Python • TypeScript • Rust • Swift
AI - Claude (Anthropic) • CoreML • Custom ML models
Architecture - FastAPI • Event-driven • Self-healing systems
JARVIS is a personal project I built for my own use. The code is private, but I wrote about the engineering challenges and solutions.
Read the technical blog posts to see how I:
- Optimized CPU usage from 80-100% down to 1-5%
- Built ML routing with 99.9% accuracy
- Created self-healing systems for zero-downtime operation
- Implemented context-aware intelligence and computer vision
- Built temporal change detection with 4-method image comparison
- Achieved 3x speedup with parallel multi-space queries
- Created voice-controlled display mirroring with OCR fallbacks
Note: Features like automated screen unlocking are designed for single-user, personal environments and would not be included in any public release.
- GitHub: @drussell23
- LinkedIn: Derek J. Russell
- Email: djamesr23@gmail.com