Hugging Face's logo
Ivan's picture

Ivan PRO

aufklarer
7 5 3
21world's profile picture pronoobie's profile picture Reza2kn's profile picture
·
https://blog.ivan.digital

AI & ML interests

GenAI

Recent Activity

updated a model about 7 hours ago
aufklarer/IndexTTS2-MLX-fp16
published a model about 7 hours ago
aufklarer/IndexTTS2-MLX-fp16
reacted to theirpost with ❤️ about 15 hours ago
A full voice agent — hearing, thinking, and speaking — now fits in ~1.2 GB on an iPhone. And on a Mac, it runs Gemma 4 at 214 tokens/sec, entirely on-device. We just measured our on-device voice-agent pipeline end to end, and the numbers are the whole story. It's one loop — voice detection → speech-to-text → language model → speech synthesis — and the only thing that changes per device is which model fills each slot: 📱 iPhone 16 Pro — whole loop resident in ~1.2 GB, speech-to-text at 0.04 RTF on the Neural Engine 📱 Galaxy S23 — ~1.5 GB, real-time on CPU 💻 Mac (M5 Pro) — ~3.6 GB, now with Gemma 4 E2B as the brain at 214 tok/s and Supertonic-3 speech synthesis at 0.03 RTF The desktop result is the new part: a real conversational model that doesn't just answer — it drives the machine by voice, opening apps and acting on them, fully local. On paper the per-model peaks add up to ~5 GB, but measured co-resident it's ~3.6 GB — the on-device weights are memory-mapped, so they don't stack. Audio and conversation state stay on the device: private by default, no server round-trip, no per-call cost. Perception, reasoning, and synthesis — all on the metal in front of you. Full write-up, with the mobile and the new desktop benchmark tables and the architecture diagram: https://soniqo.audio/blog/on-device-voice-agents Linux/WIndows: https://github.com/soniqo/speech-core Android: https://github.com/soniqo/speech-android Mac/iOS: https://github.com/soniqo/speech-swift #OnDeviceAI #EdgeAI #AppleSilicon #MLX #LocalLLM #VoiceAI #Gemma
View all activity

Organizations

Soniqo Audio's profile picture