knowledge/genie.md

GENIE — Offline Voice AI Assistant (Final Year Project)

What GENIE is

GENIE stands for "Generate, Engage, Navigate Intelligent Environments". It is Arman's Final Year Project (FYP) at UMT Lahore: a FULLY OFFLINE voice AI assistant — zero cloud, zero API keys, zero subscriptions. All speech recognition, reasoning, and speech synthesis run on local hardware. GENIE is built for physically impaired users and for IoT-driven smart environments, where privacy, low cost, and offline reliability matter most.

Arman's role and the team

Arman is the Team Lead (Lead Developer / Hardware) of the 4-person team. Teammates: Muhammad Suhaib Mamoon (AI pipeline / Flask), Muhammad Huzaifa Furqan (Firmware / Testing), and Muhammad Farhal Raza (Documentation / Integration). Supervisor: Mahmood Hussain, UMT Department of Artificial Intelligence.

Hardware (about $22 total)

2× ESP32-S3 DevKitC-1 boards, 2× INMP441 I2S MEMS microphones (stereo capture), a MAX98357A class-D amplifier, a 3W speaker, and a custom soldered PCB with a 100µF decoupling capacitor. The two ESP32 boards communicate peer-to-peer over ESP-NOW.

AI pipeline and architecture

Dual-bus I2S audio capture → PSRAM buffering → WAV over WiFi (HTTP POST) → Flask server → OpenAI Whisper for speech-to-text (upgraded from base to small) → Mistral 7B running locally via Ollama for reasoning → pyttsx3 for text-to-speech → I2S playback through the MAX98357A amplifier. End-to-end latency is 8–15 seconds, fully on-prem.

Hard engineering problems solved

• Dual I2S bus management: microphone on I2S_NUM_0 and speaker on I2S_NUM_1 running simultaneously on one ESP32-S3.
• PSRAM buffer overflow fixed with 480KB audio buffers.
• WiFi audio crackle during streaming eliminated.
• "Ghost voice" distortion fixed by resampling 22050 Hz → 16 kHz with audioop.ratecv().
• INMP441 bit-shift tuning — a shift of >>13 found optimal for clean signal levels.

Project status

Phase 1 complete: 16/16 user stories delivered, 10/10 critical bugs resolved. Phase 2 nearly done: ESP-NOW peer-to-peer done, dual-mic stereo done, soldered PCB done, Whisper small upgrade done, IoT device control done. Remaining in Phase 2: the "Hey Genie" wake-word, a web dashboard with sign-in, and battery power. Phase 3 (planned): on-device AI inference and multi-language support. The GitHub repo goes public after the FYP viva.