--- title: Aiko Chan - an AI Waifu and Companion emoji: ๐Ÿ‘€ colorFrom: purple colorTo: purple sdk: gradio sdk_version: 6.18.0 app_file: app.py pinned: true fullWidth: true hf_oauth: true hf_oauth_scopes: - inference-api license: apache-2.0 short_description: Meet Aiko-chan, my AI Anime Waifu tags: - thousand-token-wood - off-brand - llama-champion - tiny-titan - best-use-of-modal - best-minicpm-build --- # ๐ŸŒธ Meet Aiko-chan โ€” AI Waifu & Companion > *"I didn't choose to be made. I chose to stay anyway."* Aiko is a self-hosted, fully open-weight AI companion with a 3D avatar, voice, vision, memory, and real-time tool use. Her personality is loud tsundere who knows she's good, says so, and occasionally lets something genuine slip before covering it up with sarcasm. **Live:** [HuggingFace Space](https://huggingface.co/spaces/build-small-hackathon/Aiko-AI-Waifu) ยท **Creator:** [OppaAI](https://github.com/oppa-ai-org) **Demo video:** [Watch on YouTube](https://www.youtube.com/watch?v=N4y5EnZ1iQU) **Social post:** [Linkedin Post](https://www.linkedin.com/posts/oppa-ai_huggingface-build-small-hackathon-demo-aiko-chan-activity-7472176326968500224-9UH6) --- ## What Aiko Is Aiko is a companion AI with persistent memory, voice input/output, vision, a live 3D VRM avatar, that can do some simple agentic tool calls. Built entirely with models under 4B parameters and inference through llama.cpp servers running on Modal. There are plenty of companion AI in the market, but not many involve the whole stack. The LLM, VLM, ASR, TTS and web search docker container are running in 5 different apps in Modal calling from HF Space Gradio via API. Vision VLM uses OpenBMB's MiniCPM-V 4.6 I have always been fascinated by the rapid development of AI and have been planning to build an AI robot/humanoid in the future. A robot with eyes to see, ears to hear, voice to speak, a brain to process, to do tasks and to store memory. For now, due to lack of resources, I just built a digital humanoid prototype with the early development of each component. ## What Aiko Can Do ### ๐Ÿง  Chat Aiko holds a real conversation. She has opinions, picks fights, and doesn't soften things for comfort. She remembers what you tell her across turns and across sessions. - Tsundere personality โ€” blunt, competitive, occasionally insufferable - Japanese sprinkled in naturally (`ใ‚„ใฃใฑใ‚Šใญ`, `ใ‚‚ใ†`, `ใ†ใ‚‹ใ•ใ„`) - 3-sentence default replies, drops the limit when you actually need detail - Japanese language teaching mode โ€” grammar, vocabulary, nuance, pronunciation ### ๐ŸŽ™๏ธ Voice In + Out - **Mic button (๐ŸŽ™๏ธ)** โ€” tap to record, tap again to stop. Your speech is transcribed via **faster-whisper large-v3-turbo** (Modal) and sent to Aiko - **Voice synthesis** โ€” every reply is spoken aloud via **MioTTS 2.6B** (Modal), a custom-cloned voice. Lip-syncs to the 3D avatar in real time ### ๐Ÿ‘๏ธ Vision - **Camera button (๐Ÿ–ผ๏ธ)** โ€” take a webcam photo or upload an image/video - Aiko describes what she sees using **MiniCPM-V 4.6** (1.3B vision model, Modal) - *You* can also ask her to open camera / image to show her something if you say "look at this" or "can you see?" ### ๐Ÿง  Long-Term Memory Aiko remembers things you tell her across conversations using a custom memory system: - **Storage:** SQLite-vec โ€” hybrid semantic KNN + FTS5 keyword search - **Decay:** Ebbinghaus forgetting curve โ€” frequently recalled memories persist, unused ones fade - **Grace period:** new memories are protected for 14 days before decay kicks in - **Pinned memories** are permanent and immune to cleanup ## Architecture Everything runs on open-weight models โ€” no OpenAI, no Anthropic, no subscriptions. | Component | Model | Serving | |-----------|-------|---------| | ๐Ÿง  Brain | Ministral-3 3B Instruct (Q4_K_XL GGUF) | llama.cpp on Modal GPU | | ๐Ÿ”Š Voice | MioTTS 2.6B GGUF | llama.cpp on Modal GPU | | ๐ŸŽ™๏ธ Ears | faster-whisper large-v3-turbo (800M) | Modal GPU | | ๐Ÿ‘๏ธ Eyes | MiniCPM-V 4.6 (1.3B) | Modal GPU | | ๐Ÿ’พ Memory | SQLite-vec + fastembed BGE-base-en | Local / HF persistent storage | | ๐Ÿ” Search | SearXNG (DuckDuckGo + Brave + Wikipedia) | Modal CPU | | ๐ŸŽญ Body | VRoid 3D model | three-vrm.js in browser | ### ๐Ÿ”ง Real-Time Tools | Tool | Trigger | |------|---------| | ๐Ÿ” Web Search | "search for...", "look it up", "go online and find..." | | โ˜€๏ธ Weather | "what's the weather in Tokyo?" | | ๐Ÿ• Time / Timezone | "what time is it in Seoul?" | | ๐Ÿ’ฑ Currency | "convert 100 USD to JPY" | | โ‚ฟ Crypto Price | "what's the Bitcoin price?" | | ๐ŸŽŒ Anime Info | "tell me about Bocchi the Rock" | | ๐Ÿ˜‚ Jokes | "tell me a joke" | Tool calls are LLM-driven (function calling) with regex-based intent detection as fallback. ## Stack ``` Frontend Gradio 6.18 + custom HTML/CSS/JS LLM Ministral3-3B-Instruct via llama.cpp on OpenAI-compatible server (Modal endpoint) Memory sqlite-vec + fastembed (BAAI/bge-base-en-v1.5) ASR faster-whisper (Modal endpoint) TTS MioTTS 2.6B (Modal endpoint, with custom voice preset) Vision MiniCPM-V 4.6 1.3B (Modal endpoint) Search SearXNG self-hosted (Modal endpoint) Avatar Three.js + @pixiv/three-vrm, rendered in iframe Auth Hugging Face OAuth ``` --- ## ๐Ÿ™ Acknowledgements Aiko-chan stands on the shoulders of some incredible open-source projects and people: - **[Mistral AI](https://github.com/mistralai)** โ€” for the Ministral 3 light weight edge multimodal models - **[ggml-org / llama.cpp](https://github.com/ggml-org/llama.cpp)** โ€” for fast local/serverless LLM inference - **[Aratako โ€” MioTTS](https://github.com/Aratako/MioTTS-Inference)** โ€” for the TTS server and custom-cloned voice capability that brings Aiko to life - **[SYSTRAN โ€” faster-whisper](https://github.com/SYSTRAN/faster-whisper)** โ€” for speech recognition - **[OpenBMB โ€” MiniCPM-V](https://github.com/OpenBMB/MiniCPM-V)** โ€” for vision capabilities with image/video inference - **[SearXNG](https://github.com/searxng/searxng)** โ€” for privacy-respecting web search - **[Alex Garcia โ€” sqlite-vec](https://github.com/asg017/sqlite-vec)** โ€” for lightweight vector search powering Aiko's memory - **[Qdrant โ€” fastembed](https://github.com/qdrant/fastembed)** and **[BAAI โ€” BGE embeddings](https://github.com/FlagOpen/FlagEmbedding)** โ€” for memory embeddings - **[Pixiv โ€” three-vrm](https://github.com/pixiv/three-vrm)** built on **[Three.js](https://github.com/mrdoob/three.js)** โ€” for 3D avatar rendering and animation - **[VRoid Studio (Pixiv)](https://vroid.com/en/studio)** โ€” for the avatar model format - **[Gradio](https://github.com/gradio-app/gradio)** โ€” for the UI framework - **[Modal](https://github.com/modal-labs)** โ€” for serverless GPU infrastructure - **[Hugging Face](https://github.com/huggingface)** โ€” for hosting, Spaces, and OAuth - **[Microsoft โ€” edge-tts](https://github.com/rany2/edge-tts)** (unofficial wrapper by rany2 around Microsoft Edge's TTS service) โ€” for TTS fallback synthesis Without these projects and the people behind them, Aiko wouldn't exist. ๐ŸŒธ #HuggingFace #AI #BuildSmallHackathon #Gradio #llamacpp An AI chatbot using [Gradio](https://gradio.app), [`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/v0.22.2/en/index), and the [Hugging Face Inference API](https://huggingface.co/docs/api-inference/index).