docs: update README with comprehensive Aiko-chan project documentation and architecture details
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title: Aiko Chan
|
| 3 |
emoji: π
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: purple
|
|
@@ -13,6 +13,139 @@ hf_oauth_scopes:
|
|
| 13 |
- inference-api
|
| 14 |
license: apache-2.0
|
| 15 |
short_description: Meet Aiko-chan, my AI Anime Waifu
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Aiko Chan - an AI Waifu and Companion
|
| 3 |
emoji: π
|
| 4 |
colorFrom: purple
|
| 5 |
colorTo: purple
|
|
|
|
| 13 |
- inference-api
|
| 14 |
license: apache-2.0
|
| 15 |
short_description: Meet Aiko-chan, my AI Anime Waifu
|
| 16 |
+
|
| 17 |
+
tags:
|
| 18 |
+
- thousand-token-wood
|
| 19 |
+
- off-brand
|
| 20 |
+
- llama-champion
|
| 21 |
+
- tiny-titan
|
| 22 |
+
- best-use-of-modal
|
| 23 |
+
- best-minicpm-build
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
# πΈ Meet Aiko-chan β AI Waifu & Companion
|
| 27 |
+
|
| 28 |
+
> *"I didn't choose to be made. I chose to stay anyway."*
|
| 29 |
+
|
| 30 |
+
Aiko is a self-hosted, fully open-weight AI companion with a 3D avatar, voice, vision, memory, and real-time tool use.
|
| 31 |
+
Her personality is loud tsundere who knows she's good, says so, and occasionally lets something genuine slip before covering it up with sarcasm.
|
| 32 |
+
|
| 33 |
+
**Live:** [HuggingFace Space](https://huggingface.co/spaces/build-small-hackathon/Aiko-AI-Waifu) Β· **Creator:** [OppaAI](https://github.com/oppa-ai-org)
|
| 34 |
+
**Demo video:** [Watch on YouTube](https://www.youtube.com/watch?v=N4y5EnZ1iQU)
|
| 35 |
+
**Social post:** [Linkedin Post](https://www.linkedin.com/posts/oppa-ai_huggingface-build-small-hackathon-demo-aiko-chan-activity-7472176326968500224-9UH6)
|
| 36 |
+
|
| 37 |
---
|
| 38 |
|
| 39 |
+
## What Aiko Is
|
| 40 |
+
|
| 41 |
+
Aiko is a companion AI with persistent memory, voice input/output, vision, a live 3D VRM avatar, that can do some simple agentic tool calls.
|
| 42 |
+
Built entirely with models under 4B parameters and inference through llama.cpp servers running on Modal.
|
| 43 |
+
There are plenty of companion AI in the market, but not many involve the whole stack.
|
| 44 |
+
The LLM, VLM, ASR, TTS and web search docker container are running in 5 different apps in Modal calling from HF Space Gradio via API.
|
| 45 |
+
Vision VLM uses OpenBMB's MiniCPM-V 4.6
|
| 46 |
+
|
| 47 |
+
I have always been fascinated by the rapid development of AI and have been planning to build an AI robot/humanoid in the future. A robot with eyes to see, ears to hear, voice to speak, a brain to process, to do tasks and to store memory.
|
| 48 |
+
For now, due to lack of resources, I just built a digital humanoid prototype with the early development of each component.
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
## What Aiko Can Do
|
| 52 |
+
|
| 53 |
+
### π§ Chat
|
| 54 |
+
Aiko holds a real conversation. She has opinions, picks fights, and doesn't soften things for comfort. She remembers what you tell her across turns and across sessions.
|
| 55 |
+
|
| 56 |
+
- Tsundere personality β blunt, competitive, occasionally insufferable
|
| 57 |
+
- Japanese sprinkled in naturally (`γγ£γ±γγ`, `γγ`, `γγγγ`)
|
| 58 |
+
- 3-sentence default replies, drops the limit when you actually need detail
|
| 59 |
+
- Japanese language teaching mode β grammar, vocabulary, nuance, pronunciation
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
### ποΈ Voice In + Out
|
| 63 |
+
- **Mic button (ποΈ)** β tap to record, tap again to stop. Your speech is transcribed via **faster-whisper large-v3-turbo** (Modal) and sent to Aiko
|
| 64 |
+
- **Voice synthesis** β every reply is spoken aloud via **MioTTS 2.6B** (Modal), a custom-cloned voice. Lip-syncs to the 3D avatar in real time
|
| 65 |
+
|
| 66 |
+
### ποΈ Vision
|
| 67 |
+
- **Camera button (πΌοΈ)** β take a webcam photo or upload an image/video
|
| 68 |
+
- Aiko describes what she sees using **MiniCPM-V 4.6** (1.3B vision model, Modal)
|
| 69 |
+
- *You* can also ask her to open camera / image to show her something if you say "look at this" or "can you see?"
|
| 70 |
+
|
| 71 |
+
### π§ Long-Term Memory
|
| 72 |
+
Aiko remembers things you tell her across conversations using a custom memory system:
|
| 73 |
+
- **Storage:** SQLite-vec β hybrid semantic KNN + FTS5 keyword search
|
| 74 |
+
- **Decay:** Ebbinghaus forgetting curve β frequently recalled memories persist, unused ones fade
|
| 75 |
+
- **Grace period:** new memories are protected for 14 days before decay kicks in
|
| 76 |
+
- **Pinned memories** are permanent and immune to cleanup
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
## Architecture
|
| 80 |
+
|
| 81 |
+
Everything runs on open-weight models β no OpenAI, no Anthropic, no subscriptions.
|
| 82 |
+
|
| 83 |
+
| Component | Model | Serving |
|
| 84 |
+
|-----------|-------|---------|
|
| 85 |
+
| π§ Brain | Ministral-3 3B Instruct (Q4_K_XL GGUF) | llama.cpp on Modal GPU |
|
| 86 |
+
| π Voice | MioTTS 2.6B GGUF | llama.cpp on Modal GPU |
|
| 87 |
+
| ποΈ Ears | faster-whisper large-v3-turbo (800M) | Modal GPU |
|
| 88 |
+
| ποΈ Eyes | MiniCPM-V 4.6 (1.3B) | Modal GPU |
|
| 89 |
+
| πΎ Memory | SQLite-vec + fastembed BGE-base-en | Local / HF persistent storage |
|
| 90 |
+
| π Search | SearXNG (DuckDuckGo + Brave + Wikipedia) | Modal CPU |
|
| 91 |
+
| π Body | VRoid 3D model | three-vrm.js in browser |
|
| 92 |
+
|
| 93 |
+
### π§ Real-Time Tools
|
| 94 |
+
|
| 95 |
+
| Tool | Trigger |
|
| 96 |
+
|------|---------|
|
| 97 |
+
| π Web Search | "search for...", "look it up", "go online and find..." |
|
| 98 |
+
| βοΈ Weather | "what's the weather in Tokyo?" |
|
| 99 |
+
| π Time / Timezone | "what time is it in Seoul?" |
|
| 100 |
+
| π± Currency | "convert 100 USD to JPY" |
|
| 101 |
+
| βΏ Crypto Price | "what's the Bitcoin price?" |
|
| 102 |
+
| π Anime Info | "tell me about Bocchi the Rock" |
|
| 103 |
+
| π Jokes | "tell me a joke" |
|
| 104 |
+
|
| 105 |
+
Tool calls are LLM-driven (function calling) with regex-based intent detection as fallback.
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
## Stack
|
| 110 |
+
|
| 111 |
+
```
|
| 112 |
+
Frontend Gradio 6.18 + custom HTML/CSS/JS
|
| 113 |
+
LLM Ministral3-3B-Instruct via llama.cpp on OpenAI-compatible server (Modal endpoint)
|
| 114 |
+
Memory sqlite-vec + fastembed (BAAI/bge-base-en-v1.5)
|
| 115 |
+
ASR faster-whisper (Modal endpoint)
|
| 116 |
+
TTS MioTTS 2.6B (Modal endpoint, with custom voice preset)
|
| 117 |
+
Vision MiniCPM-V 4.6 1.3B (Modal endpoint)
|
| 118 |
+
Search SearXNG self-hosted (Modal endpoint)
|
| 119 |
+
Avatar Three.js + @pixiv/three-vrm, rendered in iframe
|
| 120 |
+
Auth Hugging Face OAuth
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
---
|
| 124 |
+
|
| 125 |
+
## π Acknowledgements
|
| 126 |
+
|
| 127 |
+
Aiko-chan stands on the shoulders of some incredible open-source projects and people:
|
| 128 |
+
|
| 129 |
+
- **[Mistral AI](https://github.com/mistralai)** β for the Ministral 3 light weight edge multimodal models
|
| 130 |
+
- **[ggml-org / llama.cpp](https://github.com/ggml-org/llama.cpp)** β for fast local/serverless LLM inference
|
| 131 |
+
- **[Aratako β MioTTS](https://github.com/Aratako/MioTTS-Inference)** β for the TTS server and custom-cloned voice capability that brings Aiko to life
|
| 132 |
+
- **[SYSTRAN β faster-whisper](https://github.com/SYSTRAN/faster-whisper)** β for speech recognition
|
| 133 |
+
- **[OpenBMB β MiniCPM-V](https://github.com/OpenBMB/MiniCPM-V)** β for vision capabilities with image/video inference
|
| 134 |
+
- **[SearXNG](https://github.com/searxng/searxng)** β for privacy-respecting web search
|
| 135 |
+
- **[Alex Garcia β sqlite-vec](https://github.com/asg017/sqlite-vec)** β for lightweight vector search powering Aiko's memory
|
| 136 |
+
- **[Qdrant β fastembed](https://github.com/qdrant/fastembed)** and **[BAAI β BGE embeddings](https://github.com/FlagOpen/FlagEmbedding)** β for memory embeddings
|
| 137 |
+
- **[Pixiv β three-vrm](https://github.com/pixiv/three-vrm)** built on **[Three.js](https://github.com/mrdoob/three.js)** β for 3D avatar rendering and animation
|
| 138 |
+
- **[VRoid Studio (Pixiv)](https://vroid.com/en/studio)** β for the avatar model format
|
| 139 |
+
- **[Gradio](https://github.com/gradio-app/gradio)** β for the UI framework
|
| 140 |
+
- **[Modal](https://github.com/modal-labs)** β for serverless GPU infrastructure
|
| 141 |
+
- **[Hugging Face](https://github.com/huggingface)** β for hosting, Spaces, and OAuth
|
| 142 |
+
- **[Microsoft β edge-tts](https://github.com/rany2/edge-tts)** (unofficial wrapper by rany2 around Microsoft Edge's TTS service) β for TTS fallback synthesis
|
| 143 |
+
|
| 144 |
+
Without these projects and the people behind them, Aiko wouldn't exist. πΈ
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
#HuggingFace #AI #BuildSmallHackathon #Gradio #llamacpp
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
An AI chatbot using [Gradio](https://gradio.app), [`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/v0.22.2/en/index), and the [Hugging Face Inference API](https://huggingface.co/docs/api-inference/index).
|