OppaAI commited on
Commit
9ecebbc
Β·
1 Parent(s): e5b79b3

docs: update README with comprehensive Aiko-chan project documentation and architecture details

Browse files
Files changed (1) hide show
  1. README.md +135 -2
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: Aiko Chan
3
  emoji: πŸ‘€
4
  colorFrom: purple
5
  colorTo: purple
@@ -13,6 +13,139 @@ hf_oauth_scopes:
13
  - inference-api
14
  license: apache-2.0
15
  short_description: Meet Aiko-chan, my AI Anime Waifu
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ---
17
 
18
- An example chatbot using [Gradio](https://gradio.app), [`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/v0.22.2/en/index), and the [Hugging Face Inference API](https://huggingface.co/docs/api-inference/index).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Aiko Chan - an AI Waifu and Companion
3
  emoji: πŸ‘€
4
  colorFrom: purple
5
  colorTo: purple
 
13
  - inference-api
14
  license: apache-2.0
15
  short_description: Meet Aiko-chan, my AI Anime Waifu
16
+
17
+ tags:
18
+ - thousand-token-wood
19
+ - off-brand
20
+ - llama-champion
21
+ - tiny-titan
22
+ - best-use-of-modal
23
+ - best-minicpm-build
24
+ ---
25
+
26
+ # 🌸 Meet Aiko-chan β€” AI Waifu & Companion
27
+
28
+ > *"I didn't choose to be made. I chose to stay anyway."*
29
+
30
+ Aiko is a self-hosted, fully open-weight AI companion with a 3D avatar, voice, vision, memory, and real-time tool use.
31
+ Her personality is loud tsundere who knows she's good, says so, and occasionally lets something genuine slip before covering it up with sarcasm.
32
+
33
+ **Live:** [HuggingFace Space](https://huggingface.co/spaces/build-small-hackathon/Aiko-AI-Waifu) Β· **Creator:** [OppaAI](https://github.com/oppa-ai-org)
34
+ **Demo video:** [Watch on YouTube](https://www.youtube.com/watch?v=N4y5EnZ1iQU)
35
+ **Social post:** [Linkedin Post](https://www.linkedin.com/posts/oppa-ai_huggingface-build-small-hackathon-demo-aiko-chan-activity-7472176326968500224-9UH6)
36
+
37
  ---
38
 
39
+ ## What Aiko Is
40
+
41
+ Aiko is a companion AI with persistent memory, voice input/output, vision, a live 3D VRM avatar, that can do some simple agentic tool calls.
42
+ Built entirely with models under 4B parameters and inference through llama.cpp servers running on Modal.
43
+ There are plenty of companion AI in the market, but not many involve the whole stack.
44
+ The LLM, VLM, ASR, TTS and web search docker container are running in 5 different apps in Modal calling from HF Space Gradio via API.
45
+ Vision VLM uses OpenBMB's MiniCPM-V 4.6
46
+
47
+ I have always been fascinated by the rapid development of AI and have been planning to build an AI robot/humanoid in the future. A robot with eyes to see, ears to hear, voice to speak, a brain to process, to do tasks and to store memory.
48
+ For now, due to lack of resources, I just built a digital humanoid prototype with the early development of each component.
49
+
50
+
51
+ ## What Aiko Can Do
52
+
53
+ ### 🧠 Chat
54
+ Aiko holds a real conversation. She has opinions, picks fights, and doesn't soften things for comfort. She remembers what you tell her across turns and across sessions.
55
+
56
+ - Tsundere personality β€” blunt, competitive, occasionally insufferable
57
+ - Japanese sprinkled in naturally (`γ‚„γ£γ±γ‚Šγ­`, `もう`, `うるさい`)
58
+ - 3-sentence default replies, drops the limit when you actually need detail
59
+ - Japanese language teaching mode β€” grammar, vocabulary, nuance, pronunciation
60
+
61
+
62
+ ### πŸŽ™οΈ Voice In + Out
63
+ - **Mic button (πŸŽ™οΈ)** β€” tap to record, tap again to stop. Your speech is transcribed via **faster-whisper large-v3-turbo** (Modal) and sent to Aiko
64
+ - **Voice synthesis** β€” every reply is spoken aloud via **MioTTS 2.6B** (Modal), a custom-cloned voice. Lip-syncs to the 3D avatar in real time
65
+
66
+ ### πŸ‘οΈ Vision
67
+ - **Camera button (πŸ–ΌοΈ)** β€” take a webcam photo or upload an image/video
68
+ - Aiko describes what she sees using **MiniCPM-V 4.6** (1.3B vision model, Modal)
69
+ - *You* can also ask her to open camera / image to show her something if you say "look at this" or "can you see?"
70
+
71
+ ### 🧠 Long-Term Memory
72
+ Aiko remembers things you tell her across conversations using a custom memory system:
73
+ - **Storage:** SQLite-vec β€” hybrid semantic KNN + FTS5 keyword search
74
+ - **Decay:** Ebbinghaus forgetting curve β€” frequently recalled memories persist, unused ones fade
75
+ - **Grace period:** new memories are protected for 14 days before decay kicks in
76
+ - **Pinned memories** are permanent and immune to cleanup
77
+
78
+
79
+ ## Architecture
80
+
81
+ Everything runs on open-weight models β€” no OpenAI, no Anthropic, no subscriptions.
82
+
83
+ | Component | Model | Serving |
84
+ |-----------|-------|---------|
85
+ | 🧠 Brain | Ministral-3 3B Instruct (Q4_K_XL GGUF) | llama.cpp on Modal GPU |
86
+ | πŸ”Š Voice | MioTTS 2.6B GGUF | llama.cpp on Modal GPU |
87
+ | πŸŽ™οΈ Ears | faster-whisper large-v3-turbo (800M) | Modal GPU |
88
+ | πŸ‘οΈ Eyes | MiniCPM-V 4.6 (1.3B) | Modal GPU |
89
+ | πŸ’Ύ Memory | SQLite-vec + fastembed BGE-base-en | Local / HF persistent storage |
90
+ | πŸ” Search | SearXNG (DuckDuckGo + Brave + Wikipedia) | Modal CPU |
91
+ | 🎭 Body | VRoid 3D model | three-vrm.js in browser |
92
+
93
+ ### πŸ”§ Real-Time Tools
94
+
95
+ | Tool | Trigger |
96
+ |------|---------|
97
+ | πŸ” Web Search | "search for...", "look it up", "go online and find..." |
98
+ | β˜€οΈ Weather | "what's the weather in Tokyo?" |
99
+ | πŸ• Time / Timezone | "what time is it in Seoul?" |
100
+ | πŸ’± Currency | "convert 100 USD to JPY" |
101
+ | β‚Ώ Crypto Price | "what's the Bitcoin price?" |
102
+ | 🎌 Anime Info | "tell me about Bocchi the Rock" |
103
+ | πŸ˜‚ Jokes | "tell me a joke" |
104
+
105
+ Tool calls are LLM-driven (function calling) with regex-based intent detection as fallback.
106
+
107
+
108
+
109
+ ## Stack
110
+
111
+ ```
112
+ Frontend Gradio 6.18 + custom HTML/CSS/JS
113
+ LLM Ministral3-3B-Instruct via llama.cpp on OpenAI-compatible server (Modal endpoint)
114
+ Memory sqlite-vec + fastembed (BAAI/bge-base-en-v1.5)
115
+ ASR faster-whisper (Modal endpoint)
116
+ TTS MioTTS 2.6B (Modal endpoint, with custom voice preset)
117
+ Vision MiniCPM-V 4.6 1.3B (Modal endpoint)
118
+ Search SearXNG self-hosted (Modal endpoint)
119
+ Avatar Three.js + @pixiv/three-vrm, rendered in iframe
120
+ Auth Hugging Face OAuth
121
+ ```
122
+
123
+ ---
124
+
125
+ ## πŸ™ Acknowledgements
126
+
127
+ Aiko-chan stands on the shoulders of some incredible open-source projects and people:
128
+
129
+ - **[Mistral AI](https://github.com/mistralai)** β€” for the Ministral 3 light weight edge multimodal models
130
+ - **[ggml-org / llama.cpp](https://github.com/ggml-org/llama.cpp)** β€” for fast local/serverless LLM inference
131
+ - **[Aratako β€” MioTTS](https://github.com/Aratako/MioTTS-Inference)** β€” for the TTS server and custom-cloned voice capability that brings Aiko to life
132
+ - **[SYSTRAN β€” faster-whisper](https://github.com/SYSTRAN/faster-whisper)** β€” for speech recognition
133
+ - **[OpenBMB β€” MiniCPM-V](https://github.com/OpenBMB/MiniCPM-V)** β€” for vision capabilities with image/video inference
134
+ - **[SearXNG](https://github.com/searxng/searxng)** β€” for privacy-respecting web search
135
+ - **[Alex Garcia β€” sqlite-vec](https://github.com/asg017/sqlite-vec)** β€” for lightweight vector search powering Aiko's memory
136
+ - **[Qdrant β€” fastembed](https://github.com/qdrant/fastembed)** and **[BAAI β€” BGE embeddings](https://github.com/FlagOpen/FlagEmbedding)** β€” for memory embeddings
137
+ - **[Pixiv β€” three-vrm](https://github.com/pixiv/three-vrm)** built on **[Three.js](https://github.com/mrdoob/three.js)** β€” for 3D avatar rendering and animation
138
+ - **[VRoid Studio (Pixiv)](https://vroid.com/en/studio)** β€” for the avatar model format
139
+ - **[Gradio](https://github.com/gradio-app/gradio)** β€” for the UI framework
140
+ - **[Modal](https://github.com/modal-labs)** β€” for serverless GPU infrastructure
141
+ - **[Hugging Face](https://github.com/huggingface)** β€” for hosting, Spaces, and OAuth
142
+ - **[Microsoft β€” edge-tts](https://github.com/rany2/edge-tts)** (unofficial wrapper by rany2 around Microsoft Edge's TTS service) β€” for TTS fallback synthesis
143
+
144
+ Without these projects and the people behind them, Aiko wouldn't exist. 🌸
145
+
146
+
147
+
148
+ #HuggingFace #AI #BuildSmallHackathon #Gradio #llamacpp
149
+
150
+
151
+ An AI chatbot using [Gradio](https://gradio.app), [`huggingface_hub`](https://huggingface.co/docs/huggingface_hub/v0.22.2/en/index), and the [Hugging Face Inference API](https://huggingface.co/docs/api-inference/index).