Silvestre-PO commited on
Commit
a20e038
Β·
verified Β·
1 Parent(s): 1f2f327

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +294 -7
README.md CHANGED
@@ -1,12 +1,299 @@
1
  ---
2
- title: Curiosity StoryBook
3
- emoji: 🌍
4
- colorFrom: purple
5
- colorTo: purple
6
  sdk: docker
7
  pinned: false
8
- license: apache-2.0
9
- short_description: Curiosity StoryBook is a locally-run
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Curiosity StoryBook (Socratic Edition)
3
+ emoji: πŸ“–
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: docker
7
  pinned: false
8
+ app_port: 7860
9
+ tags:
10
+ - build-small-hackathon
11
+ - track:backyard-ai
12
+ - badge:tiny-titan
13
+ - badge:best-agent
14
+ - badge:off-brand
15
+ - badge:best-demo
16
+ - quest:off-the-grid
17
+ - quest:llama-champion
18
+ - quest:sharing-is-care
19
+ - quest:field-notes
20
  ---
21
 
22
+ # πŸ“– Curiosity StoryBook (Socratic Edition)
23
+
24
+ **Curiosity StoryBook** is a locally-run, educational story generator designed for children. It features a Socratic cognitive engine alongside an illustration and audio-narration pipeline optimized to run on consumer local hardware under a strict **6 GB VRAM budget**.
25
+
26
+ The project offers two independent execution flows:
27
+ 1. **Classic Socratic Edition (`app.py`)**: A 5-page interactive journey where the AI does not provide direct answers. Instead, it uses Socratic dialogue (guided questions and clues) to help the child reason and solve the scientific mystery on their own.
28
+ 2. **Quick Response Edition (Simple Mode - `app_simple.py`)**: A streamlined, single-page interface. The child asks their curiosity question ("Why...?"), and the system instantly generates a self-contained explanatory short story, a watercolor-style illustration, and its voice narrationβ€”resolving the mystery in a single step to minimize latency.
29
+
30
+ ---
31
+
32
+ ## πŸ—οΈ System Architecture
33
+
34
+ ### πŸ“ General Layered Architecture
35
+
36
+ The project is structured into a modular layered architecture, separating the presentation web interface, session/trace orchestration, AI models inference, and hardware resource boundaries:
37
+
38
+ ```mermaid
39
+ graph TD
40
+ classDef layer fill:#f5f5f5,stroke:#9e9e9e,stroke-width:2px,stroke-dasharray: 5 5;
41
+ classDef component fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
42
+ classDef model fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px;
43
+ classDef infra fill:#eceff1,stroke:#607d8b,stroke-width:2px;
44
+
45
+ subgraph Presentation_Layer [Presentation Layer / User Interfaces]
46
+ AppClassic[Classic Socratic App - app.py / Port 7860]
47
+ AppSimple[Simple Mode App - app_simple.py / Port 7861]
48
+ end
49
+
50
+ subgraph Orchestration_Layer [Orchestration & State Management]
51
+ Coordinator[Parallel Asset Coordinator]
52
+ Context[StoryContext / agent_trace.json]
53
+ Exporter[ZIP Session Exporter]
54
+ end
55
+
56
+ subgraph Inference_Layer [Inference & Model Engines]
57
+ Evaluator[Socratic Evaluator - MiniCPM 1B CPU]
58
+ ASREngine[ASR Pipeline - NeMo 600M GPU]
59
+ Narrator[Narrative Engine - Tiny Aya 1.5B 4-bit GPU]
60
+ Illustrator[FLUX.2 Diffusion - FLUX.2 Klein 4B GPU]
61
+ TTSEngine[TTS Voice Engine - VoxCPM2 2.5B CPU]
62
+ end
63
+
64
+ subgraph Infrastructure_Layer [Infrastructure & Hardware]
65
+ WSL[WSL2 Ubuntu Linux / Python 3.10 Conda]
66
+ VRAM[Shared CUDA GPU Memory / Strict 6GB VRAM Limit]
67
+ CPU[System CPU & RAM / VoxCPM2 + MiniCPM GGUF]
68
+ end
69
+
70
+ %% Connections
71
+ AppClassic --> Context
72
+ AppSimple --> Context
73
+
74
+ Context --> Evaluator
75
+ Context --> ASREngine
76
+ Context --> Narrator
77
+
78
+ Context --> Coordinator
79
+ Coordinator --> Illustrator
80
+ Coordinator --> TTSEngine
81
+
82
+ Illustrator --> Exporter
83
+ TTSEngine --> Exporter
84
+ Context --> Exporter
85
+
86
+ %% Infrastructure bindings
87
+ Evaluator --> CPU
88
+ TTSEngine --> CPU
89
+
90
+ ASREngine --> VRAM
91
+ Narrator --> VRAM
92
+ Illustrator --> VRAM
93
+
94
+ Presentation_Layer -.-> WSL
95
+ Orchestration_Layer -.-> WSL
96
+ Inference_Layer -.-> WSL
97
+ WSL -.-> CPU
98
+ WSL -.-> VRAM
99
+
100
+ class AppClassic,AppSimple,Coordinator,Context,Exporter component;
101
+ class Evaluator,ASREngine,Narrator,Illustrator,TTSEngine model;
102
+ class WSL,VRAM,CPU infra;
103
+ ```
104
+
105
+ ### πŸ”’ Dual-Runtime Process Isolation (VRAM Management)
106
+
107
+ To run transcription, language evaluation, narrative generation, text-to-speech, and image diffusion simultaneously without exceeding the 6 GB VRAM ceiling, the system implements a **Hybrid Process Isolation Architecture**:
108
+
109
+ - **Main Process (CPU-Only)**: Hosts the Gradio web interface, the session manager/dialogue context (`StoryContext`), and the `MiniCPM-1B` cognitive evaluator running on CPU. This preserves all available VRAM for GPU rendering steps.
110
+ - **Isolated GPU Subprocesses**: Memory-heavy inference models (ASR, Story Generation with Tiny Aya, and FLUX.2) are executed in isolated Python subprocesses. Upon completing their respective inference step, the Python process exits and immediately releases 100% of the CUDA context and physical memory back to the OS.
111
+ - **Parallel Asset Generation**: To hide processing latency, audio narration synthesis (TTS on CPU) and illustration generation (FLUX on GPU) run concurrently in separate threads. This reduces per-page asset generation time from 18 seconds down to just 6-8 seconds.
112
+
113
+ ```mermaid
114
+ graph TD
115
+ classDef cpu fill:#e3f2fd,stroke:#1565c0,stroke-width:2px;
116
+ classDef gpu fill:#efebe9,stroke:#5d4037,stroke-width:2px;
117
+ classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;
118
+
119
+ User[ Child / User ] -->|Query (Voice/Text)| MainProcess[ Main Process: Gradio App ]
120
+
121
+ subgraph CPU Runtime [CPU Execution (Main Process)]
122
+ MainProcess -->|Cognitive Evaluation| MiniCPM[ MiniCPM-1B GGUF ]
123
+ MiniCPM -->|1. Detects Scientific Concept| Concept[ Concept & Language ]
124
+ MiniCPM -->|2. Assigns Semantic Companion| Companion[ Luna / Dino / Cosmo ]
125
+ end
126
+
127
+ subgraph GPU Subprocesses [GPU Execution (Isolated Subprocesses)]
128
+ MainProcess -->|Spawn ASR Subprocess| NeMo[ NVIDIA NeMo ASR ]
129
+ NeMo -->|Returns Transcribed Text| MainProcess
130
+ NeMo -.->|Releases 100% CUDA Context| GPU_Mem[ Recycled VRAM ]
131
+
132
+ MainProcess -->|Spawn Narrator Subprocess| TinyAya[ Tiny Aya Water 4-Bit ]
133
+ TinyAya -->|Generates Story & Visual Prompt| MainProcess
134
+ TinyAya -.->|Releases 100% CUDA Context| GPU_Mem
135
+
136
+ MainProcess -->|Spawn FLUX Subprocess| Flux[ FLUX.2 Klein 4B ]
137
+ Flux -->|Renders Illustration PNG| MainProcess
138
+ Flux -.->|Releases 100% CUDA Context| GPU_Mem
139
+ end
140
+
141
+ subgraph CPU/GPU Parallel Assets [Parallel Asset Generation]
142
+ ThreadA[ Thread 1: FLUX Subprocess ] -->|GPU Illustration| Visual[ page_0.png ]
143
+ ThreadB[ Thread 2: VoxCPM2 TTS ] -->|CPU Maternal Voice| Audio[ page_0.wav ]
144
+ end
145
+
146
+ MainProcess --> ThreadA
147
+ MainProcess --> ThreadB
148
+ Visual --> Exporter[ Session ZIP Exporter ]
149
+ Audio --> Exporter
150
+ Exporter -->|agent_trace.json + Assets| User
151
+
152
+ class MiniCPM,Concept,Companion cpu;
153
+ class NeMo,TinyAya,Flux gpu;
154
+ class MainProcess,ThreadA,ThreadB,Exporter process;
155
+ ```
156
+
157
+ ### πŸ”„ Inter-Process Data Flow
158
+
159
+ To ensure absolute separation of execution contexts while coordinating narrative generation, the system uses a file-based and command-line argument IPC (Inter-Process Communication) protocol:
160
+
161
+ 1. **ASR stage**: The main process records microphone input to a temporary `.wav` file, passing its path via CLI args to the ASR subprocess. NeMo writes the transcribed text to a temporary `.txt` file, which is then read by the main process.
162
+ 2. **Evaluation stage**: The main process runs MiniCPM on CPU to extract the scientific concept and detect the language, recording these metadata fields in the session state.
163
+ 3. **Narrative stage**: The main process spawns the Narrator subprocess passing the protagonist, topic, and kid's question. Tiny Aya (GPU) generates the story text and a watercolor visual prompt, writing them to a temporary `.txt` output file.
164
+ 4. **Asset Generation stage**: The main process spawns two concurrent workers: a FLUX subprocess (GPU) to render the illustration, and a VoxCPM2 TTS thread (CPU) to synthesize the speech audio. Both workers save their outputs directly as final session files.
165
+
166
+ ```mermaid
167
+ graph TD
168
+ %% Styling
169
+ classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;
170
+ classDef file fill:#efebe9,stroke:#5d4037,stroke-width:2px;
171
+ classDef data fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
172
+
173
+ %% Elements
174
+ User((Child / User))
175
+
176
+ subgraph MP [Main Process: Gradio App & Session Manager]
177
+ Gradio[Gradio UI]
178
+ Context[StoryContext / Session State]
179
+ Evaluator[Socratic Evaluator: MiniCPM CPU]
180
+ end
181
+
182
+ %% Input flow
183
+ User -->|1. Records Audio| Gradio
184
+ User -->|1. Types Query| Gradio
185
+
186
+ %% ASR subprocess data flow
187
+ subgraph ASR_Sub [ASR Subprocess]
188
+ NeMo[NVIDIA NeMo ASR]
189
+ end
190
+ Gradio -->|Writes wav| TempWav[Temp wav file]
191
+ TempWav -.->|CLI --audio| NeMo
192
+ NeMo -->|Writes text| TempASR[Temp ASR txt file]
193
+ TempASR -.->|Read text| Gradio
194
+
195
+ %% Socratic Evaluator data flow
196
+ Gradio -->|Transcribed / Typed Query| Evaluator
197
+ Evaluator -->|Cognitive Dict| Context
198
+
199
+ %% Narrator subprocess data flow
200
+ subgraph Narrator_Sub [Narrator Subprocess]
201
+ TinyAya[Tiny Aya LLM GPU]
202
+ end
203
+ Context -->|CLI arguments| TinyAya
204
+ TinyAya -->|Writes story & prompt| TempNarrative[Temp story txt file]
205
+ TempNarrative -.->|Read text| Gradio
206
+
207
+ %% Parallel Assets data flow
208
+ subgraph Assets_Gen [Parallel Asset Threads]
209
+ subgraph Flux_Sub [FLUX Subprocess]
210
+ Flux[FLUX.2 Image Gen GPU]
211
+ end
212
+ subgraph TTS_Thread [TTS Thread]
213
+ TTS[VoxCPM2 TTS CPU]
214
+ end
215
+ end
216
+
217
+ Gradio -->|Visual Prompt via CLI| Flux
218
+ Gradio -->|Narrative Text| TTS
219
+
220
+ Flux -->|Saves image| ImageFile[page_X.png]
221
+ TTS -->|Saves audio| AudioFile[page_X.wav]
222
+
223
+ %% Output flow
224
+ ImageFile --> Exporter[ZIP Exporter]
225
+ AudioFile --> Exporter
226
+ Context -->|agent_trace.json| Exporter
227
+
228
+ Exporter -->|Downloadable ZIP| Gradio
229
+ ImageFile -->|Render Illustration| Gradio
230
+ AudioFile -->|Play Voice Narration| Gradio
231
+ Gradio -->|Output screen| User
232
+
233
+ class Gradio,Context,Evaluator,NeMo,TinyAya,Flux,TTS,Exporter process;
234
+ class TempWav,TempASR,TempNarrative,ImageFile,AudioFile file;
235
+ class User data;
236
+ ```
237
+
238
+ ---
239
+
240
+ ## πŸ“Š Model Specification
241
+
242
+ All models used are kept under the **4B parameter limit** to guarantee local execution and portability:
243
+
244
+ | Component | Hugging Face Model | Parameters | Disk Size | Target Runtime | Format / Quantization | Purpose & Task |
245
+ | :--- | :--- | :---: | :---: | :---: | :---: | :--- |
246
+ | **ASR** | `nvidia/nemotron-3.5-asr-streaming-0.6b` | 600M | ~2.4 GB | GPU (Subprocess) | Float16 / NeMo | Audio-to-text transcription |
247
+ | **Evaluator** | `openbmb/MiniCPM5-1B-GGUF` | 1B | 688 MB | CPU (Process) | GGUF (Q4_K_M) | Safety, scientific concept extraction, and companion assignment |
248
+ | **Narrator** | `CohereLabs/tiny-aya-water` | 1.5B | ~3.0 GB | GPU (Subprocess) | 4-bit (BitsAndBytes) | Children's narrative generation & Socratic prompts |
249
+ | **Illustrator** | `black-forest-labs/FLUX.2-klein-4B` | 4B | ~4.2 GB | GPU (Subprocess) | BFloat16 / Diffusers | Watercolor-style digital illustration (4 steps) |
250
+ | **TTS** | `openbmb/VoxCPM2` | 2.5B | ~1.8 GB | CPU/GPU (Subprocess) | Float32 / VoxCPM | Voice synthesis with a warm, maternal tone descriptor |
251
+
252
+ ---
253
+
254
+ ## βš™οΈ Key Technical Features
255
+
256
+ - **Standalone Support (Simple Mode)**: By passing `page_index == 0`, `socratic_engine.py` leverages specialized system prompts to condense the story into 1-2 explanatory paragraphs without compromising pedagogical accuracy, removing the need for a multi-turn dialogue loop.
257
+ - **Auto-Detecting Port Binding**: The Gradio interface in `app_simple.py` performs network socket checks to detect if the standard port `7860` is already taken by the 5-page interactive app. If it is occupied, it automatically binds to port `7861`, enabling simultaneous local testing without manual configurations.
258
+ - **Zero-Config Deployment on Hugging Face Spaces**: A startup validator checks if model weights exist locally in the `models/` directory. If they are missing (standard behavior during Hugging Face Spaces build stages, where weights are ignored via `.gitignore`), the server internally triggers `download_models.py` using the `HF_TOKEN` environment secret.
259
+
260
+ ---
261
+
262
+ ## πŸ› οΈ Installation & Setup
263
+
264
+ ### 1. Environment Prerequisites
265
+ Python 3.10 is required, running inside WSL2 (Ubuntu) or a native Linux environment:
266
+ ```bash
267
+ pip install -r requirements.txt
268
+ ```
269
+
270
+ ### 2. Gated Models
271
+ You must accept the terms of service on Hugging Face to access the following models:
272
+ * [CohereLabs/tiny-aya-water](https://huggingface.co/CohereLabs/tiny-aya-water)
273
+ * [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
274
+
275
+ ### 3. Downloading Weights
276
+ Set your Hugging Face access token in your terminal and run the downloader script:
277
+ ```bash
278
+ export HF_TOKEN="your_huggingface_token"
279
+ python download_models.py
280
+ ```
281
+
282
+ ---
283
+
284
+ ## πŸš€ Running the Application
285
+
286
+ ### 1. Run the Quick Response Edition (Simple Mode - Recommended)
287
+ To start the streamlined, single-page interface on port `7860` (or `7861` as fallback):
288
+ ```bash
289
+ python app_simple.py
290
+ ```
291
+
292
+ ### 2. Run the Classic Socratic Edition (5 Interactive Steps)
293
+ To start the full-length interactive journey on port `7860`:
294
+ ```bash
295
+ python app.py
296
+ ```
297
+
298
+ ### πŸ§ͺ Simulation Mode
299
+ Both versions include a **"Simulation Mode"** checkbox in their control panels. This allows you to simulate turns and asset generation instantly on CPU using mocks. It is ideal for debugging UI styles, logical flow, and ZIP exports without loading the heavy model weights into VRAM.