--- title: Curiosity StoryBook (Socratic Edition) emoji: ๐Ÿ“– colorFrom: blue colorTo: green sdk: docker pinned: false app_port: 7860 tags: - build-small-hackathon - track:backyard-ai - badge:tiny-titan - badge:best-agent - badge:off-brand - badge:best-demo - quest:off-the-grid - quest:llama-champion - quest:sharing-is-care - quest:field-notes --- # ๐Ÿ“– Curiosity StoryBook (Socratic Edition) **Curiosity StoryBook** is a locally-run, educational story generator designed for children. It features a Socratic cognitive engine alongside an illustration and audio-narration pipeline optimized to run on consumer local hardware under a strict **6 GB VRAM budget**. The project offers two independent execution flows: 1. **Classic Socratic Edition (`app.py`)**: A 5-page interactive journey where the AI does not provide direct answers. Instead, it uses Socratic dialogue (guided questions and clues) to help the child reason and solve the scientific mystery on their own. 2. **Quick Response Edition (Simple Mode - `app_simple.py`)**: A streamlined, single-page interface. The child asks their curiosity question ("Why...?"), and the system instantly generates a self-contained explanatory short story, a watercolor-style illustration, and its voice narrationโ€”resolving the mystery in a single step to minimize latency. --- ## ๐Ÿ—๏ธ System Architecture ### ๐Ÿ“ General Layered Architecture The project is structured into a modular layered architecture, separating the presentation web interface, session/trace orchestration, AI models inference, and hardware resource boundaries: ```mermaid graph TD classDef layer fill:#f5f5f5,stroke:#9e9e9e,stroke-width:2px,stroke-dasharray: 5 5; classDef component fill:#e1f5fe,stroke:#0288d1,stroke-width:2px; classDef model fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px; classDef infra fill:#eceff1,stroke:#607d8b,stroke-width:2px; subgraph Presentation_Layer [Presentation Layer / User Interfaces] AppClassic[Classic Socratic App - app.py / Port 7860] AppSimple[Simple Mode App - app_simple.py / Port 7861] end subgraph Orchestration_Layer [Orchestration & State Management] Coordinator[Parallel Asset Coordinator] Context[StoryContext / agent_trace.json] Exporter[ZIP Session Exporter] end subgraph Inference_Layer [Inference & Model Engines] Evaluator[Socratic Evaluator - MiniCPM 1B CPU] ASREngine[ASR Pipeline - NeMo 600M GPU] Narrator[Narrative Engine - Tiny Aya 1.5B 4-bit GPU] Illustrator[FLUX.2 Diffusion - FLUX.2 Klein 4B GPU] TTSEngine[TTS Voice Engine - VoxCPM2 2.5B CPU] end subgraph Infrastructure_Layer [Infrastructure & Hardware] WSL[WSL2 Ubuntu Linux / Python 3.10 Conda] VRAM[Shared CUDA GPU Memory / Strict 6GB VRAM Limit] CPU[System CPU & RAM / VoxCPM2 + MiniCPM GGUF] end %% Connections AppClassic --> Context AppSimple --> Context Context --> Evaluator Context --> ASREngine Context --> Narrator Context --> Coordinator Coordinator --> Illustrator Coordinator --> TTSEngine Illustrator --> Exporter TTSEngine --> Exporter Context --> Exporter %% Infrastructure bindings Evaluator --> CPU TTSEngine --> CPU ASREngine --> VRAM Narrator --> VRAM Illustrator --> VRAM Presentation_Layer -.-> WSL Orchestration_Layer -.-> WSL Inference_Layer -.-> WSL WSL -.-> CPU WSL -.-> VRAM class AppClassic,AppSimple,Coordinator,Context,Exporter component; class Evaluator,ASREngine,Narrator,Illustrator,TTSEngine model; class WSL,VRAM,CPU infra; ``` ### ๐Ÿ”’ Dual-Runtime Process Isolation (VRAM Management) To run transcription, language evaluation, narrative generation, text-to-speech, and image diffusion simultaneously without exceeding the 6 GB VRAM ceiling, the system implements a **Hybrid Process Isolation Architecture**: - **Main Process (CPU-Only)**: Hosts the Gradio web interface, the session manager/dialogue context (`StoryContext`), and the `MiniCPM-1B` cognitive evaluator running on CPU. This preserves all available VRAM for GPU rendering steps. - **Isolated GPU Subprocesses**: Memory-heavy inference models (ASR, Story Generation with Tiny Aya, and FLUX.2) are executed in isolated Python subprocesses. Upon completing their respective inference step, the Python process exits and immediately releases 100% of the CUDA context and physical memory back to the OS. - **Parallel Asset Generation**: To hide processing latency, audio narration synthesis (TTS on CPU) and illustration generation (FLUX on GPU) run concurrently in separate threads. This reduces per-page asset generation time from 18 seconds down to just 6-8 seconds. ```mermaid graph TD classDef cpu fill:#e3f2fd,stroke:#1565c0,stroke-width:2px; classDef gpu fill:#efebe9,stroke:#5d4037,stroke-width:2px; classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px; User[ Child / User ] -->|Query (Voice/Text)| MainProcess[ Main Process: Gradio App ] subgraph CPU Runtime [CPU Execution (Main Process)] MainProcess -->|Cognitive Evaluation| MiniCPM[ MiniCPM-1B GGUF ] MiniCPM -->|1. Detects Scientific Concept| Concept[ Concept & Language ] MiniCPM -->|2. Assigns Semantic Companion| Companion[ Luna / Dino / Cosmo ] end subgraph GPU Subprocesses [GPU Execution (Isolated Subprocesses)] MainProcess -->|Spawn ASR Subprocess| NeMo[ NVIDIA NeMo ASR ] NeMo -->|Returns Transcribed Text| MainProcess NeMo -.->|Releases 100% CUDA Context| GPU_Mem[ Recycled VRAM ] MainProcess -->|Spawn Narrator Subprocess| TinyAya[ Tiny Aya Water 4-Bit ] TinyAya -->|Generates Story & Visual Prompt| MainProcess TinyAya -.->|Releases 100% CUDA Context| GPU_Mem MainProcess -->|Spawn FLUX Subprocess| Flux[ FLUX.2 Klein 4B ] Flux -->|Renders Illustration PNG| MainProcess Flux -.->|Releases 100% CUDA Context| GPU_Mem end subgraph CPU/GPU Parallel Assets [Parallel Asset Generation] ThreadA[ Thread 1: FLUX Subprocess ] -->|GPU Illustration| Visual[ page_0.png ] ThreadB[ Thread 2: VoxCPM2 TTS ] -->|CPU Maternal Voice| Audio[ page_0.wav ] end MainProcess --> ThreadA MainProcess --> ThreadB Visual --> Exporter[ Session ZIP Exporter ] Audio --> Exporter Exporter -->|agent_trace.json + Assets| User class MiniCPM,Concept,Companion cpu; class NeMo,TinyAya,Flux gpu; class MainProcess,ThreadA,ThreadB,Exporter process; ``` ### ๐Ÿ”„ Inter-Process Data Flow To ensure absolute separation of execution contexts while coordinating narrative generation, the system uses a file-based and command-line argument IPC (Inter-Process Communication) protocol: 1. **ASR stage**: The main process records microphone input to a temporary `.wav` file, passing its path via CLI args to the ASR subprocess. NeMo writes the transcribed text to a temporary `.txt` file, which is then read by the main process. 2. **Evaluation stage**: The main process runs MiniCPM on CPU to extract the scientific concept and detect the language, recording these metadata fields in the session state. 3. **Narrative stage**: The main process spawns the Narrator subprocess passing the protagonist, topic, and kid's question. Tiny Aya (GPU) generates the story text and a watercolor visual prompt, writing them to a temporary `.txt` output file. 4. **Asset Generation stage**: The main process spawns two concurrent workers: a FLUX subprocess (GPU) to render the illustration, and a VoxCPM2 TTS thread (CPU) to synthesize the speech audio. Both workers save their outputs directly as final session files. ```mermaid graph TD %% Styling classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px; classDef file fill:#efebe9,stroke:#5d4037,stroke-width:2px; classDef data fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px; %% Elements User((Child / User)) subgraph MP [Main Process: Gradio App & Session Manager] Gradio[Gradio UI] Context[StoryContext / Session State] Evaluator[Socratic Evaluator: MiniCPM CPU] end %% Input flow User -->|1. Records Audio| Gradio User -->|1. Types Query| Gradio %% ASR subprocess data flow subgraph ASR_Sub [ASR Subprocess] NeMo[NVIDIA NeMo ASR] end Gradio -->|Writes wav| TempWav[Temp wav file] TempWav -.->|CLI --audio| NeMo NeMo -->|Writes text| TempASR[Temp ASR txt file] TempASR -.->|Read text| Gradio %% Socratic Evaluator data flow Gradio -->|Transcribed / Typed Query| Evaluator Evaluator -->|Cognitive Dict| Context %% Narrator subprocess data flow subgraph Narrator_Sub [Narrator Subprocess] TinyAya[Tiny Aya LLM GPU] end Context -->|CLI arguments| TinyAya TinyAya -->|Writes story & prompt| TempNarrative[Temp story txt file] TempNarrative -.->|Read text| Gradio %% Parallel Assets data flow subgraph Assets_Gen [Parallel Asset Threads] subgraph Flux_Sub [FLUX Subprocess] Flux[FLUX.2 Image Gen GPU] end subgraph TTS_Thread [TTS Thread] TTS[VoxCPM2 TTS CPU] end end Gradio -->|Visual Prompt via CLI| Flux Gradio -->|Narrative Text| TTS Flux -->|Saves image| ImageFile[page_X.png] TTS -->|Saves audio| AudioFile[page_X.wav] %% Output flow ImageFile --> Exporter[ZIP Exporter] AudioFile --> Exporter Context -->|agent_trace.json| Exporter Exporter -->|Downloadable ZIP| Gradio ImageFile -->|Render Illustration| Gradio AudioFile -->|Play Voice Narration| Gradio Gradio -->|Output screen| User class Gradio,Context,Evaluator,NeMo,TinyAya,Flux,TTS,Exporter process; class TempWav,TempASR,TempNarrative,ImageFile,AudioFile file; class User data; ``` --- ## ๐Ÿ“Š Model Specification All models used are kept under the **4B parameter limit** to guarantee local execution and portability: | Component | Hugging Face Model | Parameters | Disk Size | Target Runtime | Format / Quantization | Purpose & Task | | :--- | :--- | :---: | :---: | :---: | :---: | :--- | | **ASR** | `nvidia/nemotron-3.5-asr-streaming-0.6b` | 600M | ~2.4 GB | GPU (Subprocess) | Float16 / NeMo | Audio-to-text transcription | | **Evaluator** | `openbmb/MiniCPM5-1B-GGUF` | 1B | 688 MB | CPU (Process) | GGUF (Q4_K_M) | Safety, scientific concept extraction, and companion assignment | | **Narrator** | `CohereLabs/tiny-aya-water` | 1.5B | ~3.0 GB | GPU (Subprocess) | 4-bit (BitsAndBytes) | Children's narrative generation & Socratic prompts | | **Illustrator** | `black-forest-labs/FLUX.2-klein-4B` | 4B | ~4.2 GB | GPU (Subprocess) | BFloat16 / Diffusers | Watercolor-style digital illustration (4 steps) | | **TTS** | `openbmb/VoxCPM2` | 2.5B | ~1.8 GB | CPU/GPU (Subprocess) | Float32 / VoxCPM | Voice synthesis with a warm, maternal tone descriptor | --- ## โš™๏ธ Key Technical Features - **Standalone Support (Simple Mode)**: By passing `page_index == 0`, `socratic_engine.py` leverages specialized system prompts to condense the story into 1-2 explanatory paragraphs without compromising pedagogical accuracy, removing the need for a multi-turn dialogue loop. - **Auto-Detecting Port Binding**: The Gradio interface in `app_simple.py` performs network socket checks to detect if the standard port `7860` is already taken by the 5-page interactive app. If it is occupied, it automatically binds to port `7861`, enabling simultaneous local testing without manual configurations. - **Zero-Config Deployment on Hugging Face Spaces**: A startup validator checks if model weights exist locally in the `models/` directory. If they are missing (standard behavior during Hugging Face Spaces build stages, where weights are ignored via `.gitignore`), the server internally triggers `download_models.py` using the `HF_TOKEN` environment secret. --- ## ๐Ÿ› ๏ธ Installation & Setup ### 1. Environment Prerequisites Python 3.10 is required, running inside WSL2 (Ubuntu) or a native Linux environment: ```bash pip install -r requirements.txt ``` ### 2. Gated Models You must accept the terms of service on Hugging Face to access the following models: * [CohereLabs/tiny-aya-water](https://huggingface.co/CohereLabs/tiny-aya-water) * [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) ### 3. Downloading Weights Set your Hugging Face access token in your terminal and run the downloader script: ```bash export HF_TOKEN="your_huggingface_token" python download_models.py ``` --- ## ๐Ÿš€ Running the Application ### 1. Run the Quick Response Edition (Simple Mode - Recommended) To start the streamlined, single-page interface on port `7860` (or `7861` as fallback): ```bash python app_simple.py ``` ### 2. Run the Classic Socratic Edition (5 Interactive Steps) To start the full-length interactive journey on port `7860`: ```bash python app.py ``` ### ๐Ÿงช Simulation Mode Both versions include a **"Simulation Mode"** checkbox in their control panels. This allows you to simulate turns and asset generation instantly on CPU using mocks. It is ideal for debugging UI styles, logical flow, and ZIP exports without loading the heavy model weights into VRAM.