README.md

---
title: Curiosity StoryBook (Socratic Edition)
emoji: 📖
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
app_port: 7860
tags:
- build-small-hackathon
- track:backyard-ai
- badge:tiny-titan
- badge:best-agent
- badge:off-brand
- badge:best-demo
- quest:off-the-grid
- quest:llama-champion
- quest:sharing-is-care
- quest:field-notes
---

# 📖 Curiosity StoryBook (Socratic Edition)

**Curiosity StoryBook** is a locally-run, educational story generator designed for children. It features a Socratic cognitive engine alongside an illustration and audio-narration pipeline optimized to run on consumer local hardware under a strict **6 GB VRAM budget**.

The project offers two independent execution flows:
1. **Classic Socratic Edition (`app.py`)**: A 5-page interactive journey where the AI does not provide direct answers. Instead, it uses Socratic dialogue (guided questions and clues) to help the child reason and solve the scientific mystery on their own.
2. **Quick Response Edition (Simple Mode - `app_simple.py`)**: A streamlined, single-page interface. The child asks their curiosity question ("Why...?"), and the system instantly generates a self-contained explanatory short story, a watercolor-style illustration, and its voice narration—resolving the mystery in a single step to minimize latency.

---

## 🏗️ System Architecture

### 📐 General Layered Architecture

The project is structured into a modular layered architecture, separating the presentation web interface, session/trace orchestration, AI models inference, and hardware resource boundaries:

```mermaid
graph TD
    classDef layer fill:#f5f5f5,stroke:#9e9e9e,stroke-width:2px,stroke-dasharray: 5 5;
    classDef component fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
    classDef model fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px;
    classDef infra fill:#eceff1,stroke:#607d8b,stroke-width:2px;

    subgraph Presentation_Layer [Presentation Layer / User Interfaces]
        AppClassic[Classic Socratic App - app.py / Port 7860]
        AppSimple[Simple Mode App - app_simple.py / Port 7861]
    end

    subgraph Orchestration_Layer [Orchestration & State Management]
        Coordinator[Parallel Asset Coordinator]
        Context[StoryContext / agent_trace.json]
        Exporter[ZIP Session Exporter]
    end

    subgraph Inference_Layer [Inference & Model Engines]
        Evaluator[Socratic Evaluator - MiniCPM 1B CPU]
        ASREngine[ASR Pipeline - NeMo 600M GPU]
        Narrator[Narrative Engine - Tiny Aya 1.5B 4-bit GPU]
        Illustrator[FLUX.2 Diffusion - FLUX.2 Klein 4B GPU]
        TTSEngine[TTS Voice Engine - VoxCPM2 2.5B CPU]
    end

    subgraph Infrastructure_Layer [Infrastructure & Hardware]
        WSL[WSL2 Ubuntu Linux / Python 3.10 Conda]
        VRAM[Shared CUDA GPU Memory / Strict 6GB VRAM Limit]
        CPU[System CPU & RAM / VoxCPM2 + MiniCPM GGUF]
    end

    %% Connections
    AppClassic --> Context
    AppSimple --> Context
    
    Context --> Evaluator
    Context --> ASREngine
    Context --> Narrator
    
    Context --> Coordinator
    Coordinator --> Illustrator
    Coordinator --> TTSEngine
    
    Illustrator --> Exporter
    TTSEngine --> Exporter
    Context --> Exporter

    %% Infrastructure bindings
    Evaluator --> CPU
    TTSEngine --> CPU
    
    ASREngine --> VRAM
    Narrator --> VRAM
    Illustrator --> VRAM

    Presentation_Layer -.-> WSL
    Orchestration_Layer -.-> WSL
    Inference_Layer -.-> WSL
    WSL -.-> CPU
    WSL -.-> VRAM

    class AppClassic,AppSimple,Coordinator,Context,Exporter component;
    class Evaluator,ASREngine,Narrator,Illustrator,TTSEngine model;
    class WSL,VRAM,CPU infra;
```

### 🔒 Dual-Runtime Process Isolation (VRAM Management)

To run transcription, language evaluation, narrative generation, text-to-speech, and image diffusion simultaneously without exceeding the 6 GB VRAM ceiling, the system implements a **Hybrid Process Isolation Architecture**:

- **Main Process (CPU-Only)**: Hosts the Gradio web interface, the session manager/dialogue context (`StoryContext`), and the `MiniCPM-1B` cognitive evaluator running on CPU. This preserves all available VRAM for GPU rendering steps.
- **Isolated GPU Subprocesses**: Memory-heavy inference models (ASR, Story Generation with Tiny Aya, and FLUX.2) are executed in isolated Python subprocesses. Upon completing their respective inference step, the Python process exits and immediately releases 100% of the CUDA context and physical memory back to the OS.
- **Parallel Asset Generation**: To hide processing latency, audio narration synthesis (TTS on CPU) and illustration generation (FLUX on GPU) run concurrently in separate threads. This reduces per-page asset generation time from 18 seconds down to just 6-8 seconds.

```mermaid
graph TD
    classDef cpu fill:#e3f2fd,stroke:#1565c0,stroke-width:2px;
    classDef gpu fill:#efebe9,stroke:#5d4037,stroke-width:2px;
    classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;

    User[ Child / User ] -->|Query (Voice/Text)| MainProcess[ Main Process: Gradio App ]
    
    subgraph CPU Runtime [CPU Execution (Main Process)]
        MainProcess -->|Cognitive Evaluation| MiniCPM[ MiniCPM-1B GGUF ]
        MiniCPM -->|1. Detects Scientific Concept| Concept[ Concept & Language ]
        MiniCPM -->|2. Assigns Semantic Companion| Companion[ Luna / Dino / Cosmo ]
    end
    
    subgraph GPU Subprocesses [GPU Execution (Isolated Subprocesses)]
        MainProcess -->|Spawn ASR Subprocess| NeMo[ NVIDIA NeMo ASR ]
        NeMo -->|Returns Transcribed Text| MainProcess
        NeMo -.->|Releases 100% CUDA Context| GPU_Mem[ Recycled VRAM ]
        
        MainProcess -->|Spawn Narrator Subprocess| TinyAya[ Tiny Aya Water 4-Bit ]
        TinyAya -->|Generates Story & Visual Prompt| MainProcess
        TinyAya -.->|Releases 100% CUDA Context| GPU_Mem
        
        MainProcess -->|Spawn FLUX Subprocess| Flux[ FLUX.2 Klein 4B ]
        Flux -->|Renders Illustration PNG| MainProcess
        Flux -.->|Releases 100% CUDA Context| GPU_Mem
    end

    subgraph CPU/GPU Parallel Assets [Parallel Asset Generation]
        ThreadA[ Thread 1: FLUX Subprocess ] -->|GPU Illustration| Visual[ page_0.png ]
        ThreadB[ Thread 2: VoxCPM2 TTS ] -->|CPU Maternal Voice| Audio[ page_0.wav ]
    end
    
    MainProcess --> ThreadA
    MainProcess --> ThreadB
    Visual --> Exporter[ Session ZIP Exporter ]
    Audio --> Exporter
    Exporter -->|agent_trace.json + Assets| User

    class MiniCPM,Concept,Companion cpu;
    class NeMo,TinyAya,Flux gpu;
    class MainProcess,ThreadA,ThreadB,Exporter process;
```

### 🔄 Inter-Process Data Flow

To ensure absolute separation of execution contexts while coordinating narrative generation, the system uses a file-based and command-line argument IPC (Inter-Process Communication) protocol:

1. **ASR stage**: The main process records microphone input to a temporary `.wav` file, passing its path via CLI args to the ASR subprocess. NeMo writes the transcribed text to a temporary `.txt` file, which is then read by the main process.
2. **Evaluation stage**: The main process runs MiniCPM on CPU to extract the scientific concept and detect the language, recording these metadata fields in the session state.
3. **Narrative stage**: The main process spawns the Narrator subprocess passing the protagonist, topic, and kid's question. Tiny Aya (GPU) generates the story text and a watercolor visual prompt, writing them to a temporary `.txt` output file.
4. **Asset Generation stage**: The main process spawns two concurrent workers: a FLUX subprocess (GPU) to render the illustration, and a VoxCPM2 TTS thread (CPU) to synthesize the speech audio. Both workers save their outputs directly as final session files.

```mermaid
graph TD
    %% Styling
    classDef process fill:#fff3e0,stroke:#ef6c00,stroke-width:2px;
    classDef file fill:#efebe9,stroke:#5d4037,stroke-width:2px;
    classDef data fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px;
    
    %% Elements
    User((Child / User))
    
    subgraph MP [Main Process: Gradio App & Session Manager]
        Gradio[Gradio UI]
        Context[StoryContext / Session State]
        Evaluator[Socratic Evaluator: MiniCPM CPU]
    end
    
    %% Input flow
    User -->|1. Records Audio| Gradio
    User -->|1. Types Query| Gradio
    
    %% ASR subprocess data flow
    subgraph ASR_Sub [ASR Subprocess]
        NeMo[NVIDIA NeMo ASR]
    end
    Gradio -->|Writes wav| TempWav[Temp wav file]
    TempWav -.->|CLI --audio| NeMo
    NeMo -->|Writes text| TempASR[Temp ASR txt file]
    TempASR -.->|Read text| Gradio
    
    %% Socratic Evaluator data flow
    Gradio -->|Transcribed / Typed Query| Evaluator
    Evaluator -->|Cognitive Dict| Context
    
    %% Narrator subprocess data flow
    subgraph Narrator_Sub [Narrator Subprocess]
        TinyAya[Tiny Aya LLM GPU]
    end
    Context -->|CLI arguments| TinyAya
    TinyAya -->|Writes story & prompt| TempNarrative[Temp story txt file]
    TempNarrative -.->|Read text| Gradio
    
    %% Parallel Assets data flow
    subgraph Assets_Gen [Parallel Asset Threads]
        subgraph Flux_Sub [FLUX Subprocess]
            Flux[FLUX.2 Image Gen GPU]
        end
        subgraph TTS_Thread [TTS Thread]
            TTS[VoxCPM2 TTS CPU]
        end
    end
    
    Gradio -->|Visual Prompt via CLI| Flux
    Gradio -->|Narrative Text| TTS
    
    Flux -->|Saves image| ImageFile[page_X.png]
    TTS -->|Saves audio| AudioFile[page_X.wav]
    
    %% Output flow
    ImageFile --> Exporter[ZIP Exporter]
    AudioFile --> Exporter
    Context -->|agent_trace.json| Exporter
    
    Exporter -->|Downloadable ZIP| Gradio
    ImageFile -->|Render Illustration| Gradio
    AudioFile -->|Play Voice Narration| Gradio
    Gradio -->|Output screen| User

    class Gradio,Context,Evaluator,NeMo,TinyAya,Flux,TTS,Exporter process;
    class TempWav,TempASR,TempNarrative,ImageFile,AudioFile file;
    class User data;
```

---

## 📊 Model Specification

All models used are kept under the **4B parameter limit** to guarantee local execution and portability:

| Component | Hugging Face Model | Parameters | Disk Size | Target Runtime | Format / Quantization | Purpose & Task |
| :--- | :--- | :---: | :---: | :---: | :---: | :--- |
| **ASR** | `nvidia/nemotron-3.5-asr-streaming-0.6b` | 600M | ~2.4 GB | GPU (Subprocess) | Float16 / NeMo | Audio-to-text transcription |
| **Evaluator** | `openbmb/MiniCPM5-1B-GGUF` | 1B | 688 MB | CPU (Process) | GGUF (Q4_K_M) | Safety, scientific concept extraction, and companion assignment |
| **Narrator** | `CohereLabs/tiny-aya-water` | 1.5B | ~3.0 GB | GPU (Subprocess) | 4-bit (BitsAndBytes) | Children's narrative generation & Socratic prompts |
| **Illustrator** | `black-forest-labs/FLUX.2-klein-4B` | 4B | ~4.2 GB | GPU (Subprocess) | BFloat16 / Diffusers | Watercolor-style digital illustration (4 steps) |
| **TTS** | `openbmb/VoxCPM2` | 2.5B | ~1.8 GB | CPU/GPU (Subprocess) | Float32 / VoxCPM | Voice synthesis with a warm, maternal tone descriptor |

---

## ⚙️ Key Technical Features

- **Standalone Support (Simple Mode)**: By passing `page_index == 0`, `socratic_engine.py` leverages specialized system prompts to condense the story into 1-2 explanatory paragraphs without compromising pedagogical accuracy, removing the need for a multi-turn dialogue loop.
- **Auto-Detecting Port Binding**: The Gradio interface in `app_simple.py` performs network socket checks to detect if the standard port `7860` is already taken by the 5-page interactive app. If it is occupied, it automatically binds to port `7861`, enabling simultaneous local testing without manual configurations.
- **Zero-Config Deployment on Hugging Face Spaces**: A startup validator checks if model weights exist locally in the `models/` directory. If they are missing (standard behavior during Hugging Face Spaces build stages, where weights are ignored via `.gitignore`), the server internally triggers `download_models.py` using the `HF_TOKEN` environment secret.

---

## 🛠️ Installation & Setup

### 1. Environment Prerequisites
Python 3.10 is required, running inside WSL2 (Ubuntu) or a native Linux environment:
```bash
pip install -r requirements.txt
```

### 2. Gated Models
You must accept the terms of service on Hugging Face to access the following models:
*   [CohereLabs/tiny-aya-water](https://huggingface.co/CohereLabs/tiny-aya-water)
*   [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)

### 3. Downloading Weights
Set your Hugging Face access token in your terminal and run the downloader script:
```bash
export HF_TOKEN="your_huggingface_token"
python download_models.py
```

---

## 🚀 Running the Application

### 1. Run the Quick Response Edition (Simple Mode - Recommended)
To start the streamlined, single-page interface on port `7860` (or `7861` as fallback):
```bash
python app_simple.py
```

### 2. Run the Classic Socratic Edition (5 Interactive Steps)
To start the full-length interactive journey on port `7860`:
```bash
python app.py
```

### 🧪 Simulation Mode
Both versions include a **"Simulation Mode"** checkbox in their control panels. This allows you to simulate turns and asset generation instantly on CPU using mocks. It is ideal for debugging UI styles, logical flow, and ZIP exports without loading the heavy model weights into VRAM.