pipeline_flow.md

# Agent Trace: Memory Keeper Pipeline Flow

This document outlines the end-to-end data flow and architectural pipeline for the Memory Keeper application.

## 📡 Pipeline Architecture

The system operates strictly "Off the Grid" using entirely open-weight models hosted via serverless functions. **No proprietary cloud LLM APIs (like OpenAI, Anthropic, or Gemini) are used.**

### Step 1: User Input (Hugging Face Spaces)
1. The user interacts with the completely custom `gradio.Server` interface running in a Docker container on Hugging Face Spaces.
2. The user uploads a photo, records an audio voice note, or types a text memory.
3. The FastAPI backend running on HF Spaces receives the multipart form data.

### Step 2: Parallel Perception (Modal Serverless)
The backend securely offloads the heavy perception tasks to transient Modal endpoints powered by A10G GPUs.
- **Audio Pipeline:** The audio file is sent to the `transcribe_audio` Modal endpoint. A serverless worker spins up, runs `openai/whisper-base` entirely locally, and returns the transcribed text.
- **Visual Pipeline:** The photo is sent to the `describe_photo` Modal endpoint. A worker runs `Salesforce/blip-image-captioning-base` to extract rich visual context and returns the semantic description.

### Step 3: Synthesis & Generation (Modal Serverless)
The extracted text transcripts, visual descriptions, and the user's historical profile data are gathered by the HF Spaces backend and sent to the core orchestrator.
- **LLM Pipeline:** The `build_memory_book` Modal endpoint spins up with `Qwen/Qwen2.5-7B-Instruct`.
- The LLM processes the raw contexts and generates structured JSON containing a chronological "Timeline", a narrative "Story", and a personalized "Letter".

### Step 4: Storage & Delivery
1. The HF Space backend receives the structured JSON from Qwen2.5.
2. The data is saved locally to the container's ephemeral storage (managed by an asynchronous 48-hour cleanup task).
3. The generated "Memory Book" is rendered beautifully in the custom HTML/CSS UI for the user to view and download.