docs/architecture.md

LifeOS Architecture

LifeOS is a local-first personal assistant built for the Hugging Face Build Small hackathon (Track 1 — Backyard AI). One small model, one shared memory, three life domains, zero cloud calls.

Stack: NVIDIA Nemotron-3-Nano-4B (Q4_K_M GGUF, 2.84 GB) + nomic-embed-text (146 MB GGUF), both running through the llama.cpp runtime (llama-cpp-python), behind Gradio 6 Server mode with a hand-built HTML/CSS/JS frontend. Runs on the HF Spaces free CPU tier.

1. System overview

flowchart LR
    subgraph Browser ["Browser — static/ (hand-built, zero external requests)"]
        UI["index.html + style.css + app.js<br/>4 panels: Kitchen · Health · Money · Chat"]
    end

    subgraph Server ["app.py — gradio.Server (FastAPI + Gradio API engine)"]
        API["@app.api endpoints<br/>SSE streaming, queue, concurrency=1"]
        UP["FastAPI routes<br/>/upload/flyer · /upload/flyer_text · /upload/transactions"]
    end

    subgraph Features ["features/ — deterministic logic (no LLM)"]
        FOOD["food.py<br/>deal extraction · recipe scoring"]
        HEALTH["health.py<br/>weekly pattern aggregation"]
        MONEY["money.py<br/>recurring-charge detection"]
    end

    subgraph Engine ["engine.py — reasoning"]
        PROMPT["build_prompt(domain, memory, input)"]
        GEN["generate_stream — token streaming"]
    end

    subgraph Models ["llama.cpp runtime (100% local)"]
        LLM["Nemotron-3-Nano-4B<br/>Q4_K_M · 2.84 GB"]
        EMB["nomic-embed-text-v1.5<br/>Q8_0 · 146 MB"]
    end

    subgraph Memory ["memory"]
        STM["memory.py — SHORT-TERM<br/>data/memory.json (structured)"]
        LTM["rag.py — LONG-TERM<br/>data/longterm.json (notes + vectors)"]
    end

    UI -->|"fetch + SSE"| API
    UI -->|multipart| UP
    UP --> FOOD & MONEY
    API --> FOOD & HEALTH & MONEY
    API --> PROMPT --> GEN --> LLM
    PROMPT --> STM
    PROMPT -->|"recall(query, k=5)"| LTM
    LTM -->|embeddings| EMB
    API -->|"log / detect events"| STM & LTM

Design principle — deterministic first, model last. Parsing, scoring, and detection are plain Python; the 4B model only does the judgment/explanation layer on a small curated context. That division is what makes a 4B model on 2 vCPUs feel smart: short prompts, grounded outputs, no hallucinated data.

2. Request flow per feature

sequenceDiagram
    participant B as Browser
    participant S as gr.Server
    participant F as features/*
    participant M as memory (STM+LTM)
    participant E as engine.py
    participant L as llama.cpp

    Note over B,L: 🍳 Kitchen — flyer → 3 recipe picks
    B->>S: POST /upload/flyer (PDF/image)
    S->>F: food.extract_deals (pdfplumber / pytesseract)
    F-->>B: deals chips [{item, price}]
    B->>S: call/food_recommend (SSE)
    S->>F: shortlist(deals, recent meals, prefs) → top 6
    S->>M: meals last 7d + RAG recall
    S->>E: build_prompt("food") → generate_stream
    E->>L: create_chat_completion(stream=True)
    L-->>B: tokens stream into result card

    Note over B,L: 💰 Money — CSV → cancel list
    B->>S: POST /upload/transactions (CSV)
    S->>F: parse → normalize merchants → detect_recurring (25–35d cadence, ±15% amount)
    F->>M: set_subscriptions(detected)
    F-->>B: subscription table
    B->>S: call/money_review (SSE)
    S->>E: income + budget + subs → stream CANCEL/KEEP/WATCH verdicts

Health follows the same shape (log → deterministic weekly_pattern → model recommends tomorrow's session). Chat skips the feature layer: it gets all memory slices plus conversation history.

3. Memory system — short-term + long-term

flowchart TB
    subgraph Writes
        W1["log meal / workout (UI)"]
        W2["money: detected subscriptions"]
        W3["chat: 'remember ...' statements"]
    end

    subgraph STM ["SHORT-TERM — memory.py (structured JSON)"]
        S1["meals[] · workouts[] · finances{} · user_profile{}"]
    end

    subgraph LTM ["LONG-TERM — rag.py (local RAG)"]
        L1["notes: {text, kind: fact|event|preference, vec}"]
        L2["nomic-embed via llama.cpp (embedding=True)"]
        L3["numpy cosine top-k (no vector DB)"]
    end

    subgraph Read ["Prompt assembly (every request)"]
        R1["domain slice of STM<br/>(food → meals, health → workouts, money → finances, chat → all)"]
        R2["top-5 recalled LTM notes for the query"]
        R3["[system: persona + domain task] + [user: memory + notes + request]"]
    end

    W1 --> S1
    W2 --> S1
    W1 -->|auto-summarized note| L1
    W3 -->|explicit fact| L1
    L1 --- L2 --- L3
    S1 --> R1 --> R3
    L3 --> R2 --> R3

• Short-term (data/memory.json): exact structured state — what you ate, trained, pay for. Regenerated from a seed with relative dates so the demo never goes stale.
• Long-term (data/longterm.json): durable facts, preferences, and events ("knee needs rest days between runs", "FitnessPal unused since March"). Embedded once at write time; retrieved by cosine similarity at prompt time. At demo scale (10²–10³ notes) brute-force numpy beats any vector DB — and it keeps the dependency count tiny.

4. Why these models

	Model	Size	Role
Reasoning	nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF Q4_K_M	2.84 GB	All recommendations + chat
Embeddings	nomic-ai/nomic-embed-text-v1.5-GGUF Q8_0	146 MB	Long-term memory recall

Nemotron-3-Nano is a hybrid Mamba-2 architecture (only 4 attention layers): near-constant memory per token and strong CPU throughput, which is exactly what the Spaces free tier (2 vCPU / 16 GB) needs. Reasoning traces are disabled via system prompt to keep latency down; max_tokens≈512.

5. Hackathon badge map

Badge	How
📴 Off the Grid	No cloud APIs anywhere: local GGUFs, raw-fetch frontend, system fonts, no CDN
🦙 Llama Champion	Both models run through the llama.cpp runtime
🐜 Tiny Titan	3.97B params reasoning model
🎨 Off-Brand	gr.Server + 100% hand-built frontend, no Gradio components