README.md

metadata

title: LifeOS
emoji: ⚡
colorFrom: green
colorTo: gray
sdk: gradio
sdk_version: 6.17.3
python_version: '3.13'
app_file: app.py
suggested_hardware: cpu-upgrade
pinned: false
license: other
models:
  - nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF
  - nomic-ai/nomic-embed-text-v1.5-GGUF
tags:
  - build-small-hackathon
  - local-first
  - llama-cpp
short_description: Local-first personal assistant on Nemotron-3-Nano-4B

⚡ LifeOS — your week, handled by a 4B model that never phones home

Build Small Hackathon · Track 1: Backyard AI

I'm Awais, a student. My grocery budget, my training plan, and my subscriptions all leak in different apps — and I'm not pasting my bank statements into a cloud AI. LifeOS is the fix I actually use: one small model, one shared memory, three life domains, zero cloud calls.

What it does

Tab	You give it	It gives back
🍳 Kitchen	This week's grocery flyer (PDF/image/text), or a photo of a meal/receipt (local OCR)	3 recipe picks priced off real deals, tuned to what you already cooked this week; meal-photo analysis of what you've been buying + what to buy next
💪 Health	Your workout log, preferred training days/time, and your week's calendar (work, classes)	Tomorrow's session reasoned from muscle rotation + rest balance, free workout slots that dodge your calendar, and a browser reminder for the next one
💰 Money	A bank CSV + your monthly payments	Deterministic recurring-charge detection with CANCEL / KEEP / WATCH verdicts, plus a Socratic goal coach that questions you into a realistic savings plan
🧠 Chat	Anything — typed or spoken (mic in, voice replies out)	Cross-domain answers: "Plan my week under $80, high protein, run 3×" pulls from all three
👤 Profile	Name, city, income, diet, budget	Personalizes every prompt; optional one-click local-flyer deal search for your city

Two features are optional and online — the Chat "web search" toggle and Profile "find local flyer deals" (both clearly labeled, off by default). Everything else, including voice (browser Web Speech API) and OCR, runs fully offline — the Off the Grid claim holds unless you flip those toggles.

Everything feeds two memory tiers: a structured short-term store (meals, workouts, finances) and a local RAG long-term memory — durable facts like "knee needs a rest day between runs" are embedded and recalled into every prompt. Say "remember …" in chat and it sticks.

How it's built small

• One reasoning model: NVIDIA Nemotron-3-Nano-4B (Q4_K_M GGUF, 2.84 GB) — a hybrid Mamba-2 architecture that runs on CPU.
• One embedding model: nomic-embed-text-v1.5 (Q8_0, 146 MB) for long-term memory recall.
• One runtime: both run through llama.cpp (llama-cpp-python).
• Deterministic first, model last: OCR/PDF parsing, recipe scoring, recurring-charge detection are plain Python. The 4B model only does the judgment + explanation layer on a small curated context — that's what makes a tiny model feel smart.
• Hand-built frontend on Gradio 6 Server mode (gr.Server): the browser talks to Gradio's SSE API with raw fetch — no CDN, no external fonts, no Gradio components. View source; it's three hand-written files.

flyer/CSV ──► deterministic extraction ──► curated context ─┐
                                                            ├─► Nemotron-3-Nano-4B (llama.cpp) ─► streamed answer
memory.json (short-term) + RAG recall (long-term) ──────────┘

Full architecture with diagrams: docs/architecture.md

Badges claimed

• 📴 Off the Grid — zero cloud APIs at runtime. Models are downloaded once from the Hub at startup; after that you can pull the network cable. The demo video does exactly that.
• 🦙 Llama Champion — all inference (chat and embeddings) through the llama.cpp runtime.
• 🐜 Tiny Titan — 3.97 B parameters.
• 🎨 Off-Brand — gr.Server + 100% hand-built HTML/CSS/JS.

Run it locally

No API keys, no .env, no cloud account needed. LifeOS is 100% local — it downloads the two GGUF models from the public HF Hub on first launch (~3 GB, one time) and runs entirely on your machine after that.

1. Create and activate a virtual environment

Use Python 3.13 (matches the Space).

Windows (PowerShell):

py -3.13 -m venv .venv
.\.venv\Scripts\Activate.ps1

If activation is blocked, run once: Set-ExecutionPolicy -Scope CurrentUser RemoteSigned

If py -3.13 isn't found, install Python 3.13 from python.org first.

macOS / Linux (bash):

python3.13 -m venv .venv
source .venv/bin/activate

2. Install dependencies

pip install --upgrade pip
pip install -r requirements.txt

requirements.txt points pip at the llama-cpp-python prebuilt CPU wheel index — on Windows/macOS with Python 3.13 this installs a py3-none wheel, no compiler needed. (The HF Space build doesn't use this index and instead builds llama-cpp-python from source — packages.txt installs build-essential

• cmake for that, so the Space build works too, just slower.)

OCR (image flyers & meal photos): the primary backend is EasyOCR (pip install -r requirements.txt includes it). It runs on the GPU when a CUDA-enabled PyTorch is installed and falls back to CPU otherwise. Set LIFEOS_OCR_GPU=0 to force CPU. PDF and pasted-text flyers need no OCR. Tesseract stays as an automatic fallback if EasyOCR isn't available — Windows: UB-Mannheim build (auto-detected from the default install dir, no PATH edit needed; or set TESSERACT_CMD); macOS: brew install tesseract; Debian/Ubuntu: sudo apt install tesseract-ocr.

GPU acceleration (NVIDIA/CUDA): the default CPU wheel of llama-cpp-python ignores the GPU. The prebuilt CUDA wheels from the abetlen index are compiled with AVX-512 and crash (0xC000001D illegal instruction) on CPUs that lack it — e.g. most Intel 10th-gen and earlier. The reliable route is to build from source, which adapts to your CPU (AVX2) and GPU. Prereqs: NVIDIA driver + CUDA Toolkit (nvcc), and on Windows the MSVC C++ build tools (Microsoft.VisualStudio.2022.BuildTools with the VCTools workload). Then:

CMAKE_ARGS="-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=75" \
  pip install llama-cpp-python==0.3.28 \
  --no-binary llama-cpp-python --force-reinstall --no-cache-dir

(CMAKE_CUDA_ARCHITECTURES=75 = Turing/RTX 20-series & Quadro RTX; use your card's compute capability.) On Windows, scripts/build_cuda.bat wraps this with the right vcvars/Ninja env and a short-temp workaround for the 260-char path limit. engine.py then offloads all layers by default (LIFEOS_GPU_LAYERS=-1) and auto-falls back to CPU if the GPU load fails; set LIFEOS_GPU_LAYERS=0 to force CPU, or a positive number for a partial offload on a small-VRAM card. For OCR on GPU, install a CUDA build of PyTorch (see pytorch.org) before easyocr.

3. Run with real local inference

python app.py

First launch downloads Nemotron-3-Nano-4B (Q4_K_M, 2.84 GB) and nomic-embed-text (146 MB) from the Hub, then serves the app at http://localhost:7860. The UI is reachable immediately; the models warm up in the background, and a food photo additionally pulls Qwen2.5-VL-3B (~3.8 GB incl. projector) the first time it's used.

A real install starts blank — fill in your profile, meals, workouts and finances through the UI. To explore with a populated sample persona instead, set LIFEOS_DEMO=1 (a week of meals, workouts, and subscriptions; sample flyer and bank CSV live in data/samples/).

Configuration lives in config.py; copy .env.example to .env to override the demo flag, GPU layers, model ids, host/port, etc.

Performance: engine.py uses os.cpu_count() threads automatically. Modal benchmarks on the same llama.cpp build show ~0.7 tok/s on 2 vCPU vs ~2.3 tok/s on 8 vCPU — more cores = snappier streaming.

4. Run the tests

The feature tests are pure Python (no model needed). The integration test hits a live server — start one in another shell first.

python tests/test_food.py
python tests/test_health.py
python tests/test_money.py
python tests/test_extensions.py    # fakes the model/embedder — no download
python tests/test_integration.py   # needs app.py already running (see below)

The unit tests never load the real models: test_extensions.py injects a fake LLM and a deterministic fake embedder, so they run fast and offline.

---

Dev-only: Modal verification (optional, not needed to run the app)

modal_check.py and scripts/modal_bench.py verify the GGUF loads in llama-cpp-python on Linux and benchmark tok/s — this is how the model + hardware choice was validated, and it's never imported by the app (preserving Off the Grid). Modal authenticates with a browser flow, not an API key in .env:

pip install modal
modal setup            # opens a browser to link your Modal account (one time)
modal run modal_check.py
modal run scripts/modal_bench.py

Built small, on purpose.