README.md

---
title: LifeOS
emoji: ⚡
colorFrom: green
colorTo: gray
sdk: gradio
sdk_version: 6.17.3
python_version: '3.13'
app_file: app.py
suggested_hardware: cpu-upgrade
pinned: false
license: other
models:
  - nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF
  - nomic-ai/nomic-embed-text-v1.5-GGUF
tags:
  - build-small-hackathon
  - local-first
  - llama-cpp
short_description: Local-first personal assistant on Nemotron-3-Nano-4B
---

# ⚡ LifeOS — your week, handled by a 4B model that never phones home

**Build Small Hackathon · Track 1: Backyard AI**

I'm Awais, a student. My grocery budget, my training plan, and my
subscriptions all leak in different apps — and I'm not pasting my bank
statements into a cloud AI. LifeOS is the fix I actually use: **one small
model, one shared memory, three life domains, zero cloud calls.**

## What it does

| Tab | You give it | It gives back |
|---|---|---|
| 🍳 **Kitchen** | This week's grocery flyer (PDF/image/text), or a photo of a meal/receipt (local OCR) | 3 recipe picks priced off real deals, tuned to what you *already* cooked this week; meal-photo analysis of what you've been buying + what to buy next |
| 💪 **Health** | Your workout log, preferred training days/time, and your week's calendar (work, classes) | Tomorrow's session reasoned from muscle rotation + rest balance, free workout slots that dodge your calendar, and a browser reminder for the next one |
| 💰 **Money** | A bank CSV + your monthly payments | Deterministic recurring-charge detection with CANCEL / KEEP / WATCH verdicts, plus a **Socratic goal coach** that questions you into a realistic savings plan |
| 🧠 **Chat** | Anything — typed or **spoken** (mic in, voice replies out) | Cross-domain answers: "Plan my week under $80, high protein, run 3×" pulls from all three |
| 👤 **Profile** | Name, city, income, diet, budget | Personalizes every prompt; optional one-click local-flyer deal search for your city |

> **Two features are optional and online** — the Chat "web search" toggle and
> Profile "find local flyer deals" (both clearly labeled, **off by default**).
> Everything else, including voice (browser Web Speech API) and OCR, runs
> fully offline — the Off the Grid claim holds unless you flip those toggles.

Everything feeds **two memory tiers**: a structured short-term store
(meals, workouts, finances) and a **local RAG long-term memory** — durable
facts like "knee needs a rest day between runs" are embedded and recalled
into every prompt. Say "remember …" in chat and it sticks.

## How it's built small

- **One reasoning model:** [NVIDIA Nemotron-3-Nano-4B](https://hf.co/nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF)
  (Q4_K_M GGUF, 2.84 GB) — a hybrid Mamba-2 architecture that runs on CPU.
- **One embedding model:** [nomic-embed-text-v1.5](https://hf.co/nomic-ai/nomic-embed-text-v1.5-GGUF)
  (Q8_0, 146 MB) for long-term memory recall.
- **One runtime:** both run through **llama.cpp** (`llama-cpp-python`).
- **Deterministic first, model last:** OCR/PDF parsing, recipe scoring,
  recurring-charge detection are plain Python. The 4B model only does the
  judgment + explanation layer on a small curated context — that's what
  makes a tiny model feel smart.
- **Hand-built frontend** on Gradio 6 **Server mode** (`gr.Server`): the
  browser talks to Gradio's SSE API with raw `fetch` — no CDN, no external
  fonts, no Gradio components. View source; it's three hand-written files.

```
flyer/CSV ──► deterministic extraction ──► curated context ─┐
                                                            ├─► Nemotron-3-Nano-4B (llama.cpp) ─► streamed answer
memory.json (short-term) + RAG recall (long-term) ──────────┘
```

Full architecture with diagrams: [docs/architecture.md](docs/architecture.md)

## Badges claimed

- 📴 **Off the Grid** — zero cloud APIs at runtime. Models are downloaded
  once from the Hub at startup; after that you can pull the network cable.
  The demo video does exactly that.
- 🦙 **Llama Champion** — all inference (chat *and* embeddings) through the
  llama.cpp runtime.
- 🐜 **Tiny Titan** — 3.97 B parameters.
- 🎨 **Off-Brand** — `gr.Server` + 100% hand-built HTML/CSS/JS.

## Run it locally

> **No API keys, no `.env`, no cloud account needed.** LifeOS is 100% local —
> it downloads the two GGUF models from the public HF Hub on first launch
> (~3 GB, one time) and runs entirely on your machine after that.

### 1. Create and activate a virtual environment

Use **Python 3.13** (matches the Space).

**Windows (PowerShell):**
```powershell
py -3.13 -m venv .venv
.\.venv\Scripts\Activate.ps1
```
> If activation is blocked, run once:
> `Set-ExecutionPolicy -Scope CurrentUser RemoteSigned`
>
> If `py -3.13` isn't found, install Python 3.13 from
> [python.org](https://www.python.org/downloads/) first.

**macOS / Linux (bash):**
```bash
python3.13 -m venv .venv
source .venv/bin/activate
```

### 2. Install dependencies

```bash
pip install --upgrade pip
pip install -r requirements.txt
```
> `requirements.txt` points pip at the [llama-cpp-python prebuilt CPU wheel
> index](https://abetlen.github.io/llama-cpp-python/whl/cpu) — on Windows/macOS
> with Python 3.13 this installs a `py3-none` wheel, **no compiler needed**.
> (The HF Space build doesn't use this index and instead builds
> `llama-cpp-python` from source — `packages.txt` installs `build-essential`
> + `cmake` for that, so the Space build works too, just slower.)

> **OCR (image flyers & meal photos):** the primary backend is **EasyOCR**
> (`pip install -r requirements.txt` includes it). It runs on the GPU when a
> CUDA-enabled PyTorch is installed and falls back to CPU otherwise. Set
> `LIFEOS_OCR_GPU=0` to force CPU. PDF and pasted-text flyers need no OCR.
> **Tesseract** stays as an automatic fallback if EasyOCR isn't available —
> Windows: [UB-Mannheim build](https://github.com/UB-Mannheim/tesseract/wiki)
> (auto-detected from the default install dir, no PATH edit needed; or set
> `TESSERACT_CMD`); macOS: `brew install tesseract`; Debian/Ubuntu:
> `sudo apt install tesseract-ocr`.

> **GPU acceleration (NVIDIA/CUDA):** the default CPU wheel of
> `llama-cpp-python` ignores the GPU. The prebuilt CUDA wheels from the abetlen
> index are compiled with **AVX-512** and crash (`0xC000001D` illegal
> instruction) on CPUs that lack it — e.g. most Intel 10th-gen and earlier. The
> reliable route is to **build from source**, which adapts to your CPU (AVX2)
> and GPU. Prereqs: NVIDIA driver + CUDA Toolkit (`nvcc`), and on Windows the
> MSVC C++ build tools (`Microsoft.VisualStudio.2022.BuildTools` with the VCTools
> workload). Then:
> ```bash
> CMAKE_ARGS="-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=75" \
>   pip install llama-cpp-python==0.3.28 \
>   --no-binary llama-cpp-python --force-reinstall --no-cache-dir
> ```
> (`CMAKE_CUDA_ARCHITECTURES=75` = Turing/RTX 20-series & Quadro RTX; use your
> card's compute capability.) On Windows, `scripts/build_cuda.bat` wraps this
> with the right vcvars/Ninja env and a short-temp workaround for the 260-char
> path limit. `engine.py` then offloads all layers by default
> (`LIFEOS_GPU_LAYERS=-1`) and **auto-falls back to CPU** if the GPU load fails;
> set `LIFEOS_GPU_LAYERS=0` to force CPU, or a positive number for a partial
> offload on a small-VRAM card. For OCR on GPU, install a CUDA build of PyTorch
> (see [pytorch.org](https://pytorch.org/get-started/locally/)) before `easyocr`.

### 3. Run with real local inference

```bash
python app.py
```
First launch downloads Nemotron-3-Nano-4B (Q4_K_M, 2.84 GB) and
nomic-embed-text (146 MB) from the Hub, then serves the app at
**http://localhost:7860**. The UI is reachable immediately; the models
warm up in the background, and a food photo additionally pulls
Qwen2.5-VL-3B (~3.8 GB incl. projector) the first time it's used.

A real install **starts blank** — fill in your profile, meals, workouts and
finances through the UI. To explore with a populated sample persona instead,
set `LIFEOS_DEMO=1` (a week of meals, workouts, and subscriptions; sample
flyer and bank CSV live in `data/samples/`).

Configuration lives in `config.py`; copy `.env.example` to `.env` to override
the demo flag, GPU layers, model ids, host/port, etc.

Performance: `engine.py` uses `os.cpu_count()` threads automatically.
Modal benchmarks on the same llama.cpp build show **~0.7 tok/s on 2 vCPU
vs ~2.3 tok/s on 8 vCPU** — more cores = snappier streaming.

### 4. Run the tests

The feature tests are pure Python (no model needed). The integration test
hits a live server — start one in another shell first.

```bash
python tests/test_food.py
python tests/test_health.py
python tests/test_money.py
python tests/test_extensions.py    # fakes the model/embedder — no download
python tests/test_integration.py   # needs app.py already running (see below)
```

> The unit tests never load the real models: `test_extensions.py` injects a
> fake LLM and a deterministic fake embedder, so they run fast and offline.

---

### Dev-only: Modal verification (optional, not needed to run the app)

`modal_check.py` and `scripts/modal_bench.py` verify the GGUF loads in
llama-cpp-python on Linux and benchmark tok/s — this is how the model +
hardware choice was validated, and it's **never imported by the app**
(preserving Off the Grid). Modal authenticates with a browser flow, **not
an API key in `.env`**:

```bash
pip install modal
modal setup            # opens a browser to link your Modal account (one time)
modal run modal_check.py
modal run scripts/modal_bench.py
```

*Built small, on purpose.*