GPU speedup

.env.example

@@ -3,12 +3,12 @@ API_BASE_URL=http://localhost:8000                  # Vite proxy target (api)
 VITE_API_BASE_URL=                                  # empty = use Vite proxy (recommended for `npm run dev` + Docker)
 VITE_PROXY_TARGET=http://api:8000                   # used inside Docker (frontend -> api), ignored on host
 VITE_API_KEY=replace-with-a-long-random-secret      # must match PRECIS_API_KEY (baked into frontend)
-OLLAMA_BASE_URL=http://127.0.0.1:11434              # Ollama's default — compose overrides to http://ollama:11434
 
-# host Ollama folder bind-mounted to container (reuse models)
-# This is specifically for Windows btw
-# Replace with your own Ollama repository address
-OLLAMA_HOST_DIR=C:/Users/your_username/.ollama             
+# Ollama — HOST native (Option A, GPU fast). No Docker Ollama anymore.
+# - For local `uvicorn` / `npm run dev` without Docker: http://127.0.0.1:11434 (host Ollama default)
+# - Inside Docker, `docker-compose.yml` HARD-CODES `http://host.docker.internal:11434` so this value is IGNORED there.
+OLLAMA_BASE_URL=http://127.0.0.1:11434              # Ollama host — keep 127.0.0.1 for local dev
+
 PRECIS_ALLOWED_ORIGINS=http://localhost:5173,http://localhost:8000,http://localhost:7860        # Just front end + api, keep consistent
 
 # Model addresses
@@ -19,8 +19,9 @@ DEFAULT_MODEL=phi4-mini:latest
 AVAILABLE_MODELS=phi4-mini:latest,qwen3:4b           # Only here so both front and backend have it
 
 # API stuff
-PRECIS_API_KEY=replace-with-a-long-random-secret    # Once the API is actually up, this'll be needed
+PRECIS_API_KEY=replace-with-a-long-random-secret    # Once the API is actually up, this'll be needed — `openssl rand -hex 32`
 MAX_SUMMARY_TOKENS=120
 TEMPERATURE=0.2                                     # Keep low for stability
 PRECIS_MAX_UPLOAD_BYTES=10485760
 PRECIS_MAX_TRANSCRIPT_CHARS=250000 
+OLLAMA_KEEP_ALIVE=30m                               # How long Ollama keeps model in VRAM after use

Dockerfile

@@ -61,7 +61,7 @@ COPY --from=frontend-builder /app/frontend/dist ./frontend/dist
 EXPOSE 8000 7860
 
 ENV PORT=8000 \
-    OLLAMA_BASE_URL=http://ollama:11434 \
+    OLLAMA_BASE_URL=http://host.docker.internal:11434 \
     DEFAULT_MODEL=phi4-mini:latest \
     AVAILABLE_MODELS=phi4-mini:latest \
     PRECIS_ALLOWED_ORIGINS=http://localhost:5173,http://localhost:8000,http://localhost:7860,https://*.hf.space,https://*.huggingface.co

README.md

@@ -50,16 +50,23 @@ All `/summarize/*` endpoints accept an optional `model` field to override the de
 
 ---
 
-## Quick Start (Docker)
+## Quick Start (Docker) — Host Ollama, GPU Fast
 
-**One command** to build and run the whole stack (API + frontend + Ollama). This is the main way to run Précis — no manual `ollama serve`/`uvicorn`/`vite` needed.
+**Host Ollama (Option A) is now the ONLY supported Docker setup.** No Ollama container, no 2.5GB re-download, no 600% CPU. Docker only runs `api` (FastAPI + baked frontend) + `frontend` (Vite HMR). Ollama runs natively on the host for full GPU speed.
 
 > **Python runs inside the `precis` conda environment** — same as your local setup. The `Dockerfile` uses `continuumio/miniconda3` + `environment.yml` to create `conda env precis` (Python 3.11) and installs all deps there; the container's `PATH` points to `/opt/conda/envs/precis/bin`, so `uvicorn` runs from `precis`.
 
 ### 1. Prerequisites
 
 - **Docker** + **Docker Compose** ([Install Docker Desktop](https://docs.docker.com/get-docker/))
-- That's it. Python (via `precis` conda env), Node, and Ollama are all inside the containers. No local conda/Python/Node needed.
+- **Ollama (host, native)** — [Install Ollama](https://ollama.com/download) + at least one model:
+  ```bash
+  ollama pull phi4-mini:latest
+  # optional extras:
+  ollama pull qwen3:4b
+  ollama list
+  ```
+- That's it. Python (via `precis` conda env) and Node are inside the containers. No local conda/Python/Node needed for Docker path.
 
 ### 2. Configure
 
@@ -71,39 +78,46 @@ cp .env.example .env
 # VITE_API_KEY must match PRECIS_API_KEY so the baked frontend can auth
 ```
 
-> **`.env.example` defaults for Docker:**
+> **`.env.example` defaults for Docker (host Ollama):**
 > ```ini
-> OLLAMA_BASE_URL=http://127.0.0.1:11434        # compose overrides to http://ollama:11434
+> OLLAMA_BASE_URL=http://127.0.0.1:11434        # for local `uvicorn` without Docker; Docker compose hard-codes http://host.docker.internal:11434 (host GPU)
 > DEFAULT_MODEL=phi4-mini:latest
-> AVAILABLE_MODELS=phi4-mini:latest
+> AVAILABLE_MODELS=phi4-mini:latest,qwen3:4b
 > PRECIS_API_KEY=replace-with-a-long-random-secret
 > VITE_API_KEY=replace-with-a-long-random-secret
 > PRECIS_ALLOWED_ORIGINS=http://localhost:5173,http://localhost:8000
 > VITE_API_BASE_URL=                            # empty = same origin (correct for Docker)
+> # NOTE: OLLAMA_HOST_DIR removed — no Docker Ollama container, no bind mount. Host Ollama uses its native store.
 > ```
 
-Manually update `.env.example` to match the block above if your copy is older.
-
 ### 3. Run
 
+**Terminal 1 — keep host Ollama running:**
 ```bash
-docker compose up --watch
+# If Ollama was started with default 127.0.0.1 binding and api gets 503, restart host Ollama as:
+# OLLAMA_HOST=0.0.0.0 ollama serve
+ollama serve
 ```
 
-- First build takes ~4-6 min (multi-stage: Node build + `conda env create -f environment.yml` for `precis` — cached after first build).
-- Opens: **http://localhost:5173** (Vite HMR) + API at **http://localhost:8000** (`/docs`, `/health`). With `--watch`, edits to `backend/` and `frontend/` hot-reload automatically.
-- Compose healthchecks wait for Ollama (`/api/tags`) before starting the API (which runs as `conda run -n precis uvicorn`).
+**Terminal 2 — from repo root (use -d so terminal isn't blocked):**
+```bash
+docker compose up --watch -d
+# or without -d if you want logs in this terminal (needs 2nd terminal for other commands)
+```
 
-#### Pull a model (inside Ollama container)
+- First build takes ~4-6 min (multi-stage: Node build + `conda env create -f environment.yml` for `precis` — cached after first build). Subsequent `up --watch -d` takes ~15s (no rebuild).
+- Opens: **http://localhost:5173** (Vite HMR, host Ollama GPU) + API at **http://localhost:8000** (`/docs`, `/health`). With `--watch`, edits to `backend/` and `frontend/` hot-reload automatically.
+- `GET /models` now proxies to **host** Ollama at `http://host.docker.internal:11434` — no `ollama-init` sidecar, no manifest pull. If host has no models, API 503s with `run ollama pull ...`.
 
-The API will 503 until a model is installed. In a **second terminal**:
+#### Pull a model (on HOST, not in Docker)
+
+The API will 503 until a model is installed. On **host**:
 
 ```bash
-docker exec -it precis-ollama ollama pull phi4-mini:latest
+ollama pull phi4-mini:latest
 # optional extras:
-docker exec -it precis-ollama ollama pull llama3.1:latest
-docker exec -it precis-ollama ollama pull qwen3:4b
-docker exec -it precis-ollama ollama list
+ollama pull qwen3:4b
+ollama list
 ```
 
 Then refresh http://localhost:5173 — the model selector populates automatically (`GET /models` proxied to `:8000`).
@@ -112,19 +126,18 @@ Then refresh http://localhost:5173 — the model selector populates automaticall
 
 ```bash
 docker compose up --watch -d   # detached with watch
-docker compose logs -f api     # tail API logs
-docker compose logs -f ollama
+docker compose logs -f api     # tail API logs (no ollama container anymore)
 docker compose logs -f frontend
-docker compose down            # stop
-docker compose down -v         # stop + wipe Ollama model cache (only if using named volume)
+docker compose down            # stop api+frontend
 docker compose ps
 curl http://localhost:8000/health
 curl http://localhost:8000/status
+curl http://host.docker.internal:11434/api/tags # verify host Ollama reachable from Docker
 # prod without watch/HMR (static frontend at 8000):
-docker compose -f docker-compose.yml up --build
+docker compose -f docker-compose.yml up --build -d
 ```
 
-#### Single-image run (without compose / without Ollama sidecar)
+#### Single-image run (without compose)
 
 If you just want the API image and already run Ollama natively (`ollama serve` on host):
 
@@ -132,17 +145,18 @@ If you just want the API image and already run Ollama natively (`ollama serve` o
 docker build -t precis .
 docker run --rm -p 8000:8000 --env-file .env \
   -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
+  --add-host=host.docker.internal:host-gateway \
   precis
-# Windows/Mac: host.docker.internal resolves to host. Linux: add --add-host=host.docker.internal:host-gateway
+# Windows/Mac: host.docker.internal resolves to host. Linux already adds host-gateway above.
 ```
 
 #### Hugging Face Spaces
 
-`sdk: docker` with `EXPOSE 8000 7860` + `ENV PORT=8000` + `CMD sh -c "uvicorn ... --port ${PORT:-8000}"` means the same image works everywhere: locally `http://localhost:8000` (compose sets `PORT=8000`), on HF Spaces Hugging Face sets `PORT=7860` automatically and the container listens on `7860` with no extra config.
+`sdk: docker` with `EXPOSE 8000 7860` + `ENV PORT=8000` + `CMD sh -c "uvicorn ... --port ${PORT:-8000}"` means the same image works everywhere: locally `http://localhost:8000` (compose sets `PORT=8000`), on HF Spaces Hugging Face sets `PORT=7860` automatically and the container listens on `7860` with no extra config. HF Spaces will need its own Ollama or external `OLLAMA_BASE_URL`.
 
 ---
 
-## Manual Development
+## Manual Development (without Docker)
 
 Use this if you want hot-reload for the frontend/backend separately. Docker remains the reproducible path.
 
@@ -151,14 +165,14 @@ Use this if you want hot-reload for the frontend/backend separately. Docker rema
 - **Conda** (Miniconda/Anaconda/Mamba) with env **`precis`** (`conda env create -f environment.yml` / `conda activate precis`) — Python 3.11, same env the Docker image uses
   - Alternative without conda: **Python** 3.11+ + `pip install -r requirements.txt`
 - **Node.js** 18+
-- **Ollama** (`ollama serve` to run) + at least one model: `ollama pull phi4-mini:latest`
+- **Ollama (host, native)** (`ollama serve` to run) + at least one model: `ollama pull phi4-mini:latest`
 
 ### 1. Environment
 
 ```bash
 cp .env.example .env
 # set PRECIS_API_KEY, VITE_API_KEY (same value), and for local:
-# OLLAMA_BASE_URL=http://127.0.0.1:11434
+# OLLAMA_BASE_URL=http://127.0.0.1:11434  (host native)
 # VITE_API_BASE_URL=http://localhost:8000   # or leave empty and rely on Vite proxy
 ```
 

backend/tests/test_api_fetching.py

@@ -11,7 +11,7 @@ from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
 sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "backend"))
 
-os.environ.setdefault("OLLAMA_BASE_URL", "http://ollama:11434")
+os.environ.setdefault("OLLAMA_BASE_URL", "http://127.0.0.1:11434")
 os.environ.setdefault("DEFAULT_MODEL", "phi4-mini:latest")
 os.environ.setdefault("PRECIS_API_KEY", "test-key")
 os.environ.setdefault("AVAILABLE_MODELS", "phi4-mini:latest,qwen:4b")

docker-compose.yml

@@ -1,24 +1,4 @@
 services:
-  ollama:
-    image: ollama/ollama:latest
-    container_name: precis-ollama
-    ports:
-      - "11434:11434"
-    volumes:
-      # Single copy — bind mount host Windows folder if OLLAMA_HOST_DIR is set in .env
-      # (e.g. OLLAMA_HOST_DIR=C:/Users/varak/.ollama). If unset/empty, falls back to
-      # named volume 'ollama_data' (isolated, but faster I/O than Windows bind).
-      # Short syntax with fallback: "${VAR:-default}:/target" — quoted for Windows paths.
-      - "${OLLAMA_HOST_DIR:-ollama_data}:/root/.ollama"
-    restart: unless-stopped
-    healthcheck:
-      test: ["CMD-SHELL", "ollama list >/dev/null 2>&1 || wget -qO- http://localhost:11434/api/tags >/dev/null 2>&1 || curl -fsS http://localhost:11434/api/tags >/dev/null 2>&1"]
-      interval: 15s
-      timeout: 5s
-      retries: 8
-      start_period: 20s
-      start_interval: 3s
-
   api:
     build:
       context: .
@@ -35,20 +15,19 @@ services:
     env_file:
       - .env
     environment:
-      # CRITICAL FIX for 503 "Cannot reach Ollama": inside Docker, Ollama is at http://ollama:11434,
-      # NOT 127.0.0.1 (which points to the api container itself). The .env file's
-      # OLLAMA_BASE_URL=http://127.0.0.1:11434 is for host-only dev. Hardcode here so
-      # compose .env interpolation can't override it with the host value.
+      # Host Ollama (Option A) — blazing fast GPU, no 2.5GB re-download.
+      # api inside Docker talks to host via host.docker.internal:11434.
+      # Host must have `ollama serve` running (see README). On Windows Docker Desktop this resolves automatically.
+      # If host Ollama was started with default 127.0.0.1 binding, set OLLAMA_HOST=0.0.0.0 on host before `ollama serve`.
       PORT: ${PORT:-8000}
-      OLLAMA_BASE_URL: http://ollama:11434
+      OLLAMA_BASE_URL: http://host.docker.internal:11434
       DEFAULT_MODEL: ${DEFAULT_MODEL:-phi4-mini:latest}
       AVAILABLE_MODELS: ${AVAILABLE_MODELS:-phi4-mini:latest}
       PRECIS_ALLOWED_ORIGINS: ${PRECIS_ALLOWED_ORIGINS:-http://localhost:5173,http://localhost:8000,http://localhost:7860,https://*.hf.space,https://*.huggingface.co}
       PRECIS_API_KEY: ${PRECIS_API_KEY:?PRECIS_API_KEY must be set in .env}
       OLLAMA_KEEP_ALIVE: ${OLLAMA_KEEP_ALIVE:-30m}
-    depends_on:
-      ollama:
-        condition: service_healthy
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
     volumes:
       # Mount .env read-only so config.py hot-reload isn't needed — container restarts pick up changes
       - type: bind
@@ -63,87 +42,4 @@ services:
       retries: 5
       start_period: 20s
 
-  ollama-init:
-    image: ollama/ollama:latest
-    container_name: precis-ollama-init
-    volumes:
-      - "${OLLAMA_HOST_DIR:-ollama_data}:/root/.ollama"
-    env_file:
-      - .env
-    environment:
-      OLLAMA_HOST: http://ollama:11434
-    depends_on:
-      ollama:
-        condition: service_healthy
-    entrypoint: sh
-    command:
-      - -c
-      - |
-        set -e
-        echo "ollama-init: checking local models (host bind: ${OLLAMA_HOST_DIR:-<named volume ollama_data>})"
-        echo "AVAILABLE_MODELS=$$AVAILABLE_MODELS  DEFAULT_MODEL=$$DEFAULT_MODEL"
-        echo "OLLAMA_HOST=$$OLLAMA_HOST  OLLAMA_HOST_DIR fallback: ${OLLAMA_HOST_DIR:-ollama_data}"
-        # Wait for Ollama API to actually be ready (healthcheck may pass earlier)
-        for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do
-          if curl -fsS http://ollama:11434/api/tags >/dev/null 2>&1; then
-            echo "ollama API ready after $$i tries"
-            break
-          fi
-          echo "waiting for ollama API... $$i"
-          sleep 2
-        done
-        echo "existing models (ollama list):"
-        ollama list || true
-        echo "existing models (API /api/tags):"
-        curl -fsS http://ollama:11434/api/tags 2>/dev/null || echo "API tags fetch failed"
-        # ── IDEMPOTENT PULL: only pull if not already present ──
-        # Fixes "pulling 2.5GB raw" when host bind already has models.
-        # We re-query inside the loop (not cached) because Ollama may still be
-        # importing the bind-mounted blobs on first start.
-        for m in $$(echo "$$AVAILABLE_MODELS,$$DEFAULT_MODEL" | tr ',' ' ' | tr -s ' ' '\n' | sort -u); do
-          m=$$(echo "$$m" | xargs)
-          [ -z "$$m" ] && continue
-          echo "--- checking $$m ---"
-          # Fresh check each iteration via both CLI and API (CLI may lag on Windows bind)
-          if ollama list 2>/dev/null | awk 'NR>1{print $$1}' | grep -qx "$$m"; then
-            echo "-> $$m already installed (ollama list exact) — skipping pull"
-            continue
-          fi
-          if ollama list 2>/dev/null | awk 'NR>1{print $$1}' | grep -q "^$${m%%:*}:"; then
-            echo "-> $$m already installed (ollama list prefix) — skipping pull"
-            continue
-          fi
-          if curl -fsS http://ollama:11434/api/tags 2>/dev/null | grep -q "\"name\":\"$$m\""; then
-            echo "-> $$m already installed (API exact) — skipping pull"
-            continue
-          fi
-          # Check without tag: "phi4-mini" should match "phi4-mini:latest"
-          base="$${m%%:*}"
-          if curl -fsS http://ollama:11434/api/tags 2>/dev/null | grep -q "\"name\":\"$$base:"; then
-            echo "-> $$m already installed (API prefix $$base) — skipping pull"
-            continue
-          fi
-          if ollama show "$$m" >/dev/null 2>&1; then
-            echo "-> $$m show succeeded — skipping pull"
-            continue
-          fi
-          # Also check filesystem directly (Windows bind may not be reflected in list yet)
-          # Ollama stores manifests at /root/.ollama/models/manifests/registry.ollama.ai/library/<name>/<tag>
-          manifest_path="/root/.ollama/models/manifests/registry.ollama.ai/library/$$base"
-          if [ -d "$$manifest_path" ]; then
-            echo "-> $$m manifest dir exists at $$manifest_path — skipping pull (filesystem hit)"
-            continue
-          fi
-          echo "-> ollama pull $$m (not found locally, will download)"
-          # If this still shows "pulling manifest 2.5GB", it means the model truly wasn't
-          # on the host bind. To use host's models, ensure .env has:
-          #   OLLAMA_HOST_DIR=C:/Users/YOURNAME/.ollama  (forward slashes, no trailing slash)
-          # and that host `ollama list` shows the model. Fallback named volume is empty on first run.
-          ollama pull "$$m" || echo "pull failed for $$m (may already exist or name wrong)"
-        done
-        echo "ollama-init done:"
-        ollama list || curl -fsS http://ollama:11434/api/tags || true
-    restart: "no"
-
-volumes:
-  ollama_data: {}
+volumes: {}