wip script and monorepo

.cursor/plans/hf_space_deploy_review_a7f8b3c3.plan.md

@@ -4,13 +4,13 @@ overview: "The existing uv monorepo plan is the right foundation for a Build Sma
 todos:
   - id: fix-readme-yaml
     content: "Put HF Space YAML frontmatter (sdk: docker, app_port: 7860) in root README.md, not only apps/gradio-space/README.md"
-    status: pending
+    status: completed
   - id: phase1-bootstrap
     content: "Phase 1: uv workspace + inference lib (llama_cpp only) + minimal gr.ChatInterface app"
-    status: pending
+    status: completed
   - id: phase1-docker
     content: "Phase 1: root Dockerfile (uv sync, UID 1000, port 7860) and create Space under build-small-hackathon"
-    status: pending
+    status: in_progress
   - id: phase1-verify
     content: "Phase 1: local uv sync + Gradio smoke test + confirm Space builds on CPU basic"
     status: pending

.env.example

@@ -0,0 +1,12 @@
+INFERENCE_BACKEND=llama_cpp
+MODEL_REPO=Qwen/Qwen2.5-3B-Instruct-GGUF
+MODEL_FILE=qwen2.5-3b-instruct-q4_k_m.gguf
+N_CTX=4096
+N_GPU_LAYERS=0
+
+# Optional: local GGUF path instead of Hub download
+# MODEL_PATH=./models/qwen2.5-3b-instruct-q4_k_m.gguf
+
+# Optional: transformers backend (requires inference[transformers] extra)
+# INFERENCE_BACKEND=transformers
+# MODEL_ID=Qwen/Qwen2.5-3B-Instruct

.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+*.py[cod]
+.env
+models/
+*.gguf
+.ruff_cache/
+.pytest_cache/
+*.egg-info/
+dist/
+build/

.python-version

@@ -0,0 +1 @@
+3.12

Dockerfile

@@ -0,0 +1,32 @@
+FROM python:3.12-slim
+
+ENV PYTHONUNBUFFERED=1 \
+    UV_COMPILE_BYTECODE=1 \
+    UV_LINK_MODE=copy
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    cmake \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+
+WORKDIR /app
+
+COPY pyproject.toml uv.lock .python-version ./
+COPY apps/gradio-space/pyproject.toml apps/gradio-space/
+COPY libs/inference/pyproject.toml libs/inference/
+COPY apps/gradio-space/src apps/gradio-space/src
+COPY libs/inference/src libs/inference/src
+
+RUN useradd -m -u 1000 user && \
+    uv sync --frozen --no-dev --package gradio-space && \
+    chown -R user:user /app
+
+USER user
+ENV HOME=/home/user \
+    PATH="/app/.venv/bin:$PATH"
+
+EXPOSE 7860
+
+CMD ["uv", "run", "--package", "gradio-space", "python", "-m", "gradio_space.app"]

README.md

@@ -1 +1,101 @@
-# small-model-hackathon
+---
+title: Small Model Hackathon
+emoji: 🦙
+colorFrom: blue
+colorTo: green
+sdk: docker
+app_port: 7860
+pinned: false
+license: apache-2.0
+---
+
+# Small Model Hackathon
+
+Gradio chat Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon). Runs local inference with **llama.cpp** (GGUF) by default; optional **transformers** backend via env.
+
+## Prerequisites
+
+- [uv](https://docs.astral.sh/uv/)
+- Python 3.12
+
+## Quick start
+
+```bash
+uv sync --all-packages
+cp .env.example .env   # optional: edit model settings
+
+# Download GGUF for offline dev (optional)
+uv run python scripts/download_model.py
+
+# Run Gradio locally
+uv run --package gradio-space python -m gradio_space.app
+```
+
+Open http://localhost:7860. The model downloads from Hugging Face Hub on the first chat message (or set `MODEL_PATH` to a local GGUF).
+
+## Environment variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `INFERENCE_BACKEND` | `llama_cpp` | `llama_cpp` or `transformers` |
+| `MODEL_REPO` | `Qwen/Qwen2.5-3B-Instruct-GGUF` | Hub repo for GGUF |
+| `MODEL_FILE` | `qwen2.5-3b-instruct-q4_k_m.gguf` | GGUF filename |
+| `MODEL_PATH` | — | Local GGUF path (skips Hub download) |
+| `N_CTX` | `4096` | Context window |
+| `N_GPU_LAYERS` | `0` | GPU layers for llama.cpp (0 = CPU) |
+| `MODEL_ID` | `Qwen/Qwen2.5-3B-Instruct` | Used when `INFERENCE_BACKEND=transformers` |
+
+See [`.env.example`](.env.example) for a full template.
+
+## Monorepo layout
+
+```text
+apps/gradio-space/   # Gradio UI (HF Space entrypoint)
+libs/inference/      # Swappable inference backends
+scripts/             # Dev utilities
+```
+
+### Common commands
+
+```bash
+uv add --package gradio-space <package>
+uv add --package inference <package>
+uv run --package gradio-space python -m gradio_space.app
+uv run python -c "from inference.factory import get_backend"
+```
+
+## Hugging Face Space deployment
+
+1. Create a Space under [build-small-hackathon](https://huggingface.co/build-small-hackathon) with **Docker** SDK.
+2. Link this repository (root `Dockerfile` + root `README.md` YAML above).
+3. Hardware: start with **CPU basic**; upgrade to GPU if you set `N_GPU_LAYERS > 0`.
+4. Add Space secrets: `MODEL_REPO`, `MODEL_FILE`, `N_CTX`, `N_GPU_LAYERS`.
+
+```bash
+# Optional local Docker smoke test
+docker build -t hackathon-space .
+docker run --rm -p 7860:7860 -e MODEL_REPO=Qwen/Qwen2.5-3B-Instruct-GGUF hackathon-space
+```
+
+## Hackathon checklist
+
+- [ ] Choose a track (Backyard AI or Thousand Token Wood)
+- [ ] Space live under build-small-hackathon
+- [ ] Demo video recorded
+- [ ] Social post published
+- [ ] Submission locked in by **June 15, 2026**
+
+### Badge targets
+
+- **Off-the-Grid** — local llama.cpp inference (default setup)
+- **Llama Champion** — llama.cpp + GGUF model
+- **Off-Brand** — custom UI via `gr.Server` (Phase 2)
+- **Sharing is Caring** — agent traces dataset (Phase 2)
+
+## Transformers backend (optional)
+
+```bash
+uv sync --package inference --extra transformers
+INFERENCE_BACKEND=transformers MODEL_ID=Qwen/Qwen2.5-3B-Instruct \
+  uv run --package gradio-space python -m gradio_space.app
+```

pyproject.toml

@@ -0,0 +1,26 @@
+[project]
+name = "small-model-hackathon"
+version = "0.1.0"
+description = "Build Small Hackathon — Gradio Space with local llama.cpp inference"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "gradio-space",
+    "inference",
+]
+
+[dependency-groups]
+dev = [
+    "ruff>=0.9.0",
+    "pytest>=8.0.0",
+]
+
+[tool.uv.workspace]
+members = [
+    "apps/*",
+    "libs/*",
+]
+
+[tool.uv.sources]
+gradio-space = { workspace = true }
+inference = { workspace = true }

scripts/download_model.py

@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+"""Download the configured GGUF model from Hugging Face Hub for offline dev."""
+
+from __future__ import annotations
+
+import argparse
+import os
+from pathlib import Path
+
+from huggingface_hub import hf_hub_download
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--repo",
+        default=os.environ.get("MODEL_REPO", "Qwen/Qwen2.5-3B-Instruct-GGUF"),
+        help="Hugging Face repo containing the GGUF file",
+    )
+    parser.add_argument(
+        "--file",
+        default=os.environ.get("MODEL_FILE", "qwen2.5-3b-instruct-q4_k_m.gguf"),
+        help="GGUF filename inside the repo",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=Path("models"),
+        help="Directory to copy/symlink the downloaded model into",
+    )
+    args = parser.parse_args()
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+
+    path = hf_hub_download(
+        repo_id=args.repo,
+        filename=args.file,
+        local_dir=args.output_dir,
+        local_dir_use_symlinks=False,
+    )
+    print(f"Model ready at: {path}")
+    print(f"Set MODEL_PATH={path} to use this file directly.")
+
+
+if __name__ == "__main__":
+    main()