model config wip

README.md

@@ -1,5 +1,6 @@
 ---
-title: Small Model Hackathon
+
+## title: Small Model Hackathon
 emoji: 🦙
 colorFrom: blue
 colorTo: green
@@ -7,7 +8,6 @@ sdk: docker
 app_port: 7860
 pinned: false
 license: apache-2.0
----
 
 # Small Model Hackathon
 
@@ -33,21 +33,23 @@ uv run python scripts/download_model.py
 uv run --package gradio-space python -m gradio_space.app
 ```
 
-Open http://localhost:7860. The model downloads from Hugging Face Hub on the first chat message (or set `MODEL_PATH` to a local GGUF).
+Open [http://localhost:7860](http://localhost:7860). The model downloads from Hugging Face Hub on the first chat message (or set `MODEL_PATH` to a local GGUF).
 
 ## Environment variables
 
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `INFERENCE_BACKEND` | `llama_cpp` | `llama_cpp` or `transformers` |
-| `MODEL_REPO` | `Qwen/Qwen2.5-3B-Instruct-GGUF` | Hub repo for GGUF |
-| `MODEL_FILE` | `qwen2.5-3b-instruct-q4_k_m.gguf` | GGUF filename |
-| `MODEL_PATH` | — | Local GGUF path (skips Hub download) |
-| `N_CTX` | `4096` | Context window |
-| `N_GPU_LAYERS` | `0` | GPU layers for llama.cpp (0 = CPU) |
-| `MODEL_ID` | `Qwen/Qwen2.5-3B-Instruct` | Used when `INFERENCE_BACKEND=transformers` |
 
-See [`.env.example`](.env.example) for a full template.
+| Variable            | Default                           | Description                                |
+| ------------------- | --------------------------------- | ------------------------------------------ |
+| `INFERENCE_BACKEND` | `llama_cpp`                       | `llama_cpp` or `transformers`              |
+| `MODEL_REPO`        | `Qwen/Qwen2.5-3B-Instruct-GGUF`   | Hub repo for GGUF                          |
+| `MODEL_FILE`        | `qwen2.5-3b-instruct-q4_k_m.gguf` | GGUF filename                              |
+| `MODEL_PATH`        | —                                 | Local GGUF path (skips Hub download)       |
+| `N_CTX`             | `4096`                            | Context window                             |
+| `N_GPU_LAYERS`      | `0`                               | GPU layers for llama.cpp (0 = CPU)         |
+| `MODEL_ID`          | `Qwen/Qwen2.5-3B-Instruct`        | Used when `INFERENCE_BACKEND=transformers` |
+
+
+See `[.env.example](.env.example)` for a full template.
 
 ## Monorepo layout
 
@@ -81,11 +83,11 @@ docker run --rm -p 7860:7860 -e MODEL_REPO=Qwen/Qwen2.5-3B-Instruct-GGUF hackath
 
 ## Hackathon checklist
 
-- [ ] Choose a track (Backyard AI or Thousand Token Wood)
-- [ ] Space live under build-small-hackathon
-- [ ] Demo video recorded
-- [ ] Social post published
-- [ ] Submission locked in by **June 15, 2026**
+- Choose a track (Backyard AI or Thousand Token Wood)
+- Space live under build-small-hackathon
+- Demo video recorded
+- Social post published
+- Submission locked in by **June 15, 2026**
 
 ### Badge targets
 
@@ -101,3 +103,4 @@ uv sync --package inference --extra transformers
 INFERENCE_BACKEND=transformers MODEL_ID=Qwen/Qwen2.5-3B-Instruct \
   uv run --package gradio-space python -m gradio_space.app
 ```
+

USAGE.md

@@ -45,7 +45,7 @@ MODEL_PATH=./models/qwen2.5-3b-instruct-q4_k_m.gguf
 uv run --package gradio-space python -m gradio_space.app
 ```
 
-Open http://localhost:7860.
+Open [http://localhost:7860](http://localhost:7860).
 
 The model loads on the **first chat message** unless you set `MODEL_PATH`. After code changes, restart the process to pick up updates.
 
@@ -61,16 +61,18 @@ uv run --package gradio-space python -c "from gradio_space.app import build_demo
 
 ### Local env reference
 
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `INFERENCE_BACKEND` | `llama_cpp` | `llama_cpp` or `transformers` |
-| `MODEL_REPO` | `Qwen/Qwen2.5-3B-Instruct-GGUF` | Hub repo for GGUF |
-| `MODEL_FILE` | `qwen2.5-3b-instruct-q4_k_m.gguf` | GGUF filename |
-| `MODEL_PATH` | — | Local GGUF path (skips Hub download) |
-| `N_CTX` | `4096` | Context window |
-| `N_GPU_LAYERS` | `0` | GPU layers for llama.cpp (`0` = CPU only) |
-| `PORT` | `7860` | Gradio listen port |
-| `MODEL_ID` | `Qwen/Qwen2.5-3B-Instruct` | Used when `INFERENCE_BACKEND=transformers` |
+
+| Variable            | Default                           | Description                                |
+| ------------------- | --------------------------------- | ------------------------------------------ |
+| `INFERENCE_BACKEND` | `llama_cpp`                       | `llama_cpp` or `transformers`              |
+| `MODEL_REPO`        | `Qwen/Qwen2.5-3B-Instruct-GGUF`   | Hub repo for GGUF                          |
+| `MODEL_FILE`        | `qwen2.5-3b-instruct-q4_k_m.gguf` | GGUF filename                              |
+| `MODEL_PATH`        | —                                 | Local GGUF path (skips Hub download)       |
+| `N_CTX`             | `4096`                            | Context window                             |
+| `N_GPU_LAYERS`      | `0`                               | GPU layers for llama.cpp (`0` = CPU only)  |
+| `PORT`              | `7860`                            | Gradio listen port                         |
+| `MODEL_ID`          | `Qwen/Qwen2.5-3B-Instruct`        | Used when `INFERENCE_BACKEND=transformers` |
+
 
 ### Optional: transformers backend
 
@@ -98,7 +100,7 @@ docker run --rm -p 7860:7860 \
   hackathon-space
 ```
 
-Open http://localhost:7860. Stop with `Ctrl+C`.
+Open [http://localhost:7860](http://localhost:7860). Stop with `Ctrl+C`.
 
 To use a pre-downloaded local model inside Docker, mount it and set `MODEL_PATH`:
 
@@ -142,22 +144,26 @@ hf repo create build-small-hackathon/<your-space-name> \
 
 ### 3. Configure hardware
 
-| Setting | Recommendation |
-|---------|----------------|
-| Hardware | **CPU basic** to start (llama.cpp with `N_GPU_LAYERS=0`) |
-| Upgrade | GPU Space if you set `N_GPU_LAYERS > 0` for faster inference |
+
+| Setting  | Recommendation                                               |
+| -------- | ------------------------------------------------------------ |
+| Hardware | **CPU basic** to start (llama.cpp with `N_GPU_LAYERS=0`)     |
+| Upgrade  | GPU Space if you set `N_GPU_LAYERS > 0` for faster inference |
+
 
 ### 4. Set Space environment variables
 
 In the Space **Settings → Variables and secrets**:
 
-| Variable | Value |
-|----------|-------|
-| `INFERENCE_BACKEND` | `llama_cpp` |
-| `MODEL_REPO` | `Qwen/Qwen2.5-3B-Instruct-GGUF` |
-| `MODEL_FILE` | `qwen2.5-3b-instruct-q4_k_m.gguf` |
-| `N_CTX` | `4096` |
-| `N_GPU_LAYERS` | `0` (or higher on GPU hardware) |
+
+| Variable            | Value                             |
+| ------------------- | --------------------------------- |
+| `INFERENCE_BACKEND` | `llama_cpp`                       |
+| `MODEL_REPO`        | `Qwen/Qwen2.5-3B-Instruct-GGUF`   |
+| `MODEL_FILE`        | `qwen2.5-3b-instruct-q4_k_m.gguf` |
+| `N_CTX`             | `4096`                            |
+| `N_GPU_LAYERS`      | `0` (or higher on GPU hardware)   |
+
 
 ### 5. Build and verify
 
@@ -177,14 +183,16 @@ If cold starts are too slow, attach a **Storage Bucket** in Space settings so do
 
 ## Troubleshooting
 
-| Symptom | Likely cause | Fix |
-|---------|--------------|-----|
-| First chat hangs / slow | GGUF downloading from Hub | Pre-download locally; on Space, wait or use Storage Bucket |
-| `Failed to load model` in chat | Wrong `MODEL_REPO` / `MODEL_FILE` | Check env vars match a valid GGUF on Hub |
-| Docker build fails on `llama-cpp-python` | Missing build tools | Dockerfile already installs `build-essential` and `cmake` |
-| Space build fails | Missing `uv.lock` or README YAML | Ensure `sdk: docker` is in root `README.md` frontmatter |
-| `transformers` backend error | Optional deps not installed | Run `uv sync --package inference --extra transformers` |
-| Port already in use locally | Another process on 7860 | `PORT=7861 uv run --package gradio-space python -m gradio_space.app` |
+
+| Symptom                                  | Likely cause                      | Fix                                                                  |
+| ---------------------------------------- | --------------------------------- | -------------------------------------------------------------------- |
+| First chat hangs / slow                  | GGUF downloading from Hub         | Pre-download locally; on Space, wait or use Storage Bucket           |
+| `Failed to load model` in chat           | Wrong `MODEL_REPO` / `MODEL_FILE` | Check env vars match a valid GGUF on Hub                             |
+| Docker build fails on `llama-cpp-python` | Missing build tools               | Dockerfile already installs `build-essential` and `cmake`            |
+| Space build fails                        | Missing `uv.lock` or README YAML  | Ensure `sdk: docker` is in root `README.md` frontmatter              |
+| `transformers` backend error             | Optional deps not installed       | Run `uv sync --package inference --extra transformers`               |
+| Port already in use locally              | Another process on 7860           | `PORT=7861 uv run --package gradio-space python -m gradio_space.app` |
+
 
 ---
 
@@ -196,8 +204,11 @@ All three environments use the same command:
 uv run --package gradio-space python -m gradio_space.app
 ```
 
-| Environment | How to run |
-|-------------|------------|
-| Local dev | `uv run --package gradio-space python -m gradio_space.app` |
-| Docker | `docker run -p 7860:7860 hackathon-space` |
-| HF Space | Built and started automatically from `Dockerfile` `CMD` |
+
+| Environment | How to run                                                 |
+| ----------- | ---------------------------------------------------------- |
+| Local dev   | `uv run --package gradio-space python -m gradio_space.app` |
+| Docker      | `docker run -p 7860:7860 hackathon-space`                  |
+| HF Space    | Built and started automatically from `Dockerfile` `CMD`    |
+
+

models.yaml

@@ -2,14 +2,20 @@
 # Select active preset with ACTIVE_MODEL; override any field via .env (see .env.example).
 
 defaults:
-  active_model: qwen3b-gguf
+  active_model: minicpm-v-4.6
   # Dev: set ALLOW_MODEL_SWITCH=true in .env to expose a dropdown in Gradio.
   # Space: keep false so visitors use one pinned model.
   allow_model_switch: false
 
 models:
+  minicpm-v-4.6:
+    label: MiniCPM-V 4.6 (Transformers, ~0.8B, default)
+    backend: transformers
+    model_id: openbmb/MiniCPM-V-4.6
+    trust_remote_code: true
+
   qwen3b-gguf:
-    label: Qwen 2.5 3B Instruct (GGUF, default)
+    label: Qwen 2.5 3B Instruct (GGUF)
     backend: llama_cpp
     model_repo: Qwen/Qwen2.5-3B-Instruct-GGUF
     model_file: qwen2.5-3b-instruct-q4_k_m.gguf

uv.lock

@@ -198,7 +198,7 @@ name = "cuda-bindings"
 version = "13.3.1"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "cuda-pathfinder" },
+    { name = "cuda-pathfinder", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/ce/67/5e7dba1ba576dd73da5dee894ca076ca5e959450dfff66d6d510a255d1f7/cuda_bindings-13.3.1-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7855c4868aabc0cfae28abbe83d56734bdfbd08f08fc234ac1912a12858bf49", size = 6025351, upload-time = "2026-05-29T23:11:49.685Z" },
@@ -229,34 +229,34 @@ wheels = [
 
 [package.optional-dependencies]
 cudart = [
-    { name = "nvidia-cuda-runtime", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cuda-runtime", marker = "sys_platform == 'linux'" },
 ]
 cufft = [
-    { name = "nvidia-cufft", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cufft", marker = "sys_platform == 'linux'" },
 ]
 cufile = [
     { name = "nvidia-cufile", marker = "sys_platform == 'linux'" },
 ]
 cupti = [
-    { name = "nvidia-cuda-cupti", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cuda-cupti", marker = "sys_platform == 'linux'" },
 ]
 curand = [
-    { name = "nvidia-curand", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-curand", marker = "sys_platform == 'linux'" },
 ]
 cusolver = [
-    { name = "nvidia-cusolver", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cusolver", marker = "sys_platform == 'linux'" },
 ]
 cusparse = [
-    { name = "nvidia-cusparse", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cusparse", marker = "sys_platform == 'linux'" },
 ]
 nvjitlink = [
-    { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux'" },
 ]
 nvrtc = [
-    { name = "nvidia-cuda-nvrtc", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-cuda-nvrtc", marker = "sys_platform == 'linux'" },
 ]
 nvtx = [
-    { name = "nvidia-nvtx", marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
+    { name = "nvidia-nvtx", marker = "sys_platform == 'linux'" },
 ]
 
 [[package]]
@@ -500,6 +500,7 @@ source = { editable = "libs/inference" }
 dependencies = [
     { name = "huggingface-hub" },
     { name = "llama-cpp-python" },
+    { name = "pyyaml" },
 ]
 
 [package.optional-dependencies]
@@ -514,6 +515,7 @@ requires-dist = [
     { name = "accelerate", marker = "extra == 'transformers'", specifier = ">=1.2.0" },
     { name = "huggingface-hub", specifier = ">=0.27.0" },
     { name = "llama-cpp-python", specifier = ">=0.3.0" },
+    { name = "pyyaml", specifier = ">=6.0.2" },
     { name = "torch", marker = "extra == 'transformers'", specifier = ">=2.5.0" },
     { name = "transformers", marker = "extra == 'transformers'", specifier = ">=4.47.0" },
 ]
@@ -720,7 +722,7 @@ name = "nvidia-cublas"
 version = "13.1.1.3"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "nvidia-cuda-nvrtc" },
+    { name = "nvidia-cuda-nvrtc", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/a7/a1/0bd24ee8c8d03adac032fd2909426a00c88f8c57961b1277ded97f91119f/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:b7a210458267ac818974c53038fbec2e969d5c99f305ab15c72522fa9f001dd5", size = 542848918, upload-time = "2026-04-08T18:46:22.985Z" },
@@ -759,7 +761,7 @@ name = "nvidia-cudnn-cu13"
 version = "9.20.0.48"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "nvidia-cublas" },
+    { name = "nvidia-cublas", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/56/c5/83384d846b2fd17c44bd499b36c75a45ed4f095fbbb2252294e89cea5c5c/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:e31454ae00094b0c55319d9d15b6fa2fc50a9e1c0f5c8c80fb75258234e731e1", size = 444574296, upload-time = "2026-03-09T19:28:27.751Z" },
@@ -771,7 +773,7 @@ name = "nvidia-cufft"
 version = "12.0.0.61"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "nvidia-nvjitlink" },
+    { name = "nvidia-nvjitlink", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },
@@ -801,9 +803,9 @@ name = "nvidia-cusolver"
 version = "12.0.4.66"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "nvidia-cublas" },
-    { name = "nvidia-cusparse" },
-    { name = "nvidia-nvjitlink" },
+    { name = "nvidia-cublas", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
+    { name = "nvidia-cusparse", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
+    { name = "nvidia-nvjitlink", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },
@@ -815,7 +817,7 @@ name = "nvidia-cusparse"
 version = "12.6.3.3"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "nvidia-nvjitlink" },
+    { name = "nvidia-nvjitlink", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
 ]
 wheels = [
     { url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },