Update field notes
Browse files- field_notes.md +7 -0
field_notes.md
CHANGED
|
@@ -250,6 +250,13 @@ Both Modal services (`modal_llm.py`, `modal_painter.py`) are deployed once with
|
|
| 250 |
|
| 251 |
**Why four LLM backends?** This was the hardest part of shipping. `gr.Server` (for the custom frontend) needs gradio 6.x, which forces `huggingface-hub β₯ 1.2`, which forces `transformers β₯ 5.0`. But MiniCPM4.1-8B loads via `trust_remote_code`, and its remote code is incompatible with transformers 5.x: loading needs a shim, and generation then still crashes deep in attention. The two clean escapes are to run it on **Modal** with transformers pinned to 4.x, or β better β to run it through **llama.cpp** as a quantized GGUF, which uses no `transformers` at all and is small enough (~5 GB) to sit beside FLUX on a single 24 GB GPU. The llama.cpp route also earns the Llama Champion badge.
|
| 252 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 253 |
**Engine (`engine/`):**
|
| 254 |
- `mappings.py` β affect β geometry and affect β color (OKLCH)
|
| 255 |
- `renderer.py` β geometry β polygon boundary (triangle-wave spikiness, segmented smooth interpolation)
|
|
|
|
| 250 |
|
| 251 |
**Why four LLM backends?** This was the hardest part of shipping. `gr.Server` (for the custom frontend) needs gradio 6.x, which forces `huggingface-hub β₯ 1.2`, which forces `transformers β₯ 5.0`. But MiniCPM4.1-8B loads via `trust_remote_code`, and its remote code is incompatible with transformers 5.x: loading needs a shim, and generation then still crashes deep in attention. The two clean escapes are to run it on **Modal** with transformers pinned to 4.x, or β better β to run it through **llama.cpp** as a quantized GGUF, which uses no `transformers` at all and is small enough (~5 GB) to sit beside FLUX on a single 24 GB GPU. The llama.cpp route also earns the Llama Champion badge.
|
| 252 |
|
| 253 |
+
**The deployment journey.** The four backends weren't designed up front β they're the fossil record of getting this to run on the hardware I actually had (a laptop with **16 GB RAM / 8 GB VRAM**, nowhere near enough for an 8B LLM *and* a 4B diffusion model at once):
|
| 254 |
+
|
| 255 |
+
1. **Qwen3-8B on Ollama (local) + FLUX.2 Klein on Modal.** The first working setup. The LLM ran locally through Ollama (which is itself llama.cpp under the hood); the painter β far too heavy for 8 GB β was pushed to a Modal GPU endpoint. This split kept the loop fast to iterate on while the expensive image step lived in the cloud.
|
| 256 |
+
2. **MiniCPM4.1-8B via llama.cpp + FLUX.2 Klein on Modal.** To target the OpenBMB prize I swapped the judge to MiniCPM4.1-8B, and moved to running it directly on **llama.cpp** (a quantized GGUF via `llama-cpp-python`) rather than Ollama β which also claims the Llama Champion badge. The painter is still on Modal due to previous RAM constraints.
|
| 257 |
+
3. **Both on a single ZeroGPU Space β blocked.** The goal was to host the whole thing on one free Hugging Face **ZeroGPU** Space (llama.cpp LLM + FLUX painter together). In practice this proved very difficult: ZeroGPU attributes GPU usage via an `X-IP-Token` header that the standard Gradio UI forwards automatically, but a **custom `gr.Server` frontend doesn't**, so every GPU call is scheduled *anonymously* and immediately hits a tiny per-IP quota β a constant stream of **429 Too Many Requests**, while the account's real quota sits untouched. (On a campus network the anonymous quota is also shared across everyone behind the same NAT.) The solutions are a **dedicated** GPU Space β which isn't ZeroGPU and has no such attribution problem β or keeping the heavy work on Modal. Cracking ZeroGPU for a custom-frontend app remains open.
|
| 258 |
+
*Note: The ZeroGPU Too Many Requests assumption may be mistaken and I apologize if that is the case.*
|
| 259 |
+
|
| 260 |
**Engine (`engine/`):**
|
| 261 |
- `mappings.py` β affect β geometry and affect β color (OKLCH)
|
| 262 |
- `renderer.py` β geometry β polygon boundary (triangle-wave spikiness, segmented smooth interpolation)
|