PR #2: Replace llama.cpp with transformers + spaces.GPU + temp grid
#2
by specimba - opened
No description provided.
🔴 CRITICAL: This PR fixes libcudart.so.12 build failure.
The current main branch crashes on Space boot because:
llama-cpp-pythonattempts CUDA compilation in a builder with no CUDA runtimefitz(PyMuPDF) is heavy and not needed for core inference- Custom FastAPI mounting conflicts with Gradio auto-mounting
This PR replaces it with:
transformers+bitsandbytes(pre-built wheels, no compilation)@spaces.GPUdecorator with 4-bit NF4 quantization (same as GRM-2.6-Opus)- Streaming via
TextIteratorStreamer - Built-in ChatInterface + MCP server
To merge: Click "Merge" above. The builder will deploy in ~2 minutes.
cc: @build-small-hackathon/admin — this is blocking hackathon inference for the Darwin model.
specimba changed pull request status to merged