Spaces:
Running on Zero
feat: Move the Space from CPU-only Docker to ZeroGPU (Gradio SDK)
Browse filesCPU inference on the free cpu-basic tier turned out to be genuinely too
slow for this workload (6.4k-token system prompt + tool schemas before
generation even starts) -- live testing on the Space showed a single
request taking 8+ minutes with zero tokens streamed back. No amount of
context-window or build-flag tuning fixes a fundamentally CPU-bound
problem.
Switches to ZeroGPU: drops the Docker SDK and llama-cpp-python/GGUF
path for the Space, and loads the real fine-tuned model (now uploaded
in full-precision safetensors) via transformers on a dynamically
allocated GPU inside @spaces.GPU-wrapped calls. A minimal Gradio Blocks
app is mounted at /gradio solely to satisfy the platform's Gradio SDK
requirement for ZeroGPU hardware -- the existing custom FastAPI routes
and web UI are untouched and still serve the real app at "/".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Dockerfile +0 -53
|
@@ -1,53 +0,0 @@
|
|
| 1 |
-
FROM python:3.11-slim
|
| 2 |
-
|
| 3 |
-
# Prevent interactive prompts & buffer logs
|
| 4 |
-
ENV PYTHONDONTWRITEBYTECODE=1 \
|
| 5 |
-
PYTHONUNBUFFERED=1 \
|
| 6 |
-
DEBIAN_FRONTEND=noninteractive \
|
| 7 |
-
PORT=7860 \
|
| 8 |
-
HOME=/home/user
|
| 9 |
-
|
| 10 |
-
# Install essential scientific computation libraries and compilers
|
| 11 |
-
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 12 |
-
build-essential \
|
| 13 |
-
cmake \
|
| 14 |
-
git \
|
| 15 |
-
curl \
|
| 16 |
-
liblapack-dev \
|
| 17 |
-
libblas-dev \
|
| 18 |
-
gfortran \
|
| 19 |
-
libopenblas-dev \
|
| 20 |
-
&& rm -rf /var/lib/apt/lists/*
|
| 21 |
-
|
| 22 |
-
# Set up user for Hugging Face Spaces security standard
|
| 23 |
-
RUN useradd -m -u 1000 user
|
| 24 |
-
WORKDIR /home/user/app
|
| 25 |
-
|
| 26 |
-
# Install Python dependencies. llama-cpp-python compiles from source (no
|
| 27 |
-
# prebuilt wheel is published for recent versions); cap build parallelism so
|
| 28 |
-
# the compiler doesn't spawn enough jobs to OOM-kill the Spaces build machine.
|
| 29 |
-
# Pin to a fixed AVX2+FMA target instead of NATIVE (avoids depending on the
|
| 30 |
-
# build machine's exact CPU) or full auto-detection (which was the extra
|
| 31 |
-
# compile cost that caused the OOM) -- AVX2/FMA are present on essentially
|
| 32 |
-
# every x86-64 server CPU Spaces runs on, so this keeps inference fast
|
| 33 |
-
# without the crash risk of a mismatched AVX-512 build.
|
| 34 |
-
COPY requirements.txt .
|
| 35 |
-
ENV CMAKE_BUILD_PARALLEL_LEVEL=1 \
|
| 36 |
-
CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_AVX512=OFF"
|
| 37 |
-
RUN pip install --no-cache-dir --upgrade pip && \
|
| 38 |
-
pip install --no-cache-dir -r requirements.txt
|
| 39 |
-
|
| 40 |
-
# Copy application files
|
| 41 |
-
COPY --chown=user:user . .
|
| 42 |
-
|
| 43 |
-
# Ensure storage directories exist with write access
|
| 44 |
-
RUN mkdir -p outputs/plots data/rag_index data/user_docs && \
|
| 45 |
-
chown -R user:user /home/user
|
| 46 |
-
|
| 47 |
-
USER user
|
| 48 |
-
|
| 49 |
-
# Expose standard Hugging Face Spaces port
|
| 50 |
-
EXPOSE 7860
|
| 51 |
-
|
| 52 |
-
# Launch FastAPI web console
|
| 53 |
-
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|