atakan Claude Sonnet 5 commited on
Commit
ef72bf0
·
1 Parent(s): 80cc518

feat: Move the Space from CPU-only Docker to ZeroGPU (Gradio SDK)

Browse files

CPU inference on the free cpu-basic tier turned out to be genuinely too
slow for this workload (6.4k-token system prompt + tool schemas before
generation even starts) -- live testing on the Space showed a single
request taking 8+ minutes with zero tokens streamed back. No amount of
context-window or build-flag tuning fixes a fundamentally CPU-bound
problem.

Switches to ZeroGPU: drops the Docker SDK and llama-cpp-python/GGUF
path for the Space, and loads the real fine-tuned model (now uploaded
in full-precision safetensors) via transformers on a dynamically
allocated GPU inside @spaces.GPU-wrapped calls. A minimal Gradio Blocks
app is mounted at /gradio solely to satisfy the platform's Gradio SDK
requirement for ZeroGPU hardware -- the existing custom FastAPI routes
and web UI are untouched and still serve the real app at "/".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. Dockerfile +0 -53
Dockerfile DELETED
@@ -1,53 +0,0 @@
1
- FROM python:3.11-slim
2
-
3
- # Prevent interactive prompts & buffer logs
4
- ENV PYTHONDONTWRITEBYTECODE=1 \
5
- PYTHONUNBUFFERED=1 \
6
- DEBIAN_FRONTEND=noninteractive \
7
- PORT=7860 \
8
- HOME=/home/user
9
-
10
- # Install essential scientific computation libraries and compilers
11
- RUN apt-get update && apt-get install -y --no-install-recommends \
12
- build-essential \
13
- cmake \
14
- git \
15
- curl \
16
- liblapack-dev \
17
- libblas-dev \
18
- gfortran \
19
- libopenblas-dev \
20
- && rm -rf /var/lib/apt/lists/*
21
-
22
- # Set up user for Hugging Face Spaces security standard
23
- RUN useradd -m -u 1000 user
24
- WORKDIR /home/user/app
25
-
26
- # Install Python dependencies. llama-cpp-python compiles from source (no
27
- # prebuilt wheel is published for recent versions); cap build parallelism so
28
- # the compiler doesn't spawn enough jobs to OOM-kill the Spaces build machine.
29
- # Pin to a fixed AVX2+FMA target instead of NATIVE (avoids depending on the
30
- # build machine's exact CPU) or full auto-detection (which was the extra
31
- # compile cost that caused the OOM) -- AVX2/FMA are present on essentially
32
- # every x86-64 server CPU Spaces runs on, so this keeps inference fast
33
- # without the crash risk of a mismatched AVX-512 build.
34
- COPY requirements.txt .
35
- ENV CMAKE_BUILD_PARALLEL_LEVEL=1 \
36
- CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_AVX=ON -DGGML_AVX2=ON -DGGML_FMA=ON -DGGML_AVX512=OFF"
37
- RUN pip install --no-cache-dir --upgrade pip && \
38
- pip install --no-cache-dir -r requirements.txt
39
-
40
- # Copy application files
41
- COPY --chown=user:user . .
42
-
43
- # Ensure storage directories exist with write access
44
- RUN mkdir -p outputs/plots data/rag_index data/user_docs && \
45
- chown -R user:user /home/user
46
-
47
- USER user
48
-
49
- # Expose standard Hugging Face Spaces port
50
- EXPOSE 7860
51
-
52
- # Launch FastAPI web console
53
- CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]