# CPU-only Hugging Face Space image. # Serving uses llama.cpp + a pre-merged GGUF (Q4_K_M), so there is no torch / # transformers / bitsandbytes in the production image. This keeps the image small, # cold starts fast, and RAM well under the 16 GB free-tier ceiling. FROM python:3.11-slim # System deps. build-essential + cmake are a safety net in case pip falls back to # building llama-cpp-python from source; normally the prebuilt CPU wheel is used. RUN apt-get update && apt-get install -y --no-install-recommends \ git \ curl \ build-essential \ cmake \ && rm -rf /var/lib/apt/lists/* WORKDIR /app # Build llama-cpp-python from source against THIS image's glibc toolchain. # Why not a prebuilt wheel: the abetlen CPU "wheel index" ships musl-linked wheels # for every version recent enough to support the phi3 architecture (>=0.2.63), which # fail to load on this Debian/glibc image with: # "libc.musl-x86_64.so.1: cannot open shared object file". # The only glibc wheels there (<=0.2.62) predate phi3 support. So we compile. # GGML_NATIVE=OFF -> portable CPU binary (the build host CPU may differ from the # cpu-basic runtime CPU; -march=native would risk an illegal-instruction crash). # Version note: 0.3.2's bundled llama.cpp SEGFAULTs (exit 139) at inference time on # this GGUF (produced by a much newer llama.cpp). 0.3.19 is verified to load and # generate correct SQL against the exact production GGUF, so we pin to it. ENV CMAKE_ARGS="-DGGML_NATIVE=OFF" # Compile ONE translation unit at a time. A parallel compile (default = all cores) # OOM-killed the build runner (exit 137). Single-threaded keeps peak RAM ~1-2 GB. # Slower build, but reliable on the memory-limited HF build runner. ENV CMAKE_BUILD_PARALLEL_LEVEL=1 RUN pip install --no-cache-dir --no-binary=llama-cpp-python "llama-cpp-python==0.3.19" # Install the slim CPU-serving dependencies. COPY requirements-space.txt . RUN pip install --no-cache-dir -r requirements-space.txt # Copy project source COPY . . # HF Spaces requires the app to listen on port 7860 EXPOSE 7860 # Create dirs that may not exist in the repo RUN mkdir -p data models templates static # HF Spaces runs containers as a non-root user RUN useradd -m -u 1000 appuser && chown -R appuser /app USER appuser # Point HF cache to a writable location inside the container (GGUF is cached here) ENV HF_HOME=/app/.cache/huggingface CMD ["python", "src/server.py"]