Andrew McCracken Claude commited on
Commit
2f6841c
·
1 Parent(s): cfc97b4

Switch to GPU-enabled Docker image

Browse files

- Using techdaskalos/cybersecchatbot:gpu from Docker Hub
- Enable GPU acceleration with N_GPU_LAYERS=35
- Configure MODEL_POOL_SIZE=10 for concurrent users
- Requires GPU hardware in HF Space settings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Files changed (1) hide show
  1. Dockerfile +10 -4
Dockerfile CHANGED
@@ -1,6 +1,6 @@
1
- # Use pre-built image from Docker Hub
2
- # Build this image locally with: ./build-and-push.sh
3
- FROM techdaskalos/cybersecchatbot:latest
4
 
5
  # Environment variables (already set in base image, but can override)
6
  ENV PYTHONUNBUFFERED=1
@@ -9,6 +9,12 @@ ENV MODEL_FILENAME=phi4-mini-instruct-Q4_K_M.gguf
9
  ENV USE_RAG=false
10
  ENV CACHE_ENABLED=true
11
 
 
 
 
 
 
 
12
  # Set Hugging Face cache to /data for persistence and write permissions
13
  ENV HF_HOME=/data/huggingface
14
 
@@ -20,4 +26,4 @@ RUN mkdir -p /data /app/models /app/knowledge_db /data/huggingface/hub /data/hug
20
  COPY test_interface.html /app/
21
 
22
  EXPOSE 8000
23
- CMD ["python", "main.py"]
 
1
+ # Use pre-built GPU image from Docker Hub
2
+ # Build this image locally with: docker buildx build --platform linux/amd64 -f Dockerfile.base.gpu -t techdaskalos/cybersecchatbot:latest-gpu . --push
3
+ FROM techdaskalos/cybersecchatbot:gpu
4
 
5
  # Environment variables (already set in base image, but can override)
6
  ENV PYTHONUNBUFFERED=1
 
9
  ENV USE_RAG=false
10
  ENV CACHE_ENABLED=true
11
 
12
+ # GPU configuration - offload all layers to GPU
13
+ ENV N_GPU_LAYERS=35
14
+
15
+ # Concurrent request handling - 10 model instances for 10 concurrent users
16
+ ENV MODEL_POOL_SIZE=10
17
+
18
  # Set Hugging Face cache to /data for persistence and write permissions
19
  ENV HF_HOME=/data/huggingface
20
 
 
26
  COPY test_interface.html /app/
27
 
28
  EXPOSE 8000
29
+ CMD ["python", "main.py"]