Hugging Face's logo

Spaces:
binary1ne
/
vllm-llama2
Paused

App Files Community
binary1ne commited on
Commit
4ddb93e
·
verified ·
1 Parent(s): 6da36b0

Update Dockerfile

Browse files
Files changed (1) hide show
  1. Dockerfile +8 -6
Dockerfile CHANGED
@@ -65,9 +65,11 @@ RUN pip list
65
  # -----------------------------
66
  # Start vLLM
67
  # -----------------------------
68
- CMD ["python3", "-m", "vllm.entrypoints.openai.api_server", \
69
- "--model", "unsloth/Llama-3.2-1B-bnb-4bit", \
70
- "--host", "0.0.0.0", \
71
- "--port", "7860", \
72
- "--tensor-parallel-size", "1", \
73
- "--gpu-memory-utilization", "0.0"]
 
 
 
65
  # -----------------------------
66
  # Start vLLM
67
  # -----------------------------
68
+ RUN python3 -u -m vllm.entrypoints.openai.api_server --model "unsloth/Llama-3.2-1B-bnb-4bit" --host 0.0.0.0 --port 7860 --tensor-parallel-size 1 --gpu-memory-utilization 0.0
69
+
70
+ # CMD ["python3", "-m", "vllm.entrypoints.openai.api_server", \
71
+ # "--model", "unsloth/Llama-3.2-1B-bnb-4bit", \
72
+ # "--host", "0.0.0.0", \
73
+ # "--port", "7860", \
74
+ # "--tensor-parallel-size", "1", \
75
+ # "--gpu-memory-utilization", "0.0"]