Spaces:

Sumkh
/

AgenticRAG

Sleeping

Sumkh commited on Feb 25, 2025

Commit

5c87648

verified ·

1 Parent(s): 857d06b

Update start.sh

Files changed (1) hide show

start.sh CHANGED Viewed

@@ -14,7 +14,6 @@ vllm serve unsloth/llama-3-70b-Instruct-bnb-4bit \
   --chat-template examples/tool_chat_template_llama3.1_json.jinja \
   --quantization bitsandbytes \
   --load-format bitsandbytes \
-  --dtype half \
   --enforce-eager &
 # Wait to ensure the vLLM server is fully started (adjust if needed)

   --chat-template examples/tool_chat_template_llama3.1_json.jinja \
   --quantization bitsandbytes \
   --load-format bitsandbytes \
   --enforce-eager &
 # Wait to ensure the vLLM server is fully started (adjust if needed)