Spaces:

Sumkh
/

AgenticRAG

Sleeping

Sumkh commited on Feb 25, 2025

Commit

6201226

verified ·

1 Parent(s): 004e40c

Update start.sh

Files changed (1) hide show

start.sh CHANGED Viewed

@@ -11,7 +11,8 @@ export USER_AGENT="vllm_huggingface_space"
 vllm serve unsloth/llama-3-8b-Instruct-bnb-4bit \
   --enable-auto-tool-choice \
   --tool-call-parser llama3_json \
-  --chat-template examples/tool_chat_template_llama3.1_json.jinja &
 # Wait to ensure the vLLM server is fully started (adjust if needed)
 sleep 10

 vllm serve unsloth/llama-3-8b-Instruct-bnb-4bit \
   --enable-auto-tool-choice \
   --tool-call-parser llama3_json \
+  --chat-template examples/tool_chat_template_llama3.1_json.jinja \
+  --dtype half &
 # Wait to ensure the vLLM server is fully started (adjust if needed)
 sleep 10