Spaces:

Sumkh
/

AgenticRAG

Sleeping

Sumkh commited on Feb 25, 2025

Commit

857d06b

verified ·

1 Parent(s): 49a2738

Update start.sh

Files changed (1) hide show

start.sh CHANGED Viewed

@@ -15,8 +15,7 @@ vllm serve unsloth/llama-3-70b-Instruct-bnb-4bit \
   --quantization bitsandbytes \
   --load-format bitsandbytes \
   --dtype half \
-  --enforce-eager \
-  --max-model-len 8192 &
 # Wait to ensure the vLLM server is fully started (adjust if needed)
 sleep 10

   --quantization bitsandbytes \
   --load-format bitsandbytes \
   --dtype half \
+  --enforce-eager &
 # Wait to ensure the vLLM server is fully started (adjust if needed)
 sleep 10