JamAndTeaStudios
/

Qwen2.5-14B-Instruct-1M-FP8-Dynamic

Text Generation

text-generation-inference

compressed-tensors

Model card Files Files and versions

Yudi Xue commited on Jan 29, 2025

Commit

4abdc8b

·

verified ·

1 Parent(s): f997ee9

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -26,7 +26,7 @@ This optimization reduces the number of bits per parameter from 16 to 8, reducin
 ### Use with SGLang
 ```bash
-python -m sglang.launch_server --model-path Qwen/Qwen2.5-14B-Instruct-1M \
 --port 30000 --host 0.0.0.0
 ```

 ### Use with SGLang
 ```bash
+python -m sglang.launch_server --model-path JamAndTeaStudios/Qwen2.5-14B-Instruct-1M-FP8-Dynamic \
 --port 30000 --host 0.0.0.0
 ```