Hugging Face's logo

cgg507
/
Behemoth-R1-123B-v2-q4f16_1

Model card Files
xet
Community
cgg507 commited on
Commit
c49f0ef
·
verified ·
1 Parent(s): f93a6f1

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - TheDrummer/Behemoth-R1-123B-v2
4
+ ---
5
+ python3 -m mlc_llm compile --quantization q4f16_1 --output 123b_r1.so . --overrides "tensor_parallel_shards=2" --device cuda
6
+ python3 -m mlc_llm chat --device cuda 123b_r1/ --model-lib /workspace/123b_r1.so