Hugging Face's logo

RedHatAI
/
DeepSeek-R1-Distill-Llama-70B-quantized.w8a8

Text Generation
Transformers
Safetensors
llama
deepseek
int8
vllm
llmcompressor
conversational
text-generation-inference
8-bit precision
compressed-tensors
Model card Files
xet
Community
nm-research commited on
Commit
81dc57c
·
verified ·
1 Parent(s): 8b2e640

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -1
README.md CHANGED
@@ -76,7 +76,6 @@ from llmcompressor.modifiers.quantization import QuantizationModifier
76
  from llmcompressor.modifiers.smoothquant import SmoothQuantModifier
77
  from llmcompressor.transformers import oneshot
78
  from llmcompressor.transformers.compression.helpers import calculate_offload_device_map
79
- import os
80
 
81
  # Load model
82
  model_stub = "deepseek-ai/DeepSeek-R1-Distill-Llama-70B"
 
76
  from llmcompressor.modifiers.smoothquant import SmoothQuantModifier
77
  from llmcompressor.transformers import oneshot
78
  from llmcompressor.transformers.compression.helpers import calculate_offload_device_map
 
79
 
80
  # Load model
81
  model_stub = "deepseek-ai/DeepSeek-R1-Distill-Llama-70B"