meituan
/

DeepSeek-R1-Channel-INT8

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions

Resources

View closed (4)

是否计划量化deepseek-v3.1/deepseek-v3.2模型并上传?

#13 opened 3 days ago by

can support ignore layers in w8a8_int8 quantization setting?

#12 opened 8 months ago by

Can I run this model on AMD GPU? Or is it only compatible for Nvidia GPU?

#11 opened 9 months ago by

Update inference/bf16_cast_channel_int8.py

#10 opened 10 months ago by

Update config.json

#9 opened 10 months ago by

how to achieve 2500 tps throughput？

#8 opened 10 months ago by

can this model run with `ollama` with `pure cpu` model?

#7 opened 10 months ago by

Add `quantization_config` in config.json?

#4 opened 10 months ago by

运行channel INT8后sglang报错OOM

#3 opened 10 months ago by