awq quant made with lmdeploy v0.4.2:
lmdeploy lite auto_awq sophosympatheia/Midnight-Miqu-70B-v1.5 --work-dir Midnight-Miqu-70B-v1.5-4bit
- Downloads last month
- 6
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support