Quantized Models (GGUF, IQ, Imatrix)
Collection
Various GGUF quantizations of small models. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 97 items • Updated • 71
My GGUF-Imatrix quants of DS-R1-Qwen3-8B-ArliAI-RpR-v4-Small.
Prompt format:
ChatMLNote:
Set the additional settings as per the instructions in the image at the end of the card to use the thinking setup. [1]
Reasoning setup in SillyTavern:
- Downloads last month
- 1,873
Hardware compatibility
Log In to add your hardware
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Lewdiculous/DS-R1-Qwen3-8B-ArliAI-RpR-v4-Small-GGUF-IQ-Imatrix
Base model
deepseek-ai/DeepSeek-R1-0528-Qwen3-8B