Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

4,558

Base only

Active filters: 6-bit

roleplaiapp/Dria-Agent-a-7B-Q6_K-GGUF

Text Generation • Updated Jan 19, 2025 • 11

Statuo/Allura-Qwen2.5-32b-RP-Ink-EXL2-6bpw

Updated Jan 19, 2025 • 5

Statuo/Allura-MN-12b-RP-Ink-EXL2-6bpw

Updated Jan 19, 2025 • 6

roleplaiapp/Wayfarer-12B-Q6_K-GGUF

Text Generation • 12B • Updated Jan 19, 2025 • 6

roleplaiapp/AceInstruct-72B-Q6_K-GGUF

Text Generation • 73B • Updated Jan 19, 2025 • 1

roleplaiapp/Codestral-22B-v0.1-Q6_K-GGUF

Text Generation • 22B • Updated Jan 19, 2025 • 2

roleplaiapp/AceInstruct-1.5B-Q6_K-GGUF

Text Generation • 2B • Updated Jan 19, 2025

roleplaiapp/AceInstruct-7B-Q6_K-GGUF

Text Generation • 8B • Updated Jan 19, 2025 • 25

roleplaiapp/Llama-3.1-Nemotron-70B-Instruct-HF-Q6_K-GGUF

Text Generation • 71B • Updated Jan 19, 2025 • 14

nisten/deepseek-r1-qwen32b-mlx-6bit

Text Generation • 7B • Updated Jan 20, 2025 • 18 • 14

mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-6bit

0.4B • Updated Feb 26, 2025 • 75

mlx-community/DeepSeek-R1-Distill-Qwen-7B-6bit

2B • Updated Feb 26, 2025 • 104 • 2

mlx-community/DeepSeek-R1-Distill-Qwen-14B-6bit

3B • Updated Feb 26, 2025 • 368 • 7

zhitels/DeepSeek-R1-Distill-Llama-70B-6bit

14B • Updated Jan 20, 2025 • 17

mlx-community/DeepSeek-R1-Distill-Qwen-32B-6bit

7B • Updated Feb 26, 2025 • 203 • 2

mlx-community/DeepSeek-R1-Distill-Llama-8B-6bit

2B • Updated Feb 26, 2025 • 53

adamo1139/DeepSeek-R1-Distill-Qwen-1.5B-6bpw-exl2

Updated Jan 20, 2025 • 4

zhitels/DeepSeek-R1-Distill-Qwen-32B-6bit

Text Generation • 7B • Updated Jan 21, 2025 • 124

Statuo/Deepseeker-Kunou-Qwen2.5-14b-EXL2-6bpw

Text Generation • Updated Jan 21, 2025 • 7 • 1

mlx-community/DeepSeek-R1-Distill-Llama-70B-6bit

Text Generation • 15B • Updated Feb 26, 2025 • 284 • 4

SicariusSicariiStuff/Eximius_Persona_5B-6.0bpw

Text Generation • Updated Jul 26, 2025 • 6

donniedraco/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6-mlx

7B • Updated Jan 24, 2025 • 7

MaziyarPanahi/SuperThoughts-CoT-14B-16k-o1-QwQ-GGUF

Text Generation • 15B • Updated Jan 24, 2025 • 9

MaziyarPanahi/Phi-4-Empathetic-GGUF

Text Generation • 15B • Updated Jan 24, 2025 • 77

donniedraco/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview-Q6-mlx

7B • Updated Jan 24, 2025 • 36 • 1

donniedraco/DeepSeek-R1-Distill-Qwen-32B-Q6-mlx

Text Generation • 7B • Updated Jan 24, 2025 • 30

MaziyarPanahi/Phi-4-QwQ-GGUF

Text Generation • 15B • Updated Jan 24, 2025 • 47

SicariusSicariiStuff/Wingless_Imp_8B-6.0bpw

Text Generation • Updated Jul 26, 2025 • 7

MaziyarPanahi/Llama-8B-Distill-CoT-GGUF

Text Generation • 8B • Updated Jan 24, 2025 • 18

MaziyarPanahi/DeepSeek-R1-Distill-Llama-8B-GGUF

Text Generation • 8B • Updated Jan 24, 2025 • 100