Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

4,543

Base only

Active filters: 6-bit

mlx-community/OLMoE-1B-7B-0125-Instruct-6bit

Text Generation • 2B • Updated Mar 4, 2025 • 18

alexgusevski/Phi-4-mini-instruct-Q6-mlx

Text Generation • 0.8B • Updated Mar 4, 2025 • 11

alexgusevski/Fallen-Llama-3.3-R1-70B-v1-Q6-mlx

15B • Updated Mar 5, 2025 • 43

MaziyarPanahi/granite-3.2-8b-instruct-abliterated-GGUF

Text Generation • 8B • Updated Mar 5, 2025 • 81

MaziyarPanahi/BeyondRP-8B-GGUF

Text Generation • 8B • Updated Mar 5, 2025 • 80

MaziyarPanahi/granite-3.2-2b-instruct-abliterated-GGUF

Text Generation • 3B • Updated Mar 5, 2025 • 99

mlx-community/QwQ-32B-6bit

Text Generation • 7B • Updated Mar 5, 2025 • 28 • 3

mlx-community/OLMoE-1B-7B-0125-6bit

Text Generation • 2B • Updated Mar 5, 2025 • 22

alexgusevski/QwQ-32B-6bit

Text Generation • 7B • Updated Mar 5, 2025 • 7

SlerpE/QwQ-32B-exl2-6.0bpw-h8

Text Generation • Updated Mar 6, 2025 • 10

cshared/Qwen-QwQ-32B-6.0bpw-exl2

Text Generation • Updated Mar 6, 2025 • 10

Dracones/QwQ-32B_exl2_6.0bpw

Text Generation • Updated Mar 6, 2025 • 10 • 1

LoneStriker/QwQ-32B-6.0bpw-h6-exl2

Text Generation • Updated Mar 6, 2025 • 8 • 2

mlx-community/Dolphin3.0-R1-Mistral-24B-6bit

Text Generation • 5B • Updated Mar 6, 2025 • 106

MaziyarPanahi/QwQ-32B-GGUF

Text Generation • 33B • Updated Mar 7, 2025 • 177k • 4

mlx-community/gemma-2-9b-it-6bit

Text Generation • 2B • Updated Mar 6, 2025 • 25 • 1

alexgusevski/Goppa-LogiLlama-6bit

Text Generation • 0.3B • Updated Mar 6, 2025 • 9

KYUNGYONG/DNA-R1-6bit

Text Generation • 3B • Updated Mar 8, 2025 • 11

MaziyarPanahi/Qwen-1M-DeepSeek-R1-14B-GGUF

Text Generation • 15B • Updated Mar 8, 2025 • 111 • 1

MaziyarPanahi/EmojiLlama-3.1-8B-GGUF

Text Generation • 8B • Updated Mar 8, 2025 • 101

MaziyarPanahi/Messier-Opus-14B-Elite12-GGUF

Text Generation • 15B • Updated Mar 8, 2025 • 50

MaziyarPanahi/Goppa-LogiLlama-GGUF

Text Generation • 1B • Updated Mar 8, 2025 • 55

MaziyarPanahi/Glowing-Forest-12B-GGUF

Text Generation • 12B • Updated Mar 8, 2025 • 48 • 1

MaziyarPanahi/MT-Gen8-gemma-2-9B-GGUF

Text Generation • 9B • Updated Mar 9, 2025 • 44

FluffyKaeloky/QwQ-RP-exl2-6.0bpw

Updated Mar 9, 2025 • 5

MaziyarPanahi/MT3-Gen8-gemma-2-9B-GGUF

Text Generation • 9B • Updated Mar 9, 2025 • 49

MaziyarPanahi/gemma-2-2b-fa-v2-GGUF

Text Generation • 3B • Updated Mar 9, 2025 • 44 • 1

MaziyarPanahi/tempestissimo-14b-0309-GGUF

Text Generation • 15B • Updated Mar 9, 2025 • 57

MaziyarPanahi/Hush-Qwen2.5-7B-v1.3-GGUF

Text Generation • 8B • Updated Mar 9, 2025 • 58

MaziyarPanahi/Hush-Qwen2.5-7B-RP-v1.4-1M-GGUF

Text Generation • 8B • Updated Mar 9, 2025 • 82