Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

4,570

Base only

Active filters: 6-bit

LoneStriker/Qwen2.5-14B-Instruct-6.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 11

LoneStriker/Qwen2.5-32B-Instruct-6.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 11 • 1

Zoyd/Qwen_Qwen2.5-14B-Instruct-6_0bpw_exl2

Text Generation • Updated Sep 19, 2024 • 7

LoneStriker/Qwen2.5-72B-Instruct-6.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 9 • 2

DrNicefellow/Qwen2.5-32B-Instruct-6.0bpw-exl2

Updated Sep 19, 2024 • 9

DrNicefellow/Qwen2.5-7B-Instruct-6.0bpw-exl2

Updated Sep 20, 2024 • 9

DrNicefellow/Qwen2.5-14B-Instruct-6.0bpw-exl2

Updated Sep 20, 2024 • 5

bigstorm/Qwen2.5-72B-Instruct-6.0bpw-6hb-exl2

Text Generation • Updated Sep 20, 2024 • 9

DrNicefellow/Qwen2.5-Coder-7B-Instruct-6.0bpw-exl2

Updated Sep 21, 2024 • 8

DrNicefellow/Qwen2.5-Math-7B-Instruct-6.0bpw-exl2

Updated Sep 21, 2024 • 8

bluuwhale/SeminalRP-22b-6.0bpw-h8-exl2-rpcal

Updated Sep 22, 2024 • 9

MaziyarPanahi/Llama-3.2-3B-Instruct-GGUF

Text Generation • 3B • Updated Sep 25, 2024 • 174k • 15

MaziyarPanahi/Llama-3.2-1B-Instruct-GGUF

Text Generation • 1B • Updated Sep 25, 2024 • 177k • 18

Statuo/MS-Cydonian-Gutenberg-22b-6bpw

Text Generation • Updated Sep 26, 2024 • 7 • 1

MikeRoz/ArliAI_Mistral-Small-22B-ArliAI-RPMax-v1.1-6.0bpw-h6-exl2

Updated Sep 26, 2024 • 8 • 37

Statuo/MS-ArliAI-RPMax-22b-6bpw

Updated Sep 26, 2024 • 4

NeoChen1024/gemma-2-9B-it-advanced-v2.1-exl2-6.0bpw-h8

Text Generation • Updated Sep 27, 2024 • 12 • 2

NeoChen1024/gemma-2-9b-it-abliterated-exl2-6.0bpw-h8

Updated Sep 28, 2024 • 7

BigHuggyD/gghfez_SmartMaid-123b_exl2_6.0bpw_h6

Updated Sep 28, 2024 • 8

Statuo/MS-Cydonia-V1.1-EXL2-6bpw

Updated Sep 29, 2024 • 5

riveRiPH/Lyra4-Gutenberg2-12B-6bpw-h8-exl2

Text Generation • Updated Sep 30, 2024 • 5

Jacoby746/Casual-Magnum-34B-exl2-6.0bpw

Updated Oct 3, 2024 • 6

SicariusSicariiStuff/Impish_LLAMA_3B-EXL2-6.0bpw

Text Generation • Updated Jul 26, 2025 • 13

NeoChen1024/aya-23-8B-exl2-6.0bpw-h8

Text Generation • Updated Oct 2, 2024 • 8 • 1

NeoChen1024/dolphin-2.9.3-mistral-nemo-12b-exl2-6.0bpw-h8

Updated Oct 3, 2024 • 5

Statuo/MN-12B-Vespa-x1-EXL2-6bpw

Updated Oct 5, 2024 • 3

Jellon/Mahou-Gutenberg-Nemo-12B-6bpw

Text Generation • Updated Oct 7, 2024 • 8

Statuo/MSM-MS-Cydrion-22B-EXL2-6bpw

Text Generation • Updated Oct 9, 2024 • 8

Statuo/EVA-UNIT-01_EVA-Qwen2.5-14B-v0.1-EXL2-6bpw

Updated Oct 9, 2024 • 9

Jellon/Lyra4-Gutenberg-12B-6bpw

Text Generation • Updated Oct 10, 2024 • 6