Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

2,892

Base only

Active filters: 5-bit

MaziyarPanahi/Qwen2.5-32B-Instruct-GGUF

Text Generation • 33B • Updated Sep 19, 2024 • 1.09k • 1

LoneStriker/Qwen2.5-7B-Instruct-5.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 14

LoneStriker/Qwen2.5-14B-Instruct-5.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 9

LoneStriker/Qwen2.5-32B-Instruct-5.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 15

Zoyd/Qwen_Qwen2.5-14B-Instruct-5_0bpw_exl2

Text Generation • Updated Sep 19, 2024 • 7

DrNicefellow/Qwen2.5-32B-Instruct-5.0bpw-exl2

Updated Sep 19, 2024 • 9

LoneStriker/Qwen2.5-72B-Instruct-5.0bpw-h6-exl2

Text Generation • Updated Sep 19, 2024 • 9 • 1

NobodySpecial/Qwen2.5-72B-exl2-5.0bpw

Text Generation • Updated Sep 20, 2024 • 7 • 2

DrNicefellow/Qwen2.5-7B-Instruct-5.0bpw-exl2

Updated Sep 20, 2024 • 8

DrNicefellow/Qwen2.5-14B-Instruct-5.0bpw-exl2

Updated Sep 21, 2024 • 6

DrNicefellow/Qwen2.5-Coder-7B-Instruct-5.0bpw-exl2

Updated Sep 21, 2024 • 7

DrNicefellow/Qwen2.5-Math-7B-Instruct-5.0bpw-exl2

Updated Sep 21, 2024 • 8

MaziyarPanahi/Llama-3.2-3B-Instruct-GGUF

Text Generation • 3B • Updated Sep 25, 2024 • 175k • 15

MaziyarPanahi/Llama-3.2-1B-Instruct-GGUF

Text Generation • 1B • Updated Sep 25, 2024 • 177k • 18

MikeRoz/ArliAI_Llama-3.1-70B-ArliAI-RPMax-v1.1-5.0bpw-h6-exl2

Updated Sep 27, 2024 • 4

BigHuggyD/gghfez_SmartMaid-123b_exl2_5.0bpw_h6

Updated Sep 28, 2024 • 10

riveRiPH/Lyra4-Gutenberg2-12B-5bpw-h6-exl2

Text Generation • Updated Sep 30, 2024 • 8

SicariusSicariiStuff/Impish_LLAMA_3B-EXL2-5.0bpw

Text Generation • Updated Jul 26, 2025 • 9

altomek/Llama-3.1-70B-Instruct-5bpw-EXL2

Text Generation • Updated Oct 10, 2024 • 6

MikeRoz/TheDrummer_Behemoth-123B-v1-5.0bpw-h6-exl2

Updated Oct 10, 2024 • 4 • 3

schnapper79/lumikabra-behemoth-123b-exl2-5.0bpw

Text Generation • Updated Oct 11, 2024 • 6

BigHuggyD/TheDrummer_Behemoth-123B-v1_exl2_5.0bpw_h6

Updated Oct 11, 2024 • 4

SicariusSicariiStuff/LLAMA-3_8B_Unaligned_BETA_EXL2-5.0-bpw

Text Generation • Updated Jul 26, 2025 • 11

Ya211/llamaEXL

Text Generation • Updated Oct 13, 2024 • 10

schnapper79/lumikabra_behemoth_195b-exl2-5.0bpw

Text Generation • Updated Oct 14, 2024 • 5

schnapper79/lumikabra_behemoth_195B_v2-exl2-5.0bpw

Text Generation • Updated Oct 16, 2024 • 4 • 1

MaziyarPanahi/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF

Text Generation • 71B • Updated Oct 17, 2024 • 917 • 3

softwareweaver/Twilight-Large-123B-EXL2-5bpw

Text Generation • Updated Oct 17, 2024 • 8 • 1

MaziyarPanahi/Meraj-Mini-GGUF

Text Generation • 8B • Updated Oct 17, 2024 • 683 • 2

BigHuggyD/schnapper79_lumikabra-195B_v0.3_exl2_5.0bpw_h6

Text Generation • Updated Oct 18, 2024 • 7