Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

2,890

Base only

Active filters: 5-bit

dmanary-pronavigator/gemma-2-27b-it-exl2-5.0bpw

Text Generation • Updated Jul 7, 2024 • 6

MaziyarPanahi/RYS-XLarge-GGUF

Text Generation • 78B • Updated Aug 6, 2024 • 314 • 7

MaziyarPanahi/calme-2.3-legalkit-8b-GGUF

Text Generation • 8B • Updated Aug 7, 2024 • 1.08k • 8

denru/Tess-3-Mistral-Large-2-123B-5_0bpw-h8-exl2-pippa

Updated Aug 8, 2024 • 5 • 1

MaziyarPanahi/Qwen2-Math-1.5B-Instruct-GGUF

Text Generation • 2B • Updated Aug 8, 2024 • 285 • 2

MaziyarPanahi/Qwen2-Math-7B-Instruct-GGUF

Text Generation • 8B • Updated Aug 8, 2024 • 562 • 1

MaziyarPanahi/Qwen2-Math-72B-Instruct-GGUF

Text Generation • 73B • Updated Aug 20, 2024 • 762 • 2

SicariusSicariiStuff/2B_or_not_2B-EXL2-5.0bpw

Updated Aug 11, 2024 • 3

blockblockblock/NemoReRemix-12B-bpw5-exl2

Text Generation • Updated Aug 15, 2024 • 5

LoneStriker/Hermes-3-Llama-3.1-8B-5.0bpw-h6-exl2

Updated Aug 15, 2024 • 9 • 2

LoneStriker/Hermes-3-Llama-3.1-70B-5.0bpw-h6-exl2

Updated Aug 16, 2024 • 9 • 1

HHD-QiXuan/RYS-XLarge-GGUF

Text Generation • 78B • Updated Aug 18, 2024 • 86 • 1

SicariusSicariiStuff/Dusk_Rainbow-EXL2-5.0-bpw

Updated Aug 17, 2024 • 6

MaziyarPanahi/SmolLM-1.7B-Instruct-v0.2-GGUF

Text Generation • 2B • Updated Aug 18, 2024 • 806 • 7

Proverbial1/magnum-v2-123b_exl2_5.0bpw_h8

Text Generation • Updated Aug 20, 2024 • 10

SicariusSicariiStuff/KoboldAI_LLaMA2-13B-Psyfighter2-EXL2-5.0bpw

Updated Aug 20, 2024 • 5

SicariusSicariiStuff/PygmalionAI_pygmalion-7b-EXL2-5.0bpw

Text Generation • Updated Aug 20, 2024 • 6

MaziyarPanahi/Phi-3.5-mini-instruct-GGUF

Text Generation • 4B • Updated Aug 20, 2024 • 178k • 33

LoneStriker/Phi-3.5-mini-instruct-5.0bpw-h6-exl2

Text Generation • Updated Aug 20, 2024 • 6

drexample/magnum-v2-123b-exl2-5.0bpw

Text Generation • Updated Aug 21, 2024 • 10 • 1

jburmeister/Meta-Llama-3.1-8B-Instruct-GGUF

Text Generation • 8B • Updated Aug 22, 2024 • 125

jburmeister/Meta-Llama-3.1-70B-Instruct-GGUF

Text Generation • 71B • Updated Aug 22, 2024 • 199

jburmeister/Meta-Llama-3-8B-Instruct-GGUF

Text Generation • 8B • Updated Aug 22, 2024 • 321

BigHuggyD/anthracite-org_magnum-v2-123b_exl2_5.0bpw_h6

Text Generation • Updated Aug 22, 2024 • 7

SicariusSicariiStuff/Phi-3.5-mini-instruct_Uncensored-EXL2-5.0bpw

Text Generation • Updated Jul 25, 2025 • 5

MaziyarPanahi/calme-2.1-phi3.5-4b-GGUF

Text Generation • 4B • Updated Aug 23, 2024 • 328 • 1

bullerwins/Hermes-3-Llama-3.1-8B-exl2_5.0bpw

Updated Aug 23, 2024 • 6

SicariusSicariiStuff/2B-ad-EXL2-5.0bpw

Text Generation • Updated Jul 25, 2025 • 14

MaziyarPanahi/xLAM-8x22b-r-GGUF

Text Generation • 141B • Updated Sep 6, 2024 • 1.71k • 2

pigfoot/Llama-3-Taiwan-8B-Instruct-V1-5bpw-exl2

Text Generation • Updated Aug 30, 2024 • 7