Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

32,642

Base only

Active filters: imatrix

mmnga/Llama-3.1-70B-Instruct-gguf

71B • Updated Jul 24, 2024 • 1.53k • 2

duyntnet/Mistral-Nemo-Instruct-2407-imatrix-GGUF

Text Generation • 12B • Updated Jul 24, 2024 • 746

nisten/meta-405b-instruct-cpu-optimized-gguf

410B • Updated Jul 24, 2024 • 350 • 40

legraphista/Llama-Guard-3-8B-IMat-GGUF

Text Generation • 8B • Updated Jul 23, 2024 • 896 • 5

MugenYume/phi-3.1-abliterated-Q4_K_M-GGUF

4B • Updated Jul 23, 2024 • 4

MugenYume/phi-3.1-abliterated-Q5_K_M-GGUF

4B • Updated Jul 23, 2024 • 8

MugenYume/phi-3.1-abliterated-Q5_K_S-GGUF

4B • Updated Jul 23, 2024 • 10

backyardai/Meta-Llama-3.1-70B-Instruct-GGUF

Text Generation • 71B • Updated Jul 24, 2024 • 260 • 3

MugenYume/Meta-Llama-3.1-8B-Instruct-Q4_K_S-GGUF

Text Generation • 8B • Updated Jul 24, 2024 • 22

lmstudio-community/Meta-Llama-3.1-70B-Instruct-GGUF

Text Generation • 71B • Updated Jul 28, 2024 • 1.73k • 37

Reiterate3680/Hathor_Sofit-L3-8B-v1-GGUF

Text Generation • 8B • Updated Aug 5, 2024 • 100

backyardai/L3-Horizon-Anteros-v0.1-13B-GGUF

13B • Updated Jul 24, 2024 • 11 • 1

backyardai/L3-Stheno-Maid-Blackroot-Grand-HORROR-16B-GGUF

17B • Updated Jul 24, 2024 • 281 • 3

Reiterate3680/mini-magnum-12b-v1.1-GGUF

Text Generation • 12B • Updated Aug 6, 2024 • 80 • 3

AI-Engine/gemma-2-27b-it-GGUF

27B • Updated Jul 24, 2024 • 63

legraphista/Mistral-Large-Instruct-2407-IMat-GGUF

Text Generation • 123B • Updated Jul 25, 2024 • 319 • 29

bartowski/Mistral-Large-Instruct-2407-GGUF

Text Generation • 123B • Updated Aug 27, 2024 • 1.91k • 39

MaziyarPanahi/Mistral-Large-Instruct-2407-GGUF

Text Generation • 123B • Updated Jul 26, 2024 • 3.23k • 21

mradermacher/Meta-Llama-3-8B-Instruct-function-calling-i1-GGUF

8B • Updated Aug 2, 2024 • 817

mradermacher/fusion-9b-i1-GGUF

9B • Updated Aug 2, 2024 • 66

mradermacher/not-your-advisor-9b-i1-GGUF

9B • Updated Aug 2, 2024 • 42

mradermacher/Hathor_Sofit-L3-8B-v1-i1-GGUF

8B • Updated Aug 2, 2024 • 44 • 2

mmnga/Mistral-Large-Instruct-2407-gguf

123B • Updated Jul 26, 2024 • 363

mradermacher/Llama-3-8B-Stroganoff-2.0-i1-GGUF

8B • Updated Aug 2, 2024 • 192 • 1

InferenceIllusionist/Mistral-Large-Instruct-2407-iMat-GGUF

123B • Updated Aug 4, 2024 • 515 • 3

aifeifei798/Meta-Llama-3.1-8B-Instruct-Q5_K_M-GGUF

Text Generation • 8B • Updated Jul 25, 2024 • 27

mradermacher/Honey-Yuzu-13B-i1-GGUF

12B • Updated Aug 2, 2024 • 300 • 3

mradermacher/NieR-Automata-Chatbot-none-V4-Mid-i1-GGUF

7B • Updated Aug 2, 2024 • 62

mradermacher/NieR-Automata-Chatbot-none-V5-i1-GGUF

7B • Updated Feb 12, 2025 • 83 • 1

Reiterate3680/Sekhmet_Aleph-L3.1-8B-v0.1-GGUF

Text Generation • 8B • Updated Aug 5, 2024 • 20 • 1