Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

25,589

Base only

Active filters: llama-cpp

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q3_K_S-GGUF

Text Generation • 4B • Updated May 3, 2024 • 25

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q3_K_M-GGUF

Text Generation • 4B • Updated May 3, 2024 • 47

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q3_K_L-GGUF

Text Generation • 4B • Updated May 3, 2024 • 29

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q4_0-GGUF

Text Generation • 4B • Updated May 3, 2024 • 35

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q4_K_S-GGUF

Text Generation • 4B • Updated May 3, 2024 • 36

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q4_K_M-GGUF

Text Generation • 4B • Updated May 3, 2024 • 44

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q5_0-GGUF

Text Generation • 4B • Updated May 3, 2024 • 27

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q5_K_S-GGUF

Text Generation • 4B • Updated May 3, 2024 • 23

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q5_K_M-GGUF

Text Generation • 4B • Updated May 3, 2024 • 31

igorbkz/gpt2-Q8_0-GGUF

0.2B • Updated May 4, 2024 • 263 • 3

alexcovo/codegemma-1.1-7b-it-Q4_K_M-GGUF

Text Generation • 9B • Updated May 4, 2024 • 11

alexcovo/codegemma-1.1-7b-it-Q5_K_M-GGUF

Text Generation • 9B • Updated May 4, 2024 • 12

alexcovo/Llama3-ChatQA-1.5-8B-256K-Q4_K_M-GGUF

Text Generation • 8B • Updated May 4, 2024 • 20

JoyboyBrian/gemma-2b-Q4_0-GGUF

3B • Updated May 4, 2024 • 22 • 1

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q6_K-GGUF

Text Generation • 4B • Updated May 4, 2024 • 23

AbhishekVenkatachalam06/Phi-3-mini-128k-instruct-Q8_0-GGUF

Text Generation • 4B • Updated May 4, 2024 • 23

NikolayKozloff/Llama-3-8B-Instruct-Coder-Q8_0-GGUF

8B • Updated May 4, 2024 • 28 • 5

jefferylovely/EQMaven-Q4_K_M-GGUF

7B • Updated Jun 12, 2025 • 9

JoyboyBrian/Phi-3-mini-128k-instruct-Q4_0-GGUF

Text Generation • 4B • Updated May 4, 2024 • 29

tsharish/Mistral-7B-Inst-v0.2-pubmed-1k_full-Q4_K_M-GGUF

7B • Updated May 4, 2024 • 6

DevsDoCode/LLama-3-8b-Uncensored-Q3_K_S-GGUF

8B • Updated May 6, 2024 • 138 • 2

codegood/phi-2-Q4_K_M-GGUF

Text Generation • 3B • Updated May 5, 2024 • 61 • 3

hus960/IceCoffeeRP-7b-Q4_K_M-GGUF

7B • Updated May 5, 2024 • 2

hus960/WestIceLemonTeaRP-32k-7b-Q4_K_M-GGUF

7B • Updated May 5, 2024 • 5

hus960/Eris-LelantaclesV2-7b-Q4_K_M-GGUF

7B • Updated May 5, 2024 • 21 • 1

north/llama3-verify-Q8_0-GGUF

8B • Updated May 5, 2024 • 11

iricardoxd/Meta-Llama-3-8B-Instruct-Q4_K_M-GGUF

Text Generation • 8B • Updated May 5, 2024 • 37

iricardoxd/Meta-Llama-3-8B-Q2_K-GGUF

Text Generation • 8B • Updated May 5, 2024 • 20

DevWorld/Gemago-2b-Q4_K_M-GGUF

Translation • 3B • Updated May 20, 2024 • 39

DataSoul/Dark-Miqu-70B-Q4_K_S-GGUF

69B • Updated May 5, 2024 • 11 • 1