Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

Misc

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

24,128

Base only

Active filters: llama-cpp

yh-yao/Phi-3-mini-4k-instruct-Q3_K_M-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 18

yh-yao/Phi-3-mini-4k-instruct-Q3_K_L-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 4

yh-yao/Phi-3-mini-4k-instruct-Q3_K_S-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 13

sosoai/Hansoldeco-Gemma-2-9b-v0.1-Q4_K_M-GGUF

9B • Updated Jul 2, 2024 • 9

yh-yao/Phi-3-mini-4k-instruct-Q4_0-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 25

yh-yao/Phi-3-mini-4k-instruct-Q4_K_M-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 7

yh-yao/Phi-3-mini-4k-instruct-Q4_K_S-GGUF

Text Generation • 4B • Updated Jul 2, 2024 • 18 • 1

sosoai/Hansoldeco-Gemma-2-9b-v0.1-Q6_K-GGUF

9B • Updated Jul 2, 2024 • 1

olileaf/Meta-Llama-3-8B-Q4_K_M-GGUF

Text Generation • 8B • Updated Jul 3, 2024 • 2

SheepHuan/Qwen2-7B-Q4_K_M-GGUF

Text Generation • 8B • Updated Jul 3, 2024 • 4

SheepHuan/Qwen2-0.5B-Instruct-Q4_K_M-GGUF

Text Generation • 0.5B • Updated Jul 3, 2024 • 3

victorbur/llama3-8B-DarkIdol-2.1-Uncensored-32K-Q3_K_M-GGUF

8B • Updated Jul 3, 2024 • 19 • 1

victorbur/llama3-8B-DarkIdol-2.1-Uncensored-32K-Q2_K-GGUF

8B • Updated Jul 3, 2024 • 24 • 1

win10/35b-beta-long-Q4_K_M-GGUF

35B • Updated Jul 3, 2024 • 3

e2jhiubyiiyvw/gemma-2-9b-it-Q5_K_M-GGUF

Text Generation • 9B • Updated Jul 3, 2024 • 26

BitterMiasma/L3-SthenoMaidBlackroot-8B-V1-Q6_K-GGUF

8B • Updated Jul 3, 2024 • 2 • 1

huggingkot/dolphin-2.9.3-mistral-7B-32k-Q4_K_M-GGUF

7B • Updated Jul 3, 2024 • 694 • 3

d-schmoltzi/Hermes-2-Pro-Mistral-7B-IQ3_XXS-GGUF

7B • Updated Jul 3, 2024 • 1

serkandyck/Kocdigital-LLM-8b-v0.1-Q4_K_M-GGUF

8B • Updated Jul 3, 2024 • 5

Danielouo/llama-2-7b-math-1000-epoch-Q4_K_M-GGUF

7B • Updated Jul 3, 2024 • 3

serkandyck/Trendyol-LLM-7b-chat-v1.8-Q4_K_M-GGUF

Text Generation • 7B • Updated Jul 3, 2024 • 21

joshnader/Phi-3-mini-4k-instruct-Q8_0-GGUF

Text Generation • 4B • Updated Jul 3, 2024 • 4 • 1

mrmage/Qwen2-1.5B-Instruct-Q4_K_M-GGUF

Text Generation • 2B • Updated Jul 8, 2024 • 29

mrmage/Qwen2-1.5B-Instruct-Q3_K_M-GGUF

Text Generation • 2B • Updated Jul 3, 2024 • 9

mrmage/Qwen2-1.5B-Instruct-Q3_K_L-GGUF

Text Generation • 2B • Updated Jul 3, 2024 • 15

e2jhiubyiiyvw/Meta-Llama-3-8B-Instruct-Q5_K_M-GGUF

Text Generation • 8B • Updated Jul 3, 2024 • 19

Kurgan1138/gemma2-9B-daybreak-v0.5-Q5_K_M-GGUF

9B • Updated Jul 3, 2024 • 3

aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF

7B • Updated Jul 3, 2024 • 1

aifeifei798/llm-compiler-13b-ftd-Q5_K_M-GGUF

13B • Updated Jul 3, 2024 • 2

aifeifei798/llm-compiler-7b-Q5_K_M-GGUF

7B • Updated Jul 3, 2024 • 7