Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

24,784

Base only

Active filters: llama-cpp

TouchNight/Yi-9B-200K-AEZAKMI-RAW-TOXIC-1-Q6_K-GGUF

9B • Updated May 21, 2024 • 6 • 1

ljnicol/Phi-3-mini-128k-instruct-Q4_0-GGUF

Text Generation • 4B • Updated May 21, 2024 • 24

jeiku/Nous-Capybara-3B-V1.9-Q4_K_M-GGUF

3B • Updated May 21, 2024 • 6

Ttonio/deep-solar-Rev-v3.0.4-Q4_K_M-GGUF

11B • Updated May 21, 2024 • 9

Ttonio/gemma-2b-ko-dev-pbmt192-Q4_K_M-GGUF

Text Generation • 3B • Updated May 21, 2024 • 61

ClaudioItaly/TopEvolution16-Q5_K_M-GGUF

7B • Updated May 21, 2024 • 22

Hadiseh-Mhd/Mixtral-8x7B-Instruct-v0.1-Q4_K_M-GGUF

47B • Updated May 21, 2024 • 29

Lanerdog/Llama-3-8B-Instruct-Gradient-1048k-Q5_K_M-GGUF

Text Generation • 8B • Updated May 21, 2024 • 23 • 1

pere/llama3-8B-base-instruction2-Q8_0-GGUF

8B • Updated May 21, 2024 • 13

NikolayKozloff/phi3-128k-6b-Q8_0-GGUF

6B • Updated May 21, 2024 • 11 • 2

powermove72/longchat-7b-v1.5-32k-Q4_K_M-GGUF

7B • Updated May 21, 2024 • 4

nitus-ac/nMer1-Q8_0-GGUF

7B • Updated May 21, 2024 • 2

ssmits/Falcon2-5.5B-Spanish-GGUF

5B • Updated May 21, 2024 • 14

ssmits/Falcon2-5.5B-multilingual-GGUF

5B • Updated May 21, 2024 • 13

ssmits/Phi-3-medium-4k-instruct-Q5_K_M-GGUF

Text Generation • 14B • Updated May 21, 2024 • 24

ssmits/Phi-3-medium-4k-instruct-Q4_K_M-GGUF

Text Generation • 14B • Updated May 21, 2024 • 60 • 2

ssmits/Phi-3-medium-128k-instruct-Q4_K_M-GGUF

Text Generation • 14B • Updated May 21, 2024 • 23 • 1

ssmits/Phi-3-medium-128k-instruct-Q5_K_M-GGUF

Text Generation • 14B • Updated May 21, 2024 • 19 • 2

ssmits/Phi-3-medium-128k-instruct-Q6_K-GGUF

Text Generation • 14B • Updated May 21, 2024 • 9 • 1

ssmits/Phi-3-medium-128k-instruct-Q8_0-GGUF

Text Generation • 14B • Updated May 21, 2024 • 9 • 3

ssmits/Phi-3-medium-4k-instruct-Q8_0-GGUF

Text Generation • 14B • Updated May 21, 2024 • 20 • 1

Banished-Privateer/Phi-3-medium-128k-instruct-Q8_0-GGUF

Text Generation • 14B • Updated May 21, 2024 • 22

ssmits/Phi-3-medium-4k-instruct-Q3_K_M-GGUF

Text Generation • 14B • Updated May 21, 2024 • 15

ssmits/Phi-3-medium-128k-instruct-Q2_K-GGUF

Text Generation • 14B • Updated May 21, 2024 • 11

Ransss/Noromaid-13B-0.4-DPO-Q8_0-GGUF

13B • Updated May 21, 2024 • 3 • 1

hiddenblue1/granite-34b-code-instruct-Q4_K_M-GGUF

Text Generation • 34B • Updated May 21, 2024 • 49

NikolayKozloff/Phi-3-medium-4k-instruct-Q6_K-GGUF

Text Generation • 14B • Updated May 21, 2024 • 12 • 1

NikolayKozloff/Phi-3-medium-4k-instruct-Q4_0-GGUF

Text Generation • 14B • Updated May 21, 2024 • 13 • 1

nitus-ac/nMer1d2-Q8_0-GGUF

7B • Updated May 21, 2024 • 4

NikolayKozloff/Phi-3-medium-4k-instruct-Q5_K_S-GGUF

Text Generation • 14B • Updated May 21, 2024 • 26 • 2