Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

2,892

Base only

Active filters: 5-bit

LoneStriker/LongWriter-llama3.1-8b-5.0bpw-h6-exl2

Text Generation • Updated Aug 30, 2024 • 7

bullerwins/c4ai-command-r-08-2024-exl2_5.0bpw

Text Generation • Updated Aug 31, 2024 • 12 • 1

Proverbial1/Luminum-v0.1-123B_exl2_5.0bpw_h8

Text Generation • Updated Sep 1, 2024 • 6

BigHuggyD/c4ai-command-r-plus-08-2024_exl2_5.0bpw_h6

Text Generation • Updated Sep 1, 2024 • 7

bullerwins/c4ai-command-r-plus-08-2024-exl2_5.0bpw

Text Generation • Updated Sep 2, 2024 • 7

bigstorm/Mistral-Large-Instruct-2407-5.0bpw-8hb-exl2

Updated Sep 4, 2024 • 6

BigHuggyD/FluffyKaeloky_Luminum-v0.1-123B_exl2_5.0bpw_h6

Text Generation • Updated Sep 7, 2024 • 39

LoneStriker/Yi-Coder-9B-Chat-5.0bpw-h6-exl2

Text Generation • Updated Sep 4, 2024 • 6

LoneStriker/Yi-Coder-9B-5.0bpw-h6-exl2

Updated Sep 4, 2024 • 5

bullerwins/Reflection-70B-exl2_5.0bpw

Text Generation • Updated Sep 6, 2024 • 7 • 1

LoneStriker/Reflection-Llama-3.1-70B-5.0bpw-h6-exl2

Text Generation • Updated Sep 6, 2024 • 7

MaziyarPanahi/DeepSeek-V2.5-GGUF

Text Generation • 236B • Updated Sep 11, 2024 • 752 • 3

async0x42/Meta-Llama-3.1-8B-exl2_5bpw

Text Generation • Updated Sep 8, 2024 • 5

schnapper79/lumikabra-123B_v0.4-exl2-5.0bpw

Text Generation • Updated Sep 9, 2024 • 7

riveRiPH/MN-12B-Lyra-v4-5bpw-h6-exl2

Updated Sep 9, 2024 • 6 • 1

MaziyarPanahi/ref_70_e3-GGUF

Text Generation • 71B • Updated Sep 10, 2024 • 644

schnapper79/lumikabra-123B_v0.1-exl2-5.0bpw

Text Generation • Updated Sep 10, 2024 • 8

schnapper79/lumikabra-123B_v0.3-exl2-5.0bpw

Text Generation • Updated Sep 10, 2024 • 7

async0x42/Merged-RP-Stew-V2-34B-exl2_5bpw

Updated Sep 10, 2024 • 3

MaziyarPanahi/reader-lm-0.5b-GGUF

Text Generation • 0.5B • Updated Sep 11, 2024 • 525 • 4

MaziyarPanahi/reader-lm-1.5b-GGUF

Text Generation • 2B • Updated Sep 11, 2024 • 998 • 3

SicariusSicariiStuff/ZeusLabs_Chronos-Divergence-33B-EXL2-5.0bpw

Updated Sep 11, 2024 • 9

MaziyarPanahi/solar-pro-preview-instruct-GGUF

Text Generation • 22B • Updated Sep 13, 2024 • 175k • 29

LoneStriker/Mistral-Small-Instruct-2409-5.0bpw-h6-exl2

Updated Sep 17, 2024 • 9 • 1

MaziyarPanahi/Mistral-Small-Instruct-2409-GGUF

Text Generation • 22B • Updated Jan 1, 2025 • 173k • 4

MaziyarPanahi/Qwen2.5-0.5B-Instruct-GGUF

Text Generation • 0.5B • Updated Sep 18, 2024 • 909 • 2

MaziyarPanahi/Qwen2.5-1.5B-Instruct-GGUF

Text Generation • 2B • Updated Sep 18, 2024 • 175k • 12

MaziyarPanahi/Qwen2.5-3B-Instruct-GGUF

Text Generation • 3B • Updated Sep 18, 2024 • 1.03k • 2

MaziyarPanahi/Qwen2.5-7B-Instruct-GGUF

Text Generation • 8B • Updated Sep 18, 2024 • 175k • 11

MaziyarPanahi/Qwen2.5-14B-Instruct-GGUF

Text Generation • 15B • Updated Sep 18, 2024 • 481 • 2