Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

506

Base only

Active filters: llm-compressor

JongYeop/Qwen3-30B-A3B-FP8-W8A8

Text Generation • 31B • Updated Jun 6 • 12

JongYeop/Qwen3-30B-A3B-NVFP4-W4A4

Text Generation • 17B • Updated Jun 7 • 39

soyrsoyr/Llama-3.2-1B-Instruct-W8A8-GPTQ

Text Generation • 1B • Updated Jun 7 • 15

soyrsoyr/Llama-3.2-1B-Instruct-FP8-GPTQ

Text Generation • 1B • Updated Jun 7 • 8

soyrsoyr/Llama-3.2-1B-Instruct-W4A16-GPTQ

Text Generation • 1B • Updated Jun 7 • 6

soyrsoyr/Llama-3.2-1B-Instruct-NVFP4-GPTQ

Text Generation • 0.8B • Updated Jun 7 • 10

sakamakismile/BLS-Mini-Code-1.0-NVFP4

Text Generation • 17B • Updated Jun 8 • 14 • 1

JongYeop/MegaBeam-Mistral-7B-512k-FP8-W8A8

Text Generation • 7B • Updated Jun 8 • 282

barryke/rnj-1-instruct-FP8-DYNAMIC

Text Generation • 9B • Updated Jun 10 • 18

RedHatAI/gemma-4-12B-it-NVFP4

Any-to-Any • 13B • Updated 5 days ago • 263k • 4

soyrsoyr/deepseek-moe-16b-chat-W8A8-GPTQ

Text Generation • 16B • Updated Jun 8 • 12

soyrsoyr/deepseek-moe-16b-chat-W4A16-GPTQ

Text Generation • 3B • Updated Jun 8 • 9

soyrsoyr/deepseek-moe-16b-chat-FP8-GPTQ

Text Generation • 16B • Updated Jun 8 • 4

soyrsoyr/deepseek-moe-16b-chat-NVFP4-GPTQ

Text Generation • 9B • Updated Jun 8 • 11

GulfCoastAI/Mistral-Small-4-119B-Heretic-NVFP4

68B • Updated Jun 9 • 472 • 4

palmfuture/Nex-N2-mini-NVFP4A16

Text Generation • 21B • Updated Jun 9 • 112

RedHatAI/diffusiongemma-26B-A4B-it-NVFP4

Image-Text-to-Text • 26B • Updated 11 days ago • 83.2k • 19

RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic

Image-Text-to-Text • 26B • Updated 5 days ago • 26.4k • 11

RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4

Text Generation • 303B • Updated 11 days ago • 173

sch0tten/Qwen3.6-35B-A3B-research-FP8

Text Generation • 35B • Updated Jun 12

kumar2235/Qwen3.5-4B-AWQ

Text Generation • 4B • Updated Jun 17 • 475 • 3

flukethoughts/Qwen-Image-Bench-NVFP4

Image-Text-to-Text • 17B • Updated Jun 15 • 14

amd/Mistral-7B-Instruct-v0.3-w4a16-llmcompressor-v0.10.0.2

Text Generation • 7B • Updated Jun 17 • 7

pavanperi/Voxtral-Mini-4B-Realtime-2602_W4A16_G128

Automatic Speech Recognition • 4B • Updated Jun 15 • 52

sakamakismile/Huihui-Nex-N2-mini-abliterated-MTP-NVFP4

Image-Text-to-Text • 20B • Updated Jun 16 • 204 • 6

sakamakismile/Huihui-Nex-N2-mini-abliterated-text-MTP-NVFP4

Text Generation • 20B • Updated Jun 16 • 11

sahilchachra/Qwable-v1-NVFP4A16

Text Generation • 22B • Updated Jun 17 • 493

BoneMangler/Qwen3-Coder-42B-A3B-Instruct-TOTAL-RECALL-MASTER-CODER-M-512k-ctx-W4A16-Q8_0-GGUF

Text Generation • 42B • Updated Jun 17 • 17

amd/Llama-3.3-70B-Instruct-w4a16-llmcompressor-v0.11.0

Text Generation • 11B • Updated Jun 17 • 332 • 1

BoneMangler/Qwen3-Coder-42B-A3B-Instruct-TOTAL-RECALL-MASTER-CODER-M-512k-ctx-W4A16-Q6_K-GGUF

Text Generation • 42B • Updated Jun 17 • 46