Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

463

Base only

Active filters: llmcompressor

ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-mxfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 14

mratsim/Seed-OSS-36B-Instruct-FP8-KV8

Text Generation • 36B • Updated Oct 19, 2025 • 3 • 1

devnagriai/snorTTS-Indic-v0-AWQ-W4A16

Text-to-Speech • 4B • Updated Nov 6, 2025 • 6

kaitchup/Qwen3-4B-Instruct-2507-awq-w4a16-asym

4B • Updated Dec 20, 2025 • 10 • 1

kaitchup/Qwen3-4B-Instruct-2507-fp8-dynamic

4B • Updated Nov 13, 2025 • 4

kaitchup/Qwen3-4B-Instruct-2507-gptq-w4a16-g128

4B • Updated Nov 13, 2025 • 9 • 1

kaitchup/Qwen3-4B-Instruct-2507-NVFP4

3B • Updated Dec 25, 2025 • 21

kaitchup/Qwen3-4B-Instruct-2507-w8a8-smoothquant

4B • Updated Nov 13, 2025 • 10

kaitchup/Qwen3-4B-Instruct-2507-autoround-w4a16-g128

4B • Updated Nov 14, 2025 • 6

kaitchup/Qwen3-4B-Thinking-2507-autoround-w4a16

4B • Updated Nov 14, 2025 • 4

kaitchup/Qwen3-4B-Thinking-2507-awq-w4a16-asym

4B • Updated Nov 16, 2025 • 15

kaitchup/Qwen3-4B-Thinking-2507-gptq-w4a16-g128

4B • Updated Nov 15, 2025 • 10

kaitchup/Qwen3-4B-Thinking-2507-fp8-dynamic

4B • Updated Nov 15, 2025 • 16

kaitchup/Qwen3-4B-Thinking-2507-NVFP4

3B • Updated Nov 16, 2025 • 78

kaitchup/Qwen3-4B-Thinking-2507-w8a8-smoothquant

4B • Updated Nov 16, 2025 • 12

kaitchup/Olmo-3-7B-Instruct-NVFP4

4B • Updated Dec 16, 2025 • 8

kaitchup/Olmo-3-7B-Instruct-gptq-w4a16-g128

7B • Updated Dec 16, 2025 • 6

kaitchup/Olmo-3-7B-Instruct-fp8-dynamic

7B • Updated Dec 16, 2025 • 21

kaitchup/Olmo-3-32B-Think-fp8-dynamic

32B • Updated Dec 16, 2025 • 15

kaitchup/Olmo-3-32B-Think-NVFP4

19B • Updated Dec 16, 2025 • 6

kaitchup/Olmo-3-32B-Think-gptq-w4a16-g128

5B • Updated Dec 16, 2025 • 11 • 1

bbarn4/Augmentoolkit-DataSpecialist-v0.1-AWQ

Text Generation • 7B • Updated Nov 23, 2025 • 5

TevunahAi/granite-8b-code-instruct-4k-FP8

Text Generation • 8B • Updated Dec 8, 2025 • 14

TevunahAi/granite-20b-code-instruct-8k-FP8

Text Generation • 20B • Updated Dec 8, 2025 • 8

TevunahAi/granite-34b-code-instruct-8k-FP8

Text Generation • 34B • Updated Dec 8, 2025 • 5

TevunahAi/Qwen3-Next-80B-A3B-Instruct-FP8

Text Generation • 80B • Updated Dec 8, 2025 • 19

TevunahAi/Apertus-8B-Instruct-2509-FP8

Text Generation • 8B • Updated Dec 8, 2025 • 6

Saktsant/DeepSeek-R1-Distill-Llama-8B-quantized.w4a16

Text Generation • 2B • Updated Dec 23, 2025 • 5

TevunahAi/Apertus-70B-Instruct-2509-2048-Calibration-FP8

Text Generation • 71B • Updated Dec 8, 2025 • 6

TevunahAi/gpt-oss-20b-2048-Calibration-FP8

Text Generation • 21B • Updated Dec 8, 2025 • 14 • 1