Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,238

Base only

Active filters: int4

squishai/Qwen2.5-1.5B-Instruct-bf16-squished

0.2B • Updated Jun 25 • 22 • 1

squishai/Qwen3-4B-bf16-squished

0.6B • Updated Jun 25 • 16 • 2

squishai/Qwen3-8B-bf16-squished

1B • Updated Jun 25 • 36 • 2

squishai/Llama-3.2-3B-Instruct-bf16-squished

0.5B • Updated Jun 25 • 85 • 1

squishai/Qwen2.5-7B-Instruct-bf16-squished

1B • Updated Jun 25 • 93 • 2

squishai/gemma-3-1b-it-bf16-squished

0.2B • Updated Jun 25 • 73 • 2

squishai/gemma-3-4b-it-bf16-squished

0.7B • Updated Jun 25 • 9 • 1

palmfuture/gemma-4-12B-it-INT4-W4A16

Any-to-Any • 12B • Updated Jun 4 • 360

droyster/MisoTTS-8B-torchao-int4

Text-to-Speech • Updated Jun 4 • 24 • 4

shailesh83/gemma-4-26b-a4b-int4-ov

Image-Text-to-Text • Updated Jun 5 • 11

RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-quantized.w4a16

Text Generation • 561B • Updated 5 days ago • 798 • 3

Reza2kn/Higgs-Audio-v3-TTS-4bit-ONNX

Text-to-Speech • Updated Jun 5 • 15 • 5

PeytonT/f5tts-4bit-4step-v327

Text-to-Speech • Updated Jun 5

HarmenWessels/granite-4.1-3b-int4-cw-ov

Text Generation • Updated Jun 8 • 17

HarmenWessels/granite-4.1-8b-int4-cw-ov

Text Generation • Updated Jun 8 • 8

HarmenWessels/MiniCPM5-1B-int4-g128-ov

Text Generation • Updated Jun 8 • 13

HarmenWessels/gemma-4-E2B-it-qat-int4-ov

Image-Text-to-Text • Updated Jun 8 • 38

HarmenWessels/gemma-4-E4B-it-qat-int4-ov

Image-Text-to-Text • Updated Jun 8 • 12

JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4

Text Generation • Updated Jun 6 • 11

littlebearlabs/parakeet-unified-en-0.6b-mlx-int4

Automatic Speech Recognition • 97.7M • Updated Jun 6 • 14

Kiffaz11/ministral3-3b-reasoning-torchao-int4

Image-Text-to-Text • 0.9B • Updated Jun 6 • 5

NeoChen1024/gemma-4-26B-A4B-it-qat-W4A16

Image-Text-to-Text • 29B • Updated Jun 7 • 17.2k • 3

gitarist/Qwen3-1.7B-GPTQ-Int4-g256-sym

Text Generation • 2B • Updated Jun 7 • 10

kunhunjon/gemma-4-12B-it-qat-assistant-w4a16-ct

Text Generation • 0.4B • Updated Jun 7 • 972 • 1

mattbucci/gemma-4-12B-AWQ

Image-Text-to-Text • 12B • Updated Jun 7 • 161k • 3

kunhunjon/gemma-4-26B-A4B-it-qat-assistant-w4a16-ct

Text Generation • 0.4B • Updated Jun 8 • 206

Reza2kn/sapiens2-normal-1b-INT4-G128

Reza2kn/sapiens2-matting-1b-INT4-G128

Reza2kn/sapiens2-normal-0.4b-INT4-G128

Reza2kn/sapiens2-normal-0.8b-INT4-G128