Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,251

Base only

Active filters: int4

robotflowlabs/qwen3-vl-4b-instruct-int4

Image-Text-to-Text • 5B • Updated Mar 19 • 218

cstr/F2LLM-v2-0.6B-ONNX-INT4

Sentence Similarity • Updated 18 days ago • 14

cstr/Octen-Embedding-0.6B-ONNX-INT4-FULL

Sentence Similarity • Updated 18 days ago • 12

chankhavu/Nemotron-Cascade-2-30B-A3B-AWQ-INT4

5B • Updated Mar 21 • 201 • 5

raydelossantos/OmniCoder-9B-GPTQ-Int4

Image-Text-to-Text • 9B • Updated Mar 22 • 174 • 1

CelesteImperia/Gemma-2-2B-IT-OpenVINO-INT4

Text Generation • Updated Mar 23 • 5

CelesteImperia/Llama-3.2-1B-Instruct-OpenVINO-INT4

Text Generation • Updated Mar 24 • 6

SocialLocalMobile/Qwen3.5-35B-A3B-HQQ-INT4

Image-Text-to-Text • 17B • Updated Mar 24 • 1.97k

CelesteImperia/Llama-3.2-3B-Instruct-OpenVINO-INT4

Text Generation • Updated Mar 24 • 6

CelesteImperia/Phi-3.5-mini-instruct-OpenVINO-INT4

Text Generation • Updated Mar 24 • 7

CelesteImperia/Qwen2.5-7B-Instruct-OpenVINO-INT4

Text Generation • Updated Mar 24 • 7

INSAIT-Institute/BgGPT-Gemma-3-12B-IT-GPTQ-W4A16

Image-Text-to-Text • 4B • Updated Mar 25 • 189 • 2

INSAIT-Institute/BgGPT-Gemma-3-27B-IT-GPTQ-W4A16

Image-Text-to-Text • 7B • Updated Mar 25 • 760 • 3

ginsongsong/eagle3-kimik2.5-w4a8

2B • Updated Mar 25 • 10

CelesteImperia/Mistral-Nemo-12B-Instruct-OpenVINO-INT4

Text Generation • Updated Mar 25 • 13

CelesteImperia/Qwen2.5-Coder-7B-Instruct-OpenVINO-INT4

Text Generation • Updated Mar 25 • 8

CelesteImperia/Qwen2-VL-2B-Instruct-OpenVINO-INT4-v2

Image-Text-to-Text • Updated Mar 25 • 7

CelesteImperia/MiniCPM-V-2.6-OpenVINO-INT4

Image-Text-to-Text • Updated Mar 25 • 12

happypatrick/Qwen3.5-397B-A17B-heretic-int4-AutoRound

Text Generation • Updated Mar 26 • 706 • 4

tonera/Nepotism_xii-Nunchaku

Text-to-Image • 12B • Updated Mar 27 • 20 • 1

ghecko78/Voxtral-Small-24B-2507-W4A16

Automatic Speech Recognition • 5B • Updated Mar 27 • 1.04k • 1

lbm364dl/PaddleOCR-VL-1.5-ONNX

Image-to-Text • Updated Mar 27 • 20 • 5

tonera/Qwen2.5vl-Nunchaku

Updated Apr 9 • 137 • 3

elbruno/Qwen2.5-0.5B-LocalLLMs-ToolCalling

aufklarer/Qwen3.5-0.8B-Chat-MLX

Text Generation • Updated 21 days ago • 1.16k • 1

CCSSNE/happypatrick-Qwen3.5-397B-A17B-heretic-int4-AutoRound

Text Generation • Updated Mar 29 • 12

tonera/Qwen3-text-Nunchaku

Updated Apr 9 • 435 • 9

Azaz666/Qwen2.5-VL-3B-Instruct-AWQ-INT4

4B • Updated Mar 30 • 11

mconcat/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-v2-AWQ-4bit

Text Generation • 29B • Updated Mar 30 • 1.43k • 2

caiovicentino1/Qwen3.5-9B-HLWQ-Q5

Text Generation • 9B • Updated Apr 13 • 21 • 3