Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

2,533

Base only

Active filters: quantization

tg-techie-agents/Q1D-4B-Blk0thru3-V2

Text Generation • 4B • Updated Jun 19 • 13

tg-techie-agents/Q1D-4B-Blk0thru3-V3

Text Generation • 4B • Updated Jun 19 • 8

wepiqx/Qwable-9B-Claude-Fable-5-SHQ8-GGUF

Text Generation • 9B • Updated Jun 23 • 492 • 5

tonera/Qwen-Image-2512-Lightning-Nunchaku

Text-to-Image • 20B • Updated Jun 23 • 65 • 2

kongbai-84/animagguf-quantizer

Updated Jun 20 • 2

phptop/cohere-transcribe-03-2026-int8

Automatic Speech Recognition • Updated Jun 21

RESMP-DEV/MiMo-V2.5-ASR-FP8

Automatic Speech Recognition • 8B • Updated 9 days ago • 14

barryke/granite-4.1-8b-FP8-DYNAMIC

Text Generation • 9B • Updated Jun 21 • 6

sheikhmufrad/omniASR-LLM-7B-v2-quantized

Automatic Speech Recognition • Updated Jun 22

alavicka/pollux-1152

Text Generation • Updated Jul 2

alavicka/pollux-1920

Text Generation • Updated Jul 2

FloatinggOnion/mobile-llm-quant-demo

4B • Updated Jun 21 • 28

petergilani/Qwen3.6-27B-MTP-4bit

Text Generation • 27B • Updated Jun 22 • 28

petergilani/Qwen3.6-27B-MTP-8bit

Text Generation • 8B • Updated about 15 hours ago • 74

donghyunli/Llama-2-7b-KronQ-W4A16

Text Generation • 4B • Updated 23 days ago

donghyunli/Llama-2-7b-KronQ-W2A16

Text Generation • 2B • Updated 23 days ago • 1

donghyunli/Meta-Llama-3-8B-KronQ-W4A16

Text Generation • 5B • Updated Jul 13

donghyunli/Llama-2-7b-KronQ-W3A16-fake

Text Generation • 7B • Updated 23 days ago • 230

donghyunli/Meta-Llama-3-8B-KronQ-W3A16-fake

Text Generation • 8B • Updated Jul 14 • 175

donghyunli/Meta-Llama-3-8B-KronQ-W2A16

Text Generation • 3B • Updated Jul 14

donghyunli/Llama-2-13b-KronQ-W4A16

Text Generation • 7B • Updated Jul 14

donghyunli/Llama-2-13b-KronQ-W2A16

Text Generation • 4B • Updated Jul 14

donghyunli/Llama-2-13b-KronQ-W3A16-fake

Text Generation • 13B • Updated Jul 14 • 149

wepiqx/Qwythos-9B-Claude-Mythos-5-1M-SHQ8-GGUF

Text Generation • 9B • Updated Jun 23 • 157 • 6

jupitersong/madi-whisper-turbo-v3-q8

Automatic Speech Recognition • Updated Jun 22 • 1

cod-tdq/cod-tdq

Updated Jun 22 • 1

syseeker-at-nv/nitrogen-quant

wepiqx/Qwythos-9B-Claude-Mythos-5-1M-MTP-SHQ8-GGUF

Text Generation • 9B • Updated Jun 23 • 434 • 6

wenshutang/ternlight

Sentence Similarity • Updated Jul 9 • 7

donghyunli/Llama-2-70b-KronQ-W4A16

Text Generation • 35B • Updated Jul 14