Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,303

Base only

Active filters: 4-bit

TheBlokeAI/Mixtral-tiny-GPTQ

Text Generation • 0.2B • Updated Dec 12, 2023 • 201 • 3

TheBloke/leo-hessianai-70B-AWQ

Text Generation • 69B • Updated Dec 11, 2023 • 6

TheBloke/leo-hessianai-70B-GPTQ

Text Generation • 69B • Updated Dec 11, 2023 • 92

bienpr/Llama-2-7B-Chat-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 5

Chuanming/peft-exercise-1

Updated Dec 11, 2023 • 3

TheBloke/Mixtral-8x7B-v0.1-GPTQ

Text Generation • 47B • Updated Dec 14, 2023 • 350 • 126

minhhiepcr/quantizated_Mistral-7B-Instruct-v0.1-awq

Text Generation • 7B • Updated Dec 11, 2023 • 6

iproskurina/opt-350m-GPTQ-4bit-g128

Text Generation • 95.6M • Updated Mar 29, 2025 • 13

iproskurina/opt-1.3b-GPTQ-4bit-g128

Text Generation • 0.4B • Updated Mar 29, 2025 • 13

iproskurina/opt-2.7b-GPTQ-4bit-g128

Text Generation • 0.6B • Updated Mar 29, 2025 • 13

iproskurina/opt-6.7b-GPTQ-4bit-g128

Text Generation • 1B • Updated Mar 29, 2025 • 48

ybelkada/llava-1.5-7b-hf-awq

Image-Text-to-Text • 7B • Updated Dec 11, 2023 • 385 • 2

TheBloke/LlamaGuard-7B-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 111 • 4

marcsun13/Mixtral-tiny-GPTQ

Text Generation • 0.2B • Updated Dec 11, 2023 • 9

TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ

Text Generation • 47B • Updated Dec 14, 2023 • 236 • 141

TheBloke/LlamaGuard-7B-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 18 • 2

marcsun13/Mixtral-8x7B-v0.1-GPTQ

Text Generation • 47B • Updated Dec 11, 2023 • 11

HydraIndicLM/mistral-MoQlora-punjabi-expert

Text Generation • Updated Dec 12, 2023 • 4

HydraIndicLM/mistral-MoQlora-tamil-expert

Text Generation • Updated Dec 12, 2023 • 4

TheBloke/mixtral-8x7b-v0.1-AWQ

Text Generation • 47B • Updated Dec 22, 2023 • 211 • 11

TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ

Text Generation • 47B • Updated Dec 14, 2023 • 1.05k • 59

TheBloke/Mistral-7B-Instruct-v0.2-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 4.86k • 56

TheBloke/Mistral-7B-Instruct-v0.2-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 313k • 52

TheBloke/Clover3-17B-GPTQ

Text Generation • 17B • Updated Dec 12, 2023 • 6 • 4

TheBloke/Clover3-17B-AWQ

Text Generation • 17B • Updated Dec 12, 2023 • 26 • 3

TheBloke/Rogue-Rose-103b-v0.2-GPTQ

Text Generation • 103B • Updated Dec 16, 2023 • 4 • 1

TheBloke/Rogue-Rose-103b-v0.2-AWQ

Text Generation • 103B • Updated Dec 16, 2023 • 11 • 5

TheBloke/dopeystableplats-3b-v1-GPTQ

Text Generation • 3B • Updated Dec 12, 2023 • 5

TheBloke/neural-chat-7B-v3-3-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 26

TheBloke/neural-chat-7B-v3-3-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 73 • 3