Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,303

Base only

Active filters: 4-bit

TheBloke/AmberChat-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 6 • 1

TheBloke/AmberChat-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 8 • 1

TheBloke/Amber-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 8 • 1

TheBloke/Amber-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 4 • 1

TheBloke/Seraph-7B-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 5

TheBloke/Seraph-7B-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 8

iproskurina/opt-13b-GPTQ-4bit-g128

Text Generation • 2B • Updated Mar 29, 2025 • 13

TheBloke/v1olet_marcoroni-go-bruins-merge-7B-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 15 • 1

TheBloke/v1olet_marcoroni-go-bruins-merge-7B-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 5

TheBloke/Merged-DPO-7B-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 4

TheBloke/Merged-DPO-7B-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 6 • 1

aladaf/zephyr-7b-beta_standard_v0.1_gptq

Text Generation • 7B • Updated Dec 12, 2023 • 3

TheBloke/supermario-v2-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 6 • 1

TheBloke/supermario-v2-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 5 • 1

TheBloke/Pallas-0.3-GPTQ

Text Generation • 34B • Updated Dec 12, 2023 • 4 • 1

TheBloke/Pallas-0.3-AWQ

Text Generation • 34B • Updated Dec 12, 2023 • 5 • 1

TheBloke/Synthia-MoE-v3-Mixtral-8x7B-GPTQ

Text Generation • 47B • Updated Dec 12, 2023 • 14 • 10

HydraIndicLM/mistral-MoQlora-hindi-expert

Updated Dec 12, 2023 • 7

TheBloke/Terminis-7B-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 4 • 1

TheBloke/pee-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 4

TheBloke/SeraphMarcoroni-7B-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 5

TheBloke/Marcoroni-neural-chat-7B-v1-AWQ

Text Generation • 7B • Updated Dec 12, 2023 • 6 • 1

TheBloke/Pallas-0.4-GPTQ

Text Generation • 34B • Updated Dec 12, 2023 • 13 • 1

TheBloke/Pallas-0.4-AWQ

Text Generation • 34B • Updated Dec 12, 2023 • 3 • 1

TheBloke/mixtralnt-4x7b-test-GPTQ

Text Generation • 24B • Updated Dec 12, 2023 • 11 • 5

IndicRAGware/mistral-hindi-qlora-RAG-7b

Text Generation • Updated Dec 12, 2023 • 2

TheBloke/OpenZephyrChat-AWQ

Text Generation • 7B • Updated Dec 13, 2023 • 6 • 2

adamo1139/Yi-34B-200K-AEZAKMI-v2-LoRA

Text Generation • Updated May 27, 2024 • 3 • 1

openaccess-ai-collective/SlimOrca-DPO-Mixtral-8x7B

Text Generation • Updated Dec 13, 2023 • 9 • 3

TheBloke/Terminis-7B-GPTQ

Text Generation • 7B • Updated Dec 12, 2023 • 6 • 2