Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,399

Base only

Active filters: 4-bit

TheBloke/law-LLM-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 7 • 2

TheBloke/law-LLM-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 55 • 3

TheBloke/sqlcoder-GPTQ

Text Generation • 16B • Updated Sep 27, 2023 • 10 • 9

TheBloke/Qwen-14B-Chat-GPTQ

Text Generation • 14B • Updated Oct 30, 2023 • 57 • 34

TheBloke/U-Amethyst-20B-AWQ

Text Generation • 20B • Updated Nov 9, 2023 • 6 • 3

TheBloke/U-Amethyst-20B-GPTQ

Text Generation • 20B • Updated Sep 27, 2023 • 45 • 22

CWKSC/opt-125m-gptq-4bits

Text Generation • Updated Sep 27, 2023 • 6 • 1

minhbui/viettel_v3

Text Generation • Updated Sep 27, 2023 • 9

atorsvn/TinyLlama-1.1B-Chat-v0.1-gptq-4bit

Text Generation • Updated Sep 27, 2023 • 12

TheBloke/Marcoroni-70B-v1-AWQ

Text Generation • 69B • Updated Nov 9, 2023 • 5 • 2

TheBloke/Marcoroni-70B-v1-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 8 • 2

TheBloke/Athena-v3-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 57 • 10

TheBloke/Athena-v3-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 6 • 2

TheBloke/openbuddy-openllama-7B-v12-bf16-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 10

TheBloke/openbuddy-openllama-7B-v12-bf16-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 8

casperhansen/mistral-7b-instruct-v0.1-awq

Text Generation • Updated Oct 31, 2023 • 15.1k • 2

TheBloke/Mistral-7B-v0.1-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 623 • 33

TheBloke/Mistral-7B-Instruct-v0.1-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 468 • 38

TheBlake/Llama-2-7b

Text Generation • 7B • Updated Sep 27, 2023 • 9 • 3

jphme/em_german_7b_v01_gptq

Text Generation • 7B • Updated Oct 12, 2023 • 6

TheBloke/Emerhyst-13B-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 5 • 4

TheBloke/Emerhyst-13B-GPTQ

Text Generation • 13B • Updated Sep 28, 2023 • 52 • 11

TheBloke/Emerhyst-20B-AWQ

Text Generation • 20B • Updated Nov 9, 2023 • 43 • 10

TheBloke/Emerhyst-20B-GPTQ

Text Generation • 20B • Updated Sep 28, 2023 • 45 • 14

TheBloke/leo-hessianai-13B-chat-bilingual-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 4 • 1

TheBloke/leo-hessianai-13B-chat-bilingual-GPTQ

Text Generation • 13B • Updated Sep 28, 2023 • 8 • 4

TheBloke/leo-hessianai-13B-chat-GPTQ

Text Generation • 13B • Updated Sep 28, 2023 • 12 • 1

TheBloke/leo-hessianai-13B-chat-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 6 • 1

TheBloke/leo-hessianai-13B-GPTQ

Text Generation • 13B • Updated Sep 28, 2023 • 7

TheBloke/leo-hessianai-13B-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 9