Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,294

Base only

Active filters: 4-bit

TheBloke/firefly-llama2-7B-chat-GPTQ

Text Generation • 7B • Updated Nov 17, 2023 • 9 • 2

TheBloke/firefly-llama2-7B-chat-AWQ

Text Generation • 7B • Updated Nov 17, 2023 • 5 • 1

TheBloke/alfred-40B-1023-GPTQ

Text Generation • 42B • Updated Nov 17, 2023 • 26 • 3

TheBloke/alfred-40B-1023-AWQ

Text Generation • 42B • Updated Nov 17, 2023 • 12 • 5

heka-ai/zephyr-beta-ft-1000-lr-GPTQ

Text Generation • 7B • Updated Nov 17, 2023 • 5

kaitchup/Mistral-7B-awq-4bit

Text Generation • 7B • Updated Nov 20, 2023 • 116 • 1

cshulby/Mistral-7B-OpenOrca-AlpacaFT

Text Generation • Updated Nov 17, 2023 • 6

drisspg/mathew_train_v1

Text Generation • Updated Nov 18, 2023 • 6

atorsvn/TinyLlama-1.1B-Chat-v0.4-gptq-4bit

Text Generation • 1B • Updated Nov 18, 2023 • 8

TheBloke/Euryale-1.4-L2-70B-GPTQ

Text Generation • 69B • Updated Nov 18, 2023 • 8 • 2

TheBloke/Euryale-1.4-L2-70B-AWQ

Text Generation • 69B • Updated Nov 18, 2023 • 6 • 2

Trelis/Yi-6B-200K-Llamafied-chat-SFT-AWQ

Text Generation • 6B • Updated Nov 20, 2023

patched-codes/patched-coder-34b-AWQ

Text Generation • 34B • Updated Nov 18, 2023 • 5

LoneStriker/Yi-34B-Spicyboros-3.1-3-LoRA

Text Generation • Updated Nov 19, 2023 • 5

TheBloke/opus-v0.5-70B-GPTQ

Text Generation • 69B • Updated Nov 18, 2023 • 4 • 1

TheBloke/opus-v0.5-70B-AWQ

Text Generation • 69B • Updated Nov 18, 2023 • 6 • 1

Trelis/Yi-34B-200K-Llamafied-chat-SFT-AWQ

Text Generation • Updated Nov 26, 2023

TheBloke/Nanbeige-16B-Chat-32K-GPTQ

Text Generation • 16B • Updated Nov 18, 2023 • 12 • 3

TheBloke/nsql-llama-2-7B-AWQ

Text Generation • 7B • Updated Nov 18, 2023 • 5

TheBloke/nsql-llama-2-7B-GPTQ

Text Generation • 7B • Updated Nov 18, 2023 • 7

TheBloke/nucleus-22B-token-500B-GPTQ

Text Generation • 22B • Updated Nov 18, 2023 • 8 • 2

TheBloke/nucleus-22B-token-500B-AWQ

Text Generation • 22B • Updated Nov 18, 2023 • 8 • 1

TheBloke/tigerbot-70B-chat-v4-GPTQ

Text Generation • 70B • Updated Nov 19, 2023 • 9 • 1

TheBloke/tigerbot-70B-chat-v4-AWQ

Text Generation • 70B • Updated Nov 19, 2023 • 8

TheBloke/Kaori-70B-v1-GPTQ

Text Generation • 69B • Updated Nov 19, 2023 • 10 • 2

TheBloke/Kaori-70B-v1-AWQ

Text Generation • 69B • Updated Nov 19, 2023 • 6 • 2

TheBloke/Nanbeige-16B-Base-32K-GPTQ

Text Generation • 16B • Updated Nov 19, 2023 • 14 • 1

TheBloke/Nanbeige-16B-Base-GPTQ

Text Generation • 16B • Updated Nov 19, 2023 • 10 • 2

TheBloke/Nanbeige-16B-Chat-GPTQ

Text Generation • 16B • Updated Nov 19, 2023 • 7 • 1

TheBloke/Capybara-Tess-Yi-34B-200K-AWQ

Text Generation • 34B • Updated Nov 19, 2023 • 6 • 3