Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,293

Base only

Active filters: 4-bit

TheBloke/openbuddy-deepseek-67b-v15-base-AWQ

Text Generation • 67B • Updated Dec 10, 2023 • 3

TheBloke/Marcoroni-7B-v2-AWQ

Text Generation • 7B • Updated Dec 10, 2023 • 7

TheBloke/Marcoroni-7B-v3-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 9 • 1

TheBloke/CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity-AWQ

Text Generation • 34B • Updated Dec 10, 2023 • 5 • 1

danny0122/stablelm-base-alpha-3b-gptq-4bits

Text Generation • 4B • Updated Dec 11, 2023 • 11

TheBloke/Marcoroni-7B-v2-GPTQ

Text Generation • 7B • Updated Dec 10, 2023 • 10

TheBloke/Marcoroni-7B-v3-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 5

TheBloke/CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity-GPTQ

Text Generation • 34B • Updated Dec 10, 2023 • 7 • 1

TheBloke/Merged-AGI-7B-AWQ

Text Generation • 7B • Updated Dec 10, 2023 • 9

TheBloke/Merged-AGI-7B-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 11 • 1

TheBloke/open-instruct-human-mix-65B-AWQ

Text Generation • 65B • Updated Dec 11, 2023 • 5

TheBloke/open-instruct-human-mix-65B-GPTQ

Text Generation • 65B • Updated Dec 11, 2023 • 9

danny0122/Llama-2-7b-hf-gptq-4bits

Text Generation • 7B • Updated Dec 11, 2023 • 3

TheBloke/Nethena-20B-Glued-GPTQ

Text Generation • 20B • Updated Dec 11, 2023 • 6 • 1

TheBloke/Nethena-20B-Glued-AWQ

Text Generation • 20B • Updated Dec 11, 2023 • 4 • 2

danny0122/Llama-2-7b-hf-gptq-4bitssafe

Text Generation • 7B • Updated Dec 11, 2023 • 4

TheBloke/Velara-AWQ

Text Generation • 11B • Updated Dec 11, 2023 • 5

TheBloke/Velara-GPTQ

Text Generation • 11B • Updated Dec 11, 2023 • 4 • 1

TheBloke/leo-hessianai-70B-chat-GPTQ

Text Generation • 69B • Updated Dec 11, 2023 • 17 • 1

TheBloke/leo-hessianai-70B-chat-AWQ

Text Generation • 69B • Updated Dec 11, 2023 • 4 • 1

danny0122/stablelm-base-alpha-3b-gptq-4bitssafe

Text Generation • 4B • Updated Dec 11, 2023 • 5

jamesdborin/llama2-13b-chat-4bit-AWQ

Text Generation • 13B • Updated Dec 11, 2023 • 5 • 1

jamesdborin/llama2-70b-chat-4bit-AWQ

Text Generation • 69B • Updated Dec 11, 2023 • 5 • 1

jamesdborin/llama2-7b-base-4bit-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 7

jamesdborin/llama2-13b-base-4bit-AWQ

Text Generation • 13B • Updated Dec 11, 2023 • 8

jamesdborin/llama2-70b-base-4bit-AWQ

Text Generation • 69B • Updated Dec 11, 2023 • 8

Chat-Error/Rose-Kimiko-20B

Updated Dec 12, 2023 • 7 • 2

Weni/ZeroShot-3.1.1-Mistral-7b-Multilanguage-3.0.3-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 4

TheBloke/OpenHermes-2.5-neural-chat-v3-3-Slerp-GPTQ

Text Generation • 7B • Updated Dec 11, 2023 • 7 • 3

TheBloke/OpenHermes-2.5-neural-chat-v3-3-Slerp-AWQ

Text Generation • 7B • Updated Dec 11, 2023 • 6 • 1