Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,276

Base only

Active filters: 4-bit

TheBloke/Astrid-Mistral-7B-AWQ

Text Generation • 7B • Updated Nov 28, 2023 • 7 • 2

TheBloke/Aetheria-L2-70B-AWQ

Text Generation • 69B • Updated Nov 28, 2023 • 10 • 1

TheBloke/PiVoT-0.1-Starling-LM-RP-GPTQ

Text Generation • 7B • Updated Nov 28, 2023 • 11 • 4

TheBloke/evolvedSeeker_1_3-GPTQ

Text Generation • 1B • Updated Nov 28, 2023 • 15

TheBloke/saiga_mistral_7b-GPTQ

Text Generation • 7B • Updated Nov 28, 2023 • 13 • 8

TheBloke/Astrid-Mistral-7B-GPTQ

Text Generation • 7B • Updated Nov 28, 2023 • 6 • 2

TheBloke/Aetheria-L2-70B-GPTQ

Text Generation • 69B • Updated Nov 29, 2023 • 14 • 1

TheBloke/Open-Hermes-2.5-neural-chat-3.1-frankenmerge-11b-AWQ

Text Generation • 11B • Updated Nov 30, 2023 • 7 • 2

TheBloke/SauerkrautLM-7B-HerO-AWQ

Text Generation • 7B • Updated Nov 29, 2023 • 8 • 4

TheBloke/Venus-120b-v1.0-GPTQ

Text Generation • 120B • Updated Nov 29, 2023 • 4 • 3

TheBloke/Venus-120b-v1.0-AWQ

Text Generation • 120B • Updated Nov 29, 2023 • 4 • 2

unsloth/llama-2-7b-bnb-4bit

Text Generation • 7B • Updated Sep 3, 2024 • 3.31k • 15

Sujan42024/dlite-v2-355m-bi4tQuantization

Text Generation • 0.4B • Updated Nov 29, 2023 • 5

TheBloke/psyonic-cetacean-20B-AWQ

Text Generation • 20B • Updated Nov 29, 2023 • 5

Ferrxni/finetuned_gw_mistral_GPTQ

Updated Nov 29, 2023 • 4

TheBloke/Open-Hermes-2.5-neural-chat-3.1-frankenmerge-11b-GPTQ

Text Generation • 11B • Updated Nov 30, 2023 • 18 • 1

TheBloke/psyonic-cetacean-20B-GPTQ

Text Generation • 20B • Updated Nov 29, 2023 • 5

TheBloke/SauerkrautLM-7B-HerO-GPTQ

Text Generation • 7B • Updated Nov 29, 2023 • 26 • 1

TheBloke/tigerbot-70B-chat-v2-AWQ

Text Generation • 69B • Updated Nov 30, 2023 • 11 • 2

TheBloke/deepseek-llm-67b-chat-AWQ

Text Generation • 67B • Updated Nov 29, 2023 • 136 • 5

TheBloke/deepseek-llm-67b-chat-GPTQ

Text Generation • 67B • Updated Nov 29, 2023 • 8 • 8

NobodyExistsOnTheInternet/mistral-7b-airoboros-chatml

Updated Dec 1, 2023 • 5

TheBloke/Iambe-20B-DARE-AWQ

Text Generation • 20B • Updated Nov 29, 2023 • 6 • 1

TheBloke/open-llama-3b-v2-wizard-evol-instuct-v2-196k-AWQ

Text Generation • 3B • Updated Nov 30, 2023 • 8 • 1

vachonni/falcon-1b-tech-level-fr

Text Generation • Updated Nov 29, 2023 • 5

TheBloke/deepseek-llm-7B-base-AWQ

Text Generation • 7B • Updated Nov 29, 2023 • 81 • 2

TheBloke/deepseek-llm-67b-base-AWQ

Text Generation • 67B • Updated Nov 29, 2023 • 18 • 2

TheBloke/deepseek-llm-67b-base-GPTQ

Text Generation • 67B • Updated Nov 29, 2023 • 8 • 1

TheBloke/deepseek-llm-7B-chat-AWQ

Text Generation • 7B • Updated Nov 29, 2023 • 151 • 3

TheBloke/LLaMA2-13B-Psyfighter2-AWQ

Text Generation • 13B • Updated Nov 30, 2023 • 10 • 4