Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,475

Base only

Active filters: 4-bit

TheBloke/Baichuan2-13B-Chat-GPTQ

Text Generation • 14B • Updated Sep 27, 2023 • 18 • 21

BarraHome/Llama-2-7b-GPTQ-Pacemaker

Text Generation • 7B • Updated Sep 8, 2023 • 5

jangmin/gptq-llama2-7b-chat-hf-food-order-understanding-30K

Text Generation • 7B • Updated Sep 8, 2023 • 6

Juniplayground/Mist_LLaMA-2-7B-1024_V7_GPTQ_Quantised_Version2

Text Generation • Updated Sep 8, 2023 • 6

gokul8967/Sofi-gptq

Text Generation • 13B • Updated Sep 8, 2023 • 6

TheBloke/Llama-2-PeanutButter_v19_R8-7B-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 9 • 2

TheBloke/Guanaco-7B-Uncensored-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 17 • 4

TheBloke/Guanaco-13B-Uncensored-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 7 • 5

TheBloke/Airoboros-L2-13B-2_1-YaRN-64K-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 12 • 3

Kimata/opt-125m-gptq

Text Generation • Updated Sep 8, 2023 • 8 • 1

casperhansen/vicuna-7b-v1.5-awq-gemv

Text Generation • Updated Oct 31, 2023 • 9 • 1

TheBloke/Guanaco-3B-Uncensored-v2-GPTQ

Text Generation • 3B • Updated Sep 27, 2023 • 30 • 15

TheBloke/COTHuginn-4.5-19B-GPTQ

Text Generation • 19B • Updated Sep 27, 2023 • 10 • 1

TheBloke/Spicyboros-7B-2.2-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 7 • 2

TheBloke/Falcon-180B-GPTQ

Text Generation • 179B • Updated Sep 27, 2023 • 26 • 10

Juniplayground/Mist_LLaMA-2-7B-1024_V7_GPTQ_Quantised_Version3

Text Generation • Updated Sep 9, 2023 • 7

flozi00/Llama-2-13b-german-assistant-v7-4bit-autogptq

Text Generation • 13B • Updated Sep 9, 2023 • 8

casperhansen/mpt-7b-8k-chat-awq-gemv

Text Generation • Updated Oct 31, 2023 • 8

TheBloke/Uni-TianYan-70B-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 8 • 2

TheBloke/ORCA_LLaMA_70B_QLoRA-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 10 • 3

TheBloke/Spicyboros-13B-2.2-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 55 • 18

TheBloke/Pygmalion-2-13B-SuperCOT-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 57 • 10

TheBloke/Tulpar-7B-v0-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 7 • 2

AhmedElDokmak/opt-125m-gptq-4bit

Text Generation • 0.1B • Updated Sep 20, 2023 • 7

WGNW/Llama-2-ko-7b-Chat-auto-gptq-4bit

Text Generation • 7B • Updated Sep 17, 2023 • 8 • 2

TheBloke/MLewdBoros-L2-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 59 • 26

TheBloke/ReMM-v2-L2-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 8 • 4

TheBloke/Llama-2-13B-Ensemble-v5-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 8 • 1

Nadilazev/bloom-custom-llm-exam

Text Generation • Updated Sep 11, 2023 • 14

TheBloke/Nous-Hermes-13B-Code-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 18 • 2