Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,398

Base only

Active filters: 4-bit

TheBloke/MXLewdMini-L2-13B-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 6 • 1

TheBloke/MXLewdMini-L2-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 10 • 7

TheBloke/MetaMath-13B-V1.0-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 8 • 1

TheBloke/MetaMath-13B-V1.0-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 9 • 1

TheBloke/MAmmoTH-Coder-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 10 • 3

TheBloke/MAmmoTH-Coder-13B-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 5 • 1

TheBloke/MetaMath-70B-V1.0-AWQ

Text Generation • 69B • Updated Nov 9, 2023 • 7 • 1

TheBloke/MetaMath-70B-V1.0-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 10

TheBloke/MetaMath-7B-V1.0-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 8 • 2

TheBloke/MetaMath-7B-V1.0-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 9 • 2

TheBloke/Synthia-13B-v1.2-AWQ

Text Generation • 13B • Updated Nov 9, 2023 • 6

TheBloke/Synthia-13B-v1.2-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 6 • 2

Qwen/Qwen-14B-Chat-Int4

Text Generation • 14B • Updated Dec 13, 2023 • 272 • 100

TheBloke/Synthia-7B-v1.2-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 5

TheBloke/Synthia-7B-v1.2-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 4 • 2

TheBloke/openbuddy-coder-34b-v11-bf16-AWQ

Text Generation • 34B • Updated Nov 9, 2023 • 8 • 1

TheBloke/openbuddy-coder-34b-v11-bf16-GPTQ

Text Generation • 34B • Updated Sep 27, 2023 • 11

WGNW/llama-2-7b-ko-auto-gptq-full

Text Generation • 7B • Updated Sep 24, 2023 • 8

TheBloke/openbuddy-llama2-34b-v11.1-bf16-AWQ

Text Generation • 34B • Updated Nov 9, 2023 • 7 • 3

TheBloke/openbuddy-llama2-34b-v11.1-bf16-GPTQ

Text Generation • 34B • Updated Sep 27, 2023 • 19 • 2

SidharthanRajendran/gpt2-gptq

Text Generation • Updated Sep 24, 2023 • 17

Nagharjun17/zoningLlama2-GPTQ-Old

Text Generation • Updated Sep 25, 2023 • 7

webpolis/zenos-gpt-j-6B-instruct-4bit

Text Generation • 6B • Updated Jan 19, 2024 • 8 • 1

ARahul2003/opt-125M-4bit-gptq

Text Generation • Updated Sep 25, 2023 • 6

4bit/Qwen-14B-Chat-Int4

Text Generation • 14B • Updated Sep 25, 2023 • 6 • 2

MiNeves-tops/opt-125m-gptq-4bit

Text Generation • Updated Sep 25, 2023 • 5

Guilherme34/Jenniferv2-gptq4bit

Text Generation • Updated Sep 25, 2023 • 4

jphme/em_german_13b_v01_gptq

Text Generation • 13B • Updated Oct 12, 2023 • 112 • 2

TheBloke/vicuna-33B-AWQ

Text Generation • 33B • Updated Nov 9, 2023 • 7 • 2

CJ-gyuwonpark/ch-70b-v2

Updated Sep 28, 2023 • 1