Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,393

Base only

Active filters: 4-bit

BobaZooba/Shurale7B-v1-GPTQ

Text Generation • 7B • Updated Nov 14, 2023 • 14 • 3

KimJY/wwpp-13B-gptq-4bit

Text Generation • Updated Oct 3, 2023 • 7

TheBloke/TinyLlama-1.1B-Chat-v0.3-GPTQ

Text Generation • 1B • Updated Oct 3, 2023 • 1.02M • 10

TheBloke/TinyLlama-1.1B-Chat-v0.3-AWQ

Text Generation • 1B • Updated Nov 9, 2023 • 155k • 5

TheBloke/TinyLlama-1.1B-python-v0.1-GPTQ

Text Generation • 1B • Updated Oct 3, 2023 • 385 • 2

TheBloke/TinyLlama-1.1B-python-v0.1-AWQ

Text Generation • 1B • Updated Nov 9, 2023 • 7 • 2

yschoi/opt-125m-gptq-4bit

Text Generation • Updated Oct 3, 2023 • 5

TheBloke/em_german_mistral_v01-GPTQ

Text Generation • 7B • Updated Oct 3, 2023 • 17 • 5

TheBloke/em_german_mistral_v01-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 7 • 5

TheBloke/TinyLlama-1.1B-intermediate-step-480k-1T-GPTQ

Text Generation • 1B • Updated Oct 3, 2023 • 12 • 2

TheBloke/TinyLlama-1.1B-intermediate-step-480k-1T-AWQ

Text Generation • 1B • Updated Nov 9, 2023 • 8 • 2

vsarathy/parser-info_structure-30k-no-context-llama2-7b

Text Generation • Updated Oct 3, 2023 • 4

vsarathy/parser-info_structure-30k-context-llama2-7b

Text Generation • Updated Oct 5, 2023 • 5 • 1

vsarathy/parser-info_structure-30k-no-context-falcon-7b

Updated Oct 5, 2023 • 4

vsarathy/parser-info_structure-30k-context-falcon-7b

Text Generation • Updated Oct 6, 2023 • 6

TheBloke/airoboros-mistral2.2-7B-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 9 • 2

TheBloke/airoboros-mistral2.2-7B-GPTQ

Text Generation • 7B • Updated Oct 4, 2023 • 11 • 4

CJ-gyuwonpark/ch-70b-v5

Updated Oct 6, 2023 • 1

TheBloke/CodeFuse-13B-GPTQ

Text Generation • 14B • Updated Oct 4, 2023 • 15 • 4

cadaeic/llama2-7b-recipe-qlora

Text Generation • Updated Oct 4, 2023 • 6

TheBloke/Dans-AdventurousWinds-7B-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 5 • 1

TheBloke/Dans-AdventurousWinds-7B-GPTQ

Text Generation • 7B • Updated Oct 4, 2023 • 5 • 2

TheBloke/Dans-TotSirocco-7B-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 4 • 1

TheBloke/Dans-TotSirocco-7B-GPTQ

Text Generation • 7B • Updated Oct 4, 2023 • 5 • 2

TheBloke/Mistralic-7B-1-GPTQ

Text Generation • 7B • Updated Oct 4, 2023 • 8 • 6

TheBloke/Mistralic-7B-1-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 5 • 2

vsarathy/parser-info_structure-30k-no-context-llama2-3b

Text Generation • Updated Oct 4, 2023 • 4

vsarathy/parser-info_structure-30k-context-llama2-3b

Text Generation • Updated Oct 5, 2023 • 4

TheBloke/Llama-2-7B-vietnamese-20k-AWQ

Text Generation • 7B • Updated Nov 9, 2023 • 7 • 5

TheBloke/Llama-2-7B-vietnamese-20k-GPTQ

Text Generation • 7B • Updated Oct 4, 2023 • 11 • 2