Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

48,978

Base only

Active filters: 4-bit

TheBloke/LosslessMegaCoder-Llama2-7B-Mini-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 14 • 8

TheBloke/CodeUp-Alpha-13B-HF-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 17 • 4

TheBloke/llama2-22B-daydreamer-v2-GPTQ

Text Generation • 22B • Updated Sep 27, 2023 • 14 • 8

TheBloke/PULI-GPT-3SX-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 11 • 4

TheBloke/LosslessMegaCoder-Llama2-13B-Mini-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 13 • 4

TheBloke/GodziLLa2-70B-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 20 • 4

TheBloke/Llama2-22B-Daydreamer-v3-GPTQ

Text Generation • 22B • Updated Sep 27, 2023 • 16 • 6

TheBloke/Scarlett-7B-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 9 • 1

TheBloke/Scarlett-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 11 • 5

TheBloke/scarlett-33B-GPTQ

Text Generation • 33B • Updated Sep 27, 2023 • 11 • 7

TheBloke/Carl-Llama-2-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 10 • 4

TheBloke/Carl-33B-GPTQ

Text Generation • 33B • Updated Sep 27, 2023 • 12 • 3

TheBloke/Carl-13B-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 11 • 6

TheBloke/orca_mini_v3_70B-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 49 • 13

RuterNorway/Llama-2-13b-chat-norwegian-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 51 • 10

TheBloke/Llama2-13B-MegaCode2-OASST-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 14 • 3

TheBloke/Octocoder-GPTQ

Text Generation • 16B • Updated Sep 27, 2023 • 12 • 8

TheBloke/Zarablend-L2-7B-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 99 • 10

casperhansen/falcon-7b-awq

Text Generation • Updated Nov 4, 2023 • 17 • 1

casperhansen/vicuna-7b-v1.5-awq

Text Generation • Updated Oct 31, 2023 • 13 • 3

Qwen/Qwen-7B-Chat-Int4

Text Generation • 8B • Updated Jan 4, 2024 • 383 • 75

TheBloke/LlongOrca-13B-16K-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 14 • 5

TheBloke/Samantha-1.1-70B-GPTQ

Text Generation • 69B • Updated Sep 27, 2023 • 14 • 2

TheBloke/Llama-2-7B-32K-Instruct-GPTQ

Text Generation • 7B • Updated Sep 27, 2023 • 67 • 27

flozi00/Llama-2-13b-german-assistant-v6-4bit-autogptq

Text Generation • 13B • Updated Aug 23, 2023 • 7 • 4

TheBloke/L2-MythoMax22b-Instruct-Falseblock-GPTQ

Text Generation • 22B • Updated Sep 27, 2023 • 45 • 10

anandsinghbaghel/opt-125m-gptq-4bit

Text Generation • Updated Aug 21, 2023 • 6

TheBloke/Llama2-22B-GPLATTY-GPTQ

Text Generation • 22B • Updated Sep 27, 2023 • 11 • 4

TheBloke/EverythingLM-13b-V2-16K-GPTQ

Text Generation • 13B • Updated Sep 27, 2023 • 14 • 7

TheBloke/Llama2-28B-Air03-GPTQ

Text Generation • 28B • Updated Sep 27, 2023 • 7 • 3