Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

Misc

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

45,687

Base only

Active filters: 4-bit

TheBloke/LLaMA2-13B-Psyfighter2-GPTQ

Text Generation • 13B • Updated Nov 30, 2023 • 17 • 16

TheBloke/NeuralHermes-2.5-Mistral-7B-GPTQ

Text Generation • 7B • Updated Nov 30, 2023 • 8 • 8

TheBloke/SG-Raccoon-Yi-55B-GPTQ

Text Generation • 56B • Updated Nov 30, 2023 • 7 • 1

TheBloke/SG-Raccoon-Yi-55B-AWQ

Text Generation • 56B • Updated Nov 30, 2023 • 5 • 2

Qwen/Qwen-1_8B-Chat-Int4

Text Generation • 2B • Updated Dec 13, 2023 • 160 • 36

Qwen/Qwen-72B-Chat-Int4

Text Generation • 72B • Updated Jan 4, 2024 • 302 • 47

TheBloke/tigerbot-70B-chat-v2-GPTQ

Text Generation • 69B • Updated Nov 30, 2023 • 5 • 5

mhenrichsen/context-aware-splitter-mistral-eng

Text Generation • Updated Nov 30, 2023 • 14 • 9

TheBloke/Chronomaid-Storytelling-13B-GPTQ

Text Generation • 13B • Updated Dec 4, 2023 • 11 • 9

TheBloke/Chronomaid-Storytelling-13B-AWQ

Text Generation • 13B • Updated Dec 4, 2023 • 19 • 4

gizmo-ai/OpenHermes-2.5-neural-chat-7B-v3-1-7B-AWQ

Text Generation • 7B • Updated Nov 30, 2023 • 1

AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct-4bit-gptq

Text Generation • Updated Jan 7, 2025 • 14 • 6

phatjk/vietcuna-7b-v3-AWQ

Text Generation • 8B • Updated Nov 30, 2023 • 4

TheBloke/meditron-70B-GPTQ

Text Generation • 69B • Updated Nov 30, 2023 • 13 • 5

TheBloke/meditron-70B-AWQ

Text Generation • 69B • Updated Nov 30, 2023 • 19 • 6

TheBloke/SlimOrca-13B-GPTQ

Text Generation • 13B • Updated Nov 30, 2023 • 10 • 2

TheBloke/SlimOrca-13B-AWQ

Text Generation • 13B • Updated Nov 30, 2023 • 5 • 2

TheBloke/meditron-7B-GPTQ

Text Generation • 7B • Updated Nov 30, 2023 • 7 • 3

TheBloke/meditron-7B-AWQ

Text Generation • 7B • Updated Nov 30, 2023 • 75 • 4

Medilora/guideline-adapter

Text Generation • Updated Dec 6, 2023 • 1

rycont/EleutherAI__polyglot-ko-1.3b-awq

Text Generation • 1B • Updated Dec 1, 2023 • 1

TheBloke/cinematika-7B-v0.1-AWQ

Text Generation • 7B • Updated Dec 1, 2023 • 5 • 1

TheBloke/cinematika-7B-v0.1-GPTQ

Text Generation • 7B • Updated Dec 1, 2023 • 3

TheBloke/juanako-7B-UNA-GPTQ

Text Generation • 7B • Updated Dec 1, 2023 • 13 • 2

TheBloke/juanako-7B-UNA-AWQ

Text Generation • 7B • Updated Dec 1, 2023 • 4 • 2

oieieio/Orca-2-13b-awq

Text Generation • 13B • Updated Dec 3, 2023 • 5

mzbac/Yi-34B-Llama-guanaco-awq

Text Generation • 34B • Updated Dec 2, 2023 • 3

TheBloke/SG-Raccoon-Yi-55B-200k-AWQ

Text Generation • 56B • Updated Dec 2, 2023 • 3

TheBloke/SG-Raccoon-Yi-55B-200k-GPTQ

Text Generation • 56B • Updated Dec 2, 2023 • 4

ECj/Yi-6B-200K-AWQ

Text Generation • 6B • Updated Dec 5, 2023 • 1