Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

32,667

Base only

Active filters: 8-bit

juewang/deepseek-coder-6.7b-base-trt-int4-g64-hf

Text Generation • 4B • Updated May 10, 2024 • 10

juewang/deepseek-coder-1.3b-base-trt-int4-g64-hf

Text Generation • 0.8B • Updated May 10, 2024 • 5

RichardErkhov/h2oai_-_h2o-danube-1.8b-chat-8bits

Text Generation • 2B • Updated May 10, 2024 • 6

RichardErkhov/bigscience_-_bloomz-7b1-mt-8bits

Text Generation • 7B • Updated May 10, 2024 • 4

RichardErkhov/h2oai_-_h2o-danube-1.8b-base-8bits

Text Generation • 2B • Updated May 10, 2024 • 7

RichardErkhov/cais_-_HarmBench-Llama-2-13b-cls-8bits

Text Generation • 13B • Updated May 10, 2024 • 5

RichardErkhov/argilla_-_notus-7b-v1-8bits

Text Generation • 7B • Updated May 10, 2024 • 5

ketanthakur603/mentalhealth-chatbot-finetune

Text Generation • Updated May 10, 2024 • 8

RichardErkhov/neuralmagic_-_Llama-2-7b-evolcodealpaca-8bits

Text Generation • 7B • Updated May 10, 2024 • 5

RichardErkhov/facebook_-_mbart-large-50-many-to-many-mmt-8bits

Text Generation • 0.5B • Updated May 10, 2024 • 5

RichardErkhov/speakleash_-_Bielik-7B-Instruct-v0.1-8bits

Text Generation • 7B • Updated May 11, 2024 • 5

RichardErkhov/garage-bAInd_-_Platypus2-7B-8bits

Text Generation • 7B • Updated May 11, 2024 • 4

RichardErkhov/cognitivecomputations_-_dolphin-2.8-mistral-7b-v02-8bits

Text Generation • 7B • Updated May 11, 2024 • 4 • 1

LoneStriker/dolphin-2.9.1-llama-3-8b-8.0bpw-h8-exl2

Text Generation • Updated May 11, 2024 • 2 • 3

RichardErkhov/SeaLLMs_-_SeaLLM-7B-v2-8bits

Text Generation • 7B • Updated May 11, 2024 • 4

RichardErkhov/facebook_-_mbart-large-en-ro-8bits

Text Generation • 0.5B • Updated May 11, 2024 • 5

RichardErkhov/h2oai_-_h2ogpt-gm-7b-mistral-chat-sft-dpo-rag-v1-8bits

Text Generation • 7B • Updated May 11, 2024 • 4

RichardErkhov/UCLA-AGI_-_zephyr-7b-sft-full-SPIN-iter2-8bits

Text Generation • 7B • Updated May 11, 2024 • 5

RichardErkhov/Unbabel_-_TowerInstruct-7B-v0.1-8bits

Text Generation • 7B • Updated May 11, 2024 • 5

RichardErkhov/Unbabel_-_TowerBase-7B-v0.1-8bits

Text Generation • 7B • Updated May 11, 2024 • 4

RichardErkhov/georgesung_-_llama2_7b_chat_uncensored-8bits

Text Generation • 7B • Updated May 11, 2024 • 7

RichardErkhov/AI-Sweden-Models_-_gpt-sw3-6.7b-v2-8bits

Text Generation • 7B • Updated May 11, 2024 • 8

RichardErkhov/ArthurZ_-_mamba-2.8b-8bits

Text Generation • 3B • Updated May 11, 2024 • 4

ulkaa/CodeLlama-7b-hf-nf3

Text Generation • 3B • Updated Feb 21, 2025 • 5

zaq-hack/Llama-3-Lumimaid-8B-v0.1-OAS-8bpw-exl2

Text Generation • Updated May 11, 2024 • 5

RichardErkhov/aiXcoder_-_aixcoder-7b-base-8bits

Text Generation • 7B • Updated May 11, 2024 • 5

RichardErkhov/google_-_gemma-1.1-2b-it-8bits

Text Generation • 3B • Updated May 11, 2024 • 6

RichardErkhov/HuggingFaceTB_-_cosmo-1b-8bits

Text Generation • 2B • Updated May 11, 2024 • 4

RichardErkhov/42dot_-_42dot_LLM-SFT-1.3B-8bits

Text Generation • 1B • Updated May 11, 2024 • 4

RichardErkhov/BAAI_-_Bunny-v1_0-3B-8bits

Text Generation • 3B • Updated May 11, 2024 • 5