Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,965

Base only

Active filters: int8

avans06/madlad400-7b-mt-bt-ct2-int8_float16

Updated Dec 24, 2023 • 38 • 3

ecastera/eva-mistral-dolphin-7b-spanish

Text Generation • 7B • Updated Mar 16, 2024 • 41 • 12

minuva/MiniLMv2-userflow-v2-onnx

Text Classification • Updated Apr 24, 2024 • 4 • 1

minuva/MiniLMv2-agentflow-v2-onnx

Text Classification • Updated Apr 24, 2024 • 5 • 2

ecastera/ecastera-eva-westlake-7b-spanish

Text Generation • 7B • Updated Mar 16, 2024 • 9 • 2

jvh/whisper-base-quant-ct2

Automatic Speech Recognition • Updated Mar 19, 2024 • 6 • 2

jvh/whisper-large-v2-quant-ct2

Automatic Speech Recognition • Updated Mar 19, 2024 • 10 • 3

jvh/whisper-medium-quant-ct2

Automatic Speech Recognition • Updated Mar 19, 2024 • 6 • 2

jvh/whisper-large-v3-quant-ct2

Automatic Speech Recognition • Updated Mar 19, 2024 • 15 • 1

study-hjt/Meta-Llama-3-8B-Instruct-AWQ

Text Generation • 8B • Updated Apr 23, 2024 • 8 • 1

study-hjt/Meta-Llama-3-8B-Instruct-GPTQ-Int8

Text Generation • 8B • Updated Apr 23, 2024 • 16 • 2

study-hjt/Meta-Llama-3-70B-Instruct-GPTQ-Int8

Text Generation • 71B • Updated Apr 23, 2024 • 5 • 2

avans06/Meta-Llama-3-8B-Instruct-ct2-int8_float16

Text Generation • Updated Apr 25, 2024 • 15

nitsuai/ct2fast-all-MiniLM-L6-v2

Sentence Similarity • Updated Apr 26, 2024 • 32

nitsuai/ct2fast-paraphrase-multilingual-MiniLM-L12-v2

Sentence Similarity • Updated Apr 26, 2024 • 28

study-hjt/Qwen1.5-110B-Chat-GPTQ-Int8

Text Generation • 111B • Updated Apr 27, 2024 • 7

study-hjt/Qwen1.5-32B-Chat-GPTQ-Int8

Text Generation • 33B • Updated Apr 26, 2024 • 7 • 1

study-hjt/CodeQwen1.5-7B-Chat-GPTQ-Int8

Text Generation • 7B • Updated Apr 26, 2024 • 102 • 1

Weblet/Llama-2-7b-chat-hf-ct2-int8

Text Generation • Updated Apr 30, 2024 • 5

ecastera/eva-dolphin-llama3-8b-spanish

Text Generation • Updated Jun 14, 2024 • 9 • 4

Anthonyg5005/L3-8B-Stheno-v3.1-int8-ct2

Text Generation • Updated Mar 12, 2025 • 14

Anthonyg5005/turbcat-instruct-8b-int8-ct2

Text Generation • Updated Mar 12, 2025 • 13

model-scope/glm-4-9b-chat-GPTQ-Int8

Text Generation • 9B • Updated Jul 23, 2024 • 14 • 2

RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a16

Text Generation • 8B • Updated 12 days ago • 900 • 12

RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8

Text Generation • 8B • Updated 12 days ago • 25.1k • 20

angeloc1/llama3dot1SimilarProcesses8

Text Generation • 8B • Updated Jul 27, 2024 • 10

angeloc1/llama3dot1DifferentProcesses8

Text Generation • 8B • Updated Jul 27, 2024 • 12

RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a8

Text Generation • 71B • Updated 12 days ago • 3.15k • 21

FriendliAI/Meta-Llama-3-8B-int8

Text Generation • 8B • Updated Aug 1, 2024 • 7 • 1

RedHatAI/Meta-Llama-3.1-8B-quantized.w8a16

Text Generation • 8B • Updated 12 days ago • 167 • 1