Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,991

Base only

Active filters: int8

RedHatAI/bge-large-en-v1.5-quant

Feature Extraction • Updated Nov 13, 2023 • 78 • 22

michaelfeil/ct2fast-Phind-CodeLlama-34B-v2

Text Generation • Updated Oct 8, 2023 • 7

michaelfeil/ct2fast-bge-large-en-v1.5

Feature Extraction • Updated Oct 13, 2023 • 33

michaelfeil/ct2fast-bge-base-en-v1.5

Feature Extraction • Updated Oct 13, 2023 • 34

michaelfeil/ct2fast-bge-small-en-v1.5

Feature Extraction • Updated Oct 13, 2023 • 29

michaelfeil/ct2fast-gte-large

Sentence Similarity • Updated Oct 13, 2023 • 30

michaelfeil/ct2fast-gte-base

Sentence Similarity • Updated Oct 13, 2023 • 31

michaelfeil/ct2fast-jina-embedding-l-en-v1

Sentence Similarity • Updated Oct 13, 2023 • 28

michaelfeil/ct2fast-jina-embedding-t-en-v1

Sentence Similarity • Updated Oct 13, 2023 • 29

michaelfeil/ct2fast-jina-embedding-s-en-v1

Sentence Similarity • Updated Oct 13, 2023 • 27

Anthonyg5005/WizardLM-13B-V1.2-int8-ct2

Text Generation • Updated Mar 12, 2025 • 10 • 1

KomorebiAI/nllb-200-3.3B-int8-ct2

Translation • Updated Dec 19, 2024 • 23 • 3

KomorebiAI/nllb-200-1.3B-int8-ct2

Translation • Updated Dec 19, 2024 • 20 • 2

michaelfeil/ct2fast-phi-1_5

Text Generation • Updated Nov 29, 2023 • 8

michaelfeil/ct2fast-phi-1

Text Generation • Updated Nov 30, 2023 • 5

OpenNMT/codet5p-770m-py-ct2-int8

Updated Nov 30, 2023 • 8

OpenNMT/codet5p-770m-ct2-int8

Updated Nov 30, 2023 • 9 • 1

OpenNMT/nllb-200-distilled-1.3B-ct2-int8

Translation • Updated Nov 30, 2023 • 1.46k • 15

OpenNMT/phi-1_5-ct2-int8

Text Generation • Updated Nov 30, 2023 • 8 • 1

OpenNMT/phi-1-ct2-int8

Text Generation • Updated Nov 30, 2023 • 9

OpenNMT/Llama-2-7b-hf-ct2-int8

Text Generation • Updated Dec 1, 2023 • 8

OpenNMT/nllb-200-3.3B-ct2-int8

Translation • Updated Dec 1, 2023 • 885 • 5

OpenNMT/Llama-2-7b-chat-hf-ct2-int8

Text Generation • Updated Dec 1, 2023 • 18

minuva/MiniLMv2-goemotions-v2-onnx

Text Classification • Updated Apr 24, 2024 • 116 • 2

avans06/ALMA-7B-ct2-int8_float16

Text Generation • Updated Dec 15, 2023 • 6

avans06/ALMA-13B-ct2-int8_float16

Text Generation • Updated Dec 15, 2023 • 9

minuva/MiniLMv2-toxic-jigsaw-lite-onnx

Text Classification • Updated Apr 24, 2024 • 14 • 1

minuva/MiniLMv2-toxic-jigsaw-onnx

Text Classification • Updated Apr 24, 2024 • 1.38k • 2

avans06/madlad400-7b-mt-bt-ct2-int8_float16

Updated Dec 24, 2023 • 39 • 3

ecastera/eva-mistral-dolphin-7b-spanish

Text Generation • 7B • Updated Mar 16, 2024 • 41 • 12