5-bit Quantization - a somersetent Collection

somersetent 's Collections

2-bit Quantization

3-bit Quantization

4-bit Quantization

5-bit Quantization

6-bit Quantization

8-bit Quantization

Activation-aware Weight Quantization (AWQ)

Agentic Generative AI

Agentic Reproducibility Assessment (ARA)

Audio-to-Audio (A2A)

Audio-to-Video (A2V)

Automatic-Speech-Recognition (ASR)

Captioning/Captioner

Creative Writing/Creative Writer

Diffusers/Diffusion

Distillation/Distilled

Embedding/Embed

Fine-tuning/Fine-tuned

GPT-Generated Unified Format (GGUF)

Graph Machine Learning

Hallucination-Detection

Heretic-Abliteration

i1 (llama.cpp and HuggingFace Community)

Image Feature Extraction

Image-to-Image (I2I)

Image-to-Text (I2T)

Image-to-Video (I2V)

Importance Matrix (Imatrix)

Low-Rank Adaptation (LoRA)

Masked Token Prediction (MTP)

Mixture of Experts (MoE)

MLX (Apple Silicon)

Not-For-All-Audiences

Not Safe For Work (NSFW)

Quantization/Quantized

Router-weighted Expert Activation Pruning (REAP)

Speech Language Model (Speech LMs)

Text Embeddings Inference

Text Generation Inference

Text-to-Audio (T2A)

Text-to-Image (T2I)

Text-to-Music (T2M)

Text-to-Speech (TTS)

Text-to-Text (T2T)

Text-to-Video (T2V)

Uncensored-Decensored

Uncensored-Erotica

Video-to-Audio (V2A)

Video-to-Video (V2V)

Vision-Language (VL)

Zero-Shot Image Classification

AllenAI-Open Language Models (OLMs)

Alibaba-Tongyi-Qwen

Anthropic-Claude

HuggingFace-SmolLM

Lightricks-LTX-Studio

MistralAI-Mistral NeMo

MoonshotAI-Kimi-K

NousResearch-Hermes

NVIDIA-Nemotron

5-bit Quantization

updated 3 days ago

5-bit quantization in LLMs means compressing the model's weights from high precision (like 16-bit) down to an average of 5 bits per weight.