Sparse-Llama-3.1-8B-2of4 - a nm-testing Collection

nm-testing 's Collections

KV Cache Quantization

FP8-Block Quantized Models

LLM Compressor testing

Speculators testing

Sparse-Llama-3.1-8B-2of4

Sparse-Llama-3.1-8B-2of4

updated Mar 2

RedHatAI/Sparse-Llama-3.1-8B-gsm8k-2of4

Text Generation • 8B • Updated Nov 21, 2024 • 65 • • 2
RedHatAI/Sparse-Llama-3.1-8B-evolcodealpaca-2of4-FP8-dynamic

Text Generation • 8B • Updated Dec 19, 2024 • 39
RedHatAI/Sparse-Llama-3.1-8B-2of4

Text Generation • 8B • Updated Dec 16, 2024 • 67 • • 62
RedHatAI/Sparse-Llama-3.1-8B-gsm8k-2of4-FP8-dynamic

Text Generation • 8B • Updated Dec 19, 2024 • 85 • 3
RedHatAI/Sparse-Llama-3.1-8B-ultrachat_200k-2of4

Text Generation • 8B • Updated Nov 21, 2024 • 24 • • 1
RedHatAI/Sparse-Llama-3.1-8B-evolcodealpaca-2of4

Text Generation • 8B • Updated Nov 21, 2024 • 32 • • 1
RedHatAI/Sparse-Llama-3.1-8B-ultrachat_200k-2of4-FP8-dynamic

Text Generation • 8B • Updated Dec 19, 2024 • 24 • 1
RedHatAI/Sparse-Llama-3.1-8B-ultrachat_200k-2of4-quantized.w4a16

Text Generation • 2B • Updated Dec 19, 2024 • 25 • 3
RedHatAI/Sparse-Llama-3.1-8B-evolcodealpaca-2of4-quantized.w4a16

Text Generation • 2B • Updated Dec 19, 2024 • 24