File size: 1,900 Bytes
be13d93 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
license: apache-2.0
tags:
- slidesparse
- sparse
- quantization
- int8
- fp8
- llama
- qwen
---
# SlideSparse Checkpoints
Pre-converted sparse model checkpoints using the **SlideSparse** technique.
## Overview
This repository contains model weights converted with various sparsity configurations:
- **2:4** - Standard N:M sparsity (50% sparse)
- **2:6** - Extended sparsity (67% sparse)
- **2:8** - Higher sparsity (75% sparse)
- **2:10** - Maximum sparsity (80% sparse)
## Models Included
| Base Model | Quantization | Sparsity Variants |
|------------|--------------|-------------------|
| Llama-3.2-1B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Llama-3.2-3B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Qwen2.5-7B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Qwen2.5-14B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
## Source Models
These checkpoints are derived from:
- [RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8)
- [RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8)
- [RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8)
- [RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8)
## License
- **Qwen models**: Apache 2.0
- **Llama models**: Please refer to [Meta's Llama license](https://llama.meta.com/llama3/license/)
## Usage
```bash
# Download all checkpoints
huggingface-cli download bcacdwk/slidesparse-checkpoints --local-dir ./checkpoints_slidesparse
# Download specific model
huggingface-cli download bcacdwk/slidesparse-checkpoints Llama3.2-1B-INT8-SlideSparse-2_4 --local-dir ./checkpoints_slidesparse/Llama3.2-1B-INT8-SlideSparse-2_4
```
## Citation
If you use these checkpoints, please cite the SlideSparse paper (coming soon).
|