README.md

---
license: apache-2.0
tags:
- slidesparse
- sparse
- quantization
- int8
- fp8
- llama
- qwen
---

# SlideSparse Checkpoints

Pre-converted sparse model checkpoints using the **SlideSparse** technique.

## Overview

This repository contains model weights converted with various sparsity configurations:
- **2:4** - Standard N:M sparsity (50% sparse)
- **2:6** - Extended sparsity (67% sparse)
- **2:8** - Higher sparsity (75% sparse)  
- **2:10** - Maximum sparsity (80% sparse)

## Models Included

| Base Model | Quantization | Sparsity Variants |
|------------|--------------|-------------------|
| Llama-3.2-1B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Llama-3.2-3B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Qwen2.5-7B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |
| Qwen2.5-14B | INT8, FP8 | 2:4, 2:6, 2:8, 2:10 |

## Source Models

These checkpoints are derived from:
- [RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8)
- [RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8)
- [RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8)
- [RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8](https://huggingface.co/RedHatAI/Qwen2.5-14B-Instruct-quantized.w8a8)

## License

- **Qwen models**: Apache 2.0
- **Llama models**: Please refer to [Meta's Llama license](https://llama.meta.com/llama3/license/)

## Usage

```bash
# Download all checkpoints
huggingface-cli download bcacdwk/slidesparse-checkpoints --local-dir ./checkpoints_slidesparse

# Download specific model
huggingface-cli download bcacdwk/slidesparse-checkpoints Llama3.2-1B-INT8-SlideSparse-2_4 --local-dir ./checkpoints_slidesparse/Llama3.2-1B-INT8-SlideSparse-2_4
```

## Citation

If you use these checkpoints, please cite the SlideSparse paper (coming soon).