Update README.md
Browse files
README.md
CHANGED
|
@@ -4,4 +4,13 @@ language:
|
|
| 4 |
- en
|
| 5 |
base_model:
|
| 6 |
- meta-llama/Llama-2-7b-hf
|
| 7 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
- en
|
| 5 |
base_model:
|
| 6 |
- meta-llama/Llama-2-7b-hf
|
| 7 |
+
---
|
| 8 |
+
# Model Card for Model ID
|
| 9 |
+
|
| 10 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
| 11 |
+
|
| 12 |
+
This repo contains a 2:4 sparse version of the LLaMA2-7B model. Trainied with methods from AAAI25 paper [Pruning Large Language Models with Semi-Structural Adaptive Sparse Training](https://arxiv.org/abs/2407.20584).
|
| 13 |
+
|
| 14 |
+
### Model Description
|
| 15 |
+
|
| 16 |
+
Same structured as LLaMA2-7B, but weight from linear layer conform to 2:4 sparse pattern.
|