File size: 783 Bytes
a4779a1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
```python
import sparseml.transformers
original_model_name = "Xenova/llama2.c-stories110M"
output_directory = "output/"
final_model_name = "nm-testing/llama2.c-stories110M-pruned2.4"
dataset = "open_platypus"
recipe = """
test_stage:
obcq_modifiers:
SparseGPTModifier:
sparsity: 0.5
sequential_update: true
quantize: false
mask_structure: '2:4'
targets: ['re:model.layers.\d*$']
"""
# Apply SparseGPT to the model
sparseml.transformers.oneshot(
model_name_or_path=original_model_name,
dataset_name=dataset,
recipe=recipe,
output_dir=output_directory,
)
# Upload the output model to Hugging Face Hub
from huggingface_hub import HfApi
HfApi().upload_folder(
folder_path=output_directory,
repo_id=final_model_name,
)
``` |