File size: 2,346 Bytes
b1d8897 48a55a5 b1d8897 e67b6ae b1d8897 48a55a5 b1d8897 48a55a5 f80fa0a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
title: FFG Mask Explorer
emoji: 🔬
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.0.0
app_file: app.py
pinned: false
license: apache-2.0
hardware: a10g-small
---
# FFG Mask Explorer 🔬
An interactive tool for generating and visualizing Fast Fisher Grafting (FFG) masks on fine-tuned language models.
## Features
- **Real-time mask generation** using GPU acceleration
- **Multiple grafting methods**: FFG, Magnitude, and Fish-Mask
- **Interactive visualizations** showing sparsity patterns and statistics
- **Pre-configured models** from the paper's experiments
- **Custom model support** for your own fine-tuned models
## How to Use
1. **Select a Model**: Choose from pre-configured models or enter custom model IDs
2. **Set Sparsity**: Adjust the sparsity ratio (fraction of parameters to keep)
3. **Choose Method**: Select between FFG, Magnitude, or Fish-Mask grafting
4. **Generate**: Click to create masks and visualizations in real-time
## About FFG
Fast Fisher Grafting (FFG) uses the second moments from Adam optimizer to identify important parameters in fine-tuned models. This provides more informed pruning compared to magnitude-based methods.
Based on the paper: [Harnessing Optimization Dynamics for Curvature-Informed Model Merging](https://arxiv.org/abs/2509.11167)
## Technical Details
- **GPU**: Requires GPU for efficient processing (A10G recommended)
- **Memory**: ~24GB GPU memory for 8B parameter models
- **Models**: Compatible with Llama-3.1-8B based fine-tunes
## Local Development
To run this Space locally:
```bash
# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/ffg-mask-explorer
# Install dependencies
pip install -r requirements.txt
# Copy FFG experiment suite (adjust path as needed)
cp -r /path/to/surgeon/ffg_experiment_suite .
# Run the app
python app.py
```
## Citation
```bibtex
@misc{mahdavinia2025harnessingoptimizationdynamicscurvatureinformed,
title={Harnessing Optimization Dynamics for Curvature-Informed Model Merging},
author={Pouria Mahdavinia and Hamed Mahdavi and Niloofar Mireshghallah and Mehrdad Mahdavi},
year={2025},
eprint={2509.11167},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2509.11167},
}
```
## License
Apache 2.0 # Force rebuild Sun Sep 28 19:47:52 EDT 2025
|