| | --- |
| | title: FFG Mask Explorer |
| | emoji: 🔬 |
| | colorFrom: blue |
| | colorTo: purple |
| | sdk: gradio |
| | sdk_version: 5.0.0 |
| | app_file: app.py |
| | pinned: false |
| | license: apache-2.0 |
| | hardware: a10g-small |
| | --- |
| | |
| | # FFG Mask Explorer 🔬 |
| |
|
| | An interactive tool for generating and visualizing Fast Fisher Grafting (FFG) masks on fine-tuned language models. |
| |
|
| | ## Features |
| |
|
| | - **Real-time mask generation** using GPU acceleration |
| | - **Multiple grafting methods**: FFG, Magnitude, and Fish-Mask |
| | - **Interactive visualizations** showing sparsity patterns and statistics |
| | - **Pre-configured models** from the paper's experiments |
| | - **Custom model support** for your own fine-tuned models |
| |
|
| | ## How to Use |
| |
|
| | 1. **Select a Model**: Choose from pre-configured models or enter custom model IDs |
| | 2. **Set Sparsity**: Adjust the sparsity ratio (fraction of parameters to keep) |
| | 3. **Choose Method**: Select between FFG, Magnitude, or Fish-Mask grafting |
| | 4. **Generate**: Click to create masks and visualizations in real-time |
| |
|
| | ## About FFG |
| |
|
| | Fast Fisher Grafting (FFG) uses the second moments from Adam optimizer to identify important parameters in fine-tuned models. This provides more informed pruning compared to magnitude-based methods. |
| |
|
| | Based on the paper: [Harnessing Optimization Dynamics for Curvature-Informed Model Merging](https://arxiv.org/abs/2509.11167) |
| |
|
| | ## Technical Details |
| |
|
| | - **GPU**: Requires GPU for efficient processing (A10G recommended) |
| | - **Memory**: ~24GB GPU memory for 8B parameter models |
| | - **Models**: Compatible with Llama-3.1-8B based fine-tunes |
| |
|
| | ## Local Development |
| |
|
| | To run this Space locally: |
| |
|
| | ```bash |
| | # Clone the repository |
| | git clone https://huggingface.co/spaces/YOUR_USERNAME/ffg-mask-explorer |
| | |
| | # Install dependencies |
| | pip install -r requirements.txt |
| | |
| | # Copy FFG experiment suite (adjust path as needed) |
| | cp -r /path/to/surgeon/ffg_experiment_suite . |
| | |
| | # Run the app |
| | python app.py |
| | ``` |
| |
|
| | ## Citation |
| |
|
| | ```bibtex |
| | @misc{mahdavinia2025harnessingoptimizationdynamicscurvatureinformed, |
| | title={Harnessing Optimization Dynamics for Curvature-Informed Model Merging}, |
| | author={Pouria Mahdavinia and Hamed Mahdavi and Niloofar Mireshghallah and Mehrdad Mahdavi}, |
| | year={2025}, |
| | eprint={2509.11167}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.LG}, |
| | url={https://arxiv.org/abs/2509.11167}, |
| | } |
| | ``` |
| |
|
| | ## License |
| |
|
| | Apache 2.0 # Force rebuild Sun Sep 28 19:47:52 EDT 2025 |
| |
|
