README.md

metadata

title: FFG Mask Explorer
emoji: 🔬
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.0.0
app_file: app.py
pinned: false
license: apache-2.0
hardware: a10g-small

FFG Mask Explorer 🔬

An interactive tool for generating and visualizing Fast Fisher Grafting (FFG) masks on fine-tuned language models.

Features

• Real-time mask generation using GPU acceleration
• Multiple grafting methods: FFG, Magnitude, and Fish-Mask
• Interactive visualizations showing sparsity patterns and statistics
• Pre-configured models from the paper's experiments
• Custom model support for your own fine-tuned models

How to Use

1. Select a Model: Choose from pre-configured models or enter custom model IDs
2. Set Sparsity: Adjust the sparsity ratio (fraction of parameters to keep)
3. Choose Method: Select between FFG, Magnitude, or Fish-Mask grafting
4. Generate: Click to create masks and visualizations in real-time

About FFG

Fast Fisher Grafting (FFG) uses the second moments from Adam optimizer to identify important parameters in fine-tuned models. This provides more informed pruning compared to magnitude-based methods.

Based on the paper: Harnessing Optimization Dynamics for Curvature-Informed Model Merging

Technical Details

• GPU: Requires GPU for efficient processing (A10G recommended)
• Memory: ~24GB GPU memory for 8B parameter models
• Models: Compatible with Llama-3.1-8B based fine-tunes

Local Development

To run this Space locally:

# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/ffg-mask-explorer

# Install dependencies
pip install -r requirements.txt

# Copy FFG experiment suite (adjust path as needed)
cp -r /path/to/surgeon/ffg_experiment_suite .

# Run the app
python app.py

Citation

@misc{mahdavinia2025harnessingoptimizationdynamicscurvatureinformed,
    title={Harnessing Optimization Dynamics for Curvature-Informed Model Merging}, 
    author={Pouria Mahdavinia and Hamed Mahdavi and Niloofar Mireshghallah and Mehrdad Mahdavi},
    year={2025},
    eprint={2509.11167},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    url={https://arxiv.org/abs/2509.11167}, 
}

License

Apache 2.0 # Force rebuild Sun Sep 28 19:47:52 EDT 2025