gradio/README.md

metadata

title: Robust Multimodal Foundation Models
emoji: 🛡️
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
pinned: false
license: mit
python_version: 3.11

Evaluating Robustness of Multimodal Models Against Adversarial Perturbations

This demo showcases adversarial attacks on multimodal foundation models (specifically OpenFlamingo) using APGD and SAIF algorithms.

Features

• Upload any image to generate captions
• Choose attack algorithm: APGD or SAIF
• Adjust parameters: epsilon, sparsity, iterations
• Visualize results: See original vs adversarial images and captions
• Perturbation visualization: View magnified perturbations

How to Use

1. Upload an image
2. Select attack algorithm (APGD or SAIF)
3. Adjust epsilon (max perturbation) and iterations
4. Click "Generate Captions" to see the results

Model

Uses OpenFlamingo-4B-vitl-rpj3b with adversarial attack capabilities.

Citation

If you use this work, please cite the original paper and repositories.