|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
- ru |
|
|
tags: |
|
|
- data-leakage |
|
|
- distribution shift |
|
|
- multimodal |
|
|
- membership inference attack |
|
|
--- |
|
|
# FiMMIA: scaling semantic perturbation-based membership inference across modalities |
|
|
This repository contains an implementation of **F**i**MMIA** - a modular **F**ramework for **M**ultimodal **M**embership **I**nference **A**ttacks (FiMMIA) |
|
|
|
|
|
## Description |
|
|
The system is the first collection of models and pipelines for membership inference attacks against multimodal large language models, built initially with a priority for the Russian language, and extendable to any other language or dataset. |
|
|
Pipeline supports different modalities: **Audio**, Video and Image. In our experiments, we focus on [MERA datasets](https://github.com/MERA-Evaluation/MERA), however, the model can be used to other languages. |
|
|
|
|
|
We support models for **Audio** (this), [Image](https://huggingface.co/ai-forever/FiMMIA-Image) and [Video](https://huggingface.co/ai-forever/FiMMIA-Video) modalities. |
|
|
|
|
|
Training and inference code can be obtained [here](https://github.com/ai-forever/data_leakage_detect). |
|
|
|
|
|
|
