metadata
license: mit
language:
- en
- ru
tags:
- data-leakage
- distribution shift
- multimodal
- membership inference attack
FiMMIA: scaling semantic perturbation-based membership inference across modalities
This repository contains an implementation of FiMMIA - a modular Framework for Multimodal Membership Inference Attacks (FiMMIA)
Description
The system is the first collection of models and pipelines for membership inference attacks against multimodal large language models, built initially with a priority for the Russian language, and extendable to any other language or dataset. Pipeline supports different modalities: Audio, Video and Image. In our experiments, we focus on MERA datasets, however, the model can be used to other languages.
We support models for Audio (this), Image and Video modalities.
Training and inference code can be obtained here.