Update README.md
Browse files
README.md
CHANGED
|
@@ -20,44 +20,3 @@ We support models for **Image** (this), [Audio](https://huggingface.co/ai-foreve
|
|
| 20 |
|
| 21 |
Training and inference code can be obtained [here](https://github.com/ai-forever/data_leakage_detect).
|
| 22 |
|
| 23 |
-
We report AUC-ROC for binary classification (leaked vs. clean) as shown in Table below. AUC-ROC FiMMIA performance metrics for various evaluated **Image** MLLMs.
|
| 24 |
-
|
| 25 |
-
| Origin Model | Test Model | **AUC-ROC** |
|
| 26 |
-
|-----------------------|-------------------------|-------------|
|
| 27 |
-
| Qwen2.5-VL-3B-Instruct| Qwen2.5-VL-3B-Instruct | **96.2** |
|
| 28 |
-
| Qwen2.5-VL-3B-Instruct| Qwen2-VL-7B-Instruct | 86.0 |
|
| 29 |
-
| Qwen2.5-VL-3B-Instruct| Qwen2.5-VL-7B-Instruct | 88.0 |
|
| 30 |
-
| Qwen2.5-VL-3B-Instruct| Llama3-llava-next-8b-hf | _90.2_ |
|
| 31 |
-
| Qwen2.5-VL-3B-Instruct| Gemma-3-4b-it | 65.8 |
|
| 32 |
-
| Qwen2.5-VL-3B-Instruct| Gemma-3-12b-it | 67.9 |
|
| 33 |
-
| Qwen2-VL-7B-Instruct | Qwen2.5-VL-3B-Instruct | 78.0 |
|
| 34 |
-
| Qwen2-VL-7B-Instruct | Qwen2-VL-7B-Instruct | **96.2** |
|
| 35 |
-
| Qwen2-VL-7B-Instruct | Qwen2.5-VL-7B-Instruct | _80.5_ |
|
| 36 |
-
| Qwen2-VL-7B-Instruct | Llama3-llava-next-8b-hf | 78.0 |
|
| 37 |
-
| Qwen2-VL-7B-Instruct | Gemma-3-4b-it | 77.7 |
|
| 38 |
-
| Qwen2-VL-7B-Instruct | Gemma-3-12b-it | 73.7 |
|
| 39 |
-
| Qwen2.5-VL-7B-Instruct| Qwen2.5-VL-3B-Instruct | 92.8 |
|
| 40 |
-
| Qwen2.5-VL-7B-Instruct| Qwen2-VL-7B-Instruct | 93.1 |
|
| 41 |
-
| Qwen2.5-VL-7B-Instruct| Qwen2.5-VL-7B-Instruct | **98.1** |
|
| 42 |
-
| Qwen2.5-VL-7B-Instruct| Llama3-llava-next-8b-hf | _95.8_ |
|
| 43 |
-
| Qwen2.5-VL-7B-Instruct| Gemma-3-4b-it | 95.4 |
|
| 44 |
-
| Qwen2.5-VL-7B-Instruct| Gemma-3-12b-it | 94.5 |
|
| 45 |
-
| Llama3-llava-next-8b-hf| Qwen2.5-VL-3B-Instruct | 94.6 |
|
| 46 |
-
| Llama3-llava-next-8b-hf| Qwen2-VL-7B-Instruct | 90.0 |
|
| 47 |
-
| Llama3-llava-next-8b-hf| Qwen2.5-VL-7B-Instruct | 96.6 |
|
| 48 |
-
| Llama3-llava-next-8b-hf| Llama3-llava-next-8b-hf | _97.7_ |
|
| 49 |
-
| Llama3-llava-next-8b-hf| Gemma-3-4b-it | 99.1 |
|
| 50 |
-
| Llama3-llava-next-8b-hf| Gemma-3-12b-it | **99.5** |
|
| 51 |
-
| Gemma-3-4b-it | Qwen2.5-VL-3B-Instruct | 76.0 |
|
| 52 |
-
| Gemma-3-4b-it | Qwen2-VL-7B-Instruct | 71.5 |
|
| 53 |
-
| Gemma-3-4b-it | Qwen2.5-VL-7B-Instruct | 85.2 |
|
| 54 |
-
| Gemma-3-4b-it | Llama3-llava-next-8b-hf | 86.5 |
|
| 55 |
-
| Gemma-3-4b-it | Gemma-3-4b-it | **99.4** |
|
| 56 |
-
| Gemma-3-4b-it | Gemma-3-12b-it | _98.7_ |
|
| 57 |
-
| Gemma-3-12b-it | Qwen2.5-VL-3B-Instruct | 84.1 |
|
| 58 |
-
| Gemma-3-12b-it | Qwen2-VL-7B-Instruct | 81.3 |
|
| 59 |
-
| Gemma-3-12b-it | Qwen2.5-VL-7B-Instruct | 91.2 |
|
| 60 |
-
| Gemma-3-12b-it | Llama3-llava-next-8b-hf | 93.3 |
|
| 61 |
-
| Gemma-3-12b-it | Gemma-3-4b-it | _99.4_ |
|
| 62 |
-
| Gemma-3-12b-it | Gemma-3-12b-it | **99.7** |
|
| 63 |
-
|
|
|
|
| 20 |
|
| 21 |
Training and inference code can be obtained [here](https://github.com/ai-forever/data_leakage_detect).
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|