add new lines + imagenet
Browse files
README.md
CHANGED
|
@@ -180,12 +180,22 @@ TODO: beautiful plots of shots scaling laws.
|
|
| 180 |
| | 8 | 64.8 | 55.1 | 35.7 | 46.1 | 77.6 | 114.3 | 105.7 | 76.6 | 47.9 | 58.2 | - | 67.8 | - |
|
| 181 |
| | 16 | 65.4 | 56.8 | 36.3 | 48.3 | 81.4 | 116.6 | 107.0 | 80.1 | - | 55.8 | - | 67.7 | - |
|
| 182 |
| | 32 | 65.9 | 57.8 | 36.7 | 50.0 | 82.7 | 116.6 | 107.5 | 81.1 | - | 52.5 | - | 67.3 | - |
|
|
|
|
| 183 |
| IDEFIX 9B | 0 | 50.9 | 38.4 | 25.9 | 35.5 | 25.4 | 46.0 | 36.8 | 27.3 | 48.7 | 51.7 | 44.2 | 61.8 | 5.0 (16.8/20.8) |
|
| 184 |
| | 4 | 55.4 | 45.5 | 27.6 | 36.9 | 60.0 | 93.0 | 81.3 | 59.7 | 47.9 | 50.7 | 37.4 | 62.3 | - |
|
| 185 |
| | 8 | 56.4 | 47.7 | 27.5 | 40.4 | 63.2 | 97.0 | 86.8 | 61.9 | 47.6 | 51.0 | - | 66.3 | - |
|
| 186 |
| | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
|
| 187 |
| | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
|
| 188 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 189 |
Fairness Evaluations:
|
| 190 |
| Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |
|
| 191 |
|:-----------|--------:|----------------------------:|--------------------------:|-------------------------:|
|
|
@@ -194,6 +204,7 @@ Fairness Evaluations:
|
|
| 194 |
| | 8 | 95.5 | 52.3 | 53.1 |
|
| 195 |
| | 16 | 95.7 | 47.6 | 52.8 |
|
| 196 |
| | 32 | 95.7 | 36.5 | 51.2 |
|
|
|
|
| 197 |
| IDEFIX 9B | 0 | 94.4 | 55.3 | 45.1 |
|
| 198 |
| | 4 | 93.9 | 35.3 | 44.3 |
|
| 199 |
| | 8 | 95.4 | 44.7 | 46.0 |
|
|
|
|
| 180 |
| | 8 | 64.8 | 55.1 | 35.7 | 46.1 | 77.6 | 114.3 | 105.7 | 76.6 | 47.9 | 58.2 | - | 67.8 | - |
|
| 181 |
| | 16 | 65.4 | 56.8 | 36.3 | 48.3 | 81.4 | 116.6 | 107.0 | 80.1 | - | 55.8 | - | 67.7 | - |
|
| 182 |
| | 32 | 65.9 | 57.8 | 36.7 | 50.0 | 82.7 | 116.6 | 107.5 | 81.1 | - | 52.5 | - | 67.3 | - |
|
| 183 |
+
<br>
|
| 184 |
| IDEFIX 9B | 0 | 50.9 | 38.4 | 25.9 | 35.5 | 25.4 | 46.0 | 36.8 | 27.3 | 48.7 | 51.7 | 44.2 | 61.8 | 5.0 (16.8/20.8) |
|
| 185 |
| | 4 | 55.4 | 45.5 | 27.6 | 36.9 | 60.0 | 93.0 | 81.3 | 59.7 | 47.9 | 50.7 | 37.4 | 62.3 | - |
|
| 186 |
| | 8 | 56.4 | 47.7 | 27.5 | 40.4 | 63.2 | 97.0 | 86.8 | 61.9 | 47.6 | 51.0 | - | 66.3 | - |
|
| 187 |
| | 16 | 57.0 | 48.4 | 27.9 | 42.6 | 67.4 | 99.7 | 89.4 | 64.5 | - | 50.9 | - | 67.8 | - |
|
| 188 |
| | 32 | 57.9 | 49.6 | 28.3 | 43.7 | 68.1 | 98.0 | 90.5 | 64.4 | - | 49.8 | - | 67.0 | - |
|
| 189 |
|
| 190 |
+
Imagenet Evaluation:
|
| 191 |
+
| Model | Shots | Imagenet |
|
| 192 |
+
|:-----------|--------:|-----------:|
|
| 193 |
+
| IDEFIX 80B | 16, 1k support set | 65.4 |
|
| 194 |
+
| | 16, RICES 5k support set | 72.9 |
|
| 195 |
+
<br>
|
| 196 |
+
| IDEFIX 9B | 16, 1k support set | 53.5 |
|
| 197 |
+
| | 16, RICES 5k support set | 64.5 |
|
| 198 |
+
|
| 199 |
Fairness Evaluations:
|
| 200 |
| Model | Shots | FairFaceGender (accuracy) | FairFaceRace (accuracy) | FairFaceAge (accuracy) |
|
| 201 |
|:-----------|--------:|----------------------------:|--------------------------:|-------------------------:|
|
|
|
|
| 204 |
| | 8 | 95.5 | 52.3 | 53.1 |
|
| 205 |
| | 16 | 95.7 | 47.6 | 52.8 |
|
| 206 |
| | 32 | 95.7 | 36.5 | 51.2 |
|
| 207 |
+
<br>
|
| 208 |
| IDEFIX 9B | 0 | 94.4 | 55.3 | 45.1 |
|
| 209 |
| | 4 | 93.9 | 35.3 | 44.3 |
|
| 210 |
| | 8 | 95.4 | 44.7 | 46.0 |
|