added result analysis progress and daily update

Updates/daily_updates/210042026-DailyUpdate.md

@@ -0,0 +1,35 @@
+**Daily Update — Bad Case Analysis**
+*Image embedding benchmark · Tobacco-3482*
+
+---
+
+## Completed
+
+- Fixed a result logging issue where post-processed results were overwriting the raw embedding results in the checkpoint. Separated raw and post-processed entries so both are retained independently for comparison.
+- Ran bad case analysis for both SigLIP2 SO400M and ModernVBERT Bi using top-1 kNN criterion, loading embeddings directly from cached `.npy` files.
+- Generated per-class error breakdowns for all 10 document classes across both models.
+- Sampled misclassified cases per class and retrieved top-10 nearest neighbours for each query for visual inspection.
+- Saved full error breakdown to `results/bad_case_analysis.json` and bad case report to `.txt`.
+- Created retrieval confusion heatmap and retrieval error rate summary bar chart saved to `figures\bad_cases`
+- Visualised sampled bad cases with top-10 retrieval results per query, saved to `.png` at `figures\bad_cases`.
+
+---
+
+## In Progress
+
+- Improving the visual retrieval report resolution or switching to a more readable format (HTML or PDF) for better presentation — planned for tomorrow.
+- Writing up a bad case summary covering the key failure patterns identified and potential approaches to improve embedding quality for the affected classes.
+
+---
+
+## Blockers
+
+- The visualised bad cases is non-readable.
+
+---
+
+## Key Learnings
+
+- **Per-class errors reveal more than aggregate accuracy:** SigLIP2 and ModernVBERT Bi are only ~2.6 percentage points apart in overall error rate but ModernVBERT Bi is dramatically worse on Report specifically (70.4% vs 44.4%), which aggregate accuracy alone hides. Full breakdown in the bad case report.
+- **Systematic vs random errors:** The Memo/Letter confusion is consistent across both models and almost all Memo error cases, pointing to a structural embedding space overlap rather than noise — a more actionable signal than scattered errors.
+- **Low distance does not guarantee correct retrieval:** Some Scientific and Note cases in SigLIP2 show very low cosine distances to wrong-class neighbours, meaning the model is highly confident but wrong — suggesting genuine class overlap in embedding space rather than general uncertainty.

tabacco3482_benchmarks/figures/bad_cases/ModernVBERT_Bi/ModernVBERT_Bi_bad_cases_log.txt

@@ -0,0 +1,160 @@
+BAD CASE REPORT � ModernVBERT Bi
+Criterion: top-1 kNN neighbour (cosine) has wrong label
+Embedding files: embeddings/train_emb_ModernVBERT_Bi.npy, embeddings/test_emb_ModernVBERT_Bi.npy
+====================================================================
+
+Label: ADVE  (3 bad cases / 46 total test images)
+  Sampled 3 / 3 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         611  ADVE            News                0.2318  #1 News (d=0.232) | #2 News (d=0.274) | #3 News (d=0.283) | #4 News (d=0.283) | #5 News (d=0.285) | #6 News (d=0.287) | #7 News (d=0.293) | #8 News (d=0.295) | #9 News (d=0.318) | #10 News (d=0.320)
+         350  ADVE            Letter              0.4069  #1 Letter (d=0.407) | #2 Form (d=0.408) | #3 Form (d=0.423) | #4 Report (d=0.428) | #5 Form (d=0.430) | #6 Letter (d=0.432) | #7 Form (d=0.439) | #8 Memo (d=0.446) | #9 Form (d=0.447) | #10 Form (d=0.450)
+         388  ADVE            Form                0.3931  #1 Form (d=0.393) | #2 Form (d=0.403) | #3 ADVE (d=0.412) | #4 Memo (d=0.417) | #5 Form (d=0.420) | #6 ADVE (d=0.421) | #7 ADVE (d=0.422) | #8 Form (d=0.423) | #9 Form (d=0.424) | #10 Form (d=0.427)
+
+Label: Email  (10 bad cases / 115 total test images)
+  Sampled 10 / 10 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         300  Email           Form                0.3487  #1 Form (d=0.349) | #2 Letter (d=0.358) | #3 Memo (d=0.359) | #4 Memo (d=0.359) | #5 Email (d=0.360) | #6 Memo (d=0.362) | #7 Form (d=0.362) | #8 Memo (d=0.370) | #9 Letter (d=0.375) | #10 Form (d=0.376)
+         282  Email           Note                0.1683  #1 Note (d=0.168) | #2 Note (d=0.169) | #3 Note (d=0.175) | #4 Note (d=0.189) | #5 Note (d=0.213) | #6 Note (d=0.218) | #7 Note (d=0.220) | #8 Note (d=0.224) | #9 Note (d=0.224) | #10 Email (d=0.230)
+         577  Email           Letter              0.3793  #1 Letter (d=0.379) | #2 Letter (d=0.386) | #3 Memo (d=0.394) | #4 Memo (d=0.396) | #5 Letter (d=0.399) | #6 Email (d=0.403) | #7 Letter (d=0.404) | #8 Memo (d=0.407) | #9 Memo (d=0.408) | #10 Memo (d=0.413)
+         209  Email           Letter              0.3776  #1 Letter (d=0.378) | #2 Letter (d=0.392) | #3 Letter (d=0.392) | #4 Memo (d=0.397) | #5 Email (d=0.400) | #6 Email (d=0.403) | #7 Note (d=0.407) | #8 Email (d=0.408) | #9 Letter (d=0.408) | #10 Memo (d=0.410)
+         408  Email           Memo                0.3520  #1 Memo (d=0.352) | #2 Email (d=0.354) | #3 Email (d=0.385) | #4 Email (d=0.387) | #5 Email (d=0.391) | #6 Email (d=0.395) | #7 Note (d=0.398) | #8 Email (d=0.398) | #9 Memo (d=0.403) | #10 Email (d=0.404)
+         140  Email           Memo                0.3520  #1 Memo (d=0.352) | #2 Letter (d=0.362) | #3 Email (d=0.366) | #4 Letter (d=0.368) | #5 Letter (d=0.383) | #6 Memo (d=0.386) | #7 Letter (d=0.387) | #8 Form (d=0.392) | #9 Letter (d=0.393) | #10 Memo (d=0.394)
+         591  Email           Memo                0.3462  #1 Memo (d=0.346) | #2 Email (d=0.352) | #3 Email (d=0.353) | #4 Memo (d=0.356) | #5 Letter (d=0.359) | #6 Email (d=0.361) | #7 Email (d=0.363) | #8 Email (d=0.366) | #9 Email (d=0.368) | #10 Memo (d=0.370)
+         278  Email           Letter              0.3218  #1 Letter (d=0.322) | #2 Letter (d=0.340) | #3 Letter (d=0.341) | #4 Letter (d=0.344) | #5 Letter (d=0.345) | #6 Letter (d=0.358) | #7 Memo (d=0.358) | #8 Memo (d=0.360) | #9 Memo (d=0.369) | #10 Memo (d=0.370)
+         451  Email           Letter              0.3692  #1 Letter (d=0.369) | #2 Memo (d=0.380) | #3 Email (d=0.384) | #4 Email (d=0.386) | #5 Memo (d=0.386) | #6 Memo (d=0.390) | #7 Memo (d=0.395) | #8 Memo (d=0.397) | #9 Memo (d=0.398) | #10 Email (d=0.400)
+         513  Email           Memo                0.3850  #1 Memo (d=0.385) | #2 Letter (d=0.387) | #3 Email (d=0.392) | #4 Email (d=0.400) | #5 Memo (d=0.400) | #6 Memo (d=0.400) | #7 Email (d=0.401) | #8 Memo (d=0.406) | #9 Memo (d=0.408) | #10 Memo (d=0.411)
+
+Label: Form  (7 bad cases / 79 total test images)
+  Sampled 7 / 7 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+           0  Form            ADVE                0.4955  #1 ADVE (d=0.496) | #2 Letter (d=0.531) | #3 Memo (d=0.534) | #4 Email (d=0.535) | #5 News (d=0.537) | #6 Email (d=0.541) | #7 Memo (d=0.543) | #8 Email (d=0.546) | #9 Memo (d=0.547) | #10 Memo (d=0.548)
+         687  Form            Scientific          0.4300  #1 Scientific (d=0.430) | #2 Form (d=0.431) | #3 Form (d=0.434) | #4 Form (d=0.436) | #5 Form (d=0.444) | #6 Form (d=0.451) | #7 Scientific (d=0.456) | #8 Letter (d=0.457) | #9 Form (d=0.458) | #10 Form (d=0.465)
+         216  Form            Scientific          0.3385  #1 Scientific (d=0.338) | #2 Form (d=0.350) | #3 Form (d=0.354) | #4 Form (d=0.355) | #5 Scientific (d=0.360) | #6 Scientific (d=0.391) | #7 Form (d=0.392) | #8 Letter (d=0.399) | #9 Letter (d=0.408) | #10 Form (d=0.410)
+         334  Form            Letter              0.3538  #1 Letter (d=0.354) | #2 Letter (d=0.358) | #3 Letter (d=0.362) | #4 Letter (d=0.364) | #5 Letter (d=0.389) | #6 Letter (d=0.391) | #7 Memo (d=0.408) | #8 Form (d=0.409) | #9 Letter (d=0.412) | #10 Letter (d=0.413)
+         221  Form            Scientific          0.3732  #1 Scientific (d=0.373) | #2 Form (d=0.385) | #3 Letter (d=0.386) | #4 Letter (d=0.387) | #5 Letter (d=0.389) | #6 Letter (d=0.392) | #7 Form (d=0.395) | #8 Form (d=0.398) | #9 Memo (d=0.406) | #10 Letter (d=0.409)
+         544  Form            Report              0.4013  #1 Report (d=0.401) | #2 ADVE (d=0.461) | #3 News (d=0.468) | #4 Report (d=0.471) | #5 Memo (d=0.472) | #6 Letter (d=0.480) | #7 Letter (d=0.484) | #8 Letter (d=0.489) | #9 Letter (d=0.491) | #10 Letter (d=0.491)
+         242  Form            Letter              0.2728  #1 Letter (d=0.273) | #2 Letter (d=0.280) | #3 Form (d=0.296) | #4 Letter (d=0.310) | #5 Memo (d=0.324) | #6 Letter (d=0.337) | #7 Memo (d=0.344) | #8 Letter (d=0.346) | #9 Email (d=0.348) | #10 Letter (d=0.351)
+
+Label: Letter  (14 bad cases / 114 total test images)
+  Sampled 14 / 14 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         415  Letter          Form                0.2615  #1 Form (d=0.261) | #2 Letter (d=0.289) | #3 Email (d=0.296) | #4 Letter (d=0.303) | #5 Memo (d=0.325) | #6 Letter (d=0.327) | #7 Letter (d=0.334) | #8 Letter (d=0.335) | #9 Letter (d=0.340) | #10 Letter (d=0.346)
+         406  Letter          Report              0.2746  #1 Report (d=0.275) | #2 Letter (d=0.283) | #3 Letter (d=0.296) | #4 Letter (d=0.307) | #5 Letter (d=0.308) | #6 Letter (d=0.309) | #7 Letter (d=0.314) | #8 Letter (d=0.316) | #9 Letter (d=0.317) | #10 Letter (d=0.318)
+         549  Letter          Memo                0.3119  #1 Memo (d=0.312) | #2 Letter (d=0.318) | #3 Letter (d=0.320) | #4 Letter (d=0.325) | #5 Letter (d=0.329) | #6 Memo (d=0.336) | #7 News (d=0.338) | #8 Letter (d=0.349) | #9 Memo (d=0.349) | #10 Letter (d=0.350)
+         315  Letter          Memo                0.2740  #1 Memo (d=0.274) | #2 Letter (d=0.321) | #3 Memo (d=0.358) | #4 Memo (d=0.361) | #5 Letter (d=0.362) | #6 Email (d=0.364) | #7 News (d=0.378) | #8 News (d=0.380) | #9 Letter (d=0.383) | #10 Memo (d=0.387)
+         263  Letter          Email               0.3239  #1 Email (d=0.324) | #2 Letter (d=0.326) | #3 Letter (d=0.326) | #4 Letter (d=0.330) | #5 Letter (d=0.341) | #6 Letter (d=0.345) | #7 Memo (d=0.351) | #8 Letter (d=0.354) | #9 Letter (d=0.357) | #10 Letter (d=0.358)
+         151  Letter          Report              0.2492  #1 Report (d=0.249) | #2 Letter (d=0.281) | #3 Letter (d=0.284) | #4 Letter (d=0.289) | #5 Memo (d=0.297) | #6 Letter (d=0.300) | #7 Letter (d=0.307) | #8 Letter (d=0.309) | #9 Letter (d=0.311) | #10 Memo (d=0.313)
+         285  Letter          Form                0.3094  #1 Form (d=0.309) | #2 Memo (d=0.327) | #3 Form (d=0.328) | #4 Form (d=0.342) | #5 Memo (d=0.346) | #6 Memo (d=0.351) | #7 Letter (d=0.352) | #8 Email (d=0.352) | #9 Letter (d=0.354) | #10 Memo (d=0.355)
+         187  Letter          Memo                0.2814  #1 Memo (d=0.281) | #2 Letter (d=0.332) | #3 Memo (d=0.345) | #4 Letter (d=0.348) | #5 Letter (d=0.349) | #6 Letter (d=0.351) | #7 Letter (d=0.351) | #8 Memo (d=0.354) | #9 Letter (d=0.354) | #10 Memo (d=0.356)
+         104  Letter          Memo                0.3440  #1 Memo (d=0.344) | #2 Letter (d=0.348) | #3 Letter (d=0.350) | #4 Memo (d=0.350) | #5 Letter (d=0.354) | #6 Memo (d=0.355) | #7 Memo (d=0.358) | #8 Letter (d=0.359) | #9 Letter (d=0.360) | #10 Memo (d=0.362)
+          41  Letter          Memo                0.2789  #1 Memo (d=0.279) | #2 Memo (d=0.290) | #3 Memo (d=0.294) | #4 Memo (d=0.300) | #5 Memo (d=0.302) | #6 Memo (d=0.303) | #7 Memo (d=0.319) | #8 Letter (d=0.321) | #9 Letter (d=0.324) | #10 Letter (d=0.325)
+          93  Letter          Scientific          0.2923  #1 Scientific (d=0.292) | #2 Report (d=0.330) | #3 Letter (d=0.340) | #4 Scientific (d=0.346) | #5 Scientific (d=0.347) | #6 Scientific (d=0.349) | #7 Scientific (d=0.351) | #8 Report (d=0.353) | #9 Report (d=0.362) | #10 Scientific (d=0.362)
+         199  Letter          Report              0.2437  #1 Report (d=0.244) | #2 Letter (d=0.268) | #3 Letter (d=0.274) | #4 Memo (d=0.280) | #5 Letter (d=0.284) | #6 Letter (d=0.285) | #7 Memo (d=0.290) | #8 Letter (d=0.291) | #9 Letter (d=0.292) | #10 Memo (d=0.294)
+         478  Letter          Scientific          0.3063  #1 Scientific (d=0.306) | #2 Memo (d=0.317) | #3 Note (d=0.324) | #4 Letter (d=0.328) | #5 Letter (d=0.332) | #6 Letter (d=0.334) | #7 Letter (d=0.338) | #8 Letter (d=0.341) | #9 Letter (d=0.342) | #10 Letter (d=0.342)
+         336  Letter          Note                0.3349  #1 Note (d=0.335) | #2 Letter (d=0.337) | #3 Letter (d=0.341) | #4 Letter (d=0.342) | #5 Letter (d=0.345) | #6 Letter (d=0.350) | #7 Letter (d=0.351) | #8 Letter (d=0.352) | #9 Letter (d=0.352) | #10 Letter (d=0.353)
+
+Label: Memo  (15 bad cases / 130 total test images)
+  Sampled 15 / 15 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         126  Memo            Letter              0.3098  #1 Letter (d=0.310) | #2 Letter (d=0.313) | #3 Letter (d=0.329) | #4 Letter (d=0.340) | #5 Letter (d=0.342) | #6 Memo (d=0.342) | #7 Letter (d=0.348) | #8 Letter (d=0.349) | #9 Memo (d=0.350) | #10 Memo (d=0.357)
+          63  Memo            Letter              0.3687  #1 Letter (d=0.369) | #2 Memo (d=0.375) | #3 Memo (d=0.381) | #4 Letter (d=0.382) | #5 Memo (d=0.383) | #6 Letter (d=0.386) | #7 Letter (d=0.390) | #8 Letter (d=0.394) | #9 Letter (d=0.395) | #10 Letter (d=0.395)
+         105  Memo            Letter              0.3330  #1 Letter (d=0.333) | #2 Letter (d=0.334) | #3 Letter (d=0.346) | #4 Letter (d=0.347) | #5 Memo (d=0.348) | #6 Report (d=0.349) | #7 Letter (d=0.349) | #8 Memo (d=0.353) | #9 Memo (d=0.359) | #10 Letter (d=0.359)
+         164  Memo            Letter              0.3198  #1 Letter (d=0.320) | #2 Memo (d=0.328) | #3 Letter (d=0.340) | #4 Letter (d=0.354) | #5 Memo (d=0.358) | #6 Memo (d=0.361) | #7 Letter (d=0.365) | #8 Letter (d=0.366) | #9 Memo (d=0.373) | #10 Letter (d=0.373)
+          26  Memo            Letter              0.2984  #1 Letter (d=0.298) | #2 Letter (d=0.323) | #3 Letter (d=0.330) | #4 Memo (d=0.334) | #5 Memo (d=0.336) | #6 Memo (d=0.336) | #7 Letter (d=0.339) | #8 Memo (d=0.345) | #9 Letter (d=0.346) | #10 Memo (d=0.347)
+         434  Memo            News                0.3803  #1 News (d=0.380) | #2 News (d=0.386) | #3 Memo (d=0.389) | #4 Memo (d=0.396) | #5 Memo (d=0.397) | #6 News (d=0.400) | #7 Letter (d=0.401) | #8 Letter (d=0.405) | #9 Memo (d=0.405) | #10 Memo (d=0.408)
+         186  Memo            Letter              0.3520  #1 Letter (d=0.352) | #2 Memo (d=0.357) | #3 Memo (d=0.365) | #4 Memo (d=0.372) | #5 Memo (d=0.375) | #6 Memo (d=0.377) | #7 Memo (d=0.380) | #8 Memo (d=0.399) | #9 Letter (d=0.402) | #10 Memo (d=0.405)
+         352  Memo            Letter              0.3054  #1 Letter (d=0.305) | #2 Memo (d=0.323) | #3 Memo (d=0.350) | #4 Memo (d=0.351) | #5 Letter (d=0.358) | #6 Memo (d=0.366) | #7 Memo (d=0.368) | #8 Memo (d=0.368) | #9 Memo (d=0.369) | #10 Memo (d=0.370)
+         524  Memo            ADVE                0.3383  #1 ADVE (d=0.338) | #2 ADVE (d=0.368) | #3 News (d=0.371) | #4 ADVE (d=0.387) | #5 ADVE (d=0.389) | #6 ADVE (d=0.393) | #7 Memo (d=0.397) | #8 Memo (d=0.402) | #9 Form (d=0.405) | #10 News (d=0.407)
+         220  Memo            Letter              0.2869  #1 Letter (d=0.287) | #2 Letter (d=0.299) | #3 Letter (d=0.302) | #4 Memo (d=0.307) | #5 Letter (d=0.307) | #6 Memo (d=0.313) | #7 Note (d=0.315) | #8 Memo (d=0.316) | #9 Memo (d=0.316) | #10 Letter (d=0.319)
+         529  Memo            Email               0.3564  #1 Email (d=0.356) | #2 Memo (d=0.365) | #3 Memo (d=0.368) | #4 Memo (d=0.368) | #5 Memo (d=0.369) | #6 Letter (d=0.369) | #7 Memo (d=0.372) | #8 Email (d=0.372) | #9 Email (d=0.376) | #10 Email (d=0.376)
+         469  Memo            Letter              0.3605  #1 Letter (d=0.360) | #2 Letter (d=0.366) | #3 Memo (d=0.368) | #4 Letter (d=0.368) | #5 Memo (d=0.369) | #6 Memo (d=0.372) | #7 Memo (d=0.377) | #8 Memo (d=0.378) | #9 Memo (d=0.381) | #10 Memo (d=0.383)
+          18  Memo            Letter              0.3487  #1 Letter (d=0.349) | #2 Memo (d=0.370) | #3 Letter (d=0.392) | #4 Letter (d=0.392) | #5 Memo (d=0.393) | #6 Letter (d=0.396) | #7 News (d=0.405) | #8 Letter (d=0.406) | #9 Memo (d=0.411) | #10 News (d=0.413)
+         204  Memo            Form                0.2935  #1 Form (d=0.293) | #2 Form (d=0.300) | #3 Form (d=0.310) | #4 Memo (d=0.315) | #5 Memo (d=0.317) | #6 Form (d=0.319) | #7 Memo (d=0.320) | #8 Memo (d=0.321) | #9 Form (d=0.321) | #10 Form (d=0.323)
+         519  Memo            Letter              0.3567  #1 Letter (d=0.357) | #2 Letter (d=0.381) | #3 Form (d=0.382) | #4 Form (d=0.387) | #5 Form (d=0.388) | #6 Form (d=0.388) | #7 Memo (d=0.392) | #8 Form (d=0.398) | #9 Form (d=0.399) | #10 Letter (d=0.399)
+
+Label: News  (8 bad cases / 44 total test images)
+  Sampled 8 / 8 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         567  News            ADVE                0.4363  #1 ADVE (d=0.436) | #2 Note (d=0.442) | #3 News (d=0.443) | #4 Note (d=0.446) | #5 Scientific (d=0.451) | #6 Note (d=0.451) | #7 Note (d=0.452) | #8 Note (d=0.458) | #9 Note (d=0.458) | #10 Note (d=0.463)
+           1  News            ADVE                0.2943  #1 ADVE (d=0.294) | #2 ADVE (d=0.299) | #3 News (d=0.324) | #4 News (d=0.329) | #5 News (d=0.335) | #6 News (d=0.339) | #7 News (d=0.340) | #8 News (d=0.340) | #9 ADVE (d=0.341) | #10 News (d=0.345)
+         522  News            Letter              0.4074  #1 Letter (d=0.407) | #2 Scientific (d=0.410) | #3 News (d=0.413) | #4 Scientific (d=0.417) | #5 Scientific (d=0.418) | #6 News (d=0.420) | #7 Scientific (d=0.421) | #8 Scientific (d=0.422) | #9 Scientific (d=0.422) | #10 Report (d=0.423)
+         454  News            Letter              0.3522  #1 Letter (d=0.352) | #2 Letter (d=0.379) | #3 Letter (d=0.381) | #4 Letter (d=0.386) | #5 ADVE (d=0.393) | #6 Memo (d=0.398) | #7 Letter (d=0.398) | #8 Letter (d=0.399) | #9 News (d=0.401) | #10 News (d=0.402)
+         523  News            Scientific          0.3444  #1 Scientific (d=0.344) | #2 News (d=0.344) | #3 News (d=0.351) | #4 News (d=0.355) | #5 News (d=0.356) | #6 News (d=0.358) | #7 News (d=0.360) | #8 News (d=0.364) | #9 Letter (d=0.365) | #10 News (d=0.365)
+         475  News            ADVE                0.3280  #1 ADVE (d=0.328) | #2 News (d=0.343) | #3 ADVE (d=0.347) | #4 ADVE (d=0.349) | #5 ADVE (d=0.358) | #6 ADVE (d=0.359) | #7 ADVE (d=0.359) | #8 ADVE (d=0.366) | #9 News (d=0.369) | #10 ADVE (d=0.372)
+         596  News            Letter              0.3751  #1 Letter (d=0.375) | #2 News (d=0.382) | #3 News (d=0.389) | #4 News (d=0.399) | #5 News (d=0.400) | #6 News (d=0.403) | #7 News (d=0.403) | #8 Letter (d=0.405) | #9 Letter (d=0.405) | #10 News (d=0.409)
+         234  News            Scientific          0.4692  #1 Scientific (d=0.469) | #2 Letter (d=0.477) | #3 News (d=0.489) | #4 Email (d=0.500) | #5 News (d=0.500) | #6 Scientific (d=0.500) | #7 News (d=0.500) | #8 News (d=0.502) | #9 News (d=0.502) | #10 News (d=0.504)
+
+Label: Note  (11 bad cases / 32 total test images)
+  Sampled 11 / 11 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         107  Note            Letter              0.3132  #1 Letter (d=0.313) | #2 Letter (d=0.318) | #3 Memo (d=0.325) | #4 Letter (d=0.325) | #5 Memo (d=0.325) | #6 Letter (d=0.339) | #7 Letter (d=0.339) | #8 Letter (d=0.340) | #9 Letter (d=0.345) | #10 Letter (d=0.346)
+         218  Note            Email               0.3145  #1 Email (d=0.314) | #2 Email (d=0.325) | #3 Email (d=0.328) | #4 Note (d=0.339) | #5 Email (d=0.339) | #6 Email (d=0.343) | #7 Email (d=0.343) | #8 Email (d=0.344) | #9 Note (d=0.344) | #10 Email (d=0.346)
+         247  Note            News                0.4211  #1 News (d=0.421) | #2 ADVE (d=0.438) | #3 News (d=0.441) | #4 News (d=0.447) | #5 ADVE (d=0.450) | #6 News (d=0.455) | #7 News (d=0.462) | #8 News (d=0.462) | #9 News (d=0.467) | #10 Note (d=0.467)
+         115  Note            Letter              0.4791  #1 Letter (d=0.479) | #2 Letter (d=0.496) | #3 Note (d=0.502) | #4 Note (d=0.509) | #5 Report (d=0.512) | #6 Scientific (d=0.512) | #7 Memo (d=0.519) | #8 Scientific (d=0.523) | #9 Report (d=0.525) | #10 Scientific (d=0.526)
+         270  Note            Letter              0.3990  #1 Letter (d=0.399) | #2 Letter (d=0.434) | #3 Letter (d=0.446) | #4 Note (d=0.448) | #5 Letter (d=0.449) | #6 Letter (d=0.450) | #7 Email (d=0.451) | #8 Letter (d=0.454) | #9 Letter (d=0.457) | #10 Letter (d=0.459)
+         321  Note            Letter              0.3540  #1 Letter (d=0.354) | #2 Form (d=0.355) | #3 Letter (d=0.356) | #4 Letter (d=0.370) | #5 Letter (d=0.373) | #6 Letter (d=0.384) | #7 Letter (d=0.388) | #8 Letter (d=0.390) | #9 Memo (d=0.392) | #10 Letter (d=0.393)
+         662  Note            Memo                0.4181  #1 Memo (d=0.418) | #2 Memo (d=0.421) | #3 Memo (d=0.422) | #4 Memo (d=0.425) | #5 Memo (d=0.429) | #6 Memo (d=0.439) | #7 Note (d=0.446) | #8 Memo (d=0.447) | #9 Note (d=0.451) | #10 Memo (d=0.454)
+         602  Note            Form                0.3743  #1 Form (d=0.374) | #2 Note (d=0.392) | #3 Note (d=0.397) | #4 Form (d=0.422) | #5 Note (d=0.423) | #6 Note (d=0.426) | #7 Note (d=0.427) | #8 Note (d=0.456) | #9 Note (d=0.458) | #10 Note (d=0.487)
+         260  Note            Email               0.2535  #1 Email (d=0.254) | #2 Email (d=0.258) | #3 Email (d=0.268) | #4 Email (d=0.270) | #5 Email (d=0.271) | #6 Email (d=0.281) | #7 Email (d=0.287) | #8 Email (d=0.292) | #9 Email (d=0.294) | #10 Note (d=0.298)
+         194  Note            Email               0.3181  #1 Email (d=0.318) | #2 Note (d=0.324) | #3 Email (d=0.324) | #4 Email (d=0.328) | #5 Email (d=0.331) | #6 Email (d=0.331) | #7 Email (d=0.333) | #8 Note (d=0.335) | #9 Email (d=0.335) | #10 Email (d=0.336)
+         553  Note            Letter              0.4768  #1 Letter (d=0.477) | #2 ADVE (d=0.480) | #3 Note (d=0.493) | #4 Memo (d=0.497) | #5 Memo (d=0.500) | #6 Email (d=0.502) | #7 Memo (d=0.505) | #8 Memo (d=0.506) | #9 Memo (d=0.506) | #10 Email (d=0.509)
+
+Label: Report  (38 bad cases / 54 total test images)
+  Sampled 20 / 38 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         139  Report          News                0.4460  #1 News (d=0.446) | #2 Report (d=0.466) | #3 Report (d=0.470) | #4 Letter (d=0.476) | #5 Report (d=0.479) | #6 Report (d=0.490) | #7 Report (d=0.490) | #8 News (d=0.491) | #9 News (d=0.495) | #10 Report (d=0.495)
+         496  Report          Scientific          0.4556  #1 Scientific (d=0.456) | #2 Report (d=0.516) | #3 Scientific (d=0.525) | #4 Report (d=0.529) | #5 ADVE (d=0.531) | #6 Scientific (d=0.534) | #7 Scientific (d=0.534) | #8 Memo (d=0.536) | #9 Report (d=0.541) | #10 Form (d=0.542)
+         482  Report          Memo                0.4308  #1 Memo (d=0.431) | #2 Memo (d=0.461) | #3 Scientific (d=0.465) | #4 Report (d=0.466) | #5 Report (d=0.470) | #6 Report (d=0.500) | #7 Scientific (d=0.504) | #8 Memo (d=0.513) | #9 Report (d=0.517) | #10 News (d=0.518)
+         276  Report          Form                0.4605  #1 Form (d=0.460) | #2 Letter (d=0.477) | #3 Report (d=0.486) | #4 Form (d=0.487) | #5 Report (d=0.489) | #6 Form (d=0.492) | #7 Memo (d=0.500) | #8 Scientific (d=0.503) | #9 Form (d=0.506) | #10 Report (d=0.507)
+         381  Report          Scientific          0.3691  #1 Scientific (d=0.369) | #2 Scientific (d=0.387) | #3 Letter (d=0.389) | #4 News (d=0.392) | #5 Report (d=0.394) | #6 Scientific (d=0.402) | #7 Report (d=0.404) | #8 News (d=0.405) | #9 News (d=0.405) | #10 Scientific (d=0.412)
+          77  Report          Scientific          0.4089  #1 Scientific (d=0.409) | #2 Report (d=0.414) | #3 Scientific (d=0.432) | #4 News (d=0.441) | #5 Report (d=0.449) | #6 News (d=0.455) | #7 Report (d=0.456) | #8 ADVE (d=0.456) | #9 Report (d=0.457) | #10 Scientific (d=0.457)
+         681  Report          News                0.3594  #1 News (d=0.359) | #2 Report (d=0.373) | #3 News (d=0.378) | #4 Report (d=0.388) | #5 Report (d=0.391) | #6 Report (d=0.396) | #7 Scientific (d=0.398) | #8 News (d=0.399) | #9 Scientific (d=0.399) | #10 News (d=0.404)
+         664  Report          News                0.3745  #1 News (d=0.374) | #2 News (d=0.420) | #3 Memo (d=0.423) | #4 Report (d=0.428) | #5 Letter (d=0.433) | #6 ADVE (d=0.436) | #7 News (d=0.438) | #8 Report (d=0.439) | #9 News (d=0.443) | #10 Report (d=0.446)
+         676  Report          News                0.3645  #1 News (d=0.365) | #2 Memo (d=0.389) | #3 News (d=0.412) | #4 News (d=0.415) | #5 News (d=0.416) | #6 News (d=0.418) | #7 Report (d=0.419) | #8 News (d=0.420) | #9 Report (d=0.424) | #10 Letter (d=0.429)
+         108  Report          Letter              0.2985  #1 Letter (d=0.299) | #2 Memo (d=0.299) | #3 Memo (d=0.305) | #4 Memo (d=0.310) | #5 Letter (d=0.317) | #6 Memo (d=0.318) | #7 Email (d=0.319) | #8 Memo (d=0.321) | #9 Letter (d=0.322) | #10 Memo (d=0.326)
+          84  Report          Memo                0.3184  #1 Memo (d=0.318) | #2 Form (d=0.321) | #3 Report (d=0.323) | #4 Form (d=0.332) | #5 Scientific (d=0.333) | #6 Letter (d=0.344) | #7 Form (d=0.350) | #8 Form (d=0.354) | #9 Memo (d=0.369) | #10 Form (d=0.369)
+         297  Report          Scientific          0.3634  #1 Scientific (d=0.363) | #2 Scientific (d=0.370) | #3 Scientific (d=0.378) | #4 Scientific (d=0.378) | #5 Scientific (d=0.378) | #6 Scientific (d=0.380) | #7 Scientific (d=0.382) | #8 Scientific (d=0.402) | #9 Scientific (d=0.406) | #10 Scientific (d=0.409)
+         205  Report          Memo                0.4446  #1 Memo (d=0.445) | #2 Memo (d=0.456) | #3 Memo (d=0.456) | #4 Report (d=0.458) | #5 Report (d=0.460) | #6 Letter (d=0.464) | #7 Report (d=0.468) | #8 Memo (d=0.472) | #9 Report (d=0.477) | #10 Memo (d=0.480)
+         120  Report          News                0.4246  #1 News (d=0.425) | #2 News (d=0.453) | #3 Email (d=0.462) | #4 News (d=0.462) | #5 Letter (d=0.464) | #6 Letter (d=0.465) | #7 ADVE (d=0.466) | #8 Letter (d=0.468) | #9 News (d=0.479) | #10 News (d=0.483)
+         459  Report          Memo                0.3366  #1 Memo (d=0.337) | #2 Letter (d=0.347) | #3 Memo (d=0.348) | #4 Report (d=0.348) | #5 Letter (d=0.358) | #6 Letter (d=0.359) | #7 Letter (d=0.365) | #8 Memo (d=0.367) | #9 Memo (d=0.367) | #10 Memo (d=0.368)
+         674  Report          Scientific          0.3403  #1 Scientific (d=0.340) | #2 Scientific (d=0.373) | #3 Report (d=0.378) | #4 Report (d=0.385) | #5 Scientific (d=0.386) | #6 Memo (d=0.386) | #7 Scientific (d=0.387) | #8 Scientific (d=0.388) | #9 Report (d=0.392) | #10 News (d=0.393)
+         571  Report          Letter              0.3670  #1 Letter (d=0.367) | #2 Letter (d=0.373) | #3 Letter (d=0.373) | #4 Memo (d=0.375) | #5 Memo (d=0.376) | #6 Letter (d=0.379) | #7 Report (d=0.394) | #8 Letter (d=0.402) | #9 Letter (d=0.404) | #10 Report (d=0.405)
+         558  Report          Form                0.3942  #1 Form (d=0.394) | #2 Letter (d=0.396) | #3 Memo (d=0.398) | #4 Report (d=0.399) | #5 Letter (d=0.399) | #6 Letter (d=0.400) | #7 Memo (d=0.400) | #8 Letter (d=0.405) | #9 Email (d=0.406) | #10 Letter (d=0.406)
+         582  Report          Form                0.4036  #1 Form (d=0.404) | #2 Report (d=0.405) | #3 Form (d=0.415) | #4 Form (d=0.415) | #5 Form (d=0.421) | #6 ADVE (d=0.429) | #7 Scientific (d=0.449) | #8 Form (d=0.458) | #9 Form (d=0.464) | #10 Form (d=0.470)
+          27  Report          Form                0.3438  #1 Form (d=0.344) | #2 Scientific (d=0.399) | #3 Form (d=0.401) | #4 Scientific (d=0.401) | #5 Scientific (d=0.406) | #6 Form (d=0.409) | #7 Report (d=0.415) | #8 Form (d=0.416) | #9 Report (d=0.416) | #10 Form (d=0.419)
+
+Label: Resume  (0 bad cases / 24 total test images)
+  -> 0 bad cases � perfect top-1 retrieval
+
+Label: Scientific  (21 bad cases / 59 total test images)
+  Sampled 20 / 21 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         295  Scientific      Report              0.3935  #1 Report (d=0.394) | #2 Scientific (d=0.399) | #3 Scientific (d=0.403) | #4 Scientific (d=0.429) | #5 Report (d=0.433) | #6 Scientific (d=0.440) | #7 Report (d=0.440) | #8 Scientific (d=0.442) | #9 Scientific (d=0.446) | #10 Scientific (d=0.450)
+          89  Scientific      Letter              0.3336  #1 Letter (d=0.334) | #2 Scientific (d=0.348) | #3 Scientific (d=0.354) | #4 Scientific (d=0.360) | #5 Scientific (d=0.362) | #6 Scientific (d=0.365) | #7 Letter (d=0.365) | #8 Scientific (d=0.369) | #9 Scientific (d=0.369) | #10 Letter (d=0.374)
+         377  Scientific      Report              0.3241  #1 Report (d=0.324) | #2 Report (d=0.382) | #3 Report (d=0.394) | #4 Scientific (d=0.397) | #5 Scientific (d=0.398) | #6 Resume (d=0.416) | #7 Letter (d=0.422) | #8 Letter (d=0.424) | #9 Letter (d=0.431) | #10 Letter (d=0.434)
+         447  Scientific      Form                0.4046  #1 Form (d=0.405) | #2 Scientific (d=0.441) | #3 Form (d=0.500) | #4 Form (d=0.503) | #5 Form (d=0.508) | #6 Form (d=0.511) | #7 Form (d=0.516) | #8 Note (d=0.522) | #9 Scientific (d=0.526) | #10 Note (d=0.526)
+         355  Scientific      Letter              0.3340  #1 Letter (d=0.334) | #2 Letter (d=0.342) | #3 Report (d=0.349) | #4 News (d=0.350) | #5 Report (d=0.352) | #6 Memo (d=0.353) | #7 Memo (d=0.358) | #8 Scientific (d=0.366) | #9 Letter (d=0.366) | #10 Report (d=0.371)
+         117  Scientific      News                0.4341  #1 News (d=0.434) | #2 Scientific (d=0.517) | #3 Report (d=0.519) | #4 Letter (d=0.521) | #5 Letter (d=0.525) | #6 Letter (d=0.533) | #7 Letter (d=0.536) | #8 Memo (d=0.539) | #9 Letter (d=0.542) | #10 Letter (d=0.543)
+         154  Scientific      News                0.3339  #1 News (d=0.334) | #2 News (d=0.361) | #3 News (d=0.364) | #4 Scientific (d=0.368) | #5 News (d=0.370) | #6 Report (d=0.376) | #7 News (d=0.380) | #8 News (d=0.388) | #9 Report (d=0.390) | #10 Scientific (d=0.391)
+         376  Scientific      Report              0.3370  #1 Report (d=0.337) | #2 Scientific (d=0.384) | #3 Scientific (d=0.387) | #4 Scientific (d=0.398) | #5 Report (d=0.409) | #6 Scientific (d=0.423) | #7 Scientific (d=0.423) | #8 Scientific (d=0.432) | #9 Memo (d=0.437) | #10 Scientific (d=0.438)
+         395  Scientific      Report              0.3936  #1 Report (d=0.394) | #2 Report (d=0.405) | #3 Scientific (d=0.407) | #4 Report (d=0.413) | #5 Scientific (d=0.447) | #6 Scientific (d=0.454) | #7 Report (d=0.463) | #8 Scientific (d=0.467) | #9 Scientific (d=0.468) | #10 News (d=0.469)
+         215  Scientific      News                0.2904  #1 News (d=0.290) | #2 News (d=0.303) | #3 News (d=0.310) | #4 Report (d=0.322) | #5 News (d=0.325) | #6 Scientific (d=0.343) | #7 News (d=0.346) | #8 News (d=0.347) | #9 Letter (d=0.351) | #10 News (d=0.356)
+         498  Scientific      Letter              0.4570  #1 Letter (d=0.457) | #2 News (d=0.470) | #3 News (d=0.470) | #4 Report (d=0.472) | #5 News (d=0.473) | #6 News (d=0.475) | #7 News (d=0.477) | #8 Memo (d=0.484) | #9 News (d=0.484) | #10 ADVE (d=0.485)
+         680  Scientific      Report              0.4478  #1 Report (d=0.448) | #2 Letter (d=0.460) | #3 Scientific (d=0.465) | #4 Scientific (d=0.469) | #5 Scientific (d=0.471) | #6 Report (d=0.476) | #7 Scientific (d=0.478) | #8 Report (d=0.484) | #9 Scientific (d=0.492) | #10 Scientific (d=0.492)
+         243  Scientific      Report              0.4006  #1 Report (d=0.401) | #2 Scientific (d=0.413) | #3 Report (d=0.415) | #4 Report (d=0.423) | #5 Letter (d=0.434) | #6 Letter (d=0.435) | #7 Letter (d=0.440) | #8 Letter (d=0.444) | #9 Memo (d=0.445) | #10 Letter (d=0.446)
+         450  Scientific      Letter              0.3602  #1 Letter (d=0.360) | #2 Memo (d=0.372) | #3 Memo (d=0.388) | #4 Letter (d=0.397) | #5 Letter (d=0.398) | #6 Memo (d=0.399) | #7 Memo (d=0.401) | #8 Form (d=0.403) | #9 Memo (d=0.408) | #10 Note (d=0.409)
+         641  Scientific      Letter              0.4666  #1 Letter (d=0.467) | #2 Report (d=0.469) | #3 Letter (d=0.476) | #4 Letter (d=0.487) | #5 Report (d=0.494) | #6 Report (d=0.496) | #7 Letter (d=0.500) | #8 Letter (d=0.500) | #9 Letter (d=0.502) | #10 Letter (d=0.506)
+         501  Scientific      Report              0.4275  #1 Report (d=0.428) | #2 Report (d=0.482) | #3 Report (d=0.484) | #4 Report (d=0.493) | #5 Scientific (d=0.504) | #6 Letter (d=0.505) | #7 Report (d=0.514) | #8 Report (d=0.519) | #9 News (d=0.521) | #10 Report (d=0.523)
+         508  Scientific      Letter              0.3736  #1 Letter (d=0.374) | #2 Letter (d=0.387) | #3 Letter (d=0.388) | #4 Letter (d=0.395) | #5 Letter (d=0.395) | #6 Letter (d=0.397) | #7 Letter (d=0.398) | #8 Letter (d=0.399) | #9 Letter (d=0.402) | #10 Letter (d=0.403)
+         157  Scientific      Report              0.4531  #1 Report (d=0.453) | #2 Report (d=0.468) | #3 Scientific (d=0.475) | #4 Report (d=0.481) | #5 Scientific (d=0.494) | #6 Scientific (d=0.502) | #7 Letter (d=0.502) | #8 Scientific (d=0.503) | #9 Scientific (d=0.505) | #10 Scientific (d=0.511)
+         612  Scientific      Form                0.3150  #1 Form (d=0.315) | #2 Form (d=0.331) | #3 Scientific (d=0.337) | #4 Form (d=0.349) | #5 Scientific (d=0.362) | #6 Form (d=0.392) | #7 Note (d=0.394) | #8 Form (d=0.414) | #9 Form (d=0.414) | #10 Form (d=0.415)
+         617  Scientific      News                0.4460  #1 News (d=0.446) | #2 News (d=0.453) | #3 Form (d=0.459) | #4 Report (d=0.462) | #5 ADVE (d=0.466) | #6 ADVE (d=0.472) | #7 ADVE (d=0.472) | #8 News (d=0.476) | #9 News (d=0.479) | #10 Note (d=0.480)

tabacco3482_benchmarks/figures/bad_cases/SigLIP2_SO400M_patch14-384/SigLIP2_SO400M_patch14-384_bad_cases_log.txt

@@ -0,0 +1,156 @@
+BAD CASE REPORT � SigLIP2 SO400M
+Criterion: top-1 kNN neighbour (cosine) has wrong label
+Embedding files: embeddings/train_emb_SigLIP2_SO400M_patch14-384.npy, embeddings/test_emb_SigLIP2_SO400M_patch14-384.npy
+====================================================================
+
+Label: ADVE  (5 bad cases / 46 total test images)
+  Sampled 5 / 5 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         192  ADVE            News                0.1927  #1 News (d=0.193) | #2 Note (d=0.196) | #3 Note (d=0.205) | #4 Note (d=0.219) | #5 ADVE (d=0.220) | #6 Email (d=0.222) | #7 Note (d=0.224) | #8 Note (d=0.224) | #9 Note (d=0.225) | #10 ADVE (d=0.225)
+         634  ADVE            Note                0.1382  #1 Note (d=0.138) | #2 Note (d=0.166) | #3 Email (d=0.168) | #4 Note (d=0.172) | #5 Note (d=0.174) | #6 Note (d=0.177) | #7 Note (d=0.181) | #8 ADVE (d=0.181) | #9 Note (d=0.182) | #10 Note (d=0.183)
+         388  ADVE            Memo                0.1818  #1 Memo (d=0.182) | #2 Note (d=0.190) | #3 Memo (d=0.195) | #4 News (d=0.198) | #5 Memo (d=0.211) | #6 Form (d=0.221) | #7 Note (d=0.222) | #8 Note (d=0.222) | #9 Note (d=0.223) | #10 Note (d=0.223)
+         291  ADVE            News                0.2253  #1 News (d=0.225) | #2 ADVE (d=0.234) | #3 ADVE (d=0.245) | #4 ADVE (d=0.246) | #5 ADVE (d=0.253) | #6 ADVE (d=0.256) | #7 ADVE (d=0.261) | #8 ADVE (d=0.262) | #9 ADVE (d=0.264) | #10 ADVE (d=0.266)
+         432  ADVE            Report              0.3233  #1 Report (d=0.323) | #2 Report (d=0.325) | #3 Memo (d=0.326) | #4 Report (d=0.334) | #5 Report (d=0.336) | #6 ADVE (d=0.344) | #7 Memo (d=0.351) | #8 ADVE (d=0.351) | #9 Letter (d=0.353) | #10 Report (d=0.356)
+
+Label: Email  (10 bad cases / 115 total test images)
+  Sampled 10 / 10 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         354  Email           Letter              0.3314  #1 Letter (d=0.331) | #2 Memo (d=0.339) | #3 Email (d=0.343) | #4 Email (d=0.344) | #5 Memo (d=0.345) | #6 Memo (d=0.346) | #7 Memo (d=0.348) | #8 Email (d=0.351) | #9 Email (d=0.355) | #10 Memo (d=0.356)
+         282  Email           Note                0.0687  #1 Note (d=0.069) | #2 Email (d=0.077) | #3 Email (d=0.080) | #4 Email (d=0.081) | #5 Email (d=0.087) | #6 Email (d=0.087) | #7 Email (d=0.089) | #8 Email (d=0.090) | #9 Note (d=0.091) | #10 Email (d=0.092)
+         278  Email           Letter              0.2893  #1 Letter (d=0.289) | #2 Letter (d=0.293) | #3 Email (d=0.318) | #4 Letter (d=0.321) | #5 Email (d=0.323) | #6 Letter (d=0.329) | #7 Report (d=0.330) | #8 Form (d=0.334) | #9 Letter (d=0.335) | #10 Letter (d=0.337)
+         467  Email           Memo                0.3487  #1 Memo (d=0.349) | #2 Email (d=0.356) | #3 Email (d=0.360) | #4 Email (d=0.362) | #5 Memo (d=0.364) | #6 Email (d=0.367) | #7 Email (d=0.367) | #8 Email (d=0.370) | #9 Report (d=0.372) | #10 Email (d=0.372)
+         525  Email           Note                0.2517  #1 Note (d=0.252) | #2 Letter (d=0.260) | #3 Email (d=0.263) | #4 Email (d=0.263) | #5 Email (d=0.265) | #6 Letter (d=0.266) | #7 Email (d=0.267) | #8 Letter (d=0.268) | #9 Letter (d=0.269) | #10 Email (d=0.269)
+         451  Email           Form                0.2868  #1 Form (d=0.287) | #2 Note (d=0.306) | #3 Scientific (d=0.310) | #4 Form (d=0.311) | #5 Form (d=0.317) | #6 Form (d=0.318) | #7 Scientific (d=0.319) | #8 Form (d=0.321) | #9 Scientific (d=0.322) | #10 Memo (d=0.325)
+         180  Email           Letter              0.2844  #1 Letter (d=0.284) | #2 Letter (d=0.296) | #3 Email (d=0.297) | #4 Email (d=0.298) | #5 Email (d=0.299) | #6 Letter (d=0.301) | #7 Letter (d=0.303) | #8 Email (d=0.304) | #9 Letter (d=0.309) | #10 Email (d=0.310)
+         591  Email           Memo                0.3296  #1 Memo (d=0.330) | #2 Email (d=0.332) | #3 Email (d=0.334) | #4 Email (d=0.334) | #5 Email (d=0.334) | #6 Email (d=0.334) | #7 Email (d=0.335) | #8 Email (d=0.336) | #9 Email (d=0.336) | #10 Email (d=0.337)
+         577  Email           Letter              0.3330  #1 Letter (d=0.333) | #2 Letter (d=0.340) | #3 Email (d=0.344) | #4 Letter (d=0.347) | #5 Email (d=0.349) | #6 Email (d=0.353) | #7 Email (d=0.357) | #8 Form (d=0.357) | #9 Report (d=0.358) | #10 Email (d=0.358)
+         692  Email           Memo                0.3159  #1 Memo (d=0.316) | #2 Email (d=0.326) | #3 Email (d=0.328) | #4 Email (d=0.329) | #5 Email (d=0.332) | #6 Email (d=0.333) | #7 Email (d=0.335) | #8 Memo (d=0.339) | #9 Email (d=0.342) | #10 Email (d=0.344)
+
+Label: Form  (10 bad cases / 79 total test images)
+  Sampled 10 / 10 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+           3  Form            Letter              0.2632  #1 Letter (d=0.263) | #2 Form (d=0.269) | #3 Form (d=0.281) | #4 Letter (d=0.285) | #5 Form (d=0.287) | #6 Form (d=0.295) | #7 Form (d=0.299) | #8 Form (d=0.299) | #9 Form (d=0.304) | #10 Form (d=0.304)
+          56  Form            Note                0.1567  #1 Note (d=0.157) | #2 Note (d=0.158) | #3 Memo (d=0.160) | #4 Form (d=0.172) | #5 Form (d=0.182) | #6 Note (d=0.182) | #7 Memo (d=0.184) | #8 Scientific (d=0.185) | #9 Note (d=0.186) | #10 Memo (d=0.187)
+         216  Form            Scientific          0.1636  #1 Scientific (d=0.164) | #2 Form (d=0.207) | #3 Form (d=0.216) | #4 Memo (d=0.219) | #5 Form (d=0.222) | #6 Note (d=0.222) | #7 Report (d=0.224) | #8 Form (d=0.226) | #9 Scientific (d=0.227) | #10 Form (d=0.229)
+         544  Form            Report              0.2859  #1 Report (d=0.286) | #2 Form (d=0.290) | #3 Report (d=0.292) | #4 Note (d=0.300) | #5 Email (d=0.304) | #6 Form (d=0.304) | #7 Report (d=0.306) | #8 Letter (d=0.307) | #9 Email (d=0.314) | #10 Scientific (d=0.317)
+         242  Form            Letter              0.1846  #1 Letter (d=0.185) | #2 Letter (d=0.247) | #3 Letter (d=0.258) | #4 Letter (d=0.269) | #5 Letter (d=0.275) | #6 Email (d=0.277) | #7 Memo (d=0.283) | #8 Memo (d=0.288) | #9 Report (d=0.291) | #10 Letter (d=0.294)
+         334  Form            Letter              0.2893  #1 Letter (d=0.289) | #2 Letter (d=0.290) | #3 Letter (d=0.322) | #4 Form (d=0.341) | #5 Form (d=0.347) | #6 Letter (d=0.348) | #7 Form (d=0.350) | #8 Letter (d=0.350) | #9 Letter (d=0.352) | #10 Email (d=0.354)
+         552  Form            Resume              0.2838  #1 Resume (d=0.284) | #2 Scientific (d=0.299) | #3 Note (d=0.302) | #4 Note (d=0.304) | #5 Email (d=0.306) | #6 Note (d=0.307) | #7 Scientific (d=0.309) | #8 Form (d=0.311) | #9 Form (d=0.311) | #10 Form (d=0.312)
+         110  Form            Note                0.1855  #1 Note (d=0.186) | #2 Form (d=0.188) | #3 Note (d=0.191) | #4 Form (d=0.200) | #5 Note (d=0.202) | #6 Note (d=0.213) | #7 Form (d=0.213) | #8 Note (d=0.214) | #9 Note (d=0.215) | #10 Note (d=0.216)
+         363  Form            Report              0.3172  #1 Report (d=0.317) | #2 Form (d=0.328) | #3 Form (d=0.330) | #4 Form (d=0.337) | #5 Form (d=0.340) | #6 Form (d=0.343) | #7 Form (d=0.343) | #8 Form (d=0.345) | #9 Form (d=0.347) | #10 Form (d=0.347)
+         356  Form            Report              0.2150  #1 Report (d=0.215) | #2 Form (d=0.226) | #3 Report (d=0.226) | #4 Form (d=0.229) | #5 Form (d=0.244) | #6 Form (d=0.253) | #7 Form (d=0.253) | #8 Form (d=0.261) | #9 Memo (d=0.261) | #10 Form (d=0.266)
+
+Label: Letter  (13 bad cases / 114 total test images)
+  Sampled 13 / 13 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         119  Letter          Memo                0.3092  #1 Memo (d=0.309) | #2 Report (d=0.330) | #3 Scientific (d=0.330) | #4 Memo (d=0.332) | #5 Scientific (d=0.336) | #6 Report (d=0.340) | #7 Memo (d=0.342) | #8 Letter (d=0.343) | #9 Memo (d=0.344) | #10 Memo (d=0.344)
+         285  Letter          Form                0.3254  #1 Form (d=0.325) | #2 Letter (d=0.327) | #3 Memo (d=0.330) | #4 Letter (d=0.335) | #5 Letter (d=0.344) | #6 Letter (d=0.350) | #7 Memo (d=0.350) | #8 Letter (d=0.352) | #9 Letter (d=0.360) | #10 Letter (d=0.361)
+         476  Letter          Memo                0.2676  #1 Memo (d=0.268) | #2 Memo (d=0.276) | #3 Memo (d=0.280) | #4 Memo (d=0.292) | #5 Memo (d=0.292) | #6 Memo (d=0.296) | #7 Letter (d=0.300) | #8 Memo (d=0.305) | #9 Memo (d=0.307) | #10 Memo (d=0.309)
+         151  Letter          Scientific          0.2825  #1 Scientific (d=0.283) | #2 Letter (d=0.294) | #3 Letter (d=0.296) | #4 Letter (d=0.297) | #5 Letter (d=0.298) | #6 Letter (d=0.298) | #7 Letter (d=0.299) | #8 Memo (d=0.302) | #9 Letter (d=0.303) | #10 Letter (d=0.305)
+          41  Letter          Memo                0.2083  #1 Memo (d=0.208) | #2 Memo (d=0.223) | #3 Memo (d=0.231) | #4 Memo (d=0.242) | #5 Memo (d=0.253) | #6 Memo (d=0.268) | #7 Letter (d=0.275) | #8 Memo (d=0.277) | #9 Memo (d=0.284) | #10 Letter (d=0.284)
+         106  Letter          Memo                0.2881  #1 Memo (d=0.288) | #2 Letter (d=0.291) | #3 Memo (d=0.304) | #4 Letter (d=0.306) | #5 Letter (d=0.318) | #6 Report (d=0.320) | #7 Report (d=0.321) | #8 Letter (d=0.322) | #9 Letter (d=0.324) | #10 Letter (d=0.330)
+         177  Letter          Form                0.3236  #1 Form (d=0.324) | #2 Form (d=0.327) | #3 Letter (d=0.328) | #4 Email (d=0.329) | #5 Letter (d=0.329) | #6 Letter (d=0.335) | #7 Note (d=0.336) | #8 Letter (d=0.339) | #9 Letter (d=0.340) | #10 Form (d=0.340)
+         584  Letter          Memo                0.2046  #1 Memo (d=0.205) | #2 Letter (d=0.228) | #3 Letter (d=0.232) | #4 Letter (d=0.279) | #5 Letter (d=0.284) | #6 Letter (d=0.296) | #7 Letter (d=0.296) | #8 Memo (d=0.297) | #9 Letter (d=0.297) | #10 Letter (d=0.297)
+         572  Letter          Memo                0.2932  #1 Memo (d=0.293) | #2 Letter (d=0.295) | #3 Letter (d=0.297) | #4 Letter (d=0.298) | #5 Memo (d=0.299) | #6 Letter (d=0.300) | #7 Letter (d=0.306) | #8 Memo (d=0.308) | #9 Memo (d=0.309) | #10 Letter (d=0.313)
+         530  Letter          Memo                0.2649  #1 Memo (d=0.265) | #2 Report (d=0.271) | #3 Report (d=0.279) | #4 Memo (d=0.282) | #5 Letter (d=0.296) | #6 Email (d=0.302) | #7 Report (d=0.303) | #8 Memo (d=0.305) | #9 Letter (d=0.314) | #10 Email (d=0.316)
+         413  Letter          Form                0.2709  #1 Form (d=0.271) | #2 Letter (d=0.272) | #3 Letter (d=0.282) | #4 Letter (d=0.307) | #5 Letter (d=0.316) | #6 ADVE (d=0.321) | #7 ADVE (d=0.323) | #8 Form (d=0.325) | #9 Letter (d=0.326) | #10 Letter (d=0.326)
+         225  Letter          Memo                0.2921  #1 Memo (d=0.292) | #2 Memo (d=0.299) | #3 Memo (d=0.312) | #4 Memo (d=0.319) | #5 Memo (d=0.326) | #6 Memo (d=0.328) | #7 Memo (d=0.330) | #8 Letter (d=0.330) | #9 Report (d=0.331) | #10 Memo (d=0.331)
+          93  Letter          Form                0.2529  #1 Form (d=0.253) | #2 Letter (d=0.271) | #3 Scientific (d=0.278) | #4 Memo (d=0.285) | #5 Scientific (d=0.285) | #6 Memo (d=0.287) | #7 Letter (d=0.290) | #8 Scientific (d=0.291) | #9 Scientific (d=0.292) | #10 Report (d=0.296)
+
+Label: Memo  (12 bad cases / 130 total test images)
+  Sampled 12 / 12 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+          18  Memo            Report              0.3153  #1 Report (d=0.315) | #2 Memo (d=0.352) | #3 Letter (d=0.356) | #4 Letter (d=0.360) | #5 Memo (d=0.361) | #6 Memo (d=0.367) | #7 Memo (d=0.367) | #8 Memo (d=0.369) | #9 Letter (d=0.373) | #10 Memo (d=0.374)
+         455  Memo            Letter              0.2313  #1 Letter (d=0.231) | #2 Note (d=0.246) | #3 Letter (d=0.250) | #4 Letter (d=0.252) | #5 Letter (d=0.255) | #6 Memo (d=0.256) | #7 Letter (d=0.259) | #8 Letter (d=0.260) | #9 Memo (d=0.261) | #10 Memo (d=0.263)
+         164  Memo            News                0.2161  #1 News (d=0.216) | #2 Memo (d=0.222) | #3 Memo (d=0.230) | #4 Memo (d=0.235) | #5 Memo (d=0.235) | #6 Letter (d=0.235) | #7 Memo (d=0.240) | #8 Form (d=0.244) | #9 Memo (d=0.247) | #10 Note (d=0.250)
+         290  Memo            Form                0.2820  #1 Form (d=0.282) | #2 Memo (d=0.287) | #3 Memo (d=0.290) | #4 Letter (d=0.292) | #5 Memo (d=0.295) | #6 Memo (d=0.300) | #7 Note (d=0.302) | #8 Note (d=0.306) | #9 Memo (d=0.306) | #10 Letter (d=0.307)
+         126  Memo            Letter              0.2574  #1 Letter (d=0.257) | #2 Letter (d=0.262) | #3 Letter (d=0.278) | #4 Letter (d=0.287) | #5 Letter (d=0.288) | #6 Letter (d=0.289) | #7 Letter (d=0.289) | #8 Letter (d=0.290) | #9 Letter (d=0.294) | #10 Letter (d=0.294)
+         283  Memo            Letter              0.3315  #1 Letter (d=0.332) | #2 Letter (d=0.340) | #3 Memo (d=0.351) | #4 Letter (d=0.358) | #5 Memo (d=0.363) | #6 Memo (d=0.363) | #7 Letter (d=0.366) | #8 Letter (d=0.366) | #9 Memo (d=0.368) | #10 Letter (d=0.368)
+          48  Memo            Email               0.2755  #1 Email (d=0.276) | #2 Email (d=0.276) | #3 Email (d=0.284) | #4 Email (d=0.296) | #5 Letter (d=0.299) | #6 Letter (d=0.301) | #7 Email (d=0.303) | #8 Email (d=0.303) | #9 Letter (d=0.304) | #10 Letter (d=0.304)
+          87  Memo            Note                0.2919  #1 Note (d=0.292) | #2 Report (d=0.296) | #3 Memo (d=0.299) | #4 Scientific (d=0.309) | #5 Scientific (d=0.311) | #6 Scientific (d=0.317) | #7 Letter (d=0.318) | #8 Letter (d=0.320) | #9 Letter (d=0.321) | #10 Letter (d=0.322)
+         202  Memo            Letter              0.3218  #1 Letter (d=0.322) | #2 Memo (d=0.327) | #3 Letter (d=0.328) | #4 Memo (d=0.329) | #5 Letter (d=0.340) | #6 Note (d=0.348) | #7 Memo (d=0.349) | #8 Memo (d=0.359) | #9 Memo (d=0.368) | #10 Memo (d=0.368)
+           5  Memo            Letter              0.2897  #1 Letter (d=0.290) | #2 Memo (d=0.295) | #3 Memo (d=0.296) | #4 Letter (d=0.297) | #5 Letter (d=0.298) | #6 Memo (d=0.300) | #7 Letter (d=0.302) | #8 Letter (d=0.303) | #9 Memo (d=0.307) | #10 Letter (d=0.312)
+         223  Memo            Letter              0.2641  #1 Letter (d=0.264) | #2 Memo (d=0.285) | #3 Letter (d=0.286) | #4 Letter (d=0.288) | #5 Memo (d=0.293) | #6 Memo (d=0.294) | #7 Letter (d=0.302) | #8 Letter (d=0.306) | #9 Letter (d=0.310) | #10 Memo (d=0.312)
+          75  Memo            Report              0.2817  #1 Report (d=0.282) | #2 Memo (d=0.291) | #3 Memo (d=0.298) | #4 News (d=0.299) | #5 Memo (d=0.301) | #6 Memo (d=0.303) | #7 Memo (d=0.304) | #8 Memo (d=0.304) | #9 Report (d=0.306) | #10 Memo (d=0.312)
+
+Label: News  (3 bad cases / 44 total test images)
+  Sampled 3 / 3 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         347  News            ADVE                0.2191  #1 ADVE (d=0.219) | #2 News (d=0.241) | #3 News (d=0.248) | #4 News (d=0.248) | #5 ADVE (d=0.250) | #6 ADVE (d=0.256) | #7 News (d=0.257) | #8 News (d=0.259) | #9 News (d=0.260) | #10 ADVE (d=0.263)
+          45  News            Scientific          0.3251  #1 Scientific (d=0.325) | #2 Letter (d=0.339) | #3 News (d=0.345) | #4 Letter (d=0.346) | #5 News (d=0.353) | #6 News (d=0.355) | #7 News (d=0.355) | #8 News (d=0.355) | #9 Letter (d=0.359) | #10 Letter (d=0.361)
+         567  News            ADVE                0.3074  #1 ADVE (d=0.307) | #2 ADVE (d=0.317) | #3 News (d=0.318) | #4 News (d=0.319) | #5 Scientific (d=0.323) | #6 News (d=0.325) | #7 News (d=0.327) | #8 Memo (d=0.333) | #9 News (d=0.333) | #10 News (d=0.334)
+
+Label: Note  (11 bad cases / 32 total test images)
+  Sampled 11 / 11 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         662  Note            Letter              0.3194  #1 Letter (d=0.319) | #2 Memo (d=0.320) | #3 Memo (d=0.323) | #4 Memo (d=0.327) | #5 Note (d=0.328) | #6 Memo (d=0.330) | #7 Memo (d=0.332) | #8 News (d=0.332) | #9 Memo (d=0.334) | #10 Email (d=0.337)
+         633  Note            News                0.3084  #1 News (d=0.308) | #2 Note (d=0.311) | #3 Form (d=0.328) | #4 News (d=0.335) | #5 Scientific (d=0.338) | #6 ADVE (d=0.338) | #7 Report (d=0.347) | #8 Note (d=0.347) | #9 Scientific (d=0.349) | #10 Form (d=0.349)
+         275  Note            Form                0.1896  #1 Form (d=0.190) | #2 Note (d=0.214) | #3 Form (d=0.214) | #4 Note (d=0.215) | #5 Note (d=0.222) | #6 Form (d=0.223) | #7 Note (d=0.226) | #8 Form (d=0.226) | #9 Form (d=0.227) | #10 Note (d=0.230)
+         247  Note            Scientific          0.2772  #1 Scientific (d=0.277) | #2 Report (d=0.284) | #3 Memo (d=0.286) | #4 News (d=0.298) | #5 Letter (d=0.300) | #6 Note (d=0.304) | #7 News (d=0.305) | #8 Note (d=0.306) | #9 News (d=0.307) | #10 Letter (d=0.308)
+         553  Note            News                0.2797  #1 News (d=0.280) | #2 Note (d=0.281) | #3 Note (d=0.281) | #4 Memo (d=0.281) | #5 Memo (d=0.287) | #6 Note (d=0.292) | #7 Note (d=0.293) | #8 Note (d=0.295) | #9 Letter (d=0.297) | #10 News (d=0.298)
+         107  Note            Letter              0.3229  #1 Letter (d=0.323) | #2 Memo (d=0.324) | #3 Letter (d=0.334) | #4 Letter (d=0.335) | #5 Letter (d=0.335) | #6 Memo (d=0.338) | #7 Letter (d=0.338) | #8 Memo (d=0.340) | #9 Memo (d=0.341) | #10 Letter (d=0.342)
+         400  Note            Email               0.1362  #1 Email (d=0.136) | #2 Email (d=0.138) | #3 Email (d=0.139) | #4 Note (d=0.144) | #5 Email (d=0.154) | #6 Email (d=0.156) | #7 Email (d=0.156) | #8 Email (d=0.158) | #9 Email (d=0.159) | #10 Note (d=0.159)
+         218  Note            Email               0.2466  #1 Email (d=0.247) | #2 Note (d=0.251) | #3 Note (d=0.251) | #4 Note (d=0.270) | #5 Email (d=0.279) | #6 Form (d=0.280) | #7 Note (d=0.282) | #8 Email (d=0.284) | #9 Email (d=0.289) | #10 Note (d=0.291)
+         504  Note            Email               0.1332  #1 Email (d=0.133) | #2 Email (d=0.143) | #3 Email (d=0.144) | #4 Note (d=0.160) | #5 Email (d=0.160) | #6 Note (d=0.163) | #7 Note (d=0.163) | #8 Email (d=0.165) | #9 Email (d=0.166) | #10 Email (d=0.167)
+         270  Note            Letter              0.3653  #1 Letter (d=0.365) | #2 Scientific (d=0.370) | #3 Report (d=0.372) | #4 Report (d=0.373) | #5 Letter (d=0.377) | #6 Scientific (d=0.379) | #7 Form (d=0.379) | #8 Form (d=0.381) | #9 Letter (d=0.384) | #10 Report (d=0.384)
+         245  Note            News                0.1791  #1 News (d=0.179) | #2 Note (d=0.182) | #3 Note (d=0.195) | #4 Email (d=0.199) | #5 Email (d=0.200) | #6 Memo (d=0.202) | #7 Note (d=0.205) | #8 Email (d=0.209) | #9 Memo (d=0.209) | #10 Note (d=0.213)
+
+Label: Report  (24 bad cases / 54 total test images)
+  Sampled 20 / 24 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+          77  Report          ADVE                0.1762  #1 ADVE (d=0.176) | #2 ADVE (d=0.229) | #3 ADVE (d=0.252) | #4 News (d=0.254) | #5 ADVE (d=0.254) | #6 News (d=0.262) | #7 ADVE (d=0.262) | #8 ADVE (d=0.263) | #9 Report (d=0.263) | #10 ADVE (d=0.265)
+         331  Report          Scientific          0.2857  #1 Scientific (d=0.286) | #2 Scientific (d=0.289) | #3 Scientific (d=0.291) | #4 Memo (d=0.295) | #5 Scientific (d=0.295) | #6 Memo (d=0.298) | #7 Report (d=0.299) | #8 Memo (d=0.301) | #9 Scientific (d=0.303) | #10 Report (d=0.303)
+         123  Report          Letter              0.2570  #1 Letter (d=0.257) | #2 Letter (d=0.304) | #3 Form (d=0.308) | #4 Form (d=0.314) | #5 Letter (d=0.316) | #6 Form (d=0.316) | #7 Letter (d=0.317) | #8 Memo (d=0.319) | #9 Form (d=0.321) | #10 Form (d=0.328)
+         459  Report          Scientific          0.2657  #1 Scientific (d=0.266) | #2 Scientific (d=0.269) | #3 Report (d=0.270) | #4 Scientific (d=0.271) | #5 Scientific (d=0.274) | #6 Scientific (d=0.275) | #7 Report (d=0.275) | #8 Report (d=0.279) | #9 Letter (d=0.280) | #10 Letter (d=0.282)
+         322  Report          Letter              0.2358  #1 Letter (d=0.236) | #2 Report (d=0.254) | #3 Report (d=0.263) | #4 Letter (d=0.264) | #5 Letter (d=0.268) | #6 Letter (d=0.269) | #7 Report (d=0.272) | #8 Letter (d=0.273) | #9 Report (d=0.273) | #10 Report (d=0.287)
+          96  Report          Scientific          0.2571  #1 Scientific (d=0.257) | #2 Report (d=0.257) | #3 Report (d=0.271) | #4 Form (d=0.273) | #5 News (d=0.282) | #6 News (d=0.283) | #7 Form (d=0.285) | #8 Report (d=0.287) | #9 Report (d=0.287) | #10 Email (d=0.290)
+         558  Report          Letter              0.2790  #1 Letter (d=0.279) | #2 Letter (d=0.284) | #3 Report (d=0.296) | #4 Report (d=0.315) | #5 Email (d=0.321) | #6 Letter (d=0.324) | #7 Note (d=0.327) | #8 Report (d=0.328) | #9 Report (d=0.328) | #10 Note (d=0.332)
+         533  Report          Scientific          0.2877  #1 Scientific (d=0.288) | #2 Memo (d=0.290) | #3 Memo (d=0.296) | #4 Scientific (d=0.299) | #5 Memo (d=0.303) | #6 Memo (d=0.304) | #7 Memo (d=0.307) | #8 Letter (d=0.307) | #9 Report (d=0.308) | #10 Scientific (d=0.308)
+         108  Report          Memo                0.2339  #1 Memo (d=0.234) | #2 Memo (d=0.242) | #3 News (d=0.250) | #4 Report (d=0.258) | #5 Memo (d=0.258) | #6 Letter (d=0.261) | #7 Letter (d=0.265) | #8 Memo (d=0.266) | #9 Memo (d=0.266) | #10 Letter (d=0.268)
+         297  Report          Scientific          0.2672  #1 Scientific (d=0.267) | #2 Scientific (d=0.279) | #3 Scientific (d=0.295) | #4 Scientific (d=0.298) | #5 Scientific (d=0.301) | #6 Letter (d=0.301) | #7 Scientific (d=0.303) | #8 Scientific (d=0.305) | #9 Scientific (d=0.306) | #10 Report (d=0.309)
+          84  Report          Scientific          0.2514  #1 Scientific (d=0.251) | #2 Form (d=0.252) | #3 Scientific (d=0.257) | #4 Report (d=0.259) | #5 Email (d=0.261) | #6 Scientific (d=0.263) | #7 Report (d=0.264) | #8 Form (d=0.265) | #9 Form (d=0.273) | #10 Scientific (d=0.276)
+         514  Report          Letter              0.2657  #1 Letter (d=0.266) | #2 Scientific (d=0.272) | #3 Letter (d=0.274) | #4 News (d=0.285) | #5 Letter (d=0.287) | #6 Report (d=0.288) | #7 Letter (d=0.291) | #8 Letter (d=0.293) | #9 Letter (d=0.295) | #10 Scientific (d=0.296)
+         566  Report          Memo                0.2852  #1 Memo (d=0.285) | #2 Letter (d=0.323) | #3 Letter (d=0.329) | #4 Letter (d=0.329) | #5 Report (d=0.335) | #6 Report (d=0.338) | #7 Memo (d=0.338) | #8 Scientific (d=0.339) | #9 Letter (d=0.339) | #10 Report (d=0.339)
+         582  Report          Form                0.2008  #1 Form (d=0.201) | #2 Form (d=0.213) | #3 Form (d=0.213) | #4 Form (d=0.221) | #5 Form (d=0.226) | #6 Scientific (d=0.232) | #7 Form (d=0.232) | #8 Note (d=0.233) | #9 Form (d=0.236) | #10 Form (d=0.237)
+          27  Report          Letter              0.2921  #1 Letter (d=0.292) | #2 Report (d=0.294) | #3 Scientific (d=0.310) | #4 Report (d=0.311) | #5 Report (d=0.312) | #6 Form (d=0.315) | #7 Form (d=0.316) | #8 Note (d=0.320) | #9 Form (d=0.323) | #10 Form (d=0.324)
+          62  Report          Letter              0.3210  #1 Letter (d=0.321) | #2 Report (d=0.325) | #3 Report (d=0.330) | #4 Letter (d=0.330) | #5 Email (d=0.338) | #6 Report (d=0.345) | #7 News (d=0.347) | #8 Report (d=0.347) | #9 ADVE (d=0.347) | #10 Note (d=0.348)
+         603  Report          Scientific          0.2957  #1 Scientific (d=0.296) | #2 Scientific (d=0.310) | #3 Scientific (d=0.319) | #4 Report (d=0.319) | #5 Scientific (d=0.322) | #6 Report (d=0.325) | #7 Report (d=0.327) | #8 Scientific (d=0.333) | #9 Memo (d=0.334) | #10 Form (d=0.335)
+         139  Report          Scientific          0.2324  #1 Scientific (d=0.232) | #2 Report (d=0.245) | #3 Note (d=0.260) | #4 News (d=0.263) | #5 Memo (d=0.272) | #6 Report (d=0.274) | #7 Report (d=0.275) | #8 Report (d=0.275) | #9 Scientific (d=0.277) | #10 Scientific (d=0.278)
+         120  Report          ADVE                0.2369  #1 ADVE (d=0.237) | #2 Email (d=0.300) | #3 Letter (d=0.321) | #4 Letter (d=0.342) | #5 Form (d=0.345) | #6 ADVE (d=0.345) | #7 Report (d=0.350) | #8 Letter (d=0.354) | #9 Report (d=0.357) | #10 Report (d=0.361)
+         537  Report          Letter              0.2856  #1 Letter (d=0.286) | #2 Report (d=0.291) | #3 Letter (d=0.327) | #4 Memo (d=0.327) | #5 Letter (d=0.333) | #6 Letter (d=0.334) | #7 Note (d=0.334) | #8 Report (d=0.336) | #9 Letter (d=0.337) | #10 Letter (d=0.341)
+
+Label: Resume  (0 bad cases / 24 total test images)
+  -> 0 bad cases � perfect top-1 retrieval
+
+Label: Scientific  (21 bad cases / 59 total test images)
+  Sampled 20 / 21 bad cases to visualise
+   Query idx  True label      Top-1 pred      Top-1 dist  Top-10 predictions
+  ------------------------------------------------------------------------------------------
+         395  Scientific      Report              0.2748  #1 Report (d=0.275) | #2 Scientific (d=0.278) | #3 Report (d=0.284) | #4 Report (d=0.290) | #5 Scientific (d=0.295) | #6 Scientific (d=0.302) | #7 Report (d=0.302) | #8 News (d=0.303) | #9 Scientific (d=0.305) | #10 Note (d=0.307)
+         612  Scientific      Form                0.2184  #1 Form (d=0.218) | #2 Scientific (d=0.228) | #3 Form (d=0.255) | #4 Form (d=0.259) | #5 Form (d=0.272) | #6 Scientific (d=0.279) | #7 Form (d=0.279) | #8 Form (d=0.283) | #9 Form (d=0.285) | #10 Form (d=0.286)
+         377  Scientific      Report              0.2743  #1 Report (d=0.274) | #2 Report (d=0.287) | #3 Letter (d=0.299) | #4 Scientific (d=0.300) | #5 Report (d=0.300) | #6 Letter (d=0.302) | #7 Letter (d=0.318) | #8 Scientific (d=0.324) | #9 Letter (d=0.326) | #10 Letter (d=0.329)
+         450  Scientific      Report              0.2346  #1 Report (d=0.235) | #2 Scientific (d=0.258) | #3 Report (d=0.259) | #4 Scientific (d=0.264) | #5 Letter (d=0.268) | #6 Memo (d=0.270) | #7 Form (d=0.274) | #8 Report (d=0.277) | #9 Letter (d=0.277) | #10 Report (d=0.282)
+         117  Scientific      Resume              0.3628  #1 Resume (d=0.363) | #2 Scientific (d=0.373) | #3 Resume (d=0.374) | #4 ADVE (d=0.384) | #5 Note (d=0.390) | #6 Scientific (d=0.390) | #7 Scientific (d=0.392) | #8 Scientific (d=0.393) | #9 News (d=0.396) | #10 Scientific (d=0.401)
+         617  Scientific      Memo                0.2488  #1 Memo (d=0.249) | #2 Report (d=0.265) | #3 News (d=0.266) | #4 Memo (d=0.269) | #5 Note (d=0.276) | #6 Note (d=0.277) | #7 Report (d=0.279) | #8 Memo (d=0.280) | #9 Note (d=0.285) | #10 Note (d=0.285)
+         510  Scientific      Form                0.2465  #1 Form (d=0.246) | #2 Form (d=0.257) | #3 Form (d=0.258) | #4 Scientific (d=0.264) | #5 Form (d=0.265) | #6 Scientific (d=0.279) | #7 Form (d=0.279) | #8 Form (d=0.281) | #9 Form (d=0.282) | #10 Form (d=0.283)
+          89  Scientific      Memo                0.2706  #1 Memo (d=0.271) | #2 Scientific (d=0.288) | #3 Scientific (d=0.303) | #4 Scientific (d=0.309) | #5 Report (d=0.315) | #6 Report (d=0.316) | #7 Scientific (d=0.319) | #8 Report (d=0.320) | #9 Report (d=0.322) | #10 Letter (d=0.323)
+         501  Scientific      Report              0.2448  #1 Report (d=0.245) | #2 Report (d=0.263) | #3 Report (d=0.279) | #4 Memo (d=0.282) | #5 Report (d=0.285) | #6 Report (d=0.290) | #7 Memo (d=0.293) | #8 Email (d=0.294) | #9 Report (d=0.297) | #10 Report (d=0.303)
+         364  Scientific      Report              0.2441  #1 Report (d=0.244) | #2 Report (d=0.258) | #3 Letter (d=0.272) | #4 Letter (d=0.272) | #5 Resume (d=0.285) | #6 Report (d=0.288) | #7 Letter (d=0.289) | #8 Scientific (d=0.289) | #9 Scientific (d=0.292) | #10 Scientific (d=0.294)
+         376  Scientific      Report              0.2423  #1 Report (d=0.242) | #2 Report (d=0.244) | #3 Scientific (d=0.254) | #4 Scientific (d=0.256) | #5 Note (d=0.268) | #6 Report (d=0.271) | #7 Report (d=0.274) | #8 Scientific (d=0.275) | #9 Form (d=0.287) | #10 Report (d=0.291)
+         355  Scientific      Report              0.2442  #1 Report (d=0.244) | #2 Scientific (d=0.248) | #3 Scientific (d=0.250) | #4 Report (d=0.255) | #5 Scientific (d=0.256) | #6 Report (d=0.257) | #7 Report (d=0.263) | #8 Scientific (d=0.265) | #9 Scientific (d=0.266) | #10 Report (d=0.266)
+         589  Scientific      Form                0.2227  #1 Form (d=0.223) | #2 Memo (d=0.245) | #3 Form (d=0.246) | #4 Form (d=0.248) | #5 Scientific (d=0.249) | #6 Form (d=0.256) | #7 Scientific (d=0.258) | #8 Memo (d=0.260) | #9 Note (d=0.261) | #10 Form (d=0.262)
+         264  Scientific      Memo                0.2583  #1 Memo (d=0.258) | #2 Memo (d=0.259) | #3 Memo (d=0.280) | #4 Memo (d=0.295) | #5 Memo (d=0.297) | #6 Memo (d=0.300) | #7 Scientific (d=0.307) | #8 Memo (d=0.308) | #9 Memo (d=0.309) | #10 Scientific (d=0.310)
+         447  Scientific      Form                0.1237  #1 Form (d=0.124) | #2 Note (d=0.125) | #3 Memo (d=0.131) | #4 Note (d=0.134) | #5 Form (d=0.136) | #6 Form (d=0.136) | #7 Scientific (d=0.136) | #8 Note (d=0.141) | #9 Note (d=0.142) | #10 Note (d=0.144)
+         498  Scientific      News                0.3245  #1 News (d=0.325) | #2 News (d=0.356) | #3 News (d=0.356) | #4 ADVE (d=0.360) | #5 News (d=0.362) | #6 News (d=0.363) | #7 News (d=0.364) | #8 News (d=0.367) | #9 News (d=0.368) | #10 News (d=0.368)
+         243  Scientific      Report              0.2510  #1 Report (d=0.251) | #2 Letter (d=0.269) | #3 Letter (d=0.281) | #4 Letter (d=0.282) | #5 Letter (d=0.286) | #6 Letter (d=0.293) | #7 Letter (d=0.295) | #8 Letter (d=0.296) | #9 Letter (d=0.297) | #10 Letter (d=0.298)
+         680  Scientific      Note                0.2545  #1 Note (d=0.254) | #2 Scientific (d=0.256) | #3 Scientific (d=0.262) | #4 Report (d=0.276) | #5 Report (d=0.276) | #6 Letter (d=0.276) | #7 Note (d=0.277) | #8 Report (d=0.281) | #9 Note (d=0.281) | #10 Report (d=0.284)
+         641  Scientific      Memo                0.3171  #1 Memo (d=0.317) | #2 Form (d=0.326) | #3 Memo (d=0.326) | #4 Report (d=0.326) | #5 Memo (d=0.335) | #6 Scientific (d=0.335) | #7 Memo (d=0.337) | #8 Form (d=0.341) | #9 Report (d=0.341) | #10 Form (d=0.341)
+         550  Scientific      Form                0.2259  #1 Form (d=0.226) | #2 Scientific (d=0.237) | #3 Form (d=0.241) | #4 Scientific (d=0.252) | #5 Scientific (d=0.253) | #6 Scientific (d=0.258) | #7 Scientific (d=0.259) | #8 Form (d=0.260) | #9 Form (d=0.265) | #10 Scientific (d=0.268)

tabacco3482_benchmarks/results/bad_case_analysis.json

@@ -0,0 +1,359 @@
+{
+  "SigLIP2 SO400M": {
+    "safe_name": "SigLIP2_SO400M_patch14-384",
+    "total_errors": 81,
+    "total_test": 697,
+    "error_rate": 0.1162,
+    "per_class": {
+      "ADVE": {
+        "error_rate": 0.0652,
+        "n_errors": 3,
+        "n_total": 46,
+        "confused_with": [
+          [
+            "Form",
+            1
+          ],
+          [
+            "Report",
+            1
+          ],
+          [
+            "Note",
+            1
+          ]
+        ]
+      },
+      "Email": {
+        "error_rate": 0.0522,
+        "n_errors": 6,
+        "n_total": 115,
+        "confused_with": [
+          [
+            "Letter",
+            4
+          ],
+          [
+            "Scientific",
+            1
+          ],
+          [
+            "Note",
+            1
+          ]
+        ]
+      },
+      "Form": {
+        "error_rate": 0.0759,
+        "n_errors": 6,
+        "n_total": 79,
+        "confused_with": [
+          [
+            "Letter",
+            3
+          ],
+          [
+            "Memo",
+            2
+          ],
+          [
+            "Scientific",
+            1
+          ]
+        ]
+      },
+      "Letter": {
+        "error_rate": 0.0526,
+        "n_errors": 6,
+        "n_total": 114,
+        "confused_with": [
+          [
+            "Memo",
+            4
+          ],
+          [
+            "Scientific",
+            1
+          ],
+          [
+            "Report",
+            1
+          ]
+        ]
+      },
+      "Memo": {
+        "error_rate": 0.1462,
+        "n_errors": 19,
+        "n_total": 130,
+        "confused_with": [
+          [
+            "Letter",
+            11
+          ],
+          [
+            "Note",
+            5
+          ],
+          [
+            "Email",
+            1
+          ]
+        ]
+      },
+      "News": {
+        "error_rate": 0.0227,
+        "n_errors": 1,
+        "n_total": 44,
+        "confused_with": [
+          [
+            "Report",
+            1
+          ]
+        ]
+      },
+      "Note": {
+        "error_rate": 0.25,
+        "n_errors": 8,
+        "n_total": 32,
+        "confused_with": [
+          [
+            "Letter",
+            2
+          ],
+          [
+            "Memo",
+            2
+          ],
+          [
+            "Email",
+            2
+          ]
+        ]
+      },
+      "Report": {
+        "error_rate": 0.2593,
+        "n_errors": 14,
+        "n_total": 54,
+        "confused_with": [
+          [
+            "Scientific",
+            7
+          ],
+          [
+            "Form",
+            3
+          ],
+          [
+            "Letter",
+            2
+          ]
+        ]
+      },
+      "Scientific": {
+        "error_rate": 0.3051,
+        "n_errors": 18,
+        "n_total": 59,
+        "confused_with": [
+          [
+            "Form",
+            6
+          ],
+          [
+            "Report",
+            5
+          ],
+          [
+            "Letter",
+            3
+          ]
+        ]
+      }
+    }
+  },
+  "ModernVBERT Bi": {
+    "safe_name": "ModernVBERT_Bi",
+    "total_errors": 96,
+    "total_test": 697,
+    "error_rate": 0.1377,
+    "per_class": {
+      "ADVE": {
+        "error_rate": 0.1087,
+        "n_errors": 5,
+        "n_total": 46,
+        "confused_with": [
+          [
+            "Letter",
+            1
+          ],
+          [
+            "Form",
+            1
+          ],
+          [
+            "Report",
+            1
+          ]
+        ]
+      },
+      "Email": {
+        "error_rate": 0.0435,
+        "n_errors": 5,
+        "n_total": 115,
+        "confused_with": [
+          [
+            "Note",
+            3
+          ],
+          [
+            "Letter",
+            1
+          ],
+          [
+            "Memo",
+            1
+          ]
+        ]
+      },
+      "Form": {
+        "error_rate": 0.038,
+        "n_errors": 3,
+        "n_total": 79,
+        "confused_with": [
+          [
+            "Email",
+            1
+          ],
+          [
+            "Letter",
+            1
+          ],
+          [
+            "ADVE",
+            1
+          ]
+        ]
+      },
+      "Letter": {
+        "error_rate": 0.114,
+        "n_errors": 13,
+        "n_total": 114,
+        "confused_with": [
+          [
+            "Memo",
+            8
+          ],
+          [
+            "Form",
+            2
+          ],
+          [
+            "Report",
+            2
+          ]
+        ]
+      },
+      "Memo": {
+        "error_rate": 0.1077,
+        "n_errors": 14,
+        "n_total": 130,
+        "confused_with": [
+          [
+            "Letter",
+            13
+          ],
+          [
+            "Form",
+            1
+          ]
+        ]
+      },
+      "News": {
+        "error_rate": 0.1591,
+        "n_errors": 7,
+        "n_total": 44,
+        "confused_with": [
+          [
+            "Form",
+            3
+          ],
+          [
+            "Letter",
+            1
+          ],
+          [
+            "ADVE",
+            1
+          ]
+        ]
+      },
+      "Note": {
+        "error_rate": 0.3125,
+        "n_errors": 10,
+        "n_total": 32,
+        "confused_with": [
+          [
+            "Letter",
+            3
+          ],
+          [
+            "Scientific",
+            3
+          ],
+          [
+            "Email",
+            2
+          ]
+        ]
+      },
+      "Report": {
+        "error_rate": 0.463,
+        "n_errors": 25,
+        "n_total": 54,
+        "confused_with": [
+          [
+            "Letter",
+            13
+          ],
+          [
+            "Form",
+            7
+          ],
+          [
+            "Scientific",
+            4
+          ]
+        ]
+      },
+      "Resume": {
+        "error_rate": 0.0417,
+        "n_errors": 1,
+        "n_total": 24,
+        "confused_with": [
+          [
+            "Scientific",
+            1
+          ]
+        ]
+      },
+      "Scientific": {
+        "error_rate": 0.2203,
+        "n_errors": 13,
+        "n_total": 59,
+        "confused_with": [
+          [
+            "Memo",
+            3
+          ],
+          [
+            "Report",
+            3
+          ],
+          [
+            "News",
+            2
+          ]
+        ]
+      }
+    }
+  }
+}