shawn24 commited on
Commit
329438d
Β·
verified Β·
1 Parent(s): df969ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -3
README.md CHANGED
@@ -1,3 +1,74 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # πŸͺœ LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifiers
2
+
3
+ [![Project](https://img.shields.io/badge/Project-%23dfb317)](https://shantanu-ai.github.io/projects/ACL-2025-Ladder/index.html)
4
+ [![Paper](https://img.shields.io/badge/Paper-ACL%202025-%23dfb317)](https://arxiv.org/abs/2405.12255)
5
+ [![Code](https://img.shields.io/badge/GitHub-batmanlab%2FLADDER-%2312100e)](https://github.com/batmanlab/Ladder)
6
+ [![Model](https://img.shields.io/badge/HuggingFace-Pretrained--Checkpoints-blue)](https://huggingface.co/your-model-name)
7
+ [![License](https://img.shields.io/badge/License-MIT-green)](./LICENSE)
8
+
9
+ ---
10
+
11
+ ## πŸ“Œ Summary
12
+
13
+ **LADDER** is a general framework that enables vision classifiers to automatically discover subpopulations (or "slices") of data where the model is underperforming β€” without requiring group annotations. It leverages **vision-language representations** and the **reasoning capabilities of large language models (LLMs)** to detect and rectify bias-inducing features in both natural and medical imaging domains.
14
+
15
+ ---
16
+
17
+ ## 🧠 Architecture & Components
18
+
19
+ - πŸ” **Slice Discovery** using:
20
+ - CLIP, Mammo-CLIP, and CXR-CLIP features
21
+ - BLIP and GPT-4o-generated captions
22
+ - 🧠 **Hypothesis Generation** using:
23
+ - GPT-4o, Claude, Gemini, LLaMA
24
+ - βœ… **Bias Mitigation** via reweighting & pseudo-labeling
25
+
26
+ ---
27
+
28
+ ## πŸ“Š Datasets Used
29
+
30
+ - **Natural Images**: Waterbirds, CelebA, MetaShift
31
+ - **Medical Images**: NIH ChestX-ray, RSNA Mammograms, VinDr Mammograms
32
+
33
+ ---
34
+
35
+ ## πŸ“¦ Files Included
36
+
37
+ | File | Description |
38
+ |------|-------------|
39
+ | `model.pt` | Pretrained model checkpoint |
40
+ | `feature_cache.pkl` | Cached representations (CLIP/Mammo-CLIP/CXR-CLIP) |
41
+ | `metadata.csv` | Metadata with discovered slice labels |
42
+ | `caption_blip.json` | BLIP-generated captions |
43
+ | `caption_gpt4o.json` | GPT-4o-generated captions |
44
+ | `predictions.json` | Model predictions on test set |
45
+
46
+ ---
47
+
48
+
49
+ ## πŸ§ͺ Benchmarks
50
+
51
+ LADDER outperforms traditional slice discovery methods (Domino, FACTS) across 6 datasets and >200 classifiers. It is especially effective in:
52
+
53
+ - Discovering hidden biases without explicit attribute labels
54
+ - Reasoning about non-visual factors (e.g., preprocessing artifacts)
55
+ - Operating without human-written captions
56
+
57
+ ---
58
+
59
+ ## πŸ“œ Citation
60
+
61
+ ```bibtex
62
+ @article{ghosh2024ladder,
63
+ title={LADDER: Language Driven Slice Discovery and Error Rectification},
64
+ author={Ghosh, Shantanu and Syed, Rayan and Wang, Chenyu and Poynton, Clare B and Visweswaran, Shyam and Batmanghelich, Kayhan},
65
+ journal={arXiv preprint arXiv:2408.07832},
66
+ year={2024}
67
+ }
68
+ ```
69
+
70
+ ---
71
+
72
+ ## 🀝 Acknowledgements
73
+
74
+ Boston University, Stanford University, BUMC, and the University of Pittsburgh.