AhmedSSabir commited on
Commit
1a809f8
·
verified ·
1 Parent(s): 5b0e416

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -13
README.md CHANGED
@@ -143,29 +143,19 @@ first three improvements and the first drop in a random draw from those images.
143
  | COCO 300233 | mixing bowl corn food | Ingredients and a food processor are arranged on a gray surface. (0.40) | **Ingredients and a food processor are laid out on a table. (0.47)** |
144
  | COCO 419228 (failure) | ice bear polar | **A white polar bear walks on rocks in an enclosure. (2.54)** | A large white polar bear walks on a rocky surface. (1.85) |
145
 
146
- **Summary: gain over each captioner's own top beam (CIDEr).**
147
 
148
  | Captioner | Top beam | + This model | Gain [95% CI] |
149
  | --- | ---: | ---: | --- |
150
  | BLIP-base (training captioner) | 1.0632 | 1.1470 | +0.0838 [+0.073, +0.095] |
151
  | ViLBERT (paper's candidates) | 1.1150 | 1.1475 | +0.0325 [+0.021, +0.044] |
152
  | Gemma 4 12B | 0.9624 | 0.9783 | +0.0159 [+0.009, +0.023] |
 
153
 
154
- ## Examples (ViLBERT candidates, CIDEr per caption)
155
-
156
- | Image | Best Beam | This model |
157
- | --- | --- | --- |
158
- | COCO 460621 | a white toilet sitting next to a white bath tub (0.39) | **a bathroom with a sink and a toilet (0.83)** |
159
- | COCO 43680 | a group of cows that are standing in the dirt (0.64) | **a herd of cattle walking across a dry field (1.32)** |
160
- | COCO 424432 | a white and brown dog laying on a bed (0.73) | **a dog laying on a bed with a blanket (0.86)** |
161
- | COCO 365685 (failure) | **a woman is taking a bite out of a hot dog (0.92)** | a woman sitting at a table eating a hotdog (0.11) |
162
-
163
- When the model changes the caption (1,948 of 5,000 images), the change improves
164
- CIDEr 1,110 times and lowers it 838 times.
165
 
166
  ## Limitations
167
 
168
- - **Use it as a reranker, not a relatedness classifier.** Reward training shifts
169
  the scores upward. On the paper's 41,849 held-out relatedness pairs, accuracy at
170
  the 0.5 cutoff falls from 0.893 to 0.631, and ROC-AUC from 0.962 to 0.937. For
171
  relatedness probabilities, use the stage I BERT-CNN.
 
143
  | COCO 300233 | mixing bowl corn food | Ingredients and a food processor are arranged on a gray surface. (0.40) | **Ingredients and a food processor are laid out on a table. (0.47)** |
144
  | COCO 419228 (failure) | ice bear polar | **A white polar bear walks on rocks in an enclosure. (2.54)** | A large white polar bear walks on a rocky surface. (1.85) |
145
 
146
+ <!-- **Summary: gain over each captioner's own top beam (CIDEr).**
147
 
148
  | Captioner | Top beam | + This model | Gain [95% CI] |
149
  | --- | ---: | ---: | --- |
150
  | BLIP-base (training captioner) | 1.0632 | 1.1470 | +0.0838 [+0.073, +0.095] |
151
  | ViLBERT (paper's candidates) | 1.1150 | 1.1475 | +0.0325 [+0.021, +0.044] |
152
  | Gemma 4 12B | 0.9624 | 0.9783 | +0.0159 [+0.009, +0.023] |
153
+ -->
154
 
 
 
 
 
 
 
 
 
 
 
 
155
 
156
  ## Limitations
157
 
158
+ - **Use it as a reranker, not a (relatedness classifier)[https://huggingface.co/AhmedSSabir/ModernBERT-CNN-Caption-Reranker]. ** Reward training shifts
159
  the scores upward. On the paper's 41,849 held-out relatedness pairs, accuracy at
160
  the 0.5 cutoff falls from 0.893 to 0.631, and ROC-AUC from 0.962 to 0.937. For
161
  relatedness probabilities, use the stage I BERT-CNN.