mlboydaisuke commited on
Commit
93d5166
·
verified ·
1 Parent(s): e4de434

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -92,6 +92,7 @@ decoder start token (2) at position 0, then for step `t = 0, 1, 2, ...`:
92
 
93
  ```
94
  logits = decoder(hidden, mask, window) # mask is the same one the encoder took
 
95
  next = argmax(logits[0, t])
96
  if next == 2: stop # </s>
97
  window[0, t + 1] = next
@@ -101,6 +102,13 @@ Detokenise the collected ids with the repo's tokenizer. There is no KV cache: th
101
  decoder is a plain static graph over the window, which is what makes it a single
102
  `.pte` with no state to carry between calls.
103
 
 
 
 
 
 
 
 
104
  **4. Reading a detection answer.** Grounded tasks answer with `<loc_N>` tokens,
105
  `N` in 0..999. Four in a row are a box, and each coordinate is `(N + 0.5) × side /
106
  1000` in the original image's pixels — `side` being the image's width for x and its
 
92
 
93
  ```
94
  logits = decoder(hidden, mask, window) # mask is the same one the encoder took
95
+ ban every token that would repeat a 3-gram already in the output # <- see below
96
  next = argmax(logits[0, t])
97
  if next == 2: stop # </s>
98
  window[0, t + 1] = next
 
102
  decoder is a plain static graph over the window, which is what makes it a single
103
  `.pte` with no state to carry between calls.
104
 
105
+ **Implement the 3-gram ban.** `no_repeat_ngram_size: 3` is in the model's own
106
+ `generation_config.json`, and it is not decoration: Florence-2-**large**'s decoder
107
+ returns `<s>` as its argmax three times in a row and only that ban moves it on to the
108
+ caption — without it the loop emits `<s>` forever and returns an empty string. This
109
+ base model happens to move on by itself on every image tested here, so a plain argmax
110
+ loop gives the same five captions. That is luck, not a contract.
111
+
112
  **4. Reading a detection answer.** Grounded tasks answer with `<loc_N>` tokens,
113
  `N` in 0..999. Four in a row are a box, and each coordinate is `(N + 0.5) × side /
114
  1000` in the original image's pixels — `side` being the image's width for x and its