Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -92,6 +92,7 @@ decoder start token (2) at position 0, then for step `t = 0, 1, 2, ...`:
|
|
| 92 |
|
| 93 |
```
|
| 94 |
logits = decoder(hidden, mask, window) # mask is the same one the encoder took
|
|
|
|
| 95 |
next = argmax(logits[0, t])
|
| 96 |
if next == 2: stop # </s>
|
| 97 |
window[0, t + 1] = next
|
|
@@ -101,6 +102,13 @@ Detokenise the collected ids with the repo's tokenizer. There is no KV cache: th
|
|
| 101 |
decoder is a plain static graph over the window, which is what makes it a single
|
| 102 |
`.pte` with no state to carry between calls.
|
| 103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
**4. Reading a detection answer.** Grounded tasks answer with `<loc_N>` tokens,
|
| 105 |
`N` in 0..999. Four in a row are a box, and each coordinate is `(N + 0.5) × side /
|
| 106 |
1000` in the original image's pixels — `side` being the image's width for x and its
|
|
|
|
| 92 |
|
| 93 |
```
|
| 94 |
logits = decoder(hidden, mask, window) # mask is the same one the encoder took
|
| 95 |
+
ban every token that would repeat a 3-gram already in the output # <- see below
|
| 96 |
next = argmax(logits[0, t])
|
| 97 |
if next == 2: stop # </s>
|
| 98 |
window[0, t + 1] = next
|
|
|
|
| 102 |
decoder is a plain static graph over the window, which is what makes it a single
|
| 103 |
`.pte` with no state to carry between calls.
|
| 104 |
|
| 105 |
+
**Implement the 3-gram ban.** `no_repeat_ngram_size: 3` is in the model's own
|
| 106 |
+
`generation_config.json`, and it is not decoration: Florence-2-**large**'s decoder
|
| 107 |
+
returns `<s>` as its argmax three times in a row and only that ban moves it on to the
|
| 108 |
+
caption — without it the loop emits `<s>` forever and returns an empty string. This
|
| 109 |
+
base model happens to move on by itself on every image tested here, so a plain argmax
|
| 110 |
+
loop gives the same five captions. That is luck, not a contract.
|
| 111 |
+
|
| 112 |
**4. Reading a detection answer.** Grounded tasks answer with `<loc_N>` tokens,
|
| 113 |
`N` in 0..999. Four in a row are a box, and each coordinate is `(N + 0.5) × side /
|
| 114 |
1000` in the original image's pixels — `side` being the image's width for x and its
|