| | |
| | <div style="max-width: 940px; margin: 16px 0; border:1px solid #2a2f3a; border-radius:8px; background:#0b0f19; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', 'Courier New', monospace; color:#e5e7eb;"> |
| | <div style="display:flex; align-items:center; gap:8px; padding:8px 10px; border-bottom:1px solid #1f2430; background:#111827; border-top-left-radius:8px; border-top-right-radius:8px;"> |
| | <span style="width:10px; height:10px; background:#ef4444; border-radius:50%; display:inline-block;"></span> |
| | <span style="width:10px; height:10px; background:#f59e0b; border-radius:50%; display:inline-block;"></span> |
| | <span style="width:10px; height:10px; background:#22c55e; border-radius:50%; display:inline-block;"></span> |
| | <span style="margin-left:8px; font-size:12px; color:#9ca3af;">attention-mask-visualizer</span> |
| | </div> |
| | <div style="padding:12px 14px; overflow:auto; font-size:12.5px; line-height:1.4;"> |
| | <pre style="margin:0; white-space:pre; tab-size:2;"> |
| | ATTN MASK β GPT-2 (causal) |
| | Tokens: [The, cat, sat, on, the, mat] |
| | Legend: x = can attend, . = masked (future) |
| | |
| | The cat sat on the mat |
| | The x |
| | cat x x |
| | sat x x x |
| | on x x x x |
| | the x x x x x |
| | mat x x x x x x |
| | |
| | |
| | ATTN MASK β PaliGemma-style (bidirectional prefix + causal suffix) |
| | Prefix: [<i0> <i1> <i2> <i3> <i4> What is this] |
| | Suffix: [A great duck] |
| | Legend: β = can attend, β = cannot |
| |
|
| | <i0><i1><i2><i3><i4> What is this | A great duck |
| | <i0> β β β β β β β β β β β |
| | <i1> β β β β β β β β β β β |
| | <i2> β β β β β β β β β β β |
| | <i3> β β β β β β β β β β β |
| | <i4> β β β β β β β β β β β |
| | What β β β β β β β β β β β |
| | is β β β β β β β β β β β |
| | this β β β β β β β β β β β |
| | -------------------------------------------------------------------- |
| | A β β β β β β β β β β β |
| | great β β β β β β β β β β β |
| | duck β β β β β β β β β β β |
| | </pre> |
| | </div> |
| | </div> |
| | |