| |
| <div style="max-width: 940px; margin: 16px 0; border:1px solid #2a2f3a; border-radius:8px; background:#0b0f19; font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, 'Liberation Mono', 'Courier New', monospace; color:#e5e7eb;"> |
| <div style="display:flex; align-items:center; gap:8px; padding:8px 10px; border-bottom:1px solid #1f2430; background:#111827; border-top-left-radius:8px; border-top-right-radius:8px;"> |
| <span style="width:10px; height:10px; background:#ef4444; border-radius:50%; display:inline-block;"></span> |
| <span style="width:10px; height:10px; background:#f59e0b; border-radius:50%; display:inline-block;"></span> |
| <span style="width:10px; height:10px; background:#22c55e; border-radius:50%; display:inline-block;"></span> |
| <span style="margin-left:8px; font-size:12px; color:#9ca3af;">attention-mask-visualizer</span> |
| </div> |
| <div style="padding:12px 14px; overflow:auto; font-size:12.5px; line-height:1.4;"> |
| <pre style="margin:0; white-space:pre; tab-size:2;"> |
| ATTN MASK — GPT-2 (causal) |
| Tokens: [The, cat, sat, on, the, mat] |
| Legend: x = can attend, . = masked (future) |
| |
| The cat sat on the mat |
| The x |
| cat x x |
| sat x x x |
| on x x x x |
| the x x x x x |
| mat x x x x x x |
| |
| |
| ATTN MASK — PaliGemma-style (bidirectional prefix + causal suffix) |
| Prefix: [<i0> <i1> <i2> <i3> <i4> What is this] |
| Suffix: [A great duck] |
| Legend: ✓ = can attend, ✗ = cannot |
|
|
| <i0><i1><i2><i3><i4> What is this | A great duck |
| <i0> ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| <i1> ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| <i2> ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| <i3> ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| <i4> ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| What ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| is ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| this ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ ✗ |
| -------------------------------------------------------------------- |
| A ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✗ |
| great ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ |
| duck ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ |
| </pre> |
| </div> |
| </div> |
| |
