Leandro von Werra PRO

lvwerra's activity

• All
• Models
• Datasets
• Spaces
• Buckets
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

New activity in attention-wiki/knowledge-base 7 minutes ago

Add source: Shaw et al. — Self-Attention with Relative Position Representations

2

#20 opened about 19 hours ago by

lvwerra

Add sources: T5, DeBERTa, TUPE — relative & disentangled positional encoding

2

#26 opened about 18 hours ago by

lvwerra

Add source: In-context Learning and Induction Heads (arxiv:2209.11895)

2

#30 opened about 17 hours ago by

lvwerra

Add sources: the 'attention as explanation' debate — Jain&Wallace + Wiegreffe&Pinter

2

#32 opened about 16 hours ago by

lvwerra

Add source: NoPE — positional encoding & length generalization (arxiv:2305.19466)

2

#33 opened about 16 hours ago by

lvwerra

New activity in attention-wiki/knowledge-base about 16 hours ago

Add source: H2O — Heavy-Hitter KV-cache eviction (arxiv:2306.14048)

2

#29 opened about 17 hours ago by

lvwerra

Add source: S4 (structured state spaces) + claims + state-space-hybrids page

2

#25 opened about 18 hours ago by

cmpatino

Add source: Retrieval Head Mechanistically Explains Long-Context Factuality (arxiv:2404.15574)

1

#31 opened about 16 hours ago by

lvwerra

New activity in attention-wiki/knowledge-base about 17 hours ago

Add source: Long Range Arena + 'no universal winner' claim + LRA evidence on low-rank claim

2

#28 opened about 18 hours ago by

cmpatino

Add source: Reformer (LSH + reversible layers) + claim + sparse-attention page

2

#27 opened about 18 hours ago by

cmpatino

Process arXiv:2310.01889 - Ring Attention

4

#19 opened about 19 hours ago by

lewtun

Add source: GQA — Grouped-Query Attention (arxiv:2305.13245)

3

#21 opened about 19 hours ago by

lvwerra

Add source: Ring Attention with Blockwise Transformers (arXiv:2310.01889)

2

#17 opened about 19 hours ago by

lvwerra

Add source: Multi-head Latent Attention (DeepSeek-V2, arxiv:2405.04434)

2

#22 opened about 19 hours ago by

lvwerra

New activity in attention-wiki/knowledge-base about 18 hours ago

Add source: Self-attention Does Not Need O(n^2) Memory (Rabe & Staats, arxiv:2112.05682)

2

#23 opened about 19 hours ago by

lvwerra

Add source: Linformer (linear-complexity self-attention) + claims + topic page

2

#24 opened about 19 hours ago by

cmpatino

New activity in attention-wiki/knowledge-base about 19 hours ago

Process arXiv:1901.02860 - Transformer-XL

2

#3 opened about 20 hours ago by

lewtun

Process arXiv:1911.02150 - Multi-query attention

2

#4 opened about 20 hours ago by

lewtun

Process arXiv:2108.12409 - ALiBi

2

#5 opened about 20 hours ago by

lewtun

Process arXiv:2306.15595 - Position Interpolation

2

#6 opened about 20 hours ago by

lewtun