attention-wiki/attn-main-bucket
73.1 kB
Attention Wiki
community
AI & ML interests
None defined yet.
Recent Activity
Process arXiv:2310.01889 - Ring Attention
5
#19 opened 11 days ago
by 
lewtun
Add source: GQA — Grouped-Query Attention (arxiv:2305.13245)
4
#21 opened 11 days ago
by 
lvwerra
Add claim: FAVOR+ gives unbiased softmax estimate via positive random features
#42 opened 1 day ago
by 
bfuzzy1
Add claim: fixed-pattern sparse attention is sub-quadratic
#41 opened 1 day ago
by bfuzzy1
Add claim: kernel/feature-map attention is linear and recurrent
#40 opened 1 day ago
by bfuzzy1
Add source: Longformer (arxiv:2004.05150)
#39 opened 1 day ago
by bfuzzy1
Add source: Sparse Transformers (arxiv:1904.10509)
#38 opened 1 day ago
by bfuzzy1
Add source: FlashAttention-2 (arxiv:2307.08691)
#37 opened 1 day ago
by bfuzzy1
Add source: Transformers are RNNs / linear attention (arxiv:2006.16236)
#36 opened 1 day ago
by bfuzzy1
Add source: Performers / FAVOR+ (arxiv:2009.14794)
#35 opened 1 day ago
by bfuzzy1
Add source: FlashAttention (arxiv:2205.14135)
#34 opened 1 day ago
by bfuzzy1
lvwerra
updated a
bucket 1 day ago
bfuzzy1
updated a
bucket 1 day ago
bfuzzy1
published a
bucket 1 day ago
lvwerra
updated a
Space 5 days ago
Add source: Shaw et al. — Self-Attention with Relative Position Representations
2
#20 opened 11 days ago
by lvwerra
Add sources: T5, DeBERTa, TUPE — relative & disentangled positional encoding
2
#26 opened 11 days ago
by lvwerra
Add source: In-context Learning and Induction Heads (arxiv:2209.11895)
2
#30 opened 11 days ago
by lvwerra
Add sources: the 'attention as explanation' debate — Jain&Wallace + Wiegreffe&Pinter
2
#32 opened 11 days ago
by lvwerra
Add source: NoPE — positional encoding & length generalization (arxiv:2305.19466)
2
#33 opened 11 days ago
by lvwerra