Ctrl+K
Update logbook: Reproduction: Learning When to Attend: Conditional Memory Access for Long-Context LLMs
3a601da verified - claim-1-l2a-preserves-quality-at-toy-scale-1-7b-model-8k-context
- claim-1-l2a-sparse-attention-mechanism-preserves-baseline-quality-at-toy-scale-1-7b-model-8k-context
- claim-1-l2a-sparse-attention-mechanism-preserves-baseline-quality
- claim-2-algorithmic-verification-of-l2a-gating-logic
- claim-2-l2a-skips-global-attention-for-80-of-tokens
- claim-2-l2a-triton-kernels-improve-training-throughput-by-2x
- claim-3-l2a-skips-global-attention-for-80-of-tokens
- claim-3-l2a-triton-kernels-improve-training-throughput-by-2x
- claim-4-sparse-attention-pruning-reduces-kv-cache-by-50
- conclusion
- executive-summary
- 769 Bytes