-
Sequence Parallelism: Long Sequence Training from System Perspective
Paper • 2105.13120 • Published • 6 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 13 -
Striped Attention: Faster Ring Attention for Causal Transformers
Paper • 2311.09431 • Published • 4 -
DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
Paper • 2309.14509 • Published • 22
Maozhou Ge
Gmc2
ngsnethawarya's profile picture
wbfecbe84020a24274's profile picture
webbrain-4b7f2a9c's profile picture·
AI & ML interests
None yet
Recent Activity
liked a Space 2 days ago
nouamanetazi/nccl-cluster-visualizer liked a Space 2 days ago
thomwolf/moe-training-step liked a Space 2 days ago
huggingface/moe-blogpostOrganizations
None yet