Huge Thanks for Using a Single-Stream Transformer, Porting It to a Multi-GPU ComfyUI Setup with RayLight Is a Breeze
#24
by komixenon - opened
Implementing sequence parallelism is quite easy because we do not have to deal with any weird packing and unpacking of audio, latent, and text sequences. Once the sequence enters the core model’s forward pass, only odd-length padding is necessary.
komixenon changed discussion title from Huge Thanks for Using a Single-Stream Transformer—Porting It to a Multi-GPU ComfyUI Setup with RayLight Is a Breeze to Huge Thanks for Using a Single-Stream Transformer, Porting It to a Multi-GPU ComfyUI Setup with RayLight Is a Breeze