Fix for FlashAttention RuntimeError & Triton Multi GPU fix.

#17

by Satandon1999 - opened Jun 10, 2024

base: refs/heads/main

←

from: refs/pr/17

Discussion Files changed

+91

-88

Satandon1999

Jun 10, 2024

Fix based on the discussion here: https://huggingface.co/microsoft/Phi-3-small-8k-instruct/discussions/11

Update positional_embedding.pyf5a1a1be

Satandon1999 changed pull request title from Update positional_embedding.py to Fix for FlashAttention RuntimeError Jun 10, 2024

Update triton_flash_blocksparse_attn.pyed7de9a0

Satandon1999 changed pull request title from Fix for FlashAttention RuntimeError to Fix for FlashAttention RuntimeError & Triton Multi GPU fix. Jun 18, 2024

Satandon1999

Jun 18, 2024

@damajercakms . Please review. Thanks.

Update triton_flash_blocksparse_attn.py351b4fda

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment