Fixes flash-attn import with a try/except statement

Adds support for flash-attn rotary embedding and fused dense layers.

Adds support for MQA/GQA and attention mask during training.

Update modeling_mixformer_sequential.py

Adding _set_gradient_checkpointing for compatibility (#22)

Upload modeling_mixformer_sequential.py

Add more precise license metadata (UI will be cleaner!) (#35)

Upload README.md

fix(phi-1_5): Checks length of `attention_mask`if it is passed as direct tensor.

Support for `attention_mask` in forward pass.

add _no_split_modules property (#17)

Update README.md

Update README.md

Link paper to arXiv (#5)

Update README.md

Update README.md

Update README.md

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Upload Research License.docx

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Upload tokenizer

Upload MixFormerSequentialForCausalLM

initial commit