Use attention dropout during training (#10)

Allow device auto map (#8)

Truncate to 8k by default (#5)

Set max length to 2B

Update README.md

Update README.md

Allow pytorch<2 to use without passing attn_implementation flag (#4)

chore: update from afe81ca705ca1a5bd6b7d90548fcac068850b2af

Remove triton flash implementation

Delete flash_attn_triton.py

chore: update from 896c12d73073854c513200fb74a4887cf25b2b97

chore: update from f36c08c8a58c21b5aaab523fa03fb4a24b475612

chore: update from 07ce15d58b77559fce77ea89e92d398f28663bd9

feat: allow changing flash implementation

allow math kernel

Flash attention! (#2)

Create README.md

Allow flash attn (#1)

rename to jina bert in configuration file

rename to jina bert

add basic configuration and model file

initial commit