adding llama3 fastchat conversation monkeypatch (#1539)

Add support for Gemma chat template (#1530)

Unsloth gradient checkpointing offload (#1528)

qwen2_moe support w multipack (#1455)

fix some of the edge cases for Jamba (#1452)

Remove seq_len arg in rotary_emb (#1443)

beta support for multipack with gemmoe: (#1402)

add starcoder2 (#1349)

Update fastchat_conversation_turns.py (#1294) [skip ci]

fix steps check for anneal on first cycle (#1316)

make mlflow optional (#1317)

multipack for gemma (#1313)

allow the optimizer prune ratio for ReLoRA to be configurable (#1287)

Add MPS support (#1264)

simplify haldning for newer multipack patches so they can be added in a single place (#1270)

relora: magnitude pruning of the optimizer (#1245)

support for true batches with multipack (#1230)

Respect sliding_window=None (#1214)

Mixtral fixes 20240124 (#1192) [skip ci]

Phi2 multipack (#1173)

Falcon embeddings (#1149) [skip docker]

Qwen2 (#1166)

Multipack simplify for Mixtral (#1142)

Add shifted sparse attention (#973) [skip-ci]

optimize calculation of cu_seqlens from position_ids (#1084) [skip ci]

Added chatglm3 conversation type for training models like TinyLLama (#1036)

bump transformers and update attention class map name (#1023)

remove landmark attn and xpos rope implementations (#1010)

fix mistral prompt assembly (#982)

Fix prompt assembly for llama (#952)

fix: switch to using the HuggingFace Transformers NEFT implementation (#941)

Mixtral official (#942)

adds llama and mistral dropout support (#858)

various bugfixes (#856)

refactor neft patch to be more re-usable similar to trl's impl (#796)

Hotfix for not saving correctly (#762)

Implement fused modules (#747)

Mistral: Sliding Window Attention with Flash Attention and Sample Packing (#732)

add noisy embedding (#721)

flash_attention + sample packing for stablelm 3b (#671)

fix for flash attn w mistral w/o sammple packing (#648)

Mistral flash attn packing (#646)

skip some flash attn patches unless explicitly enabled (#643)

use fastchat conversations template (#578)

update for recent transformers updates (#636)

Feat: Add support for upstream FA2 (#626)

btlm and falcon monkey patches for flash attn (#566)

Add training callback to send predictions to WandB table (#521)

reorg a bit

use flash_attn rmsnorm when available (#526)