Phi-3 conversation format, example training script and perplexity metric (#1582)

add support for rpo_alpha (#1681)

re-enable DPO for tests in modal ci (#1374)

re-enable phi for tests in modal ci (#1373)

make sure the CI fails when pytest script fails (#1669)

Generalizing the chat_template prompt strategy (#1660) [skip ci]

Switch to parallel FFD bin packing algorithm. (#1619)

Fix llama3 chat_template (extra <|eot_id|> on last turn) (#1635)

Add KTO support (#1640)

feat: Add LLaMA-3 instruct prompt strategies for fine-tuning (#1553)

ADD: warning hub model (#1301)

Add ORPO example and e2e test (#1572)

wrap prepared_ds_path in str() to avoid TypeError in fsspec package (#1548)

ORPO Trainer replacement (#1551)

fix broken linting (#1541)

feat: validate sample packing requires flash_attention (#1465)

Support loading datasets saved via save_to_disk (#1432)

make sure to capture non-null defaults from config validation (#1415)

fix(dataset): normalize tokenizer config and change hash from tokenizer class to tokenizer path (#1298)

strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428)

Feat: Add sharegpt multirole (#1137)

ORPO (#1419)

Train parameters exclusively in specific ranges (#1390)

Add Glaive conversation format support (#1365)

plain input/output prompt strategy w/o chat templates (#1346)

run tests again on Modal (#1289) [skip ci]

fix for protected model_ namespace w pydantic (#1345)

more fixes 20240228 (#1342) [skip ci]

Pydantic 2.x cfg (#1239)

make mlflow optional (#1317)

Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273)

Pretrain transforms (#1261)

relora: magnitude pruning of the optimizer (#1245)

support for true batches with multipack (#1230)

Support for additional_special_tokens (#1221) [skip ci]

Peft lotfq (#1222)

ADD: warning if hub_model_id ist set but not any save strategy (#1202)

Feat/chatml add system message (#1117)

Phi2 multipack (#1173)

DPO cleanup (#1126)

Feat(test): Add tests for alpaca chatml prompt tokenizer (#1088)

Falcon embeddings (#1149) [skip docker]

set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122) [skip ci]

Deprecate max packed sequence len (#1141)

Multipack simplify for Mixtral (#1142)

Add shifted sparse attention (#973) [skip-ci]

Add `layers_to_transform` for `lora_config` (#1118)

Enable or disable bf16 support based on availability (#1116)

keep gate in fp32 for 16 bit loras (#1105)

add gptneox embeddings, fix phi2 inputs, also fix the casting (#1083)