Commit History

Increase grad_accum to 8, add gc.collect and empty_cache before training
31b65fb
verified

ligaments-dev commited on

Manually pre-tokenize with apply_chat_template, cap seq length to 512 for OOM fix
745965e
verified

ligaments-dev commited on

Cap turns to 8, require assistant end, 5k conversations for OOM fix
0ce3e14
verified

ligaments-dev commited on

Reduce to 1 epoch, 10k conversations for faster training on larger GPU
aa1a19d
verified

ligaments-dev commited on

Remove system role, verify alternating user/assistant for Gemma compatibility
48301ee
verified

ligaments-dev commited on

Add system prompt, cap dataset, filter non-user-first conversations
8c55461
verified

ligaments-dev commited on

Filter None text values to fix TypeError in join
4778444
verified

ligaments-dev commited on

Fix alternating roles and use dtype instead of torch_dtype
938e32b
verified

ligaments-dev commited on

Merge consecutive same-role turns to fix alternating user/assistant constraint
3686614
verified

ligaments-dev commited on

Remove max_seq_length from SFTTrainer (not supported in current TRL)
80788cf
verified

ligaments-dev commited on

Fix max_seq_length placement and torch_dtype deprecation
cb6f87d
verified

ligaments-dev commited on

Upload raw training script as train.py
1642b2a
verified

ligaments-dev commited on