-
Increase grad_accum to 8, add gc.collect and empty_cache before training
-
Manually pre-tokenize with apply_chat_template, cap seq length to 512 for OOM fix
-
Cap turns to 8, require assistant end, 5k conversations for OOM fix
-
Reduce to 1 epoch, 10k conversations for faster training on larger GPU
-
Remove system role, verify alternating user/assistant for Gemma compatibility
-
Add system prompt, cap dataset, filter non-user-first conversations
-
Filter None text values to fix TypeError in join
-
Fix alternating roles and use dtype instead of torch_dtype
-
Merge consecutive same-role turns to fix alternating user/assistant constraint
-
Remove max_seq_length from SFTTrainer (not supported in current TRL)
-
Fix max_seq_length placement and torch_dtype deprecation
-
Upload raw training script as train.py
-
Add training script for gemma-2b-it telecom full fine-tuning
-
initial commit
