Commit History

Fix: add assistant_only_loss=False to prevent all labels being masked to -100
61b62c0
verified

ssdataanalysis commited on

Training in progress, step 500
eb750d6
verified

ssdataanalysis commited on

Optimal config: high-quality Hebrew data, constant LR, packing disabled, lora_dropout=0.1
960c757
verified

ssdataanalysis commited on

Switch to packing=True, batch=4, grad_acc=4, step-based checkpoints for speed
522d2a2
verified

ssdataanalysis commited on

Disable eval to fix OOM on A10G
f0b6c36
verified

ssdataanalysis commited on

Fix OOM: reduce max_length to 2048, disable packing, increase grad accum, remove liger
8792aab
verified

ssdataanalysis commited on

Add training script with QLoRA
1f5ea7f
verified

ssdataanalysis commited on

Add entrypoint script with fixed args
ac64bab
verified

ssdataanalysis commited on

Fix typo in exclude_modules and add entrypoint script
8fd9e86
verified

ssdataanalysis commited on

Upload training script
6507f53
verified

ssdataanalysis commited on

Upload training script
c572108
verified

ssdataanalysis commited on

initial commit
3ea3a00
verified

ssdataanalysis commited on