Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)
- Dataset: Wikipedia-10
- Target modules = ["query", "value"]
{
"epoch": 2.987443401951174,
"total_flos": 1.04520375926784e+18,
"train_loss": 0.8730267834961414,
"train_runtime": 69929.3152,
"train_samples": 21423,
"train_samples_per_second": 0.915,
"train_steps_per_second": 0.029
}