Final_Assignment_Template

Sleeping

mjschock commited on May 2, 2025

Commit

8bd5794

unverified ·

1 Parent(s): d577d8c

Remove unused dataset_text_field parameter from create_trainer function in train.py to streamline trainer configuration.

Files changed (1) hide show

train.py CHANGED Viewed

@@ -193,7 +193,6 @@ def create_trainer(
             tokenizer=tokenizer,
             train_dataset=dataset["train"],
             eval_dataset=dataset["validation"],
-            dataset_text_field="text",
             max_seq_length=max_seq_length,
             dataset_num_proc=2,
             packing=False,

             tokenizer=tokenizer,
             train_dataset=dataset["train"],
             eval_dataset=dataset["validation"],
             max_seq_length=max_seq_length,
             dataset_num_proc=2,
             packing=False,