Update README.md
Browse files
README.md
CHANGED
|
@@ -38,6 +38,11 @@ More information needed
|
|
| 38 |
## Training procedure
|
| 39 |
|
| 40 |
- Preprocessing function was created to tokenize the text and truncate the sequences longer than DistilBERT max seq length. Datasets [map](https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.map) function was used to apply the preprocessing func over the entire dataset.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
### Training hyperparameters
|
| 42 |
|
| 43 |
The following hyperparameters were used during training:
|
|
|
|
| 38 |
## Training procedure
|
| 39 |
|
| 40 |
- Preprocessing function was created to tokenize the text and truncate the sequences longer than DistilBERT max seq length. Datasets [map](https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.map) function was used to apply the preprocessing func over the entire dataset.
|
| 41 |
+
- DataCollatorWithPadding is more efficiently method to dynamically pad the sequences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.
|
| 42 |
+
- To evaluate model performance during training, it's quite helpful to include a metric. Load the [accuracy](https://huggingface.co/spaces/evaluate-metric/accuracy) metric from [Evaluate](https://huggingface.co/docs/evaluate/index) library.
|
| 43 |
+
- Define training hyperparameters in [TrainingArguments](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments). To push the model to the hub, we need to set `push_to_hub=True`. At the end of each epoch, the [Trainer](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.Trainer) will evaluate the accuracy and save the training checkpoint.
|
| 44 |
+
- Pass the trainingargs to the Trainer along with the model, dataset, tokenizer, data_collator.
|
| 45 |
+
- Call [train()](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.Trainer.train) to finetune your model.
|
| 46 |
### Training hyperparameters
|
| 47 |
|
| 48 |
The following hyperparameters were used during training:
|