Update README.md
Browse files
README.md
CHANGED
|
@@ -102,7 +102,7 @@ The training data was aggregated from multiple sources:
|
|
| 102 |
- **Mixed Precision Training:** Native AMP
|
| 103 |
|
| 104 |
**Label Imbalance Correction:**
|
| 105 |
-
A correction factor was computed for each topic based on the number of non-
|
| 106 |
**weight = (average non-unknown count) / (non-unknown count for the topic)**
|
| 107 |
|
| 108 |
---
|
|
@@ -152,13 +152,13 @@ After hyperparameter search, update your training arguments using the best hyper
|
|
| 152 |
|
| 153 |
```python
|
| 154 |
# After hyperparameter search:
|
| 155 |
-
best_run = trainer.hyperparameter_search(direction=
|
| 156 |
-
print(
|
| 157 |
|
| 158 |
# Update TrainingArguments accordingly
|
| 159 |
-
training_args.learning_rate = best_run.hyperparameters[
|
| 160 |
-
training_args.num_train_epochs = best_run.hyperparameters[
|
| 161 |
-
training_args.gradient_accumulation_steps = best_run.hyperparameters[
|
| 162 |
|
| 163 |
# Reinitialize the Trainer with the updated arguments
|
| 164 |
trainer = CustomTrainer(
|
|
|
|
| 102 |
- **Mixed Precision Training:** Native AMP
|
| 103 |
|
| 104 |
**Label Imbalance Correction:**
|
| 105 |
+
A correction factor was computed for each topic based on the number of non-'unknown' samples to mitigate label imbalance. The correction weight for each topic was calculated as:
|
| 106 |
**weight = (average non-unknown count) / (non-unknown count for the topic)**
|
| 107 |
|
| 108 |
---
|
|
|
|
| 152 |
|
| 153 |
```python
|
| 154 |
# After hyperparameter search:
|
| 155 |
+
best_run = trainer.hyperparameter_search(direction='maximize', hp_space=hp_space, n_trials=5)
|
| 156 |
+
print('Best hyperparameters:', best_run.hyperparameters)
|
| 157 |
|
| 158 |
# Update TrainingArguments accordingly
|
| 159 |
+
training_args.learning_rate = best_run.hyperparameters['learning_rate']
|
| 160 |
+
training_args.num_train_epochs = best_run.hyperparameters['num_train_epochs']
|
| 161 |
+
training_args.gradient_accumulation_steps = best_run.hyperparameters['gradient_accumulation_steps']
|
| 162 |
|
| 163 |
# Reinitialize the Trainer with the updated arguments
|
| 164 |
trainer = CustomTrainer(
|