Spaces:
Runtime error
Runtime error
Hajime MATSUMOTO
commited on
Commit
Β·
702c22f
1
Parent(s):
ce5bcf8
Reduce batch size to avoid OOM on L40S 48GB
Browse files
train.py
CHANGED
|
@@ -234,10 +234,10 @@ training_args = TrainingArguments(
|
|
| 234 |
num_train_epochs=2,
|
| 235 |
max_steps=-1, # -1 = γ¨γγγ―γγΌγΉ
|
| 236 |
|
| 237 |
-
# γγγγ΅γ€γΊ (L40S 48GB
|
| 238 |
-
per_device_train_batch_size=
|
| 239 |
-
per_device_eval_batch_size=
|
| 240 |
-
gradient_accumulation_steps=
|
| 241 |
|
| 242 |
# ε¦ηΏη
|
| 243 |
learning_rate=1e-4,
|
|
|
|
| 234 |
num_train_epochs=2,
|
| 235 |
max_steps=-1, # -1 = γ¨γγγ―γγΌγΉ
|
| 236 |
|
| 237 |
+
# γγγγ΅γ€γΊ (L40S 48GB + 7B QLoRA)
|
| 238 |
+
per_device_train_batch_size=2,
|
| 239 |
+
per_device_eval_batch_size=2,
|
| 240 |
+
gradient_accumulation_steps=16, # ζεΉγγγγ΅γ€γΊ: 2*16=32
|
| 241 |
|
| 242 |
# ε¦ηΏη
|
| 243 |
learning_rate=1e-4,
|