Commit
·
da08437
1
Parent(s):
cf5b212
Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,11 @@ First Version of Instruction Tuned Bloomz-7B1 model on ChatGPT dataset (85k data
|
|
| 10 |
**Training Details :**
|
| 11 |
* Epochs: 5
|
| 12 |
* Batch Size : 5 instantaneous per device x 2 gradient accumulation steps x 8 gpus = 80
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
* Machine : 8xA100 80GB
|
| 14 |
|
| 15 |
**Dataset Details :**
|
|
|
|
| 10 |
**Training Details :**
|
| 11 |
* Epochs: 5
|
| 12 |
* Batch Size : 5 instantaneous per device x 2 gradient accumulation steps x 8 gpus = 80
|
| 13 |
+
* Max Length : 512
|
| 14 |
+
* Weight Decay : 0
|
| 15 |
+
* Learning Rate : 5e-5
|
| 16 |
+
* Learning Rate Scheduler Type : Linear
|
| 17 |
+
* Number of warmup steps : 0
|
| 18 |
* Machine : 8xA100 80GB
|
| 19 |
|
| 20 |
**Dataset Details :**
|