Update README.md
Browse files
README.md
CHANGED
|
@@ -53,7 +53,7 @@ Chat-Node is a conversational variant of [Pacific-Prime 1.5B](https://huggingfac
|
|
| 53 |
|-----------|-------|
|
| 54 |
| Learning Rate | 2e-5 |
|
| 55 |
| Batch Size | 4 |
|
| 56 |
-
| Gradient Accumulation |
|
| 57 |
| Weight Decay | 0.01 |
|
| 58 |
| Warmup Ratio | 3% |
|
| 59 |
| Gradient Checkpointing | Enabled |
|
|
|
|
| 53 |
|-----------|-------|
|
| 54 |
| Learning Rate | 2e-5 |
|
| 55 |
| Batch Size | 4 |
|
| 56 |
+
| Gradient Accumulation | 16 (effective batch: 64) |
|
| 57 |
| Weight Decay | 0.01 |
|
| 58 |
| Warmup Ratio | 3% |
|
| 59 |
| Gradient Checkpointing | Enabled |
|