Update README.md
Browse files
README.md
CHANGED
|
@@ -113,7 +113,7 @@ logger = logging.getLogger(__name__)
|
|
| 113 |
|
| 114 |
MODEL_NAME = "nisten/Biggie-SmoLlm-0.15B-Base"
|
| 115 |
MAX_LENGTH = 2048
|
| 116 |
-
BATCH_SIZE =
|
| 117 |
LEARNING_RATE = 2e-4
|
| 118 |
MAX_STEPS = 3000
|
| 119 |
GRADIENT_ACCUMULATION_STEPS = 2
|
|
@@ -351,7 +351,7 @@ if __name__ == "__main__":
|
|
| 351 |
```
|
| 352 |
🚀 Now go forth and train, accelerate that code!
|
| 353 |
|
| 354 |
-
> **Note:** You'll need about
|
| 355 |
|
| 356 |
Results will appear in `./capybara_finetuned_results`
|
| 357 |
|
|
|
|
| 113 |
|
| 114 |
MODEL_NAME = "nisten/Biggie-SmoLlm-0.15B-Base"
|
| 115 |
MAX_LENGTH = 2048
|
| 116 |
+
BATCH_SIZE = 9
|
| 117 |
LEARNING_RATE = 2e-4
|
| 118 |
MAX_STEPS = 3000
|
| 119 |
GRADIENT_ACCUMULATION_STEPS = 2
|
|
|
|
| 351 |
```
|
| 352 |
🚀 Now go forth and train, accelerate that code!
|
| 353 |
|
| 354 |
+
> **Note:** You'll need about 14GB of VRAM. If you have 8GB, change to batch size 4.
|
| 355 |
|
| 356 |
Results will appear in `./capybara_finetuned_results`
|
| 357 |
|