Update README.md
Browse files
README.md
CHANGED
|
@@ -17,3 +17,11 @@ other companies may be trying to reach artificial general intelligence, but we a
|
|
| 17 |
|
| 18 |
this is a continuation of the "tuxsentience" series made by [@GrainWare](https://huggingface.co/GrainWare), however using [new advancements in AMD support in unsloth](https://github.com/unslothai/unsloth/pull/2520) we are now able to pack in significantly more grain per parameter.
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
this is a continuation of the "tuxsentience" series made by [@GrainWare](https://huggingface.co/GrainWare), however using [new advancements in AMD support in unsloth](https://github.com/unslothai/unsloth/pull/2520) we are now able to pack in significantly more grain per parameter.
|
| 19 |
|
| 20 |
+
## Recommended Settings
|
| 21 |
+
|
| 22 |
+
- **`temperature = 0.6`**
|
| 23 |
+
- `top_k = 20`
|
| 24 |
+
- `min_p = 0.00` (llama.cpp's default is 0.1)
|
| 25 |
+
- **`top_p = 0.95`**
|
| 26 |
+
- `presence_penalty = 0.0 to 2.0` (llama.cpp default turns it off, but to reduce repetitions, you can use this) Try 1.0 for example.
|
| 27 |
+
- Supports up to `262,144` context natively but you can set it to `32,768` tokens for less RAM use
|