Add note on generation parameters
Browse files
README.md
CHANGED
|
@@ -39,6 +39,10 @@ Custom GPT with RoPE, QK-norm, ReLU² activation, value embeddings (ResFormer),
|
|
| 39 |
| Vocab size | 32,768 (BPE, GPT-4 style split pattern) |
|
| 40 |
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
## Quick Start
|
| 43 |
|
| 44 |
```python
|
|
|
|
| 39 |
| Vocab size | 32,768 (BPE, GPT-4 style split pattern) |
|
| 40 |
|
| 41 |
|
| 42 |
+
## Notes
|
| 43 |
+
|
| 44 |
+
**Generation parameters:** You may need to play with temperature to get good results. The default is `0.6` with `top_k=50`.
|
| 45 |
+
|
| 46 |
## Quick Start
|
| 47 |
|
| 48 |
```python
|