Update README.md
Browse files
README.md
CHANGED
|
@@ -116,4 +116,27 @@ As I expected, it improves GSM8K, but doesn't do much to ARC.
|
|
| 116 |
- Epochs: 6
|
| 117 |
- Learning rate: 1e-5
|
| 118 |
- Learning rate schedule: One Cycle, cosine, no cycle_momentum
|
| 119 |
-
- Regularization weight: 0.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
- Epochs: 6
|
| 117 |
- Learning rate: 1e-5
|
| 118 |
- Learning rate schedule: One Cycle, cosine, no cycle_momentum
|
| 119 |
+
- Regularization weight: 0.1
|
| 120 |
+
|
| 121 |
+
## Prompt format
|
| 122 |
+
|
| 123 |
+
The format for reddit-instruct and oasst2 was:
|
| 124 |
+
|
| 125 |
+
```
|
| 126 |
+
<|user|>
|
| 127 |
+
[insert instruction here]
|
| 128 |
+
<|assistant|>
|
| 129 |
+
[insert response here]
|
| 130 |
+
<|user|>
|
| 131 |
+
...
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
The format for TinyCoT was:
|
| 135 |
+
```
|
| 136 |
+
<|user|>
|
| 137 |
+
[insert instruction here]
|
| 138 |
+
<|rationale|>
|
| 139 |
+
[insert reasoning here]
|
| 140 |
+
<|answer|>
|
| 141 |
+
[insert direct answer here]
|
| 142 |
+
```
|