Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,18 @@ The time of the great dragon war is upon us! How many different fantasy novels?
|
|
| 15 |
|
| 16 |
Trained with full text windows, followed by completion, followed by ORPO, followed by one more epoch of the full text, rotated 1/4 in the window. That last train settled everything down and it seems quite coherent.
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
To get rid of unwanted EOS's, I did the following...
|
| 19 |
|
| 20 |
```
|
|
@@ -24,4 +36,5 @@ result_dict['lm_head.weight'][2] = 0
|
|
| 24 |
model.state_dict = lambda : result_dict
|
| 25 |
```
|
| 26 |
|
| 27 |
-
So now there are no EOS's at all, ever.
|
|
|
|
|
|
| 15 |
|
| 16 |
Trained with full text windows, followed by completion, followed by ORPO, followed by one more epoch of the full text, rotated 1/4 in the window. That last train settled everything down and it seems quite coherent.
|
| 17 |
|
| 18 |
+
### How to Use
|
| 19 |
+
|
| 20 |
+
No prompt, but start with a bit of story, or a name. Maybe put an authors note of something like:
|
| 21 |
+
|
| 22 |
+
```
|
| 23 |
+
King Robb Stark and Lord Rahl are at war.
|
| 24 |
+
```
|
| 25 |
+
|
| 26 |
+
You have quite a cast of characters to draw from. Perhaps Perrin makes a stop by the Waystone Inn, or Zeddicus and Gandalf have a smoke together.
|
| 27 |
+
|
| 28 |
+
### Hacks
|
| 29 |
+
|
| 30 |
To get rid of unwanted EOS's, I did the following...
|
| 31 |
|
| 32 |
```
|
|
|
|
| 36 |
model.state_dict = lambda : result_dict
|
| 37 |
```
|
| 38 |
|
| 39 |
+
So now there are no EOS's at all, ever.
|
| 40 |
+
|