maldv commited on
Commit
f2c84df
·
verified ·
1 Parent(s): 17b3bf4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -1
README.md CHANGED
@@ -15,6 +15,18 @@ The time of the great dragon war is upon us! How many different fantasy novels?
15
 
16
  Trained with full text windows, followed by completion, followed by ORPO, followed by one more epoch of the full text, rotated 1/4 in the window. That last train settled everything down and it seems quite coherent.
17
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  To get rid of unwanted EOS's, I did the following...
19
 
20
  ```
@@ -24,4 +36,5 @@ result_dict['lm_head.weight'][2] = 0
24
  model.state_dict = lambda : result_dict
25
  ```
26
 
27
- So now there are no EOS's at all, ever.
 
 
15
 
16
  Trained with full text windows, followed by completion, followed by ORPO, followed by one more epoch of the full text, rotated 1/4 in the window. That last train settled everything down and it seems quite coherent.
17
 
18
+ ### How to Use
19
+
20
+ No prompt, but start with a bit of story, or a name. Maybe put an authors note of something like:
21
+
22
+ ```
23
+ King Robb Stark and Lord Rahl are at war.
24
+ ```
25
+
26
+ You have quite a cast of characters to draw from. Perhaps Perrin makes a stop by the Waystone Inn, or Zeddicus and Gandalf have a smoke together.
27
+
28
+ ### Hacks
29
+
30
  To get rid of unwanted EOS's, I did the following...
31
 
32
  ```
 
36
  model.state_dict = lambda : result_dict
37
  ```
38
 
39
+ So now there are no EOS's at all, ever.
40
+