Update README.md
Browse files
README.md
CHANGED
|
@@ -72,7 +72,7 @@ the decoder. The decoder operates under text generation functionality in this ca
|
|
| 72 |
LM has been shown to be an effective learning
|
| 73 |
objective to close the pretrain-finetune gap for generative downstream tasks"
|
| 74 |
|
| 75 |
-
Generally speaking, the `<tdec>` token was
|
| 76 |
|
| 77 |
```
|
| 78 |
<s><tdec> Creates a task that to retry a previously abandoned task.
|
|
@@ -81,6 +81,8 @@ Returns:
|
|
| 81 |
Task: a task that was abandoned but should be retried or None if there are
|
| 82 |
no abandoned tasks that should be retried.</s>
|
| 83 |
```
|
|
|
|
|
|
|
| 84 |
# Hyperparameters
|
| 85 |
|
| 86 |
MAX_SOURCE_LENGTH = 256 <br>
|
|
|
|
| 72 |
LM has been shown to be an effective learning
|
| 73 |
objective to close the pretrain-finetune gap for generative downstream tasks"
|
| 74 |
|
| 75 |
+
Generally speaking, the `<tdec>` token was prepended to the target (the docstring) to signal to the decoder that it is in a text generation functionality. A sample row looks like this:
|
| 76 |
|
| 77 |
```
|
| 78 |
<s><tdec> Creates a task that to retry a previously abandoned task.
|
|
|
|
| 81 |
Task: a task that was abandoned but should be retried or None if there are
|
| 82 |
no abandoned tasks that should be retried.</s>
|
| 83 |
```
|
| 84 |
+
|
| 85 |
+
This helps the decoder know under what downstream task it is currently being fine tuned in, improving the process.
|
| 86 |
# Hyperparameters
|
| 87 |
|
| 88 |
MAX_SOURCE_LENGTH = 256 <br>
|