euclaise
/

ReMask-3B

Text Generation

Model card Files Files and versions

euclaise commited on Apr 2, 2024

Commit

ec58051

·

verified ·

1 Parent(s): 00f4710

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -111,7 +111,7 @@ As I expected, it improves GSM8K, but doesn't do much to ARC.
 - Training sequence length: 256
 - Input masking probability: 40%
 - Label masking probability: 10%
-- Answer-only (full rationale masking) probability: 10%
 - Batch size: 16, accumulated to 256
 - Epochs: 6
 - Learning rate: 1e-5

 - Training sequence length: 256
 - Input masking probability: 40%
 - Label masking probability: 10%
+- Answer-only (full rationale label masking) probability: 10%
 - Batch size: 16, accumulated to 256
 - Epochs: 6
 - Learning rate: 1e-5