ucalyptus
/

prem-1B-chat-32k

Generated from Trainer

Model card Files Files and versions

Metrics Training metrics Community

ucalyptus commited on May 11, 2024

Commit

602a621

·

verified ·

1 Parent(s): 5448164

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -113,7 +113,8 @@ special_tokens:
 # prem-1B-chat-32k
-This model is a fine-tuned version of [premai-io/prem-1B-chat](https://huggingface.co/premai-io/prem-1B-chat) on the None dataset.
 It achieves the following results on the evaluation set:
 - Loss: 6.9843

 # prem-1B-chat-32k
+This model is a fine-tuned version of [premai-io/prem-1B-chat](https://huggingface.co/premai-io/prem-1B-chat) on the Argilla DPO dataset.
+32K context extended using [PoSE](https://arxiv.org/abs/2309.10400) method
 It achieves the following results on the evaluation set:
 - Loss: 6.9843