File size: 2,099 Bytes
e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 e39f580 0147828 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
base_model: hardlyworking/4Brepremover
library_name: transformers
model_name: 4Bkto
tags:
- generated_from_trainer
- axolotl
- trl
- kto
licence: license
---
# Model Card for 4Bkto
This model is a fine-tuned version of [hardlyworking/4Brepremover](https://huggingface.co/hardlyworking/4Brepremover).
It has been trained using [TRL](https://github.com/huggingface/trl).
## Quick start
```python
from transformers import pipeline
question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="hardlyworking/4Bkto", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])
```
## Training procedure
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/welchjacob254/New4B/runs/lw2hq72m)
This model was trained with KTO, a method introduced in [KTO: Model Alignment as Prospect Theoretic Optimization](https://huggingface.co/papers/2402.01306).
### Framework versions
- TRL: 0.18.2
- Transformers: 4.53.1
- Pytorch: 2.6.0+cu126
- Datasets: 3.6.0
- Tokenizers: 0.21.2
## Citations
Cite KTO as:
```bibtex
@article{ethayarajh2024kto,
title = {{KTO: Model Alignment as Prospect Theoretic Optimization}},
author = {Kawin Ethayarajh and Winnie Xu and Niklas Muennighoff and Dan Jurafsky and Douwe Kiela},
year = 2024,
eprint = {arXiv:2402.01306},
}
```
Cite TRL as:
```bibtex
@misc{vonwerra2022trl,
title = {{TRL: Transformer Reinforcement Learning}},
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
year = 2020,
journal = {GitHub repository},
publisher = {GitHub},
howpublished = {\url{https://github.com/huggingface/trl}}
}
``` |