Simple Q&A Fine Tune Dataset
#40
by
faheemraza1
- opened
The script sample_finetune.py uses HuggingFaceH4/ultrachat_200k dataset. This dataset has turn by turn conversation data in it. How important it is for LoRA fine tuning this model to have a turn-by-turn dataset? I have some books related to a very niche engineering field. It would be hard to generate a turn-by-turn dataset from them. Can I fine tune using a simple Q&A styled dataset? How effective will that be? And is there any chance that it will kill the model's ability to make back and forth conversation?