fine tune model Posible?
Thank you very much for a model with these capabilities.
I have some questions:
Is it possible to fine tune the Qlora model, flash-attention, add special tokens?
Is there an example in collaboration so I can reproduce it?
We'll try to share finetuning scripts as soon as possible !
Thank you very much for the model, it is very good. Will it not be possible to observe the evaluations with large models?
@ArthurZ not sure if this help, but I have been trying to create a finetuning example for huggingface here: https://github.com/huggingface/transformers/pull/26997
Ran into issues around padding the batches and would love some help :D
@ncoop57 the padding of batches is a bit tricky, and it will be supported in main when this PR https://github.com/huggingface/transformers/pull/27007 is done, we're on it!