how does the function ```prepare_for_finetune``` work?
#7
by timaos - opened
I cannot see the structure of the data to fine-tune cpm-bee
When I ran the accelerator-based code, I got the losses whose value was nan.
Hi timaos, I meet the same problem and solved this by change the structure of the input data. Since the cpm-bee group changed the tokenizer class, the input data should be list of dict but not the huggingface's conventional dict of list. Just need a data_prepare.py to deal with the input data.
I have put my scripts on github: https://github.com/yingli-Claire/cpm-bee-infra-finetune