Finetuned model is less in size than original model
#8
by hiiamsid - opened
I used sft trainer for finetuning this model, but to my surprise while pushing finetuned model 3 files of smaller size is save in total around 15GB but original model is around 25 GB. This is surprising
@hiiamsid the most likely reason is that this model is stored in fp32 format while the output you got is in fp16 format.
fp32 is pretty useless since its 2x the size of fp16 but fp32 is 2x slower then fp16. there is no quality difference as well.
Thats why 99% of the time everyone uses fp16 since it increases speed in inference and training, decreases vram/ram usage in inference and training and is much smaller then fp32.