The output cannot get actually value but meaningless symbol
Hello, I am really interested in FoodLMM and I think this work is useful and creative.
But it seems the model file doesn't involve a processor, so that we cannot directly use it among huggingface.
So I tried to use LISA and change the model to FoodLMMM, and I find that when I ask the model about the nutrition information. It cannot out put the actually answer about the value of nutrients, but output [MASS_TOTAL] g /[CAL_TOTAL] kcal. Which is meaningless, can you figure out what is happened?
And also I cannot generate picture... Do you know what's wrong with it?
As FoodLMM has specific heads for outputting the actual value of these tokens, you can refer to our github repository at https://github.com/YuehaoYin/FoodLMM for conducting inference with the trained model.