Set model_type
#1
by LaaZa - opened
Please add model_type to the config as DeciLM has now done. Unfortunately this changes the model type from deci_lm to deci which will break the 0.6.0 implementation of the model in AutoGPTQ, I made a PR there to change this but that means either one of those will not work there, so for future I would support the explicitly set "model_type": "deci".
Loading with AutoGPTQ directly will require manually setting it depending on the version.
Great feedback - agree 100% - thanks for highlighting this. I have updated the config.json file to include "model_type": "deci" - keep me posted and let me know if this fixes the issue with AutoGPTQ. I had similar issues with GGUF and fixing was on my to-do list.