gguf量化模型翻译时, 不遵从prompt要求的目标语言, 随机输出各种语言结果
#2
by tobchef - opened
我测试GemmaX2-28-2B-v0.1 的q6k量化, 和GemmaX2-28-9B-v0.1 的q4k/q5k量化模型, 都是不遵从prompt要求的目标语言, 随机输出各种语言结果.
但是原始的gemma-2-9b-it量化模型, 就不会有类似问题.
prompt = "Translate this from Chinese to English:\nChinese: 我爱机器翻译\nEnglish:"
I have tried imatrix q4_k_m quant from mradermacher and had similar instruction-following problem. I'd like to ask if you solved the problem.