Model does not follow or acknowledge system prompts?
#3
by RonanMcGovern - opened
Is this expected? Passing a system message results in it being ignored...
BTW the tokenizer config has the system message in there, so I'm not sure why it behaves this way. Could be the quanting...
FWIW, I can only see to run q4fp16.
I tried int8 and uint8, they load but produce garbage.