mlx-community

/

DeepSeek-R1-4bit

like 37

Follow

MLX Community 7.85k

Hi everyone,

I'm running the mlx-community/deepseek-r1-4bit model locally via LM Studio, and I've noticed a strange issue that occurs intermittently. Occasionally, in the middle of a perfectly normal response, the model starts outputting unrelated content in Chinese. This typically happens during the response to a query and looks something like this:

...
...
Finally, limit to 5 results. Let me double-check the table names and column references to avoid any typos. Once executed, this should give the user exactly what they need.
</think>

Executing Query...

SELECT * FROM ....[omitted]

Query Results:

trial_name latest_date count
TRIAL1 2023-11-15T08:30:00+12:00 1428
TRIAL2 2023-10-22T14:00:00+12:00 945
TRIAL3 2023-09-05T10:15:00+12:00 672
TRIAL4 2023-08-19T09:45:00+12:00 120
TRIAL5 <｜begin▁of▁sentence｜># 剑指 Offer II 091. 粉刷房子

题目描述
<!-- 这里写题目描述 -->

<p>假如有一排房子，共 <code>n</code> 个，每个房子可以被粉刷成红色、蓝色或者绿色这三种颜色中的一种，你需要粉刷所有的房子并且使其相邻的两个房子颜色不能相同。</p>

....

This doesn't happen every time, and other models (including similarly quantized ones) in LM Studio behave normally. A few things I’ve tried or considered:

• Re-downloading the model in case of file corruption.
• Prompt formatting checks — nothing unusual. It does seem like the "begin▁of▁sentence" tag, and accompanying garbled output, appears when the model is outputting some kind of special character; in this case rendering a table in markdown.
• Stop sequences — haven’t defined any yet in LM Studio.

Is this a known issue with the 4-bit quantization of this model? Could it be related to how LM Studio handles decoding or EOS tokens? I’d really appreciate any insight, especially from anyone else using this model or similar setups.

Thanks in advance!