Instructions to use mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir DeepSeek-R1-Distill-Qwen-1.5B-4bit mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
after updating file! model response is changed! now <think> tag is missing for this model from the beganing of the response. critical issue!
#2
by Norixxxx - opened
after updating file on your end for this model! model response is changed! now "" tag is missing for this model from the beginning of the response. critical issue!
please fix it asap!
@Norixxxx , the chat template was updated to match the chat template in the original repo. The <think> tag was added to the beginning of the model's response so that the model doesn't skip the thinking block, which can otherwise sometimes happen. You'll need to update whatever app you're using to handle this different response format.
Thank you!