Smol-reason
Collection
My first ever usage of GRPO fine tuning techniques, information learned from this model will be used on future Andy models. • 6 items • Updated
A newer version of this model is available: Sweaterdog/Smol-reason2.1-LoRA
🧠Smol-reason, a 3B model test for future models ðŸ§
Why?
When making the Andy series of models, I have been using PPO techniques to train models.
But as the bleeding edge of small models is becoming clear, reasoning models are the winners.
So, in order to learn the nuances of training models, I decided to train a small 3B model using GRPO techniques instead of PPO.
---------------------------------------------------------------------------------------------------------------------
The base model was Qwen2.5 3B, it is very smart as is, and even smarter with reasoning.
This model uses the following format while responding:
<think>
--reasoning content here--
</think>
<answer
--answer content here--
</answer>
Similar to the XML reasoning format but changed to use DeepSeek-R1 / QwQ thinking blocks.
- Downloads last month
- 56
Hardware compatibility
Log In to add your hardware
4-bit
5-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
docker model run hf.co/Sweaterdog/Smol-Reason: