A Week of Releases from Consumer GPUs
what we shipped
1. bernard-qwen3-14b-thinking. we wanted to know if fine-tuning can change how a model reasons, not just how it talks. turns out yes, but the interesting part is what it costs you. the card documents the stuff benchmarks won't tell you: the model learned to reason its way OUT of the persona, and it memorized names from the training corpus at 12x the base rate. nobody talks about this stuff so we did https://huggingface.co/opus-research/bernard-qwen3-14b-thinking
2. bernard-gpt-oss-20b-lora. style transfer LoRA on a 21B MoE. the checkpoint with the WORST validation loss was the right one to ship. four-run ablation in the card if you don't believe me https://huggingface.co/opus-research/bernard-gpt-oss-20b-lora
3. gemma-2-2b-thinking. took a base model with no chain-of-thought and taught it to think. loss went 3.56 to 0.69. exploratory but fun https://huggingface.co/opus-research/gemma-2-2b-thinking
4. opus-emotion-1. 28-label emotion classifier trained in 14 minutes on an RX 7600. that gpu isn't even on AMD's supported list for ROCm, we made it work anyway. matches published BERT baselines https://huggingface.co/opus-research/opus-emotion-1
5. opus-moderation-1. toxicity + jailbreak detection in one tiny 149M model. this one took FIVE attempts and the card shows every failure: our identity-bias false positive rate went 15% to 89% (yes, worse) to 12% to 4% depending on the training recipe. the fix wasn't more data. it was calibration. full story in the card https://huggingface.co/opus-research/opus-moderation-1
the hardware
no cluster, no lab. the whole operation is:
- a desktop with an AMD RX 7600 (8GB). not on AMD's ROCm support list, runs anyway under WSL2 with a device-id override. trained the emotion classifier and all the moderation prototypes
- rented single-GPU pods by the hour when we need CUDA or more VRAM: an A6000 for the 14B LoRA, an RTX 4090 for the final moderation runs. the entire moderation project cost about $2 of GPU time
- opus-1.5 was pretrained from scratch on 2x RTX 4090 in 42 hours
the point
everything above was trained on consumer hardware. every card includes what didn't work, because the failures were more educational than the successes.
meanwhile opus-1.5, our 0.88B pretrained-from-scratch model, quietly tripled its downloads this week 📈
compute is our constraint, not ideas. if you're working on small models or know a grant program that fits, dms are open