| --- |
| license: apache-2.0 |
| base_model: GLM-4.7-Flash |
| tags: |
| - math |
| - reasoning |
| - chain-of-thought |
| - fine-tuned |
| - glm |
| - unsloth |
| datasets: |
| - unsloth/OpenMathReasoning-mini |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # 🍡 Mochi |
|  |
|
|
| Mochi is a math-reasoning fine-tune of **GLM-4.7-Flash**, trained on the [Open Math Reasoning (mini)](https://huggingface.co/datasets/unsloth/OpenMathReasoning-mini) dataset — the same chain-of-thought data used in the winning submission to the [AI Mathematical Olympiad Progress Prize 2 (AIMO-2)](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/leaderboard) on Kaggle. |
|
|
| The goal of this fine-tune is to sharpen GLM-4.7-Flash's step-by-step mathematical reasoning while keeping the small, fast footprint of the Flash base model. |
|
|
| Looking for a quantized/local version? See [mochi-gguf](https://huggingface.co/artindnr/mochi-gguf) for GGUF builds you can run with `llama.cpp`, Ollama, or LM Studio. |
|
|
| ## Model Details |
|
|
| - **Base model:** GLM-4.7-Flash |
| - **Fine-tuning data:** [unsloth/OpenMathReasoning-mini](https://huggingface.co/datasets/unsloth/OpenMathReasoning-mini) — chain-of-thought math solutions distilled from the larger [OpenMathReasoning](https://huggingface.co/datasets/nvidia/OpenMathReasoning) dataset, which underpinned NVIDIA's AIMO-2-winning system |
| - **Objective:** Improve multi-step mathematical reasoning and answer accuracy on olympiad-style problems |
| - **Language(s):** English |
| - **License:** apache-2.0 (inherits from GLM-4.7-Flash — please verify against the base model's license before commercial use) |
|
|
| ## Intended Use |
|
|
| Mochi is intended for: |
| - Step-by-step mathematical problem solving (algebra, number theory, combinatorics, olympiad-style questions) |
| - Research into chain-of-thought fine-tuning on small/flash-class models |
| - Local or low-latency deployments that still need decent math reasoning |
|
|
| It is **not** intended as a general-purpose reasoning or safety-critical decision-making tool. As with any LLM, verify important calculations independently. |
|
|
| ## How to Use |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "artindnr/mochi" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto") |
| |
| messages = [ |
| {"role": "user", "content": "If x^2 - 5x + 6 = 0, what are the values of x?"} |
| ] |
| |
| inputs = tokenizer.apply_chat_template( |
| messages, add_generation_prompt=True, return_tensors="pt" |
| ).to(model.device) |
| |
| outputs = model.generate(inputs, max_new_tokens=512) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| You can also load Mochi with [Unsloth](https://github.com/unslothai/unsloth) for faster inference and further fine-tuning: |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name="artindnr/mochi", |
| max_seq_length=4096, |
| load_in_4bit=True, |
| ) |
| ``` |
|
|
| ## Training |
|
|
| Mochi was fine-tuned starting from **GLM-4.7-Flash** on the CoT split of `unsloth/OpenMathReasoning-mini`, using Unsloth for efficient LoRA/QLoRA training. |
|
|
| | | | |
| |---|---| |
| | Base model | GLM-4.7-Flash | |
| | Dataset | unsloth/OpenMathReasoning-mini (CoT split) | |
| | Task | Supervised fine-tuning (chain-of-thought math) | |
| | Framework | Unsloth | |
|
|
| ## Limitations |
|
|
| - Fine-tuned specifically for math reasoning; general chat/instruction-following ability may drift from the base model. |
| - Trained on a "mini" subset of OpenMathReasoning, so coverage of problem types is narrower than the full dataset. |
| - Like all LLMs, it can produce confidently incorrect derivations — always double-check final answers on problems that matter. |
|
|
| ## Citation |
|
|
| If you use this model, please also credit the underlying dataset and competition it draws from: |
|
|
| ```bibtex |
| @misc{openmathreasoning, |
| title = {OpenMathReasoning}, |
| author = {NVIDIA}, |
| year = {2025}, |
| url = {https://huggingface.co/datasets/nvidia/OpenMathReasoning} |
| } |
| ``` |
|
|
| ## Acknowledgements |
|
|
| - [GLM-4.7-Flash](https://huggingface.co) for the base model |
| - [Unsloth](https://github.com/unslothai/unsloth) for fine-tuning tooling |
| - NVIDIA's AIMO-2 team for the OpenMathReasoning dataset |
