---
license: apache-2.0
base_model: GLM-4.7-Flash
tags:
  - math
  - reasoning
  - chain-of-thought
  - fine-tuned
  - glm
  - unsloth
datasets:
  - unsloth/OpenMathReasoning-mini
language:
  - en
pipeline_tag: text-generation
---

# 🍡 Mochi
![https://cdn.media.amplience.net/i/japancentre/maker-header-1455-royal-family-food-co-ltd/maker-header-1455-royal-family-food-co-ltd?$poi$&w=583&h=230&sm=c&fmt=auto](https://cdn.media.amplience.net/i/japancentre/maker-header-1455-royal-family-food-co-ltd/maker-header-1455-royal-family-food-co-ltd?$poi$&w=583&h=230&sm=c&fmt=auto)

Mochi is a math-reasoning fine-tune of **GLM-4.7-Flash**, trained on the [Open Math Reasoning (mini)](https://huggingface.co/datasets/unsloth/OpenMathReasoning-mini) dataset — the same chain-of-thought data used in the winning submission to the [AI Mathematical Olympiad Progress Prize 2 (AIMO-2)](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/leaderboard) on Kaggle.

The goal of this fine-tune is to sharpen GLM-4.7-Flash's step-by-step mathematical reasoning while keeping the small, fast footprint of the Flash base model.

Looking for a quantized/local version? See [mochi-gguf](https://huggingface.co/artindnr/mochi-gguf) for GGUF builds you can run with `llama.cpp`, Ollama, or LM Studio.

## Model Details

- **Base model:** GLM-4.7-Flash
- **Fine-tuning data:** [unsloth/OpenMathReasoning-mini](https://huggingface.co/datasets/unsloth/OpenMathReasoning-mini) — chain-of-thought math solutions distilled from the larger [OpenMathReasoning](https://huggingface.co/datasets/nvidia/OpenMathReasoning) dataset, which underpinned NVIDIA's AIMO-2-winning system
- **Objective:** Improve multi-step mathematical reasoning and answer accuracy on olympiad-style problems
- **Language(s):** English
- **License:** apache-2.0 (inherits from GLM-4.7-Flash — please verify against the base model's license before commercial use)

## Intended Use

Mochi is intended for:
- Step-by-step mathematical problem solving (algebra, number theory, combinatorics, olympiad-style questions)
- Research into chain-of-thought fine-tuning on small/flash-class models
- Local or low-latency deployments that still need decent math reasoning

It is **not** intended as a general-purpose reasoning or safety-critical decision-making tool. As with any LLM, verify important calculations independently.

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "artindnr/mochi"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "If x^2 - 5x + 6 = 0, what are the values of x?"}
]

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

You can also load Mochi with [Unsloth](https://github.com/unslothai/unsloth) for faster inference and further fine-tuning:

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="artindnr/mochi",
    max_seq_length=4096,
    load_in_4bit=True,
)
```

## Training

Mochi was fine-tuned starting from **GLM-4.7-Flash** on the CoT split of `unsloth/OpenMathReasoning-mini`, using Unsloth for efficient LoRA/QLoRA training.

| | |
|---|---|
| Base model | GLM-4.7-Flash |
| Dataset | unsloth/OpenMathReasoning-mini (CoT split) |
| Task | Supervised fine-tuning (chain-of-thought math) |
| Framework | Unsloth |

## Limitations

- Fine-tuned specifically for math reasoning; general chat/instruction-following ability may drift from the base model.
- Trained on a "mini" subset of OpenMathReasoning, so coverage of problem types is narrower than the full dataset.
- Like all LLMs, it can produce confidently incorrect derivations — always double-check final answers on problems that matter.

## Citation

If you use this model, please also credit the underlying dataset and competition it draws from:

```bibtex
@misc{openmathreasoning,
  title  = {OpenMathReasoning},
  author = {NVIDIA},
  year   = {2025},
  url    = {https://huggingface.co/datasets/nvidia/OpenMathReasoning}
}
```

## Acknowledgements

- [GLM-4.7-Flash](https://huggingface.co) for the base model
- [Unsloth](https://github.com/unslothai/unsloth) for fine-tuning tooling
- NVIDIA's AIMO-2 team for the OpenMathReasoning dataset