README.md

---
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
tags:
- mlx
- lora
- code
- reasoning
- text-generation
- tennda
- distillation
---

# Tennda-Reason

> **A high-efficiency code & reasoning assistant fine-tuned by the Tennda Team**
> Structured `<think>` reasoning followed by complete, production-ready answers — refined thinking, reliable delivery, built for code generation, math, and logical reasoning on Apple Silicon.

---

## Model Overview

| Item | Details |
|---|---|
| **Model Name** | Tennda-Reason |
| **Developer** | Tennda Team |
| **Parameters** | 8.2B (4-bit quantized, ~4.3GB) |
| **Architecture** | Standard Transformer decoder, native `<think>` reasoning support |
| **Weight Format** | 4-bit, MLX native |
| **Framework** | MLX 0.32.1 + mlx-lm 0.31.3 (Apple Silicon Metal acceleration) |
| **Training** | QLoRA (rank=16, scale=32, last 16 layers, 19.4M trainable params / 0.237%) |
| **Training Data** | Multi-teacher distillation SFT corpus (2,000 curated samples): math 27% · code 27% · reasoning 20% · instruction 14% |
| **Context Length** | 1024 (trained), extensible via base capabilities |
| **Release Date** | 2026-08-24 |

---

## Highlights

- **Refined thinking**: `<think>` reasoning chains compressed to 300–550 token key-point style — no rambling, no wasted tokens
- **Complete delivery**: trained on "short thinking + complete answer" patterns; 0/5 test failures from runaway reasoning (baseline: 2/5)
- **Multi-domain**: balanced across math, code, logical reasoning, and instruction following
- **Apple Silicon native**: MLX 4-bit, ~6GB peak inference memory, runs on a single M-series machine

---

## Training Details

### Convergence (Loss)

| Metric | Start | Best | Final |
|---|---|---|---|
| Train loss | 1.70 | — | **0.421** |
| Val loss | 1.702 | **0.590** (iter 700) | 0.794 |

![Training curves](training_curves.png)

- 2,000 iterations ≈ 2 epochs (batch=2, seq=1024, ~67 tokens/s, ~8h on a single M4)
- GPU peak memory **8.5GB**, stable throughout, zero OOM
- **Released checkpoint: iter 700** (best validation loss), selected via blind A/B output comparison against the final checkpoint

### Checkpoint Selection

| | iter 700 (released) | iter 2000 |
|---|---|---|
| Val loss | **0.590** | 0.794 |
| Factual accuracy (networking task) | ✅ correct | ❌ detail error |
| Format compliance | ✅ verified | ✅ |

---

## Evaluation (5 prompts, temp=0.3, vs pre-training baseline)

| Task | Tennda-Reason | Baseline |
|---|---|---|
| Python quicksort | ✅ complete runnable code + complexity analysis | ❌ reasoning runaway, no answer produced |
| JS closures | ✅ full structured tutorial | ⚠️ thin output |
| SQL top salary per dept | ✅ window-function solutions | ❌ reasoning runaway, no answer produced |
| TCP 3-way handshake | ✅ vivid analogy, correct steps | ✅ concise & correct |
| Python HTTP server | ⚠️ multi-approach, minor rough edges | ✅ concise & correct |

**Summary**: format compliance 5/5; runaway-reasoning failures reduced from 2/5 (baseline) to 0/5; overall usability substantially improved.

---

## Usage (MLX)

```python
from mlx_lm import load, generate

model, tokenizer = load("MLA299/Tennda-Reason")
messages = [{"role": "user", "content": "Write a quicksort in Python"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=2048, verbose=False)
print(response)
```

Command line:

```bash
mlx_lm.generate --model MLA299/Tennda-Reason \
    --prompt "Write a SQL query: highest salary per department" \
    --max-tokens 2048
```

> Recommended sampling: temp=0.3–0.7, max_tokens ≥ 2048 (thinking chain + full answer)

---

## Limitations

- Training data is English-dominant; Chinese works but is not specifically optimized
- Post-SFT the model is more confident; factual-detail hallucinations are slightly higher than baseline — verify critical details in production
- Trained at 1024 context; longer inputs rely on native capabilities
- Contains synthetic distillation content; upstream data terms apply

---

## License

Apache-2.0. See the license terms for redistribution conditions.

---

## Citation

```bibtex
@misc{tennda-reason-2026,
  title = {Tennda-Reason: A Distillation-Fine-tuned Model for Code and Reasoning on Apple Silicon},
  author = {Tennda Team},
  year = {2026},
  month = {August},
  publisher = {Hugging Face},
  url = {https://huggingface.co/MLA299/Tennda-Reason}
}
```

---

*Tennda-Reason · © 2026 Tennda Team*