README.md

---
license: mit
language:
- en
metrics:

pipeline_tag: text-generation
tags:
- nrm
- nano
- reasoning
- thinking
- sub-1m
- lowparams
- custom_code
---

# 🧠 MiniAxion1-0.9M

**MiniAxion1-0.9M** is a Nano Reasoning Model (NRM) with ~920K parameters designed to explore the emergence of structured reasoning in extremely small neural networks.

Despite its minimal size, the model demonstrates strong consistency in reasoning format and step-based thinking using explicit `<THINK>` and `<STEP>` tokens.

---

## 🚀 Overview

* **Model Type:** Nano Reasoning Model (NRM)
* **Parameters:** ~920,833
* **Architecture:** Transformer (6 layers: 2 entry + 2 shared + 2 exit)
* **d_model:** 256
* **Heads:** 8
* **FFN size:** 512
* **LoRA Rank:** 16
* **Vocabulary Size:** 2048
* **Training Time:** ~80 minutes (CPU)

---

## 🧠 Key Capabilities

### ✅ Structured Reasoning

The model reliably produces structured reasoning traces:

```
<THINK>
<STEP> ...
<STEP> ...
</THINK>
<ANS>...</ANS>
```

* 100% usage of reasoning tokens
* Consistent multi-step formatting
* Stable output structure across tasks

---

### ⚡ Ultra-Lightweight

* Runs efficiently on CPU
* Designed for experimentation and rapid iteration
* Suitable for embedded or game-like environments

---

### 🧪 Research-Oriented Design

MiniAxion1 is not intended to compete with large-scale models. Instead, it is built to:

* Study reasoning emergence in small models
* Explore structure vs correctness trade-offs
* Enable fast iteration cycles for AI research

---

## 📊 Evaluation Results

| Task                    | Accuracy |
| ----------------------- | -------- |
| Arithmetic              | 3.3%     |
| Two-Step Arithmetic     | 10.0%    |
| Even/Odd                | 100.0%   |
| Comparison              | 5.0%     |
| Pattern Completion      | 0.0%     |
| Word Problems           | 0.0%     |
| Sorting                 | 0.0%     |
| Chain-of-Thought Format | 100.0%   |

**Average Accuracy:** 16.9%

---

## 🔍 Observations

* The model learns reasoning *structure* before reasoning *correctness*
* Chain-of-thought formatting is highly reliable
* Arithmetic and symbolic reasoning remain limited at this scale
* Evidence of partial decoupling between reasoning steps and final answers

---

## ⚠️ Limitations

* Weak performance on arithmetic and multi-step reasoning tasks
* Susceptible to incorrect intermediate reasoning steps
* Limited generalization beyond trained patterns
* Not suitable for production use in critical systems
* Due to 920k parameters, low results on evaluation is expected

---

## 🎯 Intended Use Cases

* 🧪 AI research and experimentation
* 🎮 Game AI / NPC reasoning simulation
* 📚 Educational demonstrations of reasoning structure
* ⚙️ Lightweight reasoning prototypes

---

### Quick start

```python

import torch
from model import NRMModel
from tokenizer import Tokenizer

# load
model = NRMModel.from_config("config.json")
model.load_state_dict(torch.load("model.pt"))
model.eval()

tokenizer = Tokenizer.load("tokenizer.json")

def generate(prompt):
    tokens = tokenizer.encode(prompt)
    output = model.generate(tokens)
    return tokenizer.decode(output)

print(generate("<INST>What is 2 + 2?</INST>"))
```

## 🧠 Philosophy

MiniAxion1 explores a key question:

> *Can structured reasoning emerge in extremely small models?*

This model provides early evidence that:

* Reasoning format can be learned efficiently
* Structure and correctness are separable capabilities
* Useful behavior can emerge even at sub-1M scale

---

## 🔮 Future Directions

* Improved dataset alignment for arithmetic reasoning
* Scaling parameters (1M → 10M range)
* Better coupling between reasoning and answers
* Task-specific specialization (e.g., math-only variants)
* distillation knowledge on bigger models

---

## 🤝 Acknowledgments

This model was developed as part of ongoing experimentation in nano-scale reasoning systems.
the main question was: "How low could a model think(or mimic it)?

---

## 📎 Model

👉 https://huggingface.co/AxionLab-Co/MiniAxion1-0.9M

---

## 🧪 Disclaimer

This is an experimental research model. Outputs may be incorrect even when reasoning appears structured or convincing.