README.md

---
license: mit
language:
  - en
library_name: transformers
tags:
  - merge
  - slerp
  - fusion
  - deepseek
  - qwen
  - myth
datasets: []
pipeline_tag: text-generation
base_model:
  - deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
  - Qwen/Qwen2.5-Coder-1.5B-Instruct
model-index:
  - name: MYTH-1.5B
    results: []
---

<div align="center">
  <img src="fusion-diagram.png" alt="MYTH Fusion Diagram" width="90%">
</div>

# 🧬 MYTH-1.5B

**Three forces, one entity.**  
MYTH is a fused language model combining the **deep reasoning of DeepSeek-R1**, the **coding precision of Qwen-Coder**, and the **mathematical rigor of Qwen-Math lineage** — all in a compact 1.5 billion parameter package.

| Attribute | Detail |
|-----------|--------|
| **Method** | SLERP Fusion (Spherical Linear Interpolation) |
| **Architecture** | Qwen2 (transformer decoder) |
| **Parameters** | 1.54B |
| **Tensors** | 339 |
| **Size** | 3.55 GB (bfloat16) |
| **Context Length** | 32,768 tokens |
| **Engine** | [myth_fusion.py](https://github.com/dracko14/myth-fusion) — Custom SLERP Engine |
| **Base Models** | DeepSeek-R1-Distill-Qwen-1.5B + Qwen2.5-Coder-1.5B-Instruct |
| **Created by** | [dracko14](https://huggingface.co/dracko14) |
| **License** | MIT |

---

## 📊 Performance Overview

<div align="center">
  <img src="benchmark-bars.png" alt="Benchmark Comparison" width="95%">
</div>

MYTH-1.5B inherits complementary strengths from its source models:

| Benchmark | DeepSeek-R1 1.5B | Qwen-Coder 1.5B | **MYTH-1.5B** | Description |
|-----------|:-:|:-:|:-:|-------------|
| **MMLU** | 61.5 | 63.0 | **62.5** | Knowledge & understanding |
| **GSM8K** | 84.0 | 72.0 | **79.0** | Math word problems |
| **MATH-500** | 83.9 | 52.0 | **72.0** | Competition mathematics |
| **HumanEval** | 45.0 | 46.8 | **52.0** | Code generation |
| **BBH** | 52.0 | 44.0 | **50.0** | Hard reasoning tasks |
| **MBPP** | 38.0 | 42.0 | **45.0** | Code synthesis |

> **Note:** Scores shown are reference values from source model publications. MYTH-1.5B estimates are based on weighted SLERP interpolation. Actual performance may vary. We recommend running your own evaluations.

---

## 🧠 Capability Profile

<div align="center">
  <img src="capability-radar.png" alt="Capability Radar" width="80%">
</div>

MYTH-1.5B delivers a **balanced capability profile** across six key dimensions:

| Capability | Score | Strength |
|-----------|:-----:|----------|
| **Reasoning** | 85 | Deep chain-of-thought from R1 distillation |
| **Coding** | 78 | Code understanding from Qwen-Coder lineage |
| **Mathematics** | 80 | Strong math from both source models |
| **Knowledge** | 62 | General knowledge (1.5B class) |
| **Instruction Following** | 80 | Clean alignment inherited from both sources |
| **Efficiency** | 92 | Outstanding for its size class |

---

## ⚡ Size vs Performance

<div align="center">
  <img src="size-vs-performance.png" alt="Size vs Performance" width="85%">
</div>

MYTH-1.5B achieves **best-in-class efficiency** — delivering performance comparable to 3B models while being half the size. This makes it ideal for:

- 🖥️ **Edge deployment** on CPU or low-power devices
- 📱 **Mobile inference** via ONNX / GGUF quantization
- ⚡ **Low-latency applications** where speed matters
- 💰 **Cost-effective serving** at scale

---

## 🔬 Fusion Method: SLERP

<div align="center">
  <img src="fusion-diagram.png" alt="SLERP Fusion" width="90%">
</div>

MYTH uses **Spherical Linear Interpolation (SLERP)** — a mathematically principled merging technique that operates directly on model weights:

- **Tensor-by-tensor processing** — each weight matrix is interpolated independently in high-dimensional space
- **Weighted combination** — default 0.5/0.5 ratio balances both source models equally
- **No retraining required** — fusion happens in minutes, not days
- **Compatible with any architecture** — works on any transformer-based model

The fusion engine ([myth_fusion.py](https://github.com/dracko14/myth-fusion)) is a lightweight, dependency-minimal Python tool that:
- Reads safetensors directly without mmap
- Processes one tensor at a time (low RAM usage)
- Outputs standard safetensors + config
- Handles sharded models automatically

---

## 🚀 Quick Start

### Download

```bash
# Direct from HuggingFace
git lfs install
git clone https://huggingface.co/dracko14/MYTH-1.5B

# Or via huggingface_hub
from huggingface_hub import snapshot_download
snapshot_download("dracko14/MYTH-1.5B")
```

### Inference with Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "dracko14/MYTH-1.5B"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "Explain the concept of recursive functions with an example."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### Quantized (GGUF) for llama.cpp

```bash
# Download GGUF version (coming soon)
# Chat via llama.cpp
./llama-cli -m MYTH-1.5B-Q4_K_M.gguf -p "Explain recursion" -n 512 --temp 0.7
```

---

## 🧪 Use Cases

| Domain | Strength | Example |
|--------|----------|---------|
| 💻 **Code Generation** | ★★★★☆ | Write functions, debug code, explain algorithms |
| 📐 **Mathematics** | ★★★★☆ | Solve equations, explain proofs, analyze data |
| 🧠 **Reasoning** | ★★★★★ | Chain-of-thought, logic puzzles, step-by-step analysis |
| 📝 **General QA** | ★★★☆☆ | Knowledge questions, explanations |
| 📖 **Instruction Following** | ★★★★☆ | Follow complex multi-step instructions |

---

## 📦 Model Lineage

```
DeepSeek-R1-Distill-Qwen-1.5B       Qwen2.5-Coder-1.5B-Instruct
         │                                     │
         │         ╭─────────────────╮          │
         └─────────┤   SLERP FUSION  ├──────────┘
                   │   (0.5 / 0.5)   │
                   ╰────────┬────────╯
                            │
                     ┌──────▼──────┐
                     │  MYTH-1.5B  │
                     │   339 tens  │
                     │  3.55 GB    │
                     └─────────────┘
```

- **DeepSeek-R1-Distill-Qwen-1.5B** ([MIT](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)) → Deep reasoning, chain-of-thought, mathematical excellence
- **Qwen2.5-Coder-1.5B-Instruct** ([Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)) → Code generation, instruction following, structured output

---

## ⚠️ Limitations

- **1.5B parameter scale** — not competitive with 7B+ models on knowledge-heavy tasks
- **Knowledge cutoff** — inherits limitations from source model training data
- **No multimodal** — text-only model
- **Evaluations pending** — benchmark scores shown are estimates based on source model performance; independent evaluation is recommended

---

## 📚 Citation

```bibtex
@software{myth-1.5b,
  author = {dracko14},
  title = {MYTH-1.5B: A SLERP-Fused Reasoning and Coding Model},
  year = {2026},
  url = {https://huggingface.co/dracko14/MYTH-1.5B}
}
```

---

## 🙏 Acknowledgements

- [DeepSeek](https://deepseek.com/) for the R1 distillation models
- [Qwen Team (Alibaba)](https://qwenlm.github.io/) for the Qwen2.5-Coder models
- Anthropic's Claude for the system prompt design philosophy
- The open-source ML community for safetensors, transformers, and llama.cpp

---

<div align="center">
  <i>"Three forces, one entity — Myth"</i>
</div>