README.md

metadata

license: apache-2.0
language:
  - en
tags:
  - slm
  - arithmetic
  - math
  - causal-lm
  - text-generation
  - custom_code
  - safetensors
library_name: transformers
pipeline_tag: text-generation
metrics:
  - accuracy
model-index:
  - name: Arithmetic-SLM
    results:
      - task:
          type: text-generation
          name: Arithmetic continuation
        dataset:
          type: AxiomicLabs/ArithMark-2.0
          name: ArithMark-2
        metrics:
          - type: accuracy
            name: Overall
            value: 78.6

Scores

Model	Parameters	Overall Score
Qwen/Qwen2.5-Math-1.5B	1.54B	82.08%
WhirlwindAI/Arithmetic-SLM	31.70M	78.60%
Qwen/Qwen2.5-3B	3.09B	78.44%
Qwen/Qwen2.5-1.5B	1.54B	77.72%
Qwen/Qwen2.5-Coder-1.5B	1.54B	74.88%
HuggingFaceTB/SmolLM2-1.7B	1.71B	66.12%
Qwen/Qwen2.5-0.5B	494M	63.04%
facebook/MobileLLM-R1-140M-base	140M	53.88%
SupraLabs/Supra-50M-Base	52M	27.12%

Arithmetic-SLM

Arithmetic-SLM is a small language model specialized for arithmetic continuation. It is designed to be highly efficient on numerical operations with mostly two-digit numbers in patterns such as:

a op b op c op d

where:

op = +, -, *, /

The goal is not to make a general chatbot. The goal is to train a compact model that can learn arithmetic patterns, operator priority, parentheses, and numerical continuation with very few parameters.

Calculation Patterns

1. Single operation

59 + 45 = 104
26 - 2 = 24
12 * 7 = 84
84 / 12 = 7

2. Two operations without parentheses

16 + 4 * 3 = 28
95 - 8 * 0 = 95
84 / 12 - 3 = 4

3. Two operations with parentheses

(16 / 4) + 44 = 48
(10 + 28) * 3 = 114
1 * (16 + 28) = 44

4. Three operations without parentheses

3 * 9 + 12 / 1 = 39
60 + 49 - 18 + 8 = 99
43 + 10 * 2 - 8 = 55

5. Three operations with parentheses

(132 / 12) + (46 - 15) = 42
(46 + 34) - (1 + 7) = 72
(21 + 27) * (14 - 7) = 336

6. Decimal arithmetic

0.5 * 0.5 = 0.25
1 / 10 = 0.1
7 / 2 = 3.5

Example Outputs with inference.py

Example 1 — Raw arithmetic prompt

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "59 + 45 =" \
  --max-new-tokens 32 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full

Expected style:

59 + 45 = 104

Example 2 — Production /no think format

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "0.5 * 0.5 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --repetition-penalty 1 \
  --frequency-penalty 0.0 \
  --no-repeat-ngram-size 0 \
  --seed -1 \
  --print-full

Example output:

[IM_START]user
0.5 * 0.5 = /no think[IM_END]
[IM_START]assistant
<think>
</think>
0.5 * 0.5 = 0.25[IM_END]

Example 3 — Operator priority

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "8 * 5 + 4 / 4 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full

Expected style:

8 * 5 + 4 / 4 = 41

Example 4 — Parentheses

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "(85 - 45) + 56 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.5 \
  --top-k 40 \
  --top-p 0.95 \
  --print-full

Expected style:

(85 - 45) + 56 = 96

Example 5 — Three-operation expression

python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "3 * 9 + 12 / 1 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.4 \
  --top-k 20 \
  --top-p 0.85 \
  --print-full

Expected style:

3 * 9 + 12 / 1 = 39

Next Research Directions

We will continue improving our dataset engineering, but more importantly, we want to teach the model what most models are never explicitly taught:

• Binary calculation: Neural Application Binary Interface, or NABI, with 16-bit numerical structures, including floats.
• FP16 to BASE-65,536 conversion: a float16 value is represented by 2 bytes, meaning 65,536 possible bit patterns. Base 65,536 also contains 65,536 possible integer values, making exact bit-level mapping possible.
• Dot-product learning: explicit learning of scalar products on float16 vectors with 16, 8, 4, and 2 dimensions.
• Learning the dynamics of its own learning: training the model to predict its own weights and gradients over time, including its own gradient descent dynamics.

This project does not claim to be a revolution.

It is an experiment in making small models learn precise arithmetic, numerical structure, and eventually parts of their own learning dynamics.

By Science AND FOR SCIENCE <3