--- license: apache-2.0 language: - en tags: - slm - arithmetic - math - causal-lm - text-generation - custom_code - safetensors library_name: transformers pipeline_tag: text-generation metrics: - accuracy model-index: - name: Arithmetic-SLM results: - task: type: text-generation name: Arithmetic continuation dataset: type: AxiomicLabs/ArithMark-2.0 name: ArithMark-2 metrics: - type: accuracy name: Overall value: 78.60 --- ![image](https://cdn-uploads.huggingface.co/production/uploads/6975505c60cf607407afe2c0/g4yF4mQR39XYMoUehSTsX.png) # Scores

Model	Parameters	Overall Score
`Qwen/Qwen2.5-Math-1.5B`	1.54B	82.08%
`WhirlwindAI/Arithmetic-SLM`	31.70M	78.60%
`Qwen/Qwen2.5-3B`	3.09B	78.44%
`Qwen/Qwen2.5-1.5B`	1.54B	77.72%
`Qwen/Qwen2.5-Coder-1.5B`	1.54B	74.88%
`HuggingFaceTB/SmolLM2-1.7B`	1.71B	66.12%
`Qwen/Qwen2.5-0.5B`	494M	63.04%
`facebook/MobileLLM-R1-140M-base`	140M	53.88%
`SupraLabs/Supra-50M-Base`	52M	27.12%

# Arithmetic-SLM Arithmetic-SLM is a small language model specialized for arithmetic continuation. It is designed to be highly efficient on numerical operations with mostly two-digit numbers in patterns such as: ```text a op b op c op d ``` where: ```text op = +, -, *, / ``` The goal is not to make a general chatbot. The goal is to train a compact model that can learn arithmetic patterns, operator priority, parentheses, and numerical continuation with very few parameters. ## Calculation Patterns ### 1. Single operation ```text 59 + 45 = 104 26 - 2 = 24 12 * 7 = 84 84 / 12 = 7 ``` ### 2. Two operations without parentheses ```text 16 + 4 * 3 = 28 95 - 8 * 0 = 95 84 / 12 - 3 = 4 ``` ### 3. Two operations with parentheses ```text (16 / 4) + 44 = 48 (10 + 28) * 3 = 114 1 * (16 + 28) = 44 ``` ### 4. Three operations without parentheses ```text 3 * 9 + 12 / 1 = 39 60 + 49 - 18 + 8 = 99 43 + 10 * 2 - 8 = 55 ``` ### 5. Three operations with parentheses ```text (132 / 12) + (46 - 15) = 42 (46 + 34) - (1 + 7) = 72 (21 + 27) * (14 - 7) = 336 ``` ### 6. Decimal arithmetic ```text 0.5 * 0.5 = 0.25 1 / 10 = 0.1 7 / 2 = 3.5 ``` ## Example Outputs with `inference.py` ### Example 1 — Raw arithmetic prompt ```bash python3 inference.py \ --model WhirlwindAI/Arithmetic-SLM \ --prompt "59 + 45 =" \ --max-new-tokens 32 \ --temperature 0.6 \ --top-k 50 \ --top-p 0.97 \ --print-full ``` Expected style: ```text 59 + 45 = 104 ``` ### Example 2 — Production `/no think` format ```bash python3 inference.py \ --model WhirlwindAI/Arithmetic-SLM \ --prompt "0.5 * 0.5 =" \ --no-think \ --max-new-tokens 48 \ --temperature 0.6 \ --top-k 50 \ --top-p 0.97 \ --repetition-penalty 1 \ --frequency-penalty 0.0 \ --no-repeat-ngram-size 0 \ --seed -1 \ --print-full ``` Example output: ```text [IM_START]user 0.5 * 0.5 = /no think[IM_END] [IM_START]assistant 0.5 * 0.5 = 0.25[IM_END] ``` ### Example 3 — Operator priority ```bash python3 inference.py \ --model WhirlwindAI/Arithmetic-SLM \ --prompt "8 * 5 + 4 / 4 =" \ --no-think \ --max-new-tokens 48 \ --temperature 0.6 \ --top-k 50 \ --top-p 0.97 \ --print-full ``` Expected style: ```text 8 * 5 + 4 / 4 = 41 ``` ### Example 4 — Parentheses ```bash python3 inference.py \ --model WhirlwindAI/Arithmetic-SLM \ --prompt "(85 - 45) + 56 =" \ --no-think \ --max-new-tokens 48 \ --temperature 0.5 \ --top-k 40 \ --top-p 0.95 \ --print-full ``` Expected style: ```text (85 - 45) + 56 = 96 ``` ### Example 5 — Three-operation expression ```bash python3 inference.py \ --model WhirlwindAI/Arithmetic-SLM \ --prompt "3 * 9 + 12 / 1 =" \ --no-think \ --max-new-tokens 48 \ --temperature 0.4 \ --top-k 20 \ --top-p 0.85 \ --print-full ``` Expected style: ```text 3 * 9 + 12 / 1 = 39 ``` ## Next Research Directions We will continue improving our dataset engineering, but more importantly, we want to teach the model what most models are never explicitly taught: - **Binary calculation:** Neural Application Binary Interface, or **NABI**, with 16-bit numerical structures, including floats. - **FP16 to BASE-65,536 conversion:** a `float16` value is represented by 2 bytes, meaning 65,536 possible bit patterns. Base 65,536 also contains 65,536 possible integer values, making exact bit-level mapping possible. - **Dot-product learning:** explicit learning of scalar products on `float16` vectors with 16, 8, 4, and 2 dimensions. - **Learning the dynamics of its own learning:** training the model to predict its own weights and gradients over time, including its own gradient descent dynamics. This project does not claim to be a revolution. It is an experiment in making small models learn precise arithmetic, numerical structure, and eventually parts of their own learning dynamics. **By Science AND FOR SCIENCE <3**