Update README.md

README.md

@@ -1,3 +1,305 @@
 ---
 license: apache-2.0
+language:
+  - en
+tags:
+  - slm
+  - arithmetic
+  - math
+  - causal-lm
+  - text-generation
+  - custom_code
+  - safetensors
+library_name: transformers
+pipeline_tag: text-generation
+metrics:
+  - accuracy
+model-index:
+  - name: Arithmetic-SLM
+    results:
+      - task:
+          type: text-generation
+          name: Arithmetic continuation
+        dataset:
+          type: AxiomicLabs/ArithMark-2.0
+          name: ArithMark-2
+        metrics:
+          - type: accuracy
+            name: Overall
+            value: 78.60
 ---
+
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/6975505c60cf607407afe2c0/g4yF4mQR39XYMoUehSTsX.png)
+
+# Arithmetic-SLM
+
+Arithmetic-SLM is a small language model specialized for arithmetic continuation. It is designed to be highly efficient on numerical operations with mostly two-digit numbers in patterns such as:
+
+```text
+a op b op c op d
+```
+
+where:
+
+```text
+op = +, -, *, /
+```
+
+The goal is not to make a general chatbot. The goal is to train a compact model that can learn arithmetic patterns, operator priority, parentheses, and numerical continuation with very few parameters.
+
+
+## Scores
+
+<div align="center">
+
+<table>
+  <tr>
+    <th align="center">Model</th>
+    <th align="center">Parameters</th>
+    <th align="center">Overall Score</th>
+  </tr>
+  <tr>
+    <td align="center"><code>Qwen/Qwen2.5-Math-1.5B</code></td>
+    <td align="center">1.54B</td>
+    <td align="center"><strong>82.08%</strong></td>
+  </tr>
+  <tr>
+    <td align="center"><code>PhysiQuanty/Arithmetic-SLM</code></td>
+    <td align="center">31.70M</td>
+    <td align="center"><strong>78.60%</strong></td>
+  </tr>
+  <tr>
+    <td align="center"><code>Qwen/Qwen2.5-3B</code></td>
+    <td align="center">3.09B</td>
+    <td align="center">78.44%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>Qwen/Qwen2.5-1.5B</code></td>
+    <td align="center">1.54B</td>
+    <td align="center">77.72%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>Qwen/Qwen2.5-Coder-1.5B</code></td>
+    <td align="center">1.54B</td>
+    <td align="center">74.88%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>HuggingFaceTB/SmolLM2-1.7B</code></td>
+    <td align="center">1.71B</td>
+    <td align="center">66.12%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>Qwen/Qwen2.5-0.5B</code></td>
+    <td align="center">494M</td>
+    <td align="center">63.04%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>facebook/MobileLLM-R1-140M-base</code></td>
+    <td align="center">140M</td>
+    <td align="center">53.88%</td>
+  </tr>
+  <tr>
+    <td align="center"><code>SupraLabs/Supra-50M-Base</code></td>
+    <td align="center">52M</td>
+    <td align="center">27.12%</td>
+  </tr>
+</table>
+
+</div>
+
+## Calculation Patterns
+
+### 1. Single operation
+
+```text
+59 + 45 = 104
+26 - 2 = 24
+12 * 7 = 84
+84 / 12 = 7
+```
+
+### 2. Two operations without parentheses
+
+```text
+16 + 4 * 3 = 28
+95 - 8 * 0 = 95
+84 / 12 - 3 = 4
+```
+
+### 3. Two operations with parentheses
+
+```text
+(16 / 4) + 44 = 48
+(10 + 28) * 3 = 114
+1 * (16 + 28) = 44
+```
+
+### 4. Three operations without parentheses
+
+```text
+3 * 9 + 12 / 1 = 39
+60 + 49 - 18 + 8 = 99
+43 + 10 * 2 - 8 = 55
+```
+
+### 5. Three operations with parentheses
+
+```text
+(132 / 12) + (46 - 15) = 42
+(46 + 34) - (1 + 7) = 72
+(21 + 27) * (14 - 7) = 336
+```
+
+### 6. Decimal arithmetic
+
+```text
+0.5 * 0.5 = 0.25
+1 / 10 = 0.1
+7 / 2 = 3.5
+```
+
+## Example Outputs with `inference.py`
+
+### Example 1 — Raw arithmetic prompt
+
+```bash
+python3 inference.py \
+  --model PhysiQuanty/Arithmetic-SLM \
+  --prompt "59 + 45 =" \
+  --max-new-tokens 32 \
+  --temperature 0.6 \
+  --top-k 50 \
+  --top-p 0.97 \
+  --print-full
+```
+
+Expected style:
+
+```text
+59 + 45 = 104
+```
+
+### Example 2 — Production `/no think` format
+
+```bash
+python3 inference.py \
+  --model PhysiQuanty/Arithmetic-SLM \
+  --prompt "0.5 * 0.5 =" \
+  --no-think \
+  --max-new-tokens 48 \
+  --temperature 0.6 \
+  --top-k 50 \
+  --top-p 0.97 \
+  --repetition-penalty 1 \
+  --frequency-penalty 0.0 \
+  --no-repeat-ngram-size 0 \
+  --seed -1 \
+  --print-full
+```
+
+Example output:
+
+```text
+[IM_START]user
+0.5 * 0.5 = /no think[IM_END]
+[IM_START]assistant
+<think>
+</think>
+0.5 * 0.5 = 0.25[IM_END]
+```
+
+### Example 3 — Operator priority
+
+```bash
+python3 inference.py \
+  --model PhysiQuanty/Arithmetic-SLM \
+  --prompt "8 * 5 + 4 / 4 =" \
+  --no-think \
+  --max-new-tokens 48 \
+  --temperature 0.6 \
+  --top-k 50 \
+  --top-p 0.97 \
+  --print-full
+```
+
+Expected style:
+
+```text
+8 * 5 + 4 / 4 = 41
+```
+
+### Example 4 — Parentheses
+
+```bash
+python3 inference.py \
+  --model PhysiQuanty/Arithmetic-SLM \
+  --prompt "(85 - 45) + 56 =" \
+  --no-think \
+  --max-new-tokens 48 \
+  --temperature 0.5 \
+  --top-k 40 \
+  --top-p 0.95 \
+  --print-full
+```
+
+Expected style:
+
+```text
+(85 - 45) + 56 = 96
+```
+
+### Example 5 — Three-operation expression
+
+```bash
+python3 inference.py \
+  --model PhysiQuanty/Arithmetic-SLM \
+  --prompt "3 * 9 + 12 / 1 =" \
+  --no-think \
+  --max-new-tokens 48 \
+  --temperature 0.4 \
+  --top-k 20 \
+  --top-p 0.85 \
+  --print-full
+```
+
+Expected style:
+
+```text
+3 * 9 + 12 / 1 = 39
+```
+
+### Example 6 — BOS/EOS base mode for base models
+
+Use this mode for base models that were not trained with the Qwen-style `[IM_START]user ... /no think[IM_END]` format.
+
+```bash
+python3 inference.py \
+  --model Supra-50M-Base-local \
+  --prompt "8 * 5 + 4 / 4 =" \
+  --no-qwen-format \
+  --max-new-tokens 32 \
+  --temperature 0.6 \
+  --top-k 50 \
+  --top-p 0.97 \
+  --repetition-penalty 1 \
+  --frequency-penalty 0.0 \
+  --no-repeat-ngram-size 0 \
+  --seed -1 \
+  --print-full
+```
+
+## Next Research Directions
+
+We will continue improving our dataset engineering, but more importantly, we want to teach the model what most models are never explicitly taught:
+
+- **Binary calculation:** Neural Application Binary Interface, or **NABI**, with 16-bit numerical structures, including floats.
+- **FP16 to base-65k conversion:** a `float16` value is represented by 2 bytes, meaning 65,536 possible bit patterns. Base 65,536 also contains 65,536 possible integer values, making exact bit-level mapping possible.
+- **Dot-product learning:** explicit learning of scalar products on `float16` vectors with 16, 8, 4, and 2 dimensions.
+- **Learning the dynamics of its own learning:** training the model to predict its own weights and gradients over time, including its own gradient descent dynamics.
+
+This project does not claim to be a revolution.
+
+It is an experiment in making small models learn precise arithmetic, numerical structure, and eventually parts of their own learning dynamics.
+
+**By Science AND FOR SCIENCE <3**