Update README.md

README.md

@@ -1,3 +1,196 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- pt
+---
+
+**🧠 MiniAxion1.5-3M**
+
+**Emergent reasoning in a 2.7M parameter model.
+A tiny Portuguese-first language model that learns how to think before it learns how to be correct.**
+
+**🚀 Overview**
+
+MiniAxion1.5-3M is an ultra-compact (~2.7M parameters) GPT-style language model designed to investigate reasoning emergence at extreme small scale.
+
+Unlike typical small models optimized for fluency, MiniAxion is explicitly trained to produce:
+
+Structured reasoning traces
+Step-by-step thinking (<THINK><STEP>)
+Deterministic answer formatting
+
+It operates primarily in Portuguese, making it a rare example of a non-English reasoning-first nano model.
+
+**⚡ Why This Model Is Interesting**
+
+Most models follow this trajectory:
+
+Language → Knowledge → Reasoning
+
+MiniAxion flips part of that:
+
+Structure → Reasoning format → (still learning correctness)
+
+**💡 Key insight:**
+
+The model demonstrates that reasoning structure can emerge independently of reasoning accuracy.
+
+**🧪 Evaluation**
+Task Performance
+Task	Accuracy
+Addition	10%
+Subtraction	10%
+Multiplication	0%
+Even/Odd	100%
+Comparison	5%
+Sequence Completion	0%
+Word Problems (Addition)	10%
+Word Problems (Subtraction)	0%
+Word Problems (Multiplication)	10%
+True/False	100%
+Chat/Greetings	100%
+
+**🧠 Reasoning Behavior Metrics**
+Metric	Score
+Thinking Rate	100%
+Step Format	100%
+Answer Completion	100%
+
+✔ The model always thinks
+✔ The model always structures reasoning
+✔ The model always produces an answer
+
+**📊 Interpretation**
+
+MiniAxion exhibits a clear dissociation:
+
+✅ What it learned
+Reasoning format
+Step-by-step decomposition
+Logical task patterns (parity, boolean)
+❌ What it did NOT learn
+Arithmetic correctness
+Numerical reasoning
+Multi-step computation
+
+**🔬 Core Finding**
+
+Reasoning ≠ Correctness
+
+MiniAxion shows that:
+
+Models can internalize thinking patterns
+Without actually learning how to solve problems
+
+This makes it a strong candidate for studying:
+
+Emergent reasoning
+Tiny Recursive Models (TRMs)
+Reasoning distillation
+
+**🏗️ Architecture**
+Type: GPT-style Transformer
+Parameters: ~2.7M
+Objective: Next-token prediction
+Language: Portuguese (primary)
+Specialization: Structured reasoning traces
+
+**🧠 Training Strategy**
+
+The model was trained with a reasoning-first approach:
+
+Portuguese language grounding
+Structured reasoning data (<THINK><STEP>)
+Emphasis on:
+Deterministic formats
+Multi-step thinking
+Explicit reasoning tokens
+
+🚫 No RLHF
+🚫 No instruction tuning at scale
+🚫 No large model distillation (yet)
+
+⚠️ Limitations
+1. Arithmetic Collapse
+
+Near-random performance in:
+
+Addition
+
+Subtraction
+
+Multiplication
+
+→ Indicates lack of numerical representation learning
+
+Strong dependence on:
+
+Prompt format
+
+Token patterns
+
+Seen reasoning templates
+
+**🔮 Future Work**
+
+This model is just the beginning.
+
+📈 Scaling
+
+5M / 10M / 20M versions
+
+Track emergence of correctness
+
+🧪 Distillation
+
+Inject reasoning from larger models
+
+Improve accuracy without scaling params
+
+🔁 Self-Play / Synthetic Data
+
+Generate reasoning loops
+
+Reinforce correct chains
+
+🧩 Hybrid Reasoning
+
+Combine symbolic + neural learning
+
+Fix arithmetic weakness
+
+🧾 Example Output
+
+<THINK>
+<STEP> Identifico os números
+<STEP> Tento somar os valores
+<STEP> Ajusto o resultado
+</THINK>
+<ANSWER> 74 </ANSWER>
+
+✔ Perfect reasoning structure
+❌ Incorrect answer
+
+**💡 Takeaway**
+
+MiniAxion1.5-3M proves something important:
+
+Even a 2.7M model can learn to simulate thinking before it learns to actually think correctly.
+
+**🤝 Use Cases**
+
+Research on emergent reasoning
+
+Tiny model experimentation (CPU-friendly)
+
+Educational demos of:
+
+Chain-of-Thought
+
+Reasoning failure modes
+
+Base model for:
+
+Distillation
+
+NRM experiments