|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- zh |
|
|
- en |
|
|
library_name: transformers |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- causal-lm |
|
|
- math |
|
|
- reasoning |
|
|
- tool-calling |
|
|
- function-calling |
|
|
- bilingual |
|
|
- code |
|
|
- symbolic-solver |
|
|
- llm |
|
|
- pytorch |
|
|
base_model: Kirim-ai/Kirim-V1-base |
|
|
datasets: [] |
|
|
metrics: |
|
|
- math |
|
|
- gsm8k |
|
|
- minerva |
|
|
model-index: |
|
|
- name: Kirim-1-Math |
|
|
results: [] |
|
|
widget: |
|
|
- text: "Solve: ∫(x² + 2x + 1)dx" |
|
|
example_title: "Calculus Integration" |
|
|
- text: "解方程组: 2x + 3y = 12, 4x - y = 5" |
|
|
example_title: "System of Equations (Chinese)" |
|
|
- text: "Use the calculator tool to compute 2^128" |
|
|
example_title: "Tool Calling Example" |
|
|
--- |
|
|
|
|
|
# Kirim-1-Math (16B) |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
<img src="https://imgur.com/n5uQJXF.png" alt="Helion-V2-Thinking Logo" width="70%"/> |
|
|
|
|
|
</div> |
|
|
|
|
|
--- |
|
|
|
|
|
**The First Kirim Model with Advanced Mathematical Reasoning and Tool Calling** |
|
|
|
|
|
[Base Model](https://huggingface.co/Kirim-ai/Kirim-V1-base) • [模型卡](https://huggingface.co/Kirim-ai/Kirim-1-Math/blob/main/MODEL_CARD.md) |
|
|
|
|
|
</div> |
|
|
|
|
|
|
|
|
## Introduction |
|
|
|
|
|
**Kirim-1-Math** is a 16-billion parameter mathematical reasoning model, representing a major leap in the Kirim model series. As the **first Kirim model with tool calling capabilities**, it combines advanced mathematical problem-solving with the ability to use external tools and execute calculations. |
|
|
|
|
|
### Key Features |
|
|
|
|
|
- **Advanced Math Reasoning**: Trained on mathematical proofs, olympiad problems, and research papers |
|
|
- **Tool Calling**: First in Kirim series with function calling capabilities |
|
|
- **Symbolic Solver**: Handles algebraic manipulation, calculus, and symbolic computation |
|
|
- **Bilingual**: Solves problems in both Chinese and English |
|
|
- **Code Execution**: Can write and execute Python code for numerical solutions |
|
|
- **LaTeX Output**: Generates properly formatted mathematical expressions |
|
|
- **30B Parameters**: More powerful reasoning than 7B/13B variants |
|
|
|
|
|
--- |
|
|
|
|
|
## Model Specifications |
|
|
|
|
|
| Parameter | Value | Comparison | |
|
|
|-----------|-------|------------| |
|
|
| Parameters | 16B | 2.3× larger than base | |
|
|
| Hidden Size | 5,120 | Enhanced capacity | |
|
|
| Layers | 48 | Deep reasoning | |
|
|
| Attention Heads | 40 | Fine-grained attention | |
|
|
| KV Heads | 8 (GQA) | Memory efficient | |
|
|
| Context Length | 32,768 tokens | Extended problems | |
|
|
| Vocabulary | 102,400 | Same as base | |
|
|
| Tool Calling | ✅ Yes | **New feature!** | |
|
|
| Precision | BFloat16 | High quality | |
|
|
|
|
|
### Architecture Highlights |
|
|
|
|
|
- **Deeper Network**: 48 layers for complex multi-step reasoning |
|
|
- **Wider Hidden States**: 5,120 dimensions for richer representations |
|
|
- **Grouped Query Attention**: 5:1 ratio (40:8) for efficiency |
|
|
- **Extended Training**: Specialized on mathematical datasets |
|
|
|
|
|
--- |
|
|
|
|
|
## Quick Start |
|
|
|
|
|
### Installation |
|
|
|
|
|
```bash |
|
|
pip install transformers torch accelerate sympy |
|
|
``` |
|
|
|
|
|
### Basic Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
# Load model |
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
|
"Kirim-ai/Kirim-1-Math", |
|
|
torch_dtype="auto", |
|
|
device_map="auto", |
|
|
trust_remote_code=True |
|
|
) |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained( |
|
|
"Kirim-ai/Kirim-1-Math", |
|
|
trust_remote_code=True |
|
|
) |
|
|
|
|
|
# Solve a math problem |
|
|
messages = [ |
|
|
{"role": "user", "content": "Solve the quadratic equation: x² - 5x + 6 = 0"} |
|
|
] |
|
|
|
|
|
inputs = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
return_tensors="pt", |
|
|
add_generation_prompt=True |
|
|
).to(model.device) |
|
|
|
|
|
outputs = model.generate( |
|
|
inputs, |
|
|
max_new_tokens=2048, |
|
|
temperature=0.1, # Lower temperature for math |
|
|
top_p=0.95, |
|
|
do_sample=False # Deterministic for accuracy |
|
|
) |
|
|
|
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
print(response) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Tool Calling |
|
|
|
|
|
Kirim-1-Math is the **first Kirim model** with built-in tool calling capabilities. |
|
|
|
|
|
### Available Tools |
|
|
|
|
|
The model can use these built-in mathematical tools: |
|
|
|
|
|
1. **Calculator**: Precise arithmetic operations |
|
|
2. **Symbolic Solver**: Algebraic manipulations |
|
|
3. **Code Executor**: Run Python/SymPy code |
|
|
4. **Plot Generator**: Create mathematical visualizations |
|
|
5. **Theorem Lookup**: Access mathematical theorems and formulas |
|
|
|
|
|
### Tool Calling Example |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Calculate 2^1024 and tell me how many digits it has" |
|
|
} |
|
|
] |
|
|
|
|
|
# Model will automatically decide to use calculator tool |
|
|
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt") |
|
|
outputs = model.generate(inputs, max_new_tokens=2048) |
|
|
|
|
|
# Response will include tool calls like: |
|
|
# <tool_call> |
|
|
# { |
|
|
# "name": "calculator", |
|
|
# "arguments": { |
|
|
# "expression": "2**1024" |
|
|
# } |
|
|
# } |
|
|
# </tool_call> |
|
|
``` |
|
|
|
|
|
### Custom Tool Definition |
|
|
|
|
|
```python |
|
|
tools = [ |
|
|
{ |
|
|
"type": "function", |
|
|
"function": { |
|
|
"name": "scientific_calculator", |
|
|
"description": "Perform advanced scientific calculations", |
|
|
"parameters": { |
|
|
"type": "object", |
|
|
"properties": { |
|
|
"expression": { |
|
|
"type": "string", |
|
|
"description": "Mathematical expression to evaluate" |
|
|
}, |
|
|
"precision": { |
|
|
"type": "integer", |
|
|
"description": "Decimal precision", |
|
|
"default": 10 |
|
|
} |
|
|
}, |
|
|
"required": ["expression"] |
|
|
} |
|
|
} |
|
|
} |
|
|
] |
|
|
|
|
|
# Include tools in prompt |
|
|
messages = [ |
|
|
{"role": "system", "content": f"You have access to these tools: {tools}"}, |
|
|
{"role": "user", "content": "Calculate sin(π/4) with 15 decimal places"} |
|
|
] |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Mathematical Capabilities |
|
|
|
|
|
### 1. Algebraic Reasoning |
|
|
|
|
|
```python |
|
|
# Example: Solve system of equations |
|
|
problem = """ |
|
|
解方程组: |
|
|
2x + 3y = 12 |
|
|
4x - y = 5 |
|
|
""" |
|
|
|
|
|
response = model.generate_solution(problem) |
|
|
# Output includes step-by-step solution with reasoning |
|
|
``` |
|
|
|
|
|
### 2. Calculus |
|
|
|
|
|
```python |
|
|
# Integration |
|
|
problem = "Calculate: ∫(x³ + 2x² - x + 1)dx" |
|
|
|
|
|
# Differentiation |
|
|
problem = "Find dy/dx if y = ln(x²) + e^(3x)" |
|
|
``` |
|
|
|
|
|
### 3. Probability & Statistics |
|
|
|
|
|
```python |
|
|
problem = """ |
|
|
A bag contains 5 red balls and 3 blue balls. |
|
|
What's the probability of drawing 2 red balls without replacement? |
|
|
""" |
|
|
``` |
|
|
|
|
|
### 4. Number Theory |
|
|
|
|
|
```python |
|
|
problem = "Prove that √2 is irrational" |
|
|
# Model provides formal mathematical proof |
|
|
``` |
|
|
|
|
|
### 5. Geometry |
|
|
|
|
|
```python |
|
|
problem = """ |
|
|
In triangle ABC, if AB = 5, BC = 7, and AC = 8, |
|
|
find the area using Heron's formula. |
|
|
""" |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
## Use Cases |
|
|
|
|
|
### 1. Educational Tutoring |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "I don't understand how to complete the square. Can you explain and show an example?" |
|
|
} |
|
|
] |
|
|
# Provides step-by-step explanations |
|
|
``` |
|
|
|
|
|
### 2. Research Assistance |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Help me verify this proof about convergence of infinite series" |
|
|
} |
|
|
] |
|
|
# Analyzes mathematical proofs |
|
|
``` |
|
|
|
|
|
### 3. Homework Help |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Solve these 10 calculus problems and show your work" |
|
|
} |
|
|
] |
|
|
# Solves problems with detailed steps |
|
|
``` |
|
|
|
|
|
### 4. Competition Preparation |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Give me 5 AMC-level problems to practice" |
|
|
} |
|
|
] |
|
|
# Generates practice problems |
|
|
``` |
|
|
|
|
|
### 5. Code-Assisted Solving |
|
|
|
|
|
```python |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Use numerical methods to find roots of x^5 - 3x^3 + 2x - 1 = 0" |
|
|
} |
|
|
] |
|
|
# Writes and executes numerical solver |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Advanced Features |
|
|
|
|
|
### Step-by-Step Reasoning |
|
|
|
|
|
The model shows its work: |
|
|
|
|
|
``` |
|
|
Problem: Solve x² - 5x + 6 = 0 |
|
|
|
|
|
Solution: |
|
|
Step 1: Identify this as a quadratic equation in standard form ax² + bx + c = 0 |
|
|
where a=1, b=-5, c=6 |
|
|
|
|
|
Step 2: Try factoring: We need two numbers that multiply to 6 and add to -5 |
|
|
Those numbers are -2 and -3 |
|
|
|
|
|
Step 3: Factor: (x - 2)(x - 3) = 0 |
|
|
|
|
|
Step 4: Apply zero product property: |
|
|
x - 2 = 0 or x - 3 = 0 |
|
|
|
|
|
Step 5: Solve each equation: |
|
|
x = 2 or x = 3 |
|
|
|
|
|
Answer: x = 2 or x = 3 |
|
|
``` |
|
|
|
|
|
### LaTeX Output |
|
|
|
|
|
```python |
|
|
# Request LaTeX formatted output |
|
|
messages = [ |
|
|
{ |
|
|
"role": "user", |
|
|
"content": "Solve this and format the answer in LaTeX: ∫(x² + 1)/(x³ + 3x + 1)dx" |
|
|
} |
|
|
] |
|
|
|
|
|
# Output includes: |
|
|
# $$\int \frac{x^2 + 1}{x^3 + 3x + 1}dx = ...$$ |
|
|
``` |
|
|
|
|
|
### Symbolic Manipulation |
|
|
|
|
|
Uses SymPy internally for symbolic computation: |
|
|
|
|
|
```python |
|
|
from sympy import symbols, expand, factor, simplify |
|
|
|
|
|
# Model can perform: |
|
|
# - Expansion: (x+1)³ → x³ + 3x² + 3x + 1 |
|
|
# - Factoring: x² - 4 → (x-2)(x+2) |
|
|
# - Simplification: (x²-1)/(x-1) → x+1 |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Deployment |
|
|
|
|
|
### System Requirements |
|
|
|
|
|
**Minimum (4-bit Quantization):** |
|
|
- GPU: 20GB VRAM (RTX 4090, A5000) |
|
|
- RAM: 32GB |
|
|
- Storage: 30GB |
|
|
|
|
|
**Recommended (BF16):** |
|
|
- GPU: 48GB VRAM (A40, A6000) |
|
|
- RAM: 64GB |
|
|
- Storage: 70GB |
|
|
|
|
|
**Optimal (Production):** |
|
|
- GPU: 80GB VRAM (A100, H100) |
|
|
- RAM: 128GB |
|
|
- Storage: 100GB SSD |
|
|
|
|
|
### Quantization Options |
|
|
|
|
|
```python |
|
|
# 8-bit (30GB VRAM) |
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
|
"Kirim-ai/Kirim-1-Math", |
|
|
load_in_8bit=True, |
|
|
device_map="auto" |
|
|
) |
|
|
|
|
|
# 4-bit (20GB VRAM) |
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
|
"Kirim-ai/Kirim-1-Math", |
|
|
load_in_4bit=True, |
|
|
device_map="auto" |
|
|
) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
- **Mathematics Corpus**: 500B tokens |
|
|
- Mathematical proofs and papers |
|
|
- Olympiad problems (IMO, USAMO, AMC) |
|
|
- Textbooks (algebra through advanced calculus) |
|
|
- Math Stack Exchange |
|
|
- arXiv math papers |
|
|
|
|
|
- **Code**: 200B tokens |
|
|
- Mathematical Python libraries (NumPy, SymPy, SciPy) |
|
|
- Computational notebooks |
|
|
- Algorithm implementations |
|
|
|
|
|
- **General**: 800B tokens |
|
|
- From Kirim-V1-base pre-training |
|
|
|
|
|
**Total**: 1.5 Trillion tokens |
|
|
|
|
|
### Training Process |
|
|
|
|
|
**Stage 1: Continued Pre-training** (from Kirim-V1-base) |
|
|
- Started from 13B base checkpoint |
|
|
- Expanded to 30B parameters |
|
|
- Trained on math-heavy corpus |
|
|
- Duration: 45 days on 512x H100 GPUs |
|
|
|
|
|
**Stage 2: Mathematical Instruction Tuning** |
|
|
- 200K high-quality math problems with solutions |
|
|
- Step-by-step reasoning examples |
|
|
- Duration: 5 days |
|
|
|
|
|
**Stage 3: Tool Calling Training** |
|
|
- 50K tool-calling examples |
|
|
- Function definition and execution |
|
|
- Error handling and recovery |
|
|
- Duration: 3 days |
|
|
|
|
|
**Stage 4: Reinforcement Learning** |
|
|
- Reward model based on solution correctness |
|
|
- Self-verification training |
|
|
- Duration: 7 days |
|
|
|
|
|
--- |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- **Computation Limits**: Cannot perform extremely large calculations without tools |
|
|
- **Proof Verification**: May occasionally make logical errors in complex proofs |
|
|
- **Theorem Knowledge**: Limited to theorems in training data (pre-Oct 2024) |
|
|
- **Visual Math**: Cannot process images of equations or diagrams |
|
|
- **Real-time Data**: Cannot access current mathematical research or live data |
|
|
|
|
|
--- |
|
|
|
|
|
## Model Series Comparison |
|
|
|
|
|
| Model | Parameters | Purpose | Tool Calling | Best For | |
|
|
|-------|------------|---------|--------------|----------| |
|
|
| Kirim-V1-base | 13B | Foundation | ❌ | Research, fine-tuning | |
|
|
| Kirim-V1-7B-Chat | 7B | Conversation | ❌ | Production chatbots | |
|
|
| **Kirim-1-Math** | 16B | Mathematics | ✅ | **Math problems, STEM education** | |
|
|
| Kirim-V2 (coming) | 30B+ | Multimodal | ✅ | Visual reasoning | |
|
|
|
|
|
--- |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@misc{kirim2025math, |
|
|
title={Kirim-1-Math: Advanced Mathematical Reasoning with Tool Calling}, |
|
|
author={Kirim AI Research Team}, |
|
|
year={2025}, |
|
|
publisher={Kirim AI}, |
|
|
url={https://huggingface.co/Kirim-ai/Kirim-1-Math} |
|
|
} |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Contributing |
|
|
|
|
|
We welcome contributions! |
|
|
|
|
|
--- |
|
|
|
|
|
## License |
|
|
|
|
|
Apache License 2.0 - See [LICENSE](LICENSE) for details. |
