|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: unsloth/qwen3-0.6b-unsloth-bnb-4bit |
|
|
tags: |
|
|
- math |
|
|
- reasoning |
|
|
- qwen3 |
|
|
- unsloth |
|
|
- fine-tuned |
|
|
datasets: |
|
|
- openai/gsm8k |
|
|
- lighteval/MATH |
|
|
model-index: |
|
|
- name: Shadow-V2 |
|
|
results: |
|
|
- task: |
|
|
type: math-reasoning |
|
|
dataset: |
|
|
name: GSM8K |
|
|
type: openai/gsm8k |
|
|
metrics: |
|
|
- name: accuracy |
|
|
type: accuracy |
|
|
value: TBD |
|
|
--- |
|
|
|
|
|
# Shadow-V2 |
|
|
|
|
|
Fine-tuned Qwen3-0.6B for mathematical reasoning. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
| Property | Value | |
|
|
|----------|-------| |
|
|
| Base Model | Qwen3-0.6B | |
|
|
| Parameters | 636M total, 40M trainable (6.34%) | |
|
|
| Precision | BF16 | |
|
|
| Training Method | LoRA via Unsloth | |
|
|
| Context Length | 2048 | |
|
|
|
|
|
## Training |
|
|
|
|
|
| Config | Value | |
|
|
|--------|-------| |
|
|
| Dataset | 25,000 examples | |
|
|
| Epochs | 1 | |
|
|
| Batch Size | 16 (2 × 8 accum) | |
|
|
| Steps | 1,200 | |
|
|
| Hardware | Tesla T4 16GB | |
|
|
| Time | 1.35 hours | |
|
|
| Final Loss | 0.43 | |
|
|
|
|
|
## Benchmarks |
|
|
|
|
|
| Benchmark | Shadow-V2 | Qwen3-0.6B (base) | |
|
|
|-----------|-----------|-------------------| |
|
|
| GSM8K (5-shot) | TBD | 42.3 | |
|
|
| MATH (4-shot) | TBD | 18.2 | |
|
|
| HumanEval (0-shot) | TBD | 28.0 | |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("Redhanuman/Shadow-V2") |
|
|
tokenizer = AutoTokenizer.from_pretrained("Redhanuman/Shadow-V2") |
|
|
|
|
|
prompt = "Solve: If 3x + 7 = 22, find x.\nAnswer:" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
