File size: 1,535 Bytes
478b054 923c4b7 478b054 923c4b7 478b054 923c4b7 478b054 923c4b7 478b054 923c4b7 478b054 923c4b7 478b054 923c4b7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
---
license: apache-2.0
base_model: unsloth/qwen3-0.6b-unsloth-bnb-4bit
tags:
- math
- reasoning
- qwen3
- unsloth
- fine-tuned
datasets:
- openai/gsm8k
- lighteval/MATH
model-index:
- name: Shadow-V2
results:
- task:
type: math-reasoning
dataset:
name: GSM8K
type: openai/gsm8k
metrics:
- name: accuracy
type: accuracy
value: TBD
---
# Shadow-V2
Fine-tuned Qwen3-0.6B for mathematical reasoning.
## Model Details
| Property | Value |
|----------|-------|
| Base Model | Qwen3-0.6B |
| Parameters | 636M total, 40M trainable (6.34%) |
| Precision | BF16 |
| Training Method | LoRA via Unsloth |
| Context Length | 2048 |
## Training
| Config | Value |
|--------|-------|
| Dataset | 25,000 examples |
| Epochs | 1 |
| Batch Size | 16 (2 × 8 accum) |
| Steps | 1,200 |
| Hardware | Tesla T4 16GB |
| Time | 1.35 hours |
| Final Loss | 0.43 |
## Benchmarks
| Benchmark | Shadow-V2 | Qwen3-0.6B (base) |
|-----------|-----------|-------------------|
| GSM8K (5-shot) | TBD | 42.3 |
| MATH (4-shot) | TBD | 18.2 |
| HumanEval (0-shot) | TBD | 28.0 |
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Redhanuman/Shadow-V2")
tokenizer = AutoTokenizer.from_pretrained("Redhanuman/Shadow-V2")
prompt = "Solve: If 3x + 7 = 22, find x.\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |