Upload BENCHMARK.md with huggingface_hub
Browse files- BENCHMARK.md +41 -0
BENCHMARK.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Fox1.3 Benchmark Results
|
| 2 |
+
|
| 3 |
+
## Performance Metrics
|
| 4 |
+
|
| 5 |
+
### Speed Benchmark
|
| 6 |
+
| Test | Tokens | Time | Speed |
|
| 7 |
+
|------|--------|------|-------|
|
| 8 |
+
| Capital of France | 105 | 1.97s | 53.3 tok/s |
|
| 9 |
+
| Photosynthesis | 107 | 1.70s | 62.8 tok/s |
|
| 10 |
+
| Math problem (3x+15=45) | 114 | 1.68s | 67.7 tok/s |
|
| 11 |
+
| Python code (prime check) | 109 | 1.70s | 64.3 tok/s |
|
| 12 |
+
| AI vs Machine Learning | 110 | 1.76s | 62.4 tok/s |
|
| 13 |
+
|
| 14 |
+
**Average Speed: 61.8 tokens/sec**
|
| 15 |
+
|
| 16 |
+
## Comparison with Claude Opus 4.5
|
| 17 |
+
|
| 18 |
+
| Metric | Fox1.3 | Claude Opus 4.5 |
|
| 19 |
+
|--------|---------|------------------|
|
| 20 |
+
| **Parameters** | ~850M | ~175B |
|
| 21 |
+
| **Speed** | **61.8 tok/s** | ~20 tok/s |
|
| 22 |
+
| **Size** | 1.7 GB | ~350 GB |
|
| 23 |
+
| **Time to first token** | ~50ms | ~1840ms |
|
| 24 |
+
| **Context window** | 32K | 200K |
|
| 25 |
+
| **Running cost** | Free (local) | $5-$25/1M tokens |
|
| 26 |
+
|
| 27 |
+
**Note:** Fox1.3 is ~3x faster than Opus 4.5, but Opus is more intelligent due to 200x more parameters.
|
| 28 |
+
|
| 29 |
+
## Hardware
|
| 30 |
+
- GPU: NVIDIA RTX 3060
|
| 31 |
+
- Memory: 12GB VRAM
|
| 32 |
+
|
| 33 |
+
## Model Details
|
| 34 |
+
- Base: Qwen2.5-1B-Instruct
|
| 35 |
+
- Training: LoRA fine-tuning on 150+ examples
|
| 36 |
+
- Trained: 2026-04-01
|
| 37 |
+
- Dataset includes: general knowledge, science, math, reasoning, coding
|
| 38 |
+
|
| 39 |
+
## Links
|
| 40 |
+
- Model: https://huggingface.co/teolm30/fox1.3
|
| 41 |
+
- Training data: Available in repository
|