File size: 663 Bytes
3e8fae6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Evaluation

The selected model was evaluated against its published Base source using frozen
packed validation/test data plus completion and FIM slices.

| Metric | Published Base loss | Coder-Base loss | Relative change |
|---|---:|---:|---:|
| Packed validation | 2.572834 | 1.488147 | -42.16% |
| Packed test | 2.604606 | 1.798295 | -30.96% |
| Completion validation | 2.286497 | 1.521212 | -33.47% |
| FIM validation | 2.408383 | 1.394937 | -42.08% |

All four frozen loss metrics improved. These tests measure a narrow local code
corpus and do not establish correctness on HumanEval, MBPP, repository-level
work, security tasks, or long-context generation.