Model card: simple rule scores next to v8.0 and v1.0
Browse files
README.md
CHANGED
|
@@ -256,7 +256,7 @@ Four automated checks run before training: removal of curves copied from a gener
|
|
| 256 |
|
| 257 |
The legacy v8.0 model was a seven-model stacking ensemble trained on 179 rows from 15 games, with one column per game and no store requirements. It scored R² 0.984 on a random split, and its shipped lookup table scored R² 0.0932 on unseen games. This model gets no game or GPU identifiers, which is why it holds up when grouped.
|
| 258 |
|
| 259 |
-
Running the v8.0 ensemble itself on the same independent benchmarks (`scripts/ds_pipeline/evaluate_legacy_v8.py`), the median error was 79.8% on TechPowerUp (this model: 18.9%), 75.1% on TechSpot (36.7%) and 39.8% on OpenML (38.8%). v8.0 overestimated new games by about +90% to +110%. On older popular games the two are roughly tied.
|
| 260 |
|
| 261 |
## Architecture
|
| 262 |
|
|
|
|
| 256 |
|
| 257 |
The legacy v8.0 model was a seven-model stacking ensemble trained on 179 rows from 15 games, with one column per game and no store requirements. It scored R² 0.984 on a random split, and its shipped lookup table scored R² 0.0932 on unseen games. This model gets no game or GPU identifiers, which is why it holds up when grouped.
|
| 258 |
|
| 259 |
+
Running the v8.0 ensemble itself on the same independent benchmarks (`scripts/ds_pipeline/evaluate_legacy_v8.py`), the median error was 79.8% on TechPowerUp (this model: 18.9%), 75.1% on TechSpot (36.7%) and 39.8% on OpenML (38.8%). v8.0 overestimated new games by about +90% to +110%. On older popular games the two are roughly tied. The simple spec rule scored 19.8%, 37.5% and 36.7% on the same three tests, so on OpenML it beat both models.
|
| 260 |
|
| 261 |
## Architecture
|
| 262 |
|