grasant commited on
Commit
0dff000
·
verified ·
1 Parent(s): 4532591

Model card: simple rule scores next to v8.0 and v1.0

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -256,7 +256,7 @@ Four automated checks run before training: removal of curves copied from a gener
256
 
257
  The legacy v8.0 model was a seven-model stacking ensemble trained on 179 rows from 15 games, with one column per game and no store requirements. It scored R² 0.984 on a random split, and its shipped lookup table scored R² 0.0932 on unseen games. This model gets no game or GPU identifiers, which is why it holds up when grouped.
258
 
259
- Running the v8.0 ensemble itself on the same independent benchmarks (`scripts/ds_pipeline/evaluate_legacy_v8.py`), the median error was 79.8% on TechPowerUp (this model: 18.9%), 75.1% on TechSpot (36.7%) and 39.8% on OpenML (38.8%). v8.0 overestimated new games by about +90% to +110%. On older popular games the two are roughly tied.
260
 
261
  ## Architecture
262
 
 
256
 
257
  The legacy v8.0 model was a seven-model stacking ensemble trained on 179 rows from 15 games, with one column per game and no store requirements. It scored R² 0.984 on a random split, and its shipped lookup table scored R² 0.0932 on unseen games. This model gets no game or GPU identifiers, which is why it holds up when grouped.
258
 
259
+ Running the v8.0 ensemble itself on the same independent benchmarks (`scripts/ds_pipeline/evaluate_legacy_v8.py`), the median error was 79.8% on TechPowerUp (this model: 18.9%), 75.1% on TechSpot (36.7%) and 39.8% on OpenML (38.8%). v8.0 overestimated new games by about +90% to +110%. On older popular games the two are roughly tied. The simple spec rule scored 19.8%, 37.5% and 36.7% on the same three tests, so on OpenML it beat both models.
260
 
261
  ## Architecture
262