GAMBIT: Generalization or Memorization? Brittleness Testing for Chess-Trained Language Models
arXiv link COMING SOON
Variants
KINGPT-Woodpecker
KINGPT variant trained on 13,341,057 unique puzzle positions (FEN + best move pairs).
Achieved train loss 0.3590, val loss 0.3704 on puzzles corpus after training for ~500B tokens.
KINGPT-Beaver
KINGPT variant trained on 54,681 unique positions generated from 1050 Stockfish 18 self-play games.
Achieved train loss 0.0974, val loss 1.7554 (overfitting due to small dataset size) on selfplay corpus after training for ~25B tokens.
KINGPT-Chimera
KINGPT variant trained on combined dataset of 13,395,738 Woodpecker and Beaver variant positions.
Achieved train loss 0.3594, val loss 0.3710 on combined corpus after training for ~500B tokens.
Citation
Citation COMING SOON!