README.md

metadata

language:
  - en
  - zh
tags:
  - game24
  - qwen2.5
  - sft
  - grpo
  - lora
  - reasoning
library_name: transformers
base_model: Qwen/Qwen2.5-1.5B-Instruct

game24-rl

This repository contains two checkpoints from the game24-rl course project:

• sft-final/: full fine-tuned Qwen/Qwen2.5-1.5B-Instruct checkpoint for 24-game reasoning.
• grpo-lora-final/: LoRA adapter trained from the SFT checkpoint with GRPO.

Code and experiment documentation: https://github.com/ElysiaFollower/game24-rl

Training Route

Qwen2.5-1.5B-Instruct -> full SFT final -> GRPO LoRA adapter -> decoding/eval

Notes

The project focuses on standard 24-point game solving. The model is expected to produce reasoning and a final answer expression that can be checked by the repository verifier.

The GRPO artifact is a PEFT LoRA adapter, not a standalone full model. Load the SFT checkpoint first, then apply the adapter.

Reported Repo-Local Results

Under the repo-local train/validation/test split documented in the GitHub repository:

• Base model full-data direct eval: 16/1362 = 1.17%
• SFT final, validation, 1024 token budget: 110/136 = 80.88%
• GRPO LoRA, validation, 1024 token budget: 116/136 = 85.29%
• SFT final, validation, 4096 token budget: 123/136 = 90.44%
• SFT final, test, 4096 token budget: 128/137 = 93.43%
• GRPO LoRA, validation, 4096 token budget: 126/136 = 92.65%
• GRPO LoRA, test, 4096 token budget: 129/137 = 94.16%

See the GitHub handoff and experiment docs for split details, verifier details, decoding settings, and caveats.