YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
CastleNav-DDQN
CastleNav is a compact deep-reinforcement-learning benchmark modeled as four castle rooms connected by narrow doors. The agent must find a key and then reach a portal. Starts, key positions, and portal positions vary between episodes.
Method
- dueling Q-network;
- Double-DQN targets;
- replay memory;
- target-network updates;
- epsilon-greedy exploration;
- potential-based distance shaping;
- held-out deterministic evaluation.
uv run python projects/castle-nav-rl/train.py
uv run python projects/castle-nav-rl/evaluate.py
The trained policy is stored as Safetensors alongside a JSON architecture config, training metrics, deterministic evaluation results, and example rollouts.
Ablations and final result
Four policy variants were trained:
- plain Dueling Double-DQN;
- shortest-path reward shaping;
- demonstration pretraining with continual imitation loss;
- geodesic action features plus demonstrations and online RL.
The final 35,909-parameter policy achieved 100% success over 500 held-out episodes with an average path length of 20.886 steps. Earlier variants are preserved as ablations rather than presented as successful models.
MicroPolicy distillation
The final teacher generated 55,286 observations through a mixture of greedy and random exploration. A two-hidden-layer student learned its soft Q distribution and greedy actions.
| Policy | Parameters | Teacher agreement | 500-castle success | Average steps |
|---|---|---|---|---|
| Geodesic DQfD teacher | 35,909 | 100% | 100% | 20.886 |
| Distilled MicroPolicy | 1,764 | 95.04% | 100% | 20.818 |
The student is 20.36 times smaller and retained the teacher's full deterministic success rate on a separate 500-episode seed stream.
uv run python projects/castle-nav-rl/distill_policy.py
- Downloads last month
- 23