Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+result-50M-frames.gif filter=lfs diff=lfs merge=lfs -text
+training-curves.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,118 @@
+---
+license: mit
+tags:
+- reinforcement-learning
+- dqn
+- deep-q-learning
+- atari
+- ale
+- space-invaders
+- pytorch
+library_name: pytorch
+pipeline_tag: reinforcement-learning
+---
+
+# DQN — ALE/SpaceInvaders-v5
+
+<p align="center">
+  <img src="./result-50M-frames.gif" alt="Trained agent playing Space Invaders"/>
+</p>
+
+A from-scratch PyTorch reimplementation of DeepMind's DQN Atari papers ([Mnih et al.,
+2013](https://arxiv.org/pdf/1312.5602.pdf), [Mnih et al.,
+2015](https://www.nature.com/articles/nature14236/)), trained on `ALE/SpaceInvaders-v5`.
+
+- **Code / training pipeline:** https://github.com/colichar/deep-q-learning
+- **Architecture:** DeepMind's original conv/FC Q-network (3 conv layers + 2 FC layers), operating on stacks of 4
+  grayscale 84x84 frames.
+- **Framework:** PyTorch.
+
+## Training setup
+
+Trained for 50M frames (~12.5 hours) on a single RTX 4070 Ti Super, with a single `gymnasium` environment
+(`num_envs=1`). Hyperparameters match the DeepMind 2015 paper, with a centered RMSprop optimizer:
+
+| Hyperparameter | Value |
+| --- | --- |
+| Optimizer | RMSprop (centered, `alpha=0.95`, `eps=0.01`) |
+| Learning rate | 0.00025 |
+| Discount (gamma) | 0.99 |
+| Replay memory size | 1,000,000 frames |
+| Memory warmup | 50,000 frames |
+| Batch size | 32 |
+| Target network sync | every 10,000 frames |
+| Main network update | every 4 frames |
+| Frame skip | 4 (with max-pooling over the last 2 frames for flicker removal) |
+| Reward clipping | {-1, 0, 1} |
+
+See the [repo README](https://github.com/colichar/deep-q-learning#start-training) for the full flag list.
+
+## Results
+
+Over the last 1M frames of training (501 episodes): **average episode reward ≈ 1968.6**, max episode reward 4155.
+
+<p align="center">
+  <img src="./training-curves.png" alt="Episode reward and training loss over 50M frames"/>
+</p>
+
+Reward is the 200-episode rolling mean (shaded band: raw per-episode reward); loss is the Huber loss averaged
+every 400 frames, on a log scale.
+
+## Files
+
+- `model.pth` — `torch.save` dict with `model_state_dict` and `optimizer_state_dict` for the main Q-network.
+- `episodes.csv` — per-episode `frame_num,episode_num,episode_reward,epsilon,wall_clock_elapsed_seconds`, full
+  training run.
+- `losses.csv` — `frame_num,avg_loss`, Huber loss averaged every 400 frames.
+- `training-curves.png` — the plot above, generated from the two CSVs.
+
+The replay memory buffer (~14GB) used to resume training locally is intentionally **not** included here — it's
+only needed to resume training, not to run or evaluate the model.
+
+## Usage
+
+The checkpoint's state dict matches `src.models.cnn_model_py.CNNModelPY` in the linked repo and is loaded through
+`SpaceInvaderAgent`, not standalone `torch.load` + a bare `nn.Module` — see
+[`scripts/evaluate.py`](https://github.com/colichar/deep-q-learning/blob/main/scripts/evaluate.py):
+
+```bash
+git clone https://github.com/colichar/deep-q-learning
+cd deep-q-learning
+uv sync --extra cpu   # or --extra cuda
+uv run AutoROM --accept-license -y
+
+# download this checkpoint's model/ dir from the Hub into, e.g., checkpoint/model/
+uv run python scripts/evaluate.py --checkpoint checkpoint --optimizer rmsprop --episodes 5
+```
+
+## Limitations
+
+- Trained and evaluated on a single game (`ALE/SpaceInvaders-v5`); the network is not general-purpose across
+  Atari games without retraining.
+- Intended for research/education (reproducing the DQN papers), not as a production game-playing agent.
+
+## License
+
+MIT — see [LICENSE](https://github.com/colichar/deep-q-learning/blob/main/LICENSE) in the linked repo.
+
+## Citation
+
+```bibtex
+@article{mnih2013playing,
+  title={Playing atari with deep reinforcement learning},
+  author={Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Graves, Alex and Antonoglou, Ioannis and Wierstra, Daan and Riedmiller, Martin},
+  journal={arXiv preprint arXiv:1312.5602},
+  year={2013}
+}
+
+@article{mnih2015human,
+  title={Human-level control through deep reinforcement learning},
+  author={Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Rusu, Andrei A and Veness, Joel and Bellemare, Marc G and Graves, Alex and Riedmiller, Martin and Fidjeland, Andreas K and Ostrovski, Georg and others},
+  journal={Nature},
+  volume={518},
+  number={7540},
+  pages={529--533},
+  year={2015},
+  publisher={Nature Publishing Group}
+}
+```