Upload World Models for Space Invaders

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+sample_gameplay.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,78 @@
+---
+tags:
+- reinforcement-learning
+- world-models
+- atari
+- space-invaders
+- deep-learning
+library_name: pytorch
+---
+
+# World Models for Space Invaders
+
+This is a World Models agent trained on the `SpaceInvadersNoFrameskip-v4` environment.
+
+## Model Description
+
+World Models is a model-based reinforcement learning approach that learns a compressed representation 
+of the environment and trains a controller to maximize reward in the learned model.
+
+The architecture consists of three components:
+- **V (Vision)**: Variational Autoencoder that compresses 64x64 RGB frames to 32-dimensional latent vectors
+- **M (Memory)**: MDN-RNN that predicts the next latent state given current state and action
+- **C (Controller)**: Linear policy trained with CMA-ES evolution strategy
+
+## Training Details
+
+### Hyperparameters
+- VAE Latent Dimension: 32
+- RNN Hidden Dimension: 256
+- Number of Gaussian Mixtures: 5
+- Population Size (CMA-ES): 64
+- Training Episodes: 100
+- VAE Epochs: 10
+- RNN Epochs: 20
+- Controller Generations: 10
+
+## Evaluation Results
+
+- **Mean Reward**: 506.50 ± 34.57
+- **Max Reward**: 565.00
+- **Mean Episode Length**: 852.00
+
+## Usage
+
+```python
+import torch
+import gymnasium as gym
+
+# Load models
+vae = VAE(latent_dim=32)
+vae.load_state_dict(torch.load('vae_model.pt'))
+
+rnn = MDNRNN(latent_dim=32, action_dim=6)
+rnn.load_state_dict(torch.load('mdnrnn_model.pt'))
+
+controller = Controller(latent_dim=32, hidden_dim=256)
+controller.load_state_dict(torch.load('controller_model.pt'))
+
+# Run agent
+env = gym.make('SpaceInvadersNoFrameskip-v4')
+# ... (see repository for full inference code)
+```
+
+## References
+
+- Paper: [World Models (Ha & Schmidhuber, 2018)](https://worldmodels.github.io/)
+- Code: Based on the original World Models implementation
+
+## Citation
+
+```bibtex
+@article{ha2018worldmodels,
+  title={World Models},
+  author={Ha, David and Schmidhuber, J{\"u}rgen},
+  journal={arXiv preprint arXiv:1803.10122},
+  year={2018}
+}
+```

config.json

@@ -0,0 +1,22 @@
+{
+  "env_name": "ALE/SpaceInvaders-v5",
+  "max_episode_steps": 1000,
+  "latent_dim": 32,
+  "hidden_dim": 256,
+  "num_mixtures": 5,
+  "action_dim": 6,
+  "vae_epochs": 10,
+  "vae_batch_size": 32,
+  "vae_lr": 0.0001,
+  "num_random_episodes": 100,
+  "rnn_epochs": 20,
+  "rnn_batch_size": 16,
+  "rnn_lr": 0.0001,
+  "sequence_length": 32,
+  "population_size": 64,
+  "num_generations": 10,
+  "sigma": 0.5,
+  "eval_episodes": 10,
+  "record_video_every": 10,
+  "device": "cuda"
+}

controller_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83c1183938a96735212d6b52013a043f55857b2c48001669288ba2d02e3c53a3
+size 8941

eval_stats.json

@@ -0,0 +1,7 @@
+{
+  "mean_reward": 506.5,
+  "std_reward": 34.57238782612506,
+  "mean_length": 852.0,
+  "min_reward": 460.0,
+  "max_reward": 565.0
+}

mdnrnn_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:469d6dde4ed5c223b1af9685430526cc6bc8d6c6954ec41fa1dd70a9426c6d12
+size 1549070

sample_gameplay.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1af84b887edca9c959663e80a493c724429698b1e40e91486a17a9a73ee96f8
+size 228198

vae_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f14b7c69a35616ee0422da3c102d974e86023b7063b5ad4ff79be7a88bc8e7a9
+size 7117721