Upload World Models for Space Invaders

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+sample_gameplay.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,78 @@
+---
+tags:
+- reinforcement-learning
+- world-models
+- atari
+- space-invaders
+- deep-learning
+library_name: pytorch
+---
+
+# World Models for Space Invaders
+
+This is a World Models agent trained on the `SpaceInvadersNoFrameskip-v4` environment.
+
+## Model Description
+
+World Models is a model-based reinforcement learning approach that learns a compressed representation 
+of the environment and trains a controller to maximize reward in the learned model.
+
+The architecture consists of three components:
+- **V (Vision)**: Variational Autoencoder that compresses 64x64 RGB frames to 32-dimensional latent vectors
+- **M (Memory)**: MDN-RNN that predicts the next latent state given current state and action
+- **C (Controller)**: Linear policy trained with CMA-ES evolution strategy
+
+## Training Details
+
+### Hyperparameters
+- VAE Latent Dimension: 32
+- RNN Hidden Dimension: 256
+- Number of Gaussian Mixtures: 5
+- Population Size (CMA-ES): 64
+- Training Episodes: 100
+- VAE Epochs: 10
+- RNN Epochs: 20
+- Controller Generations: 10
+
+## Evaluation Results
+
+- **Mean Reward**: 155.00 ± 0.00
+- **Max Reward**: 155.00
+- **Mean Episode Length**: 1000.00
+
+## Usage
+
+```python
+import torch
+import gymnasium as gym
+
+# Load models
+vae = VAE(latent_dim=32)
+vae.load_state_dict(torch.load('vae_model.pt'))
+
+rnn = MDNRNN(latent_dim=32, action_dim=6)
+rnn.load_state_dict(torch.load('mdnrnn_model.pt'))
+
+controller = Controller(latent_dim=32, hidden_dim=256)
+controller.load_state_dict(torch.load('controller_model.pt'))
+
+# Run agent
+env = gym.make('SpaceInvadersNoFrameskip-v4')
+# ... (see repository for full inference code)
+```
+
+## References
+
+- Paper: [World Models (Ha & Schmidhuber, 2018)](https://worldmodels.github.io/)
+- Code: Based on the original World Models implementation
+
+## Citation
+
+```bibtex
+@article{ha2018worldmodels,
+  title={World Models},
+  author={Ha, David and Schmidhuber, J{\"u}rgen},
+  journal={arXiv preprint arXiv:1803.10122},
+  year={2018}
+}
+```

config.json

@@ -0,0 +1,22 @@
+{
+  "env_name": "SpaceInvadersNoFrameskip-v4",
+  "max_episode_steps": 1000,
+  "latent_dim": 32,
+  "hidden_dim": 256,
+  "num_mixtures": 5,
+  "action_dim": 6,
+  "vae_epochs": 10,
+  "vae_batch_size": 32,
+  "vae_lr": 0.0001,
+  "num_random_episodes": 100,
+  "rnn_epochs": 20,
+  "rnn_batch_size": 16,
+  "rnn_lr": 0.0001,
+  "sequence_length": 32,
+  "population_size": 64,
+  "num_generations": 10,
+  "sigma": 0.5,
+  "eval_episodes": 10,
+  "record_video_every": 10,
+  "device": "cuda"
+}

controller_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac7c86f564aef60093bf7fcd22b86e3b5b9f07e59a0b43c5822566203ad3e44c
+size 8941

eval_stats.json

@@ -0,0 +1,7 @@
+{
+  "mean_reward": 155.0,
+  "std_reward": 0.0,
+  "mean_length": 1000.0,
+  "min_reward": 155.0,
+  "max_reward": 155.0
+}

mdnrnn_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0962ecad39b967e2b3d34d6954f51687a5ca30d7ad97edbe2032daaecd0259a1
+size 1549070

sample_gameplay.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05b84dc8b3b4a842081ac4d3277ca78e7b86e45a14e5ab3814ad8a693c8548ca
+size 109782

vae_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9bf44de7a3baa9fab029a96395ddc1ae28b7931bef2682e11cc3b12eb23cd9a
+size 7117721