---
library_name: stable-baselines3
tags:
- SpaceInvadersNoFrameskip-v4
- deep-reinforcement-learning
- reinforcement-learning
- stable-baselines3
model-index:
- name: DQN
  results:
  - task:
      type: reinforcement-learning
      name: reinforcement-learning
    dataset:
      name: SpaceInvadersNoFrameskip-v4
      type: SpaceInvadersNoFrameskip-v4
    metrics:
    - type: mean_reward
      value: 29.00 +/- 64.30
      name: mean_reward
      verified: false
---

# **DQN** Agent playing **SpaceInvadersNoFrameskip-v4** 🚀
Imagine you have a robot friend who loves playing a game called Space Invaders. We've taught this robot how to play the game really well using a special technique called Deep Q-Network (DQN). The robot can now play Space Invaders without missing a single frame!

We used a special library called Stable Baselines3 to help teach the robot how to play. It's like a school for robots to learn cool things. And to make things even easier, we have a special place called the RL Zoo where the robot can practice and get even better at playing the game.

So, what you're seeing is a video of our robot friend playing Space Invaders like a pro, thanks to the training it received using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3) and the [RL Zoo](https://github.com/DLR-RM/rl-baselines3-zoo). It's like watching a robot master a game after lots of practice and learning in a special training ground!

## Usage (with SB3 RL Zoo) 📚

RL Zoo: https://github.com/DLR-RM/rl-baselines3-zoo<br/>
SB3: https://github.com/DLR-RM/stable-baselines3<br/>
SB3 Contrib: https://github.com/Stable-Baselines-Team/stable-baselines3-contrib
SBX (SB3 + Jax): https://github.com/araffin/sbx

Install the RL Zoo (with SB3 and SB3-Contrib):
```bash
pip install rl_zoo3
```

```
# Download model and save it into the logs/ folder
python -m rl_zoo3.load_from_hub --algo dqn --env SpaceInvadersNoFrameskip-v4 -orga LimTara -f logs/
python -m rl_zoo3.enjoy --algo dqn --env SpaceInvadersNoFrameskip-v4  -f logs/
```

If you installed the RL Zoo3 via pip (`pip install rl_zoo3`), from anywhere you can do:
```
python -m rl_zoo3.load_from_hub --algo dqn --env SpaceInvadersNoFrameskip-v4 -orga LimTara -f logs/
python -m rl_zoo3.enjoy --algo dqn --env SpaceInvadersNoFrameskip-v4  -f logs/
```

## Training (with the RL Zoo) 👾
The code segment provided is a set of commands used to train a DQN agent on the SpaceInvaders game environment, store the training logs, and then upload the trained model to a specific organization repository, potentially generating a video in the process.
```
python -m rl_zoo3.train --algo dqn --env SpaceInvadersNoFrameskip-v4 -f logs/
# Upload the model and generate video (when possible)
python -m rl_zoo3.push_to_hub --algo dqn --env SpaceInvadersNoFrameskip-v4 -f logs/ -orga LimTara
```

## Hyperparameters 🔗
Hyperparameters are used in order to control the behavior of the training model. 
In this project, some of the hyperparameters include the number of times the model is simulated (n_timesteps). The higher the number, the more times the model is simulated and the higher the accuracy but the longer it takes to run. ⏰
You can also control how risky it is willing to go in order to explore new things. The higher the risk, the more the model will explore and step out of their comfort zone (exploration_final_eps)! 💥

To create and edit the hyperparameters, you need to create a file called "dqn.yml" by going to the left bar and click on the file icon which is right below the key icon. Then all you need to do is simply paste the following into the "dqn.yml" file! 📁 
```python
OrderedDict([('batch_size', 32),
             ('buffer_size', 100000),
             ('env_wrapper',
              ['stable_baselines3.common.atari_wrappers.AtariWrapper']),
             ('exploration_final_eps', 0.01),
             ('exploration_fraction', 0.1),
             ('frame_stack', 4),
             ('gradient_steps', 1),
             ('learning_rate', 0.0001),
             ('learning_starts', 100000),
             ('n_timesteps', 100000),
             ('optimize_memory_usage', False),
             ('policy', 'CnnPolicy'),
             ('target_update_interval', 1000),
             ('train_freq', 4),
             ('normalize', False)])
```

# Environment Arguments
```python
{'render_mode': 'rgb_array'}
```

# Display 👀
Finally, we now have to display our model so the audience can see what we've done! To do that, first you must install the following libraries and run a virtual screen:
```%%capture
!apt install python-opengl
!apt install xvfb
!pip3 install pyvirtualdisplay
```

```
from pyvirtualdisplay import Display

virtual_display = Display(visible=0, size=(1400, 900))
virtual_display.start()
```

Once you're in your model repository page, go over to "Files and Versions" and create a new folder called "replay.mp4" and simply upload your video and BOOM the virtual display will be right there in your model card! 🖥️

# Saving your Changes 💾
Oftentimes we are in the middle of coding a model, but suddenly we close the tab and everything is gone! Certainly, that is something that nobody wants to experience right?
So in order to prevent that from happening, go over to the "file" on the top left and click "Save a copy in Drive". Once you've done that, the changes you make will be saved even when you close your tab. Now you can freely edit your code without worrying that your precious hard work will disappear! Phew!

# Next Steps 🎉
Congratulations! You've now completed the beginner's guide to creating a Deep Q-Learning model! Now it's your turn, try and create your own model and remember that errors are bound to happen so be patient and have fun exploring! 🤩