---
license: apache-2.0
tags:
  - reinforcement-learning
  - stable-baselines3
  - ppo
  - game
  - gymnasium
  - pygame
library_name: stable-baselines3
task: reinforcement-learning
---

# PixelCopter RL Agent

![PixelCopter](https://raw.githubusercontent.com/jatinror/ppo-pixelcopter/main/pixelcopter_screenshot.png)

## Overview

PixelCopter is a side-scrolling reinforcement learning environment inspired by the classic helicopter game. The agent:

- Controls a **pixel copter** navigating through a scrolling cavern.
- Avoids colliding with walls that move from right to left.
- Gains positive rewards for surviving timesteps; receives negative rewards on collision.

This repository contains:

- `train_pixelcopter.py` – Training script  
- `test_pixelcopter.py` – Test script with Pygame visualization  
- `ppo_pixelcopter_cert.zip` – Pre-trained PPO agent  

---

## Environment Details

- **Observation Space:** `[copter_y, wall_gap_positions...]`  
- **Action Space:**  
  - `0`: Do nothing (gravity pulls the copter down)  
  - `1`: Apply lift (move upwards)
- **Rewards:**  
  - +1 for surviving a timestep  
  - -5 for collision
- **Medium Difficulty Settings:**  
  - Gap size: 6  
  - Gravity: 0.25  
  - Max timesteps per episode: 500  

---

## Installation

```bash
pip install stable-baselines3 gymnasium pygame numpy