|
|
--- |
|
|
language: "en" |
|
|
license: mit |
|
|
tags: |
|
|
- reinforcement-learning |
|
|
- q-learning |
|
|
- gridworld |
|
|
- mountain-car |
|
|
- openai-gym |
|
|
datasets: |
|
|
- gym |
|
|
--- |
|
|
|
|
|
# Q-Learning in Reinforcement Learning Environments |
|
|
|
|
|
## Project Overview |
|
|
This project implements the Q-Learning algorithm in two different reinforcement learning environments: a custom-built Gridworld and the MountainCar-v0 environment from the OpenAI Gym library. |
|
|
|
|
|
### Part 1: Custom Gridworld Environment |
|
|
A simple 5x5 Gridworld where the goal is to navigate from the start state to the goal state, avoiding obstacles and minimizing the number of steps: |
|
|
- **Grid Layout**: |
|
|
- Start at (0, 0) |
|
|
- Goal at (4, 4) |
|
|
- Obstacles at (2, 2) and (3, 3) |
|
|
- **Rewards**: |
|
|
- Goal: +100 |
|
|
- Obstacle: -10 |
|
|
- Each step: -1 |
|
|
|
|
|
### Part 2: MountainCar-v0 Environment |
|
|
Utilizing the standard MountainCar-v0 environment from the gym library, where the agent must drive a car up a steep hill: |
|
|
- **Environment Features**: |
|
|
- The agent learns to balance momentum and gravity to reach the peak. |
|
|
- Visualize the environment using `env.render()`. |
|
|
|
|
|
## Video Preview |
|
|
Watch the demonstration video on [YouTube](https://www.youtube.com/watch?v=eLnngYsdluo). |
|
|
|
|
|
|
|
|
|
|
|
## Implementation Details |
|
|
- **Q-Learning**: |
|
|
- Initialize and update the Q-table using the Bellman equation. |
|
|
- An epsilon-greedy policy is used for action selection. |
|
|
- **Training and Testing**: |
|
|
- Train the agent over multiple episodes. |
|
|
- Visualize and analyze the learned policy to evaluate performance. |
|
|
|
|
|
## How to Use |
|
|
1. **Requirements**: Ensure you have Python and the necessary libraries installed, including `gym` for the MountainCar environment. |
|
|
2. **Setup**: |
|
|
- Clone the repository: `git clone https://huggingface.co/Karim2211/ReinforcementLearningModels/tree/main` |
|
|
- Install dependencies: `pip install -r requirements.txt` |
|
|
3. **Running the Code**: |
|
|
- Navigate to the project directory. |
|
|
- For Gridworld: `python gridworld.py` |
|
|
- For MountainCar: `python mountain_car.py` |
|
|
4. **Loading Models**: |
|
|
- Models are saved in the `models/` directory. |
|
|
- To load and test models: |
|
|
```python |
|
|
import pickle |
|
|
# For Gridworld model |
|
|
with open('models/gridworld_model.pkl', 'rb') as f: |
|
|
gridworld_model = pickle.load(f) |
|
|
# For MountainCar model |
|
|
with open('models/mountain_car_model.pkl', 'rb') as f: |
|
|
mountain_car_model = pickle.load(f) |
|
|
``` |
|
|
5. **Testing the Models**: |
|
|
- Use the loaded models to make predictions or to evaluate the policies. |
|
|
- Example testing code: |
|
|
```python |
|
|
# Test Gridworld model |
|
|
gridworld.test_policy(gridworld_model) |
|
|
# Test MountainCar model |
|
|
mountain_car.test_policy(mountain_car_model) |
|
|
``` |
|
|
|
|
|
## Dependencies |
|
|
- Python 3.x |
|
|
- gym |
|
|
- numpy |
|
|
- matplotlib (optional for visualization) |
|
|
|
|
|
|
|
|
## Authors |
|
|
- Karim Ashraf |
|
|
- Shiref Elhalawany |
|
|
- Mahitab Waleed |
|
|
|
|
|
## License |
|
|
This project is licensed under the MIT License. |
