---
library_name: ml-agents
tags:
- Huggy
- deep-reinforcement-learning
- reinforcement-learning
- ML-Agents-Huggy
- PPO
- Unity
---

# 🕹 PPO Agent: Playing **Huggy**

This repository contains a trained **PPO** agent for the Huggy environment, built using the [Unity ML-Agents Toolkit](https://github.com/Unity-Technologies/ml-agents).

## 📊 Training Details
- **Trainer type**: PPO  
- **Hyperparameters**:  
  - Batch size: 2048  
  - Buffer size: 20480  
  - Learning rate: 0.0003 (linear schedule)  
  - Beta: 0.005 (linear schedule)  
  - Epsilon: 0.2 (linear schedule)  
  - Lambda: 0.95  
  - Epochs: 3  
- **Network architecture**:  
  - Hidden units: 512  
  - Layers: 3  
  - Normalization: Enabled  
- **Reward signal**: Extrinsic (γ = 0.995, strength = 1.0)  
- **Max steps**: 2,000,000  
- **Checkpoint interval**: 200,000  
- **Exported models**: ONNX checkpoints at multiple intervals (e.g., 199996, 399914, …, 2000042)

During training, the agent’s **mean reward improved steadily** from ~1.8 at 50k steps to ~3.9 at 1.4M+ steps, stabilizing around ~3.7–3.9 with variance ~1.9–2.0.

## 🚀 Usage (with ML-Agents)
Documentation: [ML-Agents Toolkit Docs](https://unity-technologies.github.io/ml-agents/ML-Agents-Toolkit-Documentation/)

### Resume Training
```bash
mlagents-learn <your_config.yaml> --run-id=<run_id> --resume
```

### Watch Your Agent Play
You can watch the agent directly in your browser:
1. Go to [Unity models on Hugging Face](https://huggingface.co/unity).
2. Find this model: **KraTUZen/HuggyTheStickFetcher**.
3. Select the exported `.onnx` file.
4. Click **Watch the agent play 👀**.

## 📚 Tutorials
- [Short tutorial](https://huggingface.co/learn/deep-rl-course/unitbonus1/introduction): Teach Huggy the Dog 🐶 to fetch the stick and play in-browser.  
- [Longer tutorial](https://huggingface.co/learn/deep-rl-course/unit5/introduction): Deep dive into ML-Agents training and deployment.

---