---
library_name: pytorch
pipeline_tag: reinforcement-learning
tags:
- reinforcement-learning
- dqn
- onnx
- fruitbox
- gamesaien
- chrome-extension
- gymnasium
---

# AlphaApple - FruitBox DQN

This model plays the FruitBox (Fruit Box) puzzle game hosted on Gamesaien. It predicts Q-values over all axis-aligned rectangles on a 10x17 board. A valid action is a rectangle whose cell sum is exactly 10, so you must apply an action mask to filter invalid rectangles before selecting the best move.

## Quick facts
- Board: 10x17, values 0-9 (0 means empty)
- Action space: 8415 axis-aligned rectangles
- Input: one-hot board with shape `[1, 10, 10, 17]`
- Output: Q-values for all rectangles
- Masking: required to remove invalid rectangles

## Files in this repo
- `model.pth`: PyTorch checkpoint dict with `policy_net`, `target_net`, `optimizer`
- `model.onnx`: Exported ONNX model for browser/runtime inference

## How to use (PyTorch)
```python
# Model definition is in https://github.com/kbsooo/AlphaApple (src/models.py)
import torch
from src.models import FruitBoxDQN

rows, cols = 10, 17
n_actions = 55 * 153  # (rows*(rows+1)/2) * (cols*(cols+1)/2) = 8415
model = FruitBoxDQN(rows, cols, n_actions)

ckpt = torch.load("model.pth", map_location="cpu")
state = ckpt["policy_net"] if "policy_net" in ckpt else ckpt
model.load_state_dict(state)
model.eval()
```

## How to use (ONNX / browser)
```js
const session = await ort.InferenceSession.create("model.onnx");
// input: Float32Array with shape [1, 10, 10, 17]
const output = await session.run({ input });
// output.output.data: Q-values for 8415 rectangles
```

## Action masking (required)
You must mask invalid rectangles before selecting an action. A rectangle is valid if the sum of its cells equals 10. Without the mask, the model can pick illegal moves.

## Training details
- Environment: FruitBoxEnv (implemented in `envs/fruitbox_env.py`, class `FruitBoxEnvImproved`)
- Board generator: BackwardBoardGenerator (solvable boards)
- Curriculum: target coverage ramps from 0.3 to 0.95 in steps of 0.1
- Optimizer: Adam, gamma=0.99, lr=1e-4
- Episodes: 10k (Colab integrated script)

## Limitations
- Trained on generated boards; performance may vary on edge cases.
- Requires an accurate action mask and correct board extraction.

## Links
- Game: https://en.gamesaien.com/game/fruit_box/
- Project: https://github.com/kbsooo/AlphaApple