openenv/pr_813/en/environments/chess.md

Chess Environment

A chess reinforcement learning environment for OpenEnv, powered by the moonfish chess engine.

Features

• Full chess rules via python-chess library
• Configurable opponent: moonfish engine, random moves, or self-play
• Position evaluation: Uses moonfish's PSQT-based evaluation
• Standard OpenEnv interface: reset(), step(), state

Quick Start

Using Docker

# Build the image
docker build -t chess-env:latest -f envs/chess_env/server/Dockerfile .

# Run the server
docker run -p 8000:8000 chess-env:latest

Using the Client

The client is async by default:

import asyncio
from chess_env import ChessEnv, ChessAction

async def main():
    async with ChessEnv(base_url="http://localhost:8000") as env:
        # Reset for a new game
        result = await env.reset()
        print(f"Starting position: {result.observation.fen}")
        print(f"Legal moves: {result.observation.legal_moves}")

        # Make a move
        result = await env.step(ChessAction(move="e2e4"))
        print(f"Reward: {result.reward}, Done: {result.done}")

        # Play until game ends
        while not result.done:
            move = result.observation.legal_moves[0]
            result = await env.step(ChessAction(move=move))

        print(f"Game result: {result.observation.result}")

asyncio.run(main())

For synchronous usage, use the .sync() wrapper:

from chess_env import ChessEnv, ChessAction

with ChessEnv(base_url="http://localhost:8000").sync() as env:
    result = env.reset()
    result = env.step(ChessAction(move="e2e4"))
    print(f"Reward: {result.reward}")

Observation Space

Field	Type	Description
fen	str	Board position in FEN notation
legal_moves	List[str]	Legal moves in UCI format
is_check	bool	Whether current player is in check
done	bool	Whether game has ended
reward	float	Reward for last action
result	str	Game result ("1-0", "0-1", "1/2-1/2")

Action Space

Field	Type	Description
move	str	UCI format move (e.g., "e2e4", "e7e8q")

Rewards

Outcome	Reward
Win	+1.0
Loss	-1.0
Draw	0.0
Illegal move	-0.1

Configuration

The environment supports these configuration options:

Parameter	Default	Description
opponent	"moonfish"	Opponent type: "moonfish", "random", or None
opponent_depth	2	Search depth for moonfish opponent
max_moves	500	Maximum half-moves before draw
agent_color	None	Agent color: "white", "black", or None (alternate each episode)
gamma	0.99	Discount factor for temporal credit assignment

Temporal Discounting

For RL training, the environment computes temporally discounted rewards at episode end. This helps with credit assignment in long games where only the final outcome is known.

When an episode ends, the terminal observation's metadata includes:

• discounted_rewards: List of discounted rewards for each agent move
• gamma: The discount factor used

The formula is r_t = γ^(T-1-t) × R_final where:

• T = total agent moves
• t = move index (0-indexed)
• R_final = terminal reward (+1, -1, or 0)

Example for a 5-move win with γ=0.99:

Move 0: 0.99^4 × 1.0 = 0.961
Move 1: 0.99^3 × 1.0 = 0.970
Move 2: 0.99^2 × 1.0 = 0.980
Move 3: 0.99^1 × 1.0 = 0.990
Move 4: 0.99^0 × 1.0 = 1.000

Links

• moonfish GitHub
• Play online