---
library_name: transformers
tags:
- chess
- llm-course
- chess-challenge
license: mit
---

# chess-ooooooooo

A chess transformer model trained for the LLM Course Chess Challenge.

## Model Architecture

This model uses a GPT-style transformer architecture optimized for chess move prediction:

- **Parameters**: 948,352 (0.95M)
- **Vocabulary size**: 85
- **Embedding dimension**: 128
- **Number of layers**: 6
- **Attention heads**: 4
- **Feed-forward dimension**: 320
- **Context length**: 256
- **Dropout**: 0.1

## Training

The model was trained on a subset of the Lichess 2025 dataset, focusing on learning valid chess move sequences. The architecture was carefully tuned to stay within the 1M parameter constraint while maintaining reasonable performance.

## Usage

```python
from transformers import AutoModelForCausalLM
from src.tokenizer import ChessTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "LLM-course/chess-ooooooooo",
    trust_remote_code=True
)
tokenizer = ChessTokenizer.from_pretrained(
    "LLM-course/chess-ooooooooo",
    trust_remote_code=True
)

# Generate moves
input_text = "[BOS] WPe2e4"
input_ids = tokenizer.encode(input_text)
outputs = model.generate(input_ids, max_length=50)
predicted_moves = tokenizer.decode(outputs[0])
```

## Submission

Submitted by [etienneLefranc](https://huggingface.co/etienneLefranc) for the LLM Course Chess Challenge.