Chess Challenge submission by rubenifrah

README.md

@@ -0,0 +1,26 @@
+---
+library_name: transformers
+tags:
+- chess
+- llm-course
+- chess-challenge
+license: mit
+---
+
+# chess-coord-v1
+
+Chess model submitted to the LLM Course Chess Challenge.
+
+## Submission Info
+
+- **Submitted by**: [rubenifrah](https://huggingface.co/rubenifrah)
+- **Parameters**: 909,824
+- **Organization**: LLM-course
+
+## Model Details
+
+- **Architecture**: Chess Transformer (GPT-style)
+- **Vocab size**: 1682
+- **Embedding dim**: 128
+- **Layers**: 4
+- **Heads**: 4

config.json

@@ -0,0 +1,20 @@
+{
+  "architectures": [
+    "ChessForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "dropout": 0.1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "chess_transformer",
+  "n_ctx": 256,
+  "n_embd": 128,
+  "n_head": 4,
+  "n_inner": 384,
+  "n_layer": 4,
+  "pad_token_id": 0,
+  "tie_weights": true,
+  "transformers_version": "4.57.3",
+  "vocab_size": 1682
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fcb65c515d1bf40361be24e8229d3ac7322033e319370295bd12576eff9388b
+size 3643696

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "[BOS]",
+  "eos_token": "[EOS]",
+  "pad_token": "[PAD]",
+  "unk_token": "[UNK]"
+}

src/__init__.py

@@ -0,0 +1,18 @@
+"""
+Chess Challenge - LLM Training Exercise
+
+Train a 1M parameter model to play chess using custom tokenizers and architectures.
+"""
+
+from src.tokenizer import ChessTokenizer
+from src.model import ChessConfig, ChessForCausalLM
+from src.utils import count_parameters, print_parameter_budget
+
+__version__ = "0.1.0"
+__all__ = [
+    "ChessTokenizer",
+    "ChessConfig", 
+    "ChessForCausalLM",
+    "count_parameters",
+    "print_parameter_budget",
+]

src/chess.sh

@@ -0,0 +1,73 @@
+#!/bin/bash
+#SBATCH -p mesonet
+#SBATCH -N 1
+#SBATCH -c 8              # Increased CPUs for data loading
+#SBATCH --gres=gpu:1      # 1 GPU per job
+#SBATCH --time=12:00:00   # Sufficient for 3 epochs
+#SBATCH --mem=32G         
+#SBATCH --account=m25146
+#SBATCH --job-name=chess_train
+#SBATCH --output=logs/%x_%j.out
+#SBATCH --error=logs/%x_%j.err
+
+export PYTHONUNBUFFERED=1
+export HF_DATASETS_OFFLINE=1
+export TRANSFORMERS_OFFLINE=1
+export CUDA_LAUNCH_BLOCKING=1
+
+# Exit immediately if a command exits with a non-zero status
+set -e
+
+# Load modules (adjust based on your cluster, e.g., module load cuda/11.8)
+# module load cuda/12.1  <-- Uncomment if needed
+
+# Activate venv
+if [ ! -d "venv" ]; then
+    echo "Creating venv..."
+    python3 -m venv venv
+    source venv/bin/activate
+    pip install --upgrade pip
+    pip install -e .
+else
+    source venv/bin/activate
+fi
+
+# Define paths
+MODEL_DIR="./output_chess_model"
+mkdir -p logs
+
+echo "============================================"
+echo "Starting Chess Training Pipeline"
+echo "Job ID: $SLURM_JOB_ID"
+echo "Output Dir: $MODEL_DIR"
+echo "============================================"
+
+# 1. Training
+echo "[1/2] Starting Training..."
+python -m src.train \
+    --output_dir $MODEL_DIR \
+    --num_train_epochs 3 \
+    --per_device_train_batch_size 64 \
+    --save_steps 2000
+
+# 2. Evaluation
+echo "--------------------------------------------"
+echo "[2/2] Starting Evaluation..."
+
+# Legal moves check
+echo "Running legal move evaluation..."
+python -m src.evaluate \
+    --model_path "${MODEL_DIR}/final_model" \
+    --mode legal \
+    --n_positions 1000
+
+# Win rate check
+echo "Running win rate check vs Stockfish..."
+python -m src.evaluate \
+    --model_path "${MODEL_DIR}/final_model" \
+    --mode winrate \
+    --n_games 50 \
+    --stockfish_level 1
+
+echo "============================================"
+echo "Pipeline Completed!"

src/chess_challenge.egg-info/PKG-INFO

@@ -0,0 +1,198 @@
+Metadata-Version: 2.4
+Name: chess-challenge
+Version: 0.1.0
+Summary: LLM Chess Challenge - Train a 1M parameter model to play chess
+Author-email: Nathanaël Fijalkow <nathanael.fijalkow@gmail.com>
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Education
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=2.0.0
+Requires-Dist: transformers>=4.40.0
+Requires-Dist: accelerate>=0.26.0
+Requires-Dist: datasets>=2.14.0
+Requires-Dist: python-chess>=1.999
+Requires-Dist: huggingface-hub>=0.20.0
+Requires-Dist: tqdm>=4.65.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: wandb>=0.15.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Provides-Extra: eval
+Requires-Dist: stockfish>=3.28.0; extra == "eval"
+
+# Chess Challenge
+
+Train a 1M parameter LLM to play chess!
+
+## Objective
+
+Design and train a transformer-based language model to predict chess moves. Your model must:
+
+1. **Stay under 1M parameters** - This is the hard constraint!
+2. **Use a custom tokenizer** - Design an efficient move-level tokenizer
+3. **Play legal chess** - The model should learn to generate valid moves
+4. **Beat Stockfish** - Your ELO will be measured against Stockfish Level 1
+
+## Dataset
+
+We use the Lichess dataset: [`dlouapre/lichess_2025-01_1M`](https://huggingface.co/datasets/dlouapre/lichess_2025-01_1M)
+
+The dataset uses an extended UCI notation:
+- `W`/`B` prefix for White/Black
+- Piece letter: `P`=Pawn, `N`=Knight, `B`=Bishop, `R`=Rook, `Q`=Queen, `K`=King
+- Source and destination squares (e.g., `e2e4`)
+- Special suffixes: `(x)`=capture, `(+)`=check, `(+*)`=checkmate, `(o)`/`(O)`=castling
+
+Example game:
+```
+WPe2e4 BPe7e5 WNg1f3 BNb8c6 WBf1b5 BPa7a6 WBb5c6(x) BPd7c6(x) ...
+```
+
+## Quick Start
+
+### Installation
+
+```bash
+# Clone the template
+git clone https://github.com/nathanael-fijalkow/ChessChallengeTemplate.git
+cd ChessChallengeTemplate
+
+# Create virtual environment
+uv venv
+source .venv/bin/activate  # On Windows: venv\Scripts\activate
+
+# Install dependencies
+uv pip install -e .
+```
+
+### Train a Model
+
+```bash
+# Basic training
+python -m src.train \
+    --output_dir ./my_model \
+    --num_train_epochs 3 \
+    --per_device_train_batch_size 32
+```
+
+### Evaluate Your Model
+
+Evaluation happens in two phases:
+
+```bash
+# Phase 1: Legal Move Evaluation (quick sanity check)
+python -m src.evaluate \
+    --model_path ./my_model/final_model \
+    --mode legal \
+    --n_positions 500
+
+# Phase 2: Win Rate Evaluation (full games against Stockfish)
+python -m src.evaluate \
+    --model_path ./my_model/final_model \
+    --mode winrate \
+    --n_games 100 \
+    --stockfish_level 1
+
+# Or run both phases:
+python -m src.evaluate \
+    --model_path ./my_model/final_model \
+    --mode both
+```
+
+## Parameter Budget
+
+Use the utility function to check your budget:
+
+```python
+from src import ChessConfig, print_parameter_budget
+
+config = ChessConfig(
+    vocab_size=1200,
+    n_embd=128,
+    n_layer=4,
+    n_head=4,
+)
+print_parameter_budget(config)
+```
+
+### Pro Tips
+
+1. **Weight Tying**: The default config ties the embedding and output layer weights, saving ~154k parameters
+2. **Vocabulary Size**: Keep it small! ~1200 tokens covers all moves
+3. **Depth vs Width**: With limited parameters, experiment with shallow-but-wide vs deep-but-narrow
+
+## Customization
+
+### Custom Tokenizer
+
+The template provides a move-level tokenizer that builds vocabulary from the actual dataset.
+Feel free to try different approaches!
+
+### Custom Architecture
+
+Modify the model in `src/model.py`:
+
+```python
+from src import ChessConfig, ChessForCausalLM
+
+# Customize configuration
+config = ChessConfig(
+    vocab_size=1200,
+    n_embd=128,      # Try 96, 128, or 192
+    n_layer=4,       # Try 3, 4, or 6
+    n_head=4,        # Try 4 or 8
+    n_inner=384,     # Feed-forward dimension (default: 3*n_embd)
+    dropout=0.1,
+    tie_weights=True,
+)
+
+model = ChessForCausalLM(config)
+```
+
+## Evaluation Metrics
+
+### Phase 1: Legal Move Evaluation
+
+Tests if your model generates valid chess moves:
+
+| Metric | Description |
+|--------|-------------|
+| **Legal Rate (1st try)** | % of legal moves on first attempt |
+| **Legal Rate (with retry)** | % of legal moves within 3 attempts |
+
+> **Target**: >90% legal rate before proceeding to Phase 2
+
+### Phase 2: Win Rate Evaluation
+
+Full games against Stockfish to measure playing strength:
+
+| Metric | Description |
+|--------|-------------|
+| **Win Rate** | % of games won against Stockfish |
+| **ELO Rating** | Estimated rating based on game results |
+| **Avg Game Length** | Average number of moves per game |
+| **Illegal Move Rate** | % of illegal moves during games |
+
+
+## Submission
+
+1. Train your model
+2. Log in to Hugging Face: `hf auth login`
+3. Submit your model using the submission script:
+
+```bash
+python submit.py --model_path ./my_model/final_model --model_name your-model-name
+```
+
+The script will:
+- Upload your model to the LLM-course organization
+- Include your HF username in the model card for tracking

src/chess_challenge.egg-info/SOURCES.txt

@@ -0,0 +1,8 @@
+README.md
+pyproject.toml
+src/chess_challenge.egg-info/PKG-INFO
+src/chess_challenge.egg-info/SOURCES.txt
+src/chess_challenge.egg-info/dependency_links.txt
+src/chess_challenge.egg-info/entry_points.txt
+src/chess_challenge.egg-info/requires.txt
+src/chess_challenge.egg-info/top_level.txt

src/chess_challenge.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

src/chess_challenge.egg-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+chess-eval = src.evaluate:main
+chess-train = src.train:main

src/chess_challenge.egg-info/requires.txt

@@ -0,0 +1,17 @@
+torch>=2.0.0
+transformers>=4.40.0
+accelerate>=0.26.0
+datasets>=2.14.0
+python-chess>=1.999
+huggingface-hub>=0.20.0
+tqdm>=4.65.0
+numpy>=1.24.0
+wandb>=0.15.0
+
+[dev]
+pytest>=7.0.0
+black>=23.0.0
+ruff>=0.1.0
+
+[eval]
+stockfish>=3.28.0

src/chess_challenge.egg-info/top_level.txt

@@ -0,0 +1 @@
+

src/data.py

@@ -0,0 +1,259 @@
+"""
+Data loading utilities for the Chess Challenge.
+
+This module provides functions to load and process chess game data
+from the Lichess dataset on Hugging Face.
+"""
+
+from __future__ import annotations
+
+from typing import Dict, Iterator, List, Optional
+
+import torch
+from torch.utils.data import Dataset
+
+
+class ChessDataset(Dataset):
+    """
+    PyTorch Dataset for chess games.
+    
+    This dataset loads games from a Hugging Face dataset and prepares
+    them for language modeling training.
+    
+    Each game is tokenized and truncated/padded to max_length.
+    The labels are shifted by one position for next-token prediction.
+    
+    Example:
+        >>> from src.tokenizer import ChessTokenizer
+        >>> tokenizer = ChessTokenizer.build_vocab_from_dataset()
+        >>> dataset = ChessDataset(tokenizer, max_length=256)
+        >>> sample = dataset[0]
+        >>> print(sample["input_ids"].shape)  # (256,)
+    """
+    
+    def __init__(
+        self,
+        tokenizer,
+        dataset_name: str = "dlouapre/lichess_2025-01_1M",
+        split: str = "train",
+        column: str = "text",
+        max_length: int = 256,
+        max_samples: Optional[int] = None,
+    ):
+        """
+        Initialize the chess dataset.
+        
+        Args:
+            tokenizer: The chess tokenizer to use.
+            dataset_name: Name of the dataset on Hugging Face Hub.
+            split: Dataset split to use.
+            column: Column containing the game strings.
+            max_length: Maximum sequence length.
+            max_samples: Maximum number of samples to load.
+        """
+        from datasets import load_dataset
+        
+        self.tokenizer = tokenizer
+        self.max_length = max_length
+        self.column = column
+        
+        # Load dataset
+        dataset = load_dataset(dataset_name, split=split)
+        
+        if max_samples is not None:
+            dataset = dataset.select(range(min(max_samples, len(dataset))))
+        
+        self.data = dataset
+    
+    def __len__(self) -> int:
+        return len(self.data)
+    
+    def __getitem__(self, idx: int) -> Dict[str, torch.Tensor]:
+        game = self.data[idx][self.column]
+        
+        # Prepend BOS token for proper language modeling
+        game_with_bos = self.tokenizer.bos_token + " " + game
+        
+        # Tokenize
+        encoding = self.tokenizer(
+            game_with_bos,
+            truncation=True,
+            max_length=self.max_length,
+            padding="max_length",
+            return_tensors="pt",
+        )
+        
+        # Squeeze batch dimension
+        input_ids = encoding["input_ids"].squeeze(0)
+        attention_mask = encoding["attention_mask"].squeeze(0)
+
+        # SAFETY CHECK: Clamp tokens to be within vocab range
+        # This prevents "device-side assert triggered" if tokenizer produces unexpected IDs
+        if input_ids.max() >= self.tokenizer.vocab_size:
+            print(f"WARNING: Found token ID {input_ids.max()} >= vocab size {self.tokenizer.vocab_size}. Clamping.")
+            input_ids = torch.clamp(input_ids, max=self.tokenizer.vocab_size - 1)
+        
+        # Labels are the same as input_ids (model will shift internally)
+        labels = input_ids.clone()
+        
+        # Set padding tokens to -100 to ignore in loss
+        labels[attention_mask == 0] = -100
+        
+        return {
+            "input_ids": input_ids,
+            "attention_mask": attention_mask,
+            "labels": labels,
+        }
+
+
+class ChessDataCollator:
+    """
+    Data collator for chess games.
+    
+    This collator pads sequences to the same length within a batch
+    and creates the appropriate attention masks.
+    """
+    
+    def __init__(self, tokenizer, max_length: int = 256):
+        self.tokenizer = tokenizer
+        self.max_length = max_length
+    
+    def __call__(self, features: List[Dict]) -> Dict[str, torch.Tensor]:
+        # Stack tensors
+        input_ids = torch.stack([f["input_ids"] for f in features])
+        attention_mask = torch.stack([f["attention_mask"] for f in features])
+        labels = torch.stack([f["labels"] for f in features])
+        
+        return {
+            "input_ids": input_ids,
+            "attention_mask": attention_mask,
+            "labels": labels,
+        }
+
+
+def create_train_val_datasets(
+    tokenizer,
+    dataset_name: str = "dlouapre/lichess_2025-01_1M",
+    max_length: int = 256,
+    train_samples: Optional[int] = None,
+    val_samples: int = 5000,
+    val_ratio: float = 0.05,
+):
+    """
+    Create training and validation datasets.
+    
+    Args:
+        tokenizer: The chess tokenizer.
+        dataset_name: Name of the dataset.
+        max_length: Maximum sequence length.
+        train_samples: Maximum training samples (None for all).
+        val_samples: Number of validation samples.
+        val_ratio: Ratio of validation samples (used if train_samples is None).
+    
+    Returns:
+        Tuple of (train_dataset, val_dataset).
+    """
+    from datasets import load_dataset
+    
+    # Load full dataset
+    full_dataset = load_dataset(dataset_name, split="train")
+    
+    # Determine split sizes
+    total = len(full_dataset)
+    
+    if train_samples is not None:
+        n_train = min(train_samples, total - val_samples)
+    else:
+        n_train = int(total * (1 - val_ratio))
+    
+    n_val = min(val_samples, total - n_train)
+    
+    # Split dataset
+    train_data = full_dataset.select(range(n_train))
+    val_data = full_dataset.select(range(n_train, n_train + n_val))
+    
+    # Create dataset objects
+    train_dataset = ChessDataset(
+        tokenizer=tokenizer,
+        dataset_name=dataset_name,
+        max_length=max_length,
+    )
+    train_dataset.data = train_data
+    
+    val_dataset = ChessDataset(
+        tokenizer=tokenizer,
+        dataset_name=dataset_name,
+        max_length=max_length,
+    )
+    val_dataset.data = val_data
+    
+    return train_dataset, val_dataset
+
+
+def stream_games(
+    dataset_name: str = "dlouapre/lichess_2025-01_1M",
+    split: str = "train",
+    column: str = "text",
+) -> Iterator[str]:
+    """
+    Stream games from the dataset for memory-efficient processing.
+    
+    Args:
+        dataset_name: Name of the dataset on Hugging Face Hub.
+        split: Dataset split to use.
+        column: Column containing the game strings.
+    
+    Yields:
+        Game strings one at a time.
+    """
+    from datasets import load_dataset
+    
+    dataset = load_dataset(dataset_name, split=split, streaming=True)
+    
+    for example in dataset:
+        yield example[column]
+
+
+def analyze_dataset_statistics(
+    dataset_name: str = "dlouapre/lichess_2025-01_1M",
+    max_samples: int = 10000,
+) -> Dict:
+    """
+    Analyze statistics of the chess dataset.
+    
+    Args:
+        dataset_name: Name of the dataset.
+        max_samples: Maximum number of samples to analyze.
+    
+    Returns:
+        Dictionary containing dataset statistics.
+    """
+    from collections import Counter
+    from datasets import load_dataset
+    
+    dataset = load_dataset(dataset_name, split="train")
+    dataset = dataset.select(range(min(max_samples, len(dataset))))
+    
+    game_lengths = []
+    move_counts = Counter()
+    opening_moves = Counter()
+    
+    for example in dataset:
+        moves = example["text"].strip().split()
+        game_lengths.append(len(moves))
+        move_counts.update(moves)
+        
+        # Track common openings (first 4 moves)
+        if len(moves) >= 4:
+            opening = " ".join(moves[:4])
+            opening_moves[opening] += 1
+    
+    return {
+        "total_games": len(dataset),
+        "avg_game_length": sum(game_lengths) / len(game_lengths),
+        "min_game_length": min(game_lengths),
+        "max_game_length": max(game_lengths),
+        "unique_moves": len(move_counts),
+        "most_common_moves": move_counts.most_common(20),
+        "most_common_openings": opening_moves.most_common(10),
+    }

src/evaluate.py

@@ -0,0 +1,702 @@
+"""
+Evaluation script for the Chess Challenge.
+
+This script evaluates a trained chess model by playing games against
+Stockfish and computing ELO ratings.
+"""
+
+from __future__ import annotations
+
+import argparse
+import random
+from dataclasses import dataclass
+from typing import List, Optional, Tuple
+
+import torch
+
+
+@dataclass
+class GameResult:
+    """Result of a single game."""
+    moves: List[str]
+    result: str  # "1-0", "0-1", or "1/2-1/2"
+    model_color: str  # "white" or "black"
+    termination: str  # "checkmate", "stalemate", "illegal_move", "max_moves", etc.
+    illegal_move_count: int
+
+
+class ChessEvaluator:
+    """
+    Evaluator for chess models.
+    
+    This class handles playing games between a trained model and Stockfish,
+    tracking results, and computing ELO ratings.
+    """
+    
+    def __init__(
+        self,
+        model,
+        tokenizer,
+        stockfish_path: Optional[str] = None,
+        stockfish_level: int = 1,
+        max_retries: int = 3,
+        device: str = "cuda" if torch.cuda.is_available() else "cpu",
+    ):
+        """
+        Initialize the evaluator.
+        
+        Args:
+            model: The trained chess model.
+            tokenizer: The chess tokenizer.
+            stockfish_path: Path to Stockfish executable.
+            stockfish_level: Stockfish skill level (0-20).
+            max_retries: Maximum retries for illegal moves.
+            device: Device to run the model on.
+        """
+        self.model = model.to(device)
+        self.tokenizer = tokenizer
+        self.max_retries = max_retries
+        self.device = device
+        
+        # Initialize Stockfish
+        try:
+            import chess
+            import chess.engine
+            
+            self.chess = chess
+            
+            if stockfish_path is None:
+                # Try common paths
+                import shutil
+                stockfish_path = shutil.which("stockfish")
+            
+            if stockfish_path:
+                self.engine = chess.engine.SimpleEngine.popen_uci(stockfish_path)
+                self.engine.configure({"Skill Level": stockfish_level})
+            else:
+                print("WARNING: Stockfish not found. Install it for full evaluation.")
+                self.engine = None
+                
+        except ImportError:
+            raise ImportError(
+                "python-chess is required for evaluation. "
+                "Install it with: pip install python-chess"
+            )
+    
+    def __del__(self):
+        """Clean up Stockfish engine."""
+        if hasattr(self, 'engine') and self.engine:
+            self.engine.quit()
+    
+    def _convert_board_to_moves(self, board) -> str:
+        """Convert board move history to model input format."""
+        moves = []
+        temp_board = self.chess.Board()
+        
+        for move in board.move_stack:
+            # Get piece and color
+            color = "W" if temp_board.turn == self.chess.WHITE else "B"
+            piece = temp_board.piece_at(move.from_square)
+            piece_letter = piece.symbol().upper() if piece else "P"
+            
+            # Get squares
+            from_sq = self.chess.square_name(move.from_square)
+            to_sq = self.chess.square_name(move.to_square)
+            
+            move_str = f"{color}{piece_letter}{from_sq}{to_sq}"
+            
+            # Add promotion
+            if move.promotion:
+                move_str += f"={self.chess.piece_symbol(move.promotion).upper()}"
+            
+            # Add capture suffix
+            if temp_board.is_capture(move):
+                move_str += "(x)"
+            
+            # Add check/checkmate suffix
+            temp_board.push(move)
+            if temp_board.is_checkmate():
+                move_str = move_str.replace("(x)", "(x+*)") if "(x)" in move_str else move_str + "(+*)"
+            elif temp_board.is_check():
+                move_str = move_str.replace("(x)", "(x+)") if "(x)" in move_str else move_str + "(+)"
+            
+            # Handle castling
+            if piece_letter == "K" and abs(ord(from_sq[0]) - ord(to_sq[0])) > 1:
+                if to_sq[0] == 'g':  # Kingside
+                    move_str = move_str.split("(")[0] + "(o)"
+                else:  # Queenside
+                    move_str = move_str.split("(")[0] + "(O)"
+            
+            moves.append(move_str)
+        
+        return " ".join(moves)
+    
+    def _is_separator_token(self, token_str: str) -> bool:
+        """
+        Check if a token represents a separator (whitespace, EOS, etc.).
+        
+        This allows the evaluator to work with different tokenization strategies:
+        - Move-level tokenizers: each move is one token, no separators generated
+        - Character-level tokenizers: space character marks end of move
+        - BPE/subword tokenizers: may generate partial moves
+        
+        Args:
+            token_str: The decoded token string.
+        
+        Returns:
+            True if this token indicates end of a move.
+        """
+        # Check for EOS token
+        if hasattr(self.tokenizer, 'eos_token') and token_str == self.tokenizer.eos_token:
+            return True
+        
+        # Check for whitespace (space, newline, etc.)
+        if token_str.strip() == "" and len(token_str) > 0:
+            return True
+        
+        # Check if the token ends with whitespace (some tokenizers include trailing space)
+        if token_str != token_str.rstrip():
+            return True
+        
+        return False
+
+    def _generate_move_tokens(
+        self,
+        input_ids: torch.Tensor,
+        temperature: float = 0.7,
+        top_k: int = 10,
+        max_tokens: int = 20,
+    ) -> str:
+        """
+        Generate tokens until a separator (whitespace/EOS) is encountered.
+        
+        This method supports different tokenization strategies:
+        - For move-level tokenizers: generates one token (the full move)
+        - For character/subword tokenizers: generates until whitespace
+        
+        Args:
+            input_ids: The input token IDs.
+            temperature: Sampling temperature.
+            top_k: Top-k filtering parameter.
+            max_tokens: Maximum tokens to generate for a single move.
+        
+        Returns:
+            The generated move string (without trailing separator).
+        """
+        generated_tokens = []
+        current_ids = input_ids.clone()
+        
+        for _ in range(max_tokens):
+            with torch.no_grad():
+                outputs = self.model(input_ids=current_ids)
+                logits = outputs.logits[:, -1, :] / temperature
+                
+                # Apply top-k filtering
+                if top_k > 0:
+                    top_k_values = torch.topk(logits, min(top_k, logits.size(-1)))[0]
+                    indices_to_remove = logits < top_k_values[..., -1, None]
+                    logits[indices_to_remove] = float("-inf")
+                
+                # Sample
+                probs = torch.softmax(logits, dim=-1)
+                next_token = torch.multinomial(probs, num_samples=1)  # Shape: [1, 1]
+            
+            # Decode the token
+            token_str = self.tokenizer.decode(next_token[0])
+            
+            # Check if this is a separator token
+            if self._is_separator_token(token_str):
+                break
+            
+            generated_tokens.append(next_token[0])  # Store [1] tensor
+            
+            # Append to input for next iteration (next_token is already [1, 1])
+            current_ids = torch.cat([current_ids, next_token], dim=-1)
+            
+            # For move-level tokenizers, a single non-separator token is the full move
+            # We can detect this by checking if the token looks like a complete move
+            # (starts with W or B, has enough characters for a move)
+            if len(token_str) >= 6 and token_str[0] in "WB":
+                break
+        
+        # Decode all generated tokens together
+        if generated_tokens:
+            all_tokens = torch.cat(generated_tokens, dim=0)
+            move_str = self.tokenizer.decode(all_tokens, skip_special_tokens=True)
+            return move_str.strip()
+        
+        return ""
+
+    def _get_model_move(
+        self,
+        board,
+        temperature: float = 0.7,
+        top_k: int = 10,
+    ) -> Tuple[Optional[str], int]:
+        """
+        Get the model's next move prediction.
+        
+        This method generates tokens until a separator (whitespace/EOS) is produced,
+        allowing it to work with different tokenization strategies:
+        - Move-level tokenizers: each move is a single token
+        - Character-level tokenizers: moves are generated character by character
+        - BPE/subword tokenizers: moves may be split into subwords
+        
+        Returns:
+            Tuple of (UCI move string, number of retries used).
+        """
+        self.model.eval()
+        
+        # Convert board to input format
+        moves_str = self._convert_board_to_moves(board)
+        
+        # Add BOS token if no moves yet
+        if not moves_str:
+            input_text = self.tokenizer.bos_token
+        else:
+            input_text = self.tokenizer.bos_token + " " + moves_str
+        
+        # Tokenize
+        inputs = self.tokenizer(
+            input_text,
+            return_tensors="pt",
+            truncation=True,
+            max_length=self.model.config.n_ctx - 10,  # Leave room for generated tokens
+        ).to(self.device)
+        
+        # Try to generate a legal move
+        for retry in range(self.max_retries):
+            # Generate tokens until separator
+            move_token = self._generate_move_tokens(
+                inputs["input_ids"],
+                temperature=temperature,
+                top_k=top_k,
+            )
+            
+            # Convert to UCI
+            if len(move_token) >= 6:
+                uci_move = move_token[2:4] + move_token[4:6]
+                
+                # Handle promotion
+                if "=" in move_token:
+                    promo_idx = move_token.index("=")
+                    uci_move += move_token[promo_idx + 1].lower()
+                
+                try:
+                    move = self.chess.Move.from_uci(uci_move)
+                    if move in board.legal_moves:
+                        return uci_move, retry
+                except (ValueError, self.chess.InvalidMoveError):
+                    pass
+        
+        return None, self.max_retries
+    
+    def _get_stockfish_move(self, board, time_limit: float = 0.1) -> str:
+        """Get Stockfish's move."""
+        if self.engine is None:
+            raise RuntimeError("Stockfish engine not initialized")
+        
+        result = self.engine.play(board, self.chess.engine.Limit(time=time_limit))
+        return result.move.uci()
+    
+    def play_game(
+        self,
+        model_color: str = "white",
+        max_moves: int = 200,
+        temperature: float = 0.7,
+    ) -> GameResult:
+        """
+        Play a single game between the model and Stockfish.
+        
+        Args:
+            model_color: "white" or "black".
+            max_moves: Maximum number of moves before draw.
+            temperature: Sampling temperature for model.
+        
+        Returns:
+            GameResult with the game details.
+        """
+        board = self.chess.Board()
+        moves = []
+        illegal_move_count = 0
+        
+        model_is_white = model_color == "white"
+        
+        while not board.is_game_over() and len(moves) < max_moves:
+            is_model_turn = (board.turn == self.chess.WHITE) == model_is_white
+            
+            if is_model_turn:
+                # Model's turn
+                uci_move, retries = self._get_model_move(board, temperature)
+                illegal_move_count += retries
+                
+                if uci_move is None:
+                    # Model couldn't find a legal move
+                    return GameResult(
+                        moves=moves,
+                        result="0-1" if model_is_white else "1-0",
+                        model_color=model_color,
+                        termination="illegal_move",
+                        illegal_move_count=illegal_move_count + 1,
+                    )
+                
+                move = self.chess.Move.from_uci(uci_move)
+            else:
+                # Stockfish's turn
+                if self.engine:
+                    uci_move = self._get_stockfish_move(board)
+                    move = self.chess.Move.from_uci(uci_move)
+                else:
+                    # Random move if no engine
+                    move = random.choice(list(board.legal_moves))
+            
+            board.push(move)
+            moves.append(move.uci())
+        
+        # Determine result
+        if board.is_checkmate():
+            if board.turn == self.chess.WHITE:
+                result = "0-1"  # Black wins
+            else:
+                result = "1-0"  # White wins
+            termination = "checkmate"
+        elif board.is_stalemate():
+            result = "1/2-1/2"
+            termination = "stalemate"
+        elif board.is_insufficient_material():
+            result = "1/2-1/2"
+            termination = "insufficient_material"
+        elif board.can_claim_draw():
+            result = "1/2-1/2"
+            termination = "draw_claim"
+        elif len(moves) >= max_moves:
+            result = "1/2-1/2"
+            termination = "max_moves"
+        else:
+            result = "1/2-1/2"
+            termination = "unknown"
+        
+        return GameResult(
+            moves=moves,
+            result=result,
+            model_color=model_color,
+            termination=termination,
+            illegal_move_count=illegal_move_count,
+        )
+    
+    def evaluate_legal_moves(
+        self,
+        n_positions: int = 1000,
+        temperature: float = 0.7,
+        verbose: bool = True,
+    ) -> dict:
+        """
+        Evaluate the model's ability to generate legal moves.
+        
+        This evaluation only checks if the model generates legal moves,
+        without playing full games. Useful as a first-pass evaluation.
+        
+        Args:
+            n_positions: Number of positions to test.
+            temperature: Sampling temperature.
+            verbose: Whether to print progress.
+        
+        Returns:
+            Dictionary with legal move statistics.
+        """
+        results = {
+            "total_positions": 0,
+            "legal_first_try": 0,
+            "legal_with_retry": 0,
+            "illegal_all_retries": 0,
+            "positions": [],
+        }
+        
+        # Generate random positions by playing random moves
+        for i in range(n_positions):
+            board = self.chess.Board()
+            
+            # Play random number of moves (5-40) to get varied positions
+            n_random_moves = random.randint(5, 40)
+            for _ in range(n_random_moves):
+                if board.is_game_over():
+                    break
+                move = random.choice(list(board.legal_moves))
+                board.push(move)
+            
+            if board.is_game_over():
+                continue  # Skip terminal positions
+            
+            results["total_positions"] += 1
+            
+            # Test model's move generation
+            uci_move, retries = self._get_model_move(board, temperature)
+            
+            position_result = {
+                "fen": board.fen(),
+                "move_number": len(board.move_stack),
+                "legal": uci_move is not None,
+                "retries": retries,
+            }
+            results["positions"].append(position_result)
+            
+            if uci_move is not None:
+                if retries == 0:
+                    results["legal_first_try"] += 1
+                else:
+                    results["legal_with_retry"] += 1
+            else:
+                results["illegal_all_retries"] += 1
+            
+            if verbose and (i + 1) % 100 == 0:
+                legal_rate = (results["legal_first_try"] + results["legal_with_retry"]) / results["total_positions"]
+                print(f"  Positions: {i + 1}/{n_positions} | Legal rate: {legal_rate:.1%}")
+        
+        # Calculate statistics
+        total = results["total_positions"]
+        if total > 0:
+            results["legal_rate_first_try"] = results["legal_first_try"] / total
+            results["legal_rate_with_retry"] = (results["legal_first_try"] + results["legal_with_retry"]) / total
+            results["illegal_rate"] = results["illegal_all_retries"] / total
+        else:
+            results["legal_rate_first_try"] = 0
+            results["legal_rate_with_retry"] = 0
+            results["illegal_rate"] = 1
+        
+        return results
+    
+    def evaluate(
+        self,
+        n_games: int = 100,
+        temperature: float = 0.7,
+        verbose: bool = True,
+    ) -> dict:
+        """
+        Run a full win-rate evaluation of the model against Stockfish.
+        
+        Args:
+            n_games: Number of games to play.
+            temperature: Sampling temperature.
+            verbose: Whether to print progress.
+        
+        Returns:
+            Dictionary with evaluation metrics.
+        """
+        results = {
+            "wins": 0,
+            "losses": 0,
+            "draws": 0,
+            "illegal_moves": 0,
+            "total_moves": 0,
+            "games": [],
+        }
+        
+        for i in range(n_games):
+            # Alternate colors
+            model_color = "white" if i % 2 == 0 else "black"
+            
+            game = self.play_game(
+                model_color=model_color,
+                temperature=temperature,
+            )
+            
+            results["games"].append(game)
+            results["total_moves"] += len(game.moves)
+            results["illegal_moves"] += game.illegal_move_count
+            
+            # Count result
+            if game.result == "1/2-1/2":
+                results["draws"] += 1
+            elif (game.result == "1-0" and model_color == "white") or \
+                 (game.result == "0-1" and model_color == "black"):
+                results["wins"] += 1
+            else:
+                results["losses"] += 1
+            
+            if verbose and (i + 1) % 10 == 0:
+                print(f"  Games: {i + 1}/{n_games} | "
+                      f"W: {results['wins']} L: {results['losses']} D: {results['draws']}")
+        
+        # Calculate statistics
+        total = results["wins"] + results["losses"] + results["draws"]
+        results["win_rate"] = results["wins"] / total if total > 0 else 0
+        results["draw_rate"] = results["draws"] / total if total > 0 else 0
+        results["loss_rate"] = results["losses"] / total if total > 0 else 0
+
+        total_attempts = results["total_moves"] + results["illegal_moves"]
+
+        # Average length counts both legal moves and illegal attempts so early illegal terminations
+        # don't show as near-zero length games.
+        results["avg_game_length"] = total_attempts / total if total > 0 else 0
+
+        # Illegal move rate: illegal attempts over total attempts
+        results["illegal_move_rate"] = results["illegal_moves"] / total_attempts if total_attempts > 0 else 0
+        
+        # Estimate ELO (simplified)
+        # Stockfish Level 1 is approximately 1350 ELO
+        stockfish_elo = 1350
+        if results["win_rate"] > 0 or results["loss_rate"] > 0:
+            score = results["wins"] + 0.5 * results["draws"]
+            expected = total * 0.5  # Expected score against equal opponent
+            
+            # Simple ELO estimation
+            if score > 0:
+                win_ratio = score / total
+                if win_ratio > 0 and win_ratio < 1:
+                    elo_diff = -400 * (1 - 2 * win_ratio) / (1 if win_ratio > 0.5 else -1)
+                    results["estimated_elo"] = stockfish_elo + elo_diff
+                else:
+                    results["estimated_elo"] = stockfish_elo + (400 if win_ratio >= 1 else -400)
+            else:
+                results["estimated_elo"] = stockfish_elo - 400
+        else:
+            results["estimated_elo"] = None
+        
+        return results
+
+
+def load_model_from_hub(model_id: str, device: str = "auto"):
+    """
+    Load a model from the Hugging Face Hub.
+    
+    Args:
+        model_id: Model ID on Hugging Face Hub.
+        device: Device to load the model on.
+    
+    Returns:
+        Tuple of (model, tokenizer).
+    """
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+    
+    # Import to register custom classes
+    from src.model import ChessConfig, ChessForCausalLM
+    
+    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+    model = AutoModelForCausalLM.from_pretrained(
+        model_id,
+        trust_remote_code=True,
+        device_map=device,
+    )
+    
+    return model, tokenizer
+
+
+def main():
+    """Main evaluation function."""
+    parser = argparse.ArgumentParser(description="Evaluate a chess model")
+    
+    parser.add_argument(
+        "--model_path", type=str, required=True,
+        help="Path to the model or Hugging Face model ID"
+    )
+    parser.add_argument(
+        "--mode", type=str, default="legal", choices=["legal", "winrate", "both"],
+        help="Evaluation mode: 'legal' for legal move rate, 'winrate' for games, 'both' for both"
+    )
+    parser.add_argument(
+        "--stockfish_path", type=str, default=None,
+        help="Path to Stockfish executable"
+    )
+    parser.add_argument(
+        "--stockfish_level", type=int, default=1,
+        help="Stockfish skill level (0-20)"
+    )
+    parser.add_argument(
+        "--n_positions", type=int, default=500,
+        help="Number of positions for legal move evaluation"
+    )
+    parser.add_argument(
+        "--n_games", type=int, default=100,
+        help="Number of games to play for win rate evaluation"
+    )
+    parser.add_argument(
+        "--temperature", type=float, default=0.7,
+        help="Sampling temperature"
+    )
+    
+    args = parser.parse_args()
+    
+    print("=" * 60)
+    print("CHESS CHALLENGE - EVALUATION")
+    print("=" * 60)
+    
+    # Load model
+    print(f"\nLoading model from: {args.model_path}")
+    
+    if "/" in args.model_path and not args.model_path.startswith("."):
+        # Assume Hugging Face model ID
+        model, tokenizer = load_model_from_hub(args.model_path)
+    else:
+        # Local path
+        from transformers import AutoModelForCausalLM
+        from src.tokenizer import ChessTokenizer
+        from src.model import ChessConfig, ChessForCausalLM
+        
+        tokenizer = ChessTokenizer.from_pretrained(args.model_path)
+        model = AutoModelForCausalLM.from_pretrained(args.model_path)
+    
+    # Create evaluator
+    print(f"\nSetting up evaluator...")
+    evaluator = ChessEvaluator(
+        model=model,
+        tokenizer=tokenizer,
+        stockfish_path=args.stockfish_path,
+        stockfish_level=args.stockfish_level,
+    )
+    
+    # Run legal move evaluation
+    if args.mode in ["legal", "both"]:
+        print(f"\n" + "=" * 60)
+        print("PHASE 1: LEGAL MOVE EVALUATION")
+        print("=" * 60)
+        print(f"Testing {args.n_positions} random positions...")
+        
+        legal_results = evaluator.evaluate_legal_moves(
+            n_positions=args.n_positions,
+            temperature=args.temperature,
+            verbose=True,
+        )
+        
+        print("\n" + "-" * 40)
+        print("LEGAL MOVE RESULTS")
+        print("-" * 40)
+        print(f"  Positions tested:     {legal_results['total_positions']}")
+        print(f"  Legal (1st try):      {legal_results['legal_first_try']} ({legal_results['legal_rate_first_try']:.1%})")
+        print(f"  Legal (with retry):   {legal_results['legal_first_try'] + legal_results['legal_with_retry']} ({legal_results['legal_rate_with_retry']:.1%})")
+        print(f"  Always illegal:       {legal_results['illegal_all_retries']} ({legal_results['illegal_rate']:.1%})")
+    
+    # Run win rate evaluation
+    if args.mode in ["winrate", "both"]:
+        print(f"\n" + "=" * 60)
+        print("PHASE 2: WIN RATE EVALUATION")
+        print("=" * 60)
+        print(f"Playing {args.n_games} games against Stockfish (Level {args.stockfish_level})...")
+        
+        winrate_results = evaluator.evaluate(
+            n_games=args.n_games,
+            temperature=args.temperature,
+            verbose=True,
+        )
+        
+        print("\n" + "-" * 40)
+        print("WIN RATE RESULTS")
+        print("-" * 40)
+        print(f"  Wins:   {winrate_results['wins']}")
+        print(f"  Losses: {winrate_results['losses']}")
+        print(f"  Draws:  {winrate_results['draws']}")
+        print(f"\n  Win Rate:  {winrate_results['win_rate']:.1%}")
+        print(f"  Draw Rate: {winrate_results['draw_rate']:.1%}")
+        print(f"  Loss Rate: {winrate_results['loss_rate']:.1%}")
+        print(f"\n  Avg Game Length: {winrate_results['avg_game_length']:.1f} moves")
+        print(f"  Illegal Move Rate: {winrate_results['illegal_move_rate']:.2%}")
+        
+        if winrate_results["estimated_elo"]:
+            print(f"\n  Estimated ELO: {winrate_results['estimated_elo']:.0f}")
+    
+    print("\n" + "=" * 60)
+    print("EVALUATION COMPLETE")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    main()

src/model.py

@@ -0,0 +1,444 @@
+"""
+Chess Transformer Model for the Chess Challenge.
+
+This module provides a simple GPT-style transformer architecture
+designed to fit within the 1M parameter constraint.
+
+Key components:
+- ChessConfig: Configuration class for model hyperparameters
+- ChessForCausalLM: The main model class for next-move prediction
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+from typing import Optional, Tuple, Union
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import PretrainedConfig, PreTrainedModel
+from transformers.modeling_outputs import CausalLMOutputWithPast
+
+
+class ChessConfig(PretrainedConfig):
+    """
+    Configuration class for the Chess Transformer model.
+    
+    This configuration is designed for a ~1M parameter model.
+    Students can adjust these values to explore different architectures.
+    
+    Parameter budget breakdown (with default values):
+    - Embeddings (vocab): 1200 x 128 = 153,600
+    - Position Embeddings: 256 x 128 = 32,768
+    - Transformer Layers: 6 x ~120,000 = ~720,000
+    - LM Head (with weight tying): 0 (shared with embeddings)
+    - Total: ~906,000 parameters
+    
+    Attributes:
+        vocab_size: Size of the vocabulary (number of unique moves).
+        n_embd: Embedding dimension (d_model).
+        n_layer: Number of transformer layers.
+        n_head: Number of attention heads.
+        n_ctx: Maximum sequence length (context window).
+        n_inner: Feed-forward inner dimension (default: 3 * n_embd).
+        dropout: Dropout probability.
+        layer_norm_epsilon: Epsilon for layer normalization.
+        tie_weights: Whether to tie embedding and output weights.
+    """
+    
+    model_type = "chess_transformer"
+    
+    def __init__(
+        self,
+        vocab_size: int = 1200,
+        n_embd: int = 128,
+        n_layer: int = 6,
+        n_head: int = 4,
+        n_ctx: int = 256,
+        n_inner: Optional[int] = None,
+        dropout: float = 0.1,
+        layer_norm_epsilon: float = 1e-5,
+        tie_weights: bool = True,
+        pad_token_id: int = 0,
+        bos_token_id: int = 1,
+        eos_token_id: int = 2,
+        **kwargs,
+    ):
+        super().__init__(
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            eos_token_id=eos_token_id,
+            **kwargs,
+        )
+        
+        self.vocab_size = vocab_size
+        self.n_embd = n_embd
+        self.n_layer = n_layer
+        self.n_head = n_head
+        self.n_ctx = n_ctx
+        self.n_inner = n_inner if n_inner is not None else 3 * n_embd  # Reduced from 4x to 3x
+        self.dropout = dropout
+        self.layer_norm_epsilon = layer_norm_epsilon
+        self.tie_weights = tie_weights
+        # Inform HF base class about tying behavior
+        self.tie_word_embeddings = bool(tie_weights)
+
+
+class MultiHeadAttention(nn.Module):
+    """
+    Multi-head self-attention module.
+    
+    This is a standard scaled dot-product attention implementation
+    with causal masking for autoregressive generation.
+    """
+    
+    def __init__(self, config: ChessConfig):
+        super().__init__()
+        
+        assert config.n_embd % config.n_head == 0, \
+            f"n_embd ({config.n_embd}) must be divisible by n_head ({config.n_head})"
+        
+        self.n_head = config.n_head
+        self.n_embd = config.n_embd
+        self.head_dim = config.n_embd // config.n_head
+        
+        # Combined QKV projection for efficiency
+        self.c_attn = nn.Linear(config.n_embd, 3 * config.n_embd)
+        self.c_proj = nn.Linear(config.n_embd, config.n_embd)
+        
+        self.dropout = nn.Dropout(config.dropout)
+        
+        # Causal mask (will be created on first forward pass)
+        self.register_buffer(
+            "bias",
+            torch.tril(torch.ones(config.n_ctx, config.n_ctx)).view(
+                1, 1, config.n_ctx, config.n_ctx
+            ),
+            persistent=False,
+        )
+    
+    def forward(
+        self,
+        x: torch.Tensor,
+        attention_mask: Optional[torch.Tensor] = None,
+    ) -> torch.Tensor:
+        batch_size, seq_len, _ = x.size()
+        
+        # Compute Q, K, V
+        qkv = self.c_attn(x)
+        q, k, v = qkv.split(self.n_embd, dim=2)
+        
+        # Reshape for multi-head attention
+        q = q.view(batch_size, seq_len, self.n_head, self.head_dim).transpose(1, 2)
+        k = k.view(batch_size, seq_len, self.n_head, self.head_dim).transpose(1, 2)
+        v = v.view(batch_size, seq_len, self.n_head, self.head_dim).transpose(1, 2)
+        
+        # Scaled dot-product attention
+        attn_weights = torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(self.head_dim)
+        
+        # Apply causal mask
+        causal_mask = self.bias[:, :, :seq_len, :seq_len]
+        attn_weights = attn_weights.masked_fill(causal_mask == 0, float("-inf"))
+        
+        # Apply attention mask (for padding)
+        if attention_mask is not None:
+            # attention_mask shape: (batch_size, seq_len) -> (batch_size, 1, 1, seq_len)
+            attention_mask = attention_mask.unsqueeze(1).unsqueeze(2)
+            attn_weights = attn_weights.masked_fill(attention_mask == 0, float("-inf"))
+        
+        attn_weights = F.softmax(attn_weights, dim=-1)
+        attn_weights = self.dropout(attn_weights)
+        
+        # Apply attention to values
+        attn_output = torch.matmul(attn_weights, v)
+        
+        # Reshape back
+        attn_output = attn_output.transpose(1, 2).contiguous().view(
+            batch_size, seq_len, self.n_embd
+        )
+        
+        # Output projection
+        attn_output = self.c_proj(attn_output)
+        
+        return attn_output
+
+
+class FeedForward(nn.Module):
+    """
+    Feed-forward network (MLP) module.
+    
+    Standard two-layer MLP with GELU activation.
+    """
+    
+    def __init__(self, config: ChessConfig):
+        super().__init__()
+        
+        self.c_fc = nn.Linear(config.n_embd, config.n_inner)
+        self.c_proj = nn.Linear(config.n_inner, config.n_embd)
+        self.dropout = nn.Dropout(config.dropout)
+    
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = self.c_fc(x)
+        x = F.gelu(x)
+        x = self.c_proj(x)
+        x = self.dropout(x)
+        return x
+
+
+class TransformerBlock(nn.Module):
+    """
+    A single transformer block with attention and feed-forward layers.
+    
+    Uses pre-normalization (LayerNorm before attention/FFN) for better
+    training stability.
+    """
+    
+    def __init__(self, config: ChessConfig):
+        super().__init__()
+        
+        self.ln_1 = nn.LayerNorm(config.n_embd, eps=config.layer_norm_epsilon)
+        self.attn = MultiHeadAttention(config)
+        self.ln_2 = nn.LayerNorm(config.n_embd, eps=config.layer_norm_epsilon)
+        self.mlp = FeedForward(config)
+    
+    def forward(
+        self,
+        x: torch.Tensor,
+        attention_mask: Optional[torch.Tensor] = None,
+    ) -> torch.Tensor:
+        # Pre-norm attention
+        x = x + self.attn(self.ln_1(x), attention_mask=attention_mask)
+        # Pre-norm FFN
+        x = x + self.mlp(self.ln_2(x))
+        return x
+
+
+class ChessForCausalLM(PreTrainedModel):
+    """
+    Chess Transformer for Causal Language Modeling (next-move prediction).
+    
+    This model is designed to predict the next chess move given a sequence
+    of previous moves. It uses a GPT-style architecture with:
+    - Token embeddings for chess moves
+    - Learned positional embeddings
+    - Stacked transformer blocks
+    - Linear head for next-token prediction
+    
+    The model supports weight tying between the embedding layer and the
+    output projection to save parameters.
+    
+    Example:
+        >>> config = ChessConfig(vocab_size=1200, n_embd=128, n_layer=6)
+        >>> model = ChessForCausalLM(config)
+        >>> inputs = {"input_ids": torch.tensor([[1, 42, 87]])}
+        >>> outputs = model(**inputs)
+        >>> next_move_logits = outputs.logits[:, -1, :]
+    """
+    
+    config_class = ChessConfig
+    base_model_prefix = "transformer"
+    supports_gradient_checkpointing = True
+    # Suppress missing-key warning for tied lm_head when loading
+    keys_to_ignore_on_load_missing = ["lm_head.weight"]
+    
+    def __init__(self, config: ChessConfig):
+        super().__init__(config)
+        
+        # Token and position embeddings
+        self.wte = nn.Embedding(config.vocab_size, config.n_embd)
+        self.wpe = nn.Embedding(config.n_ctx, config.n_embd)
+        
+        self.drop = nn.Dropout(config.dropout)
+        
+        # Transformer blocks
+        self.h = nn.ModuleList([
+            TransformerBlock(config) for _ in range(config.n_layer)
+        ])
+        
+        # Final layer norm
+        self.ln_f = nn.LayerNorm(config.n_embd, eps=config.layer_norm_epsilon)
+        
+        # Output head
+        self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False)
+        
+        # Declare tied weights for proper serialization
+        if config.tie_weights:
+            self._tied_weights_keys = ["lm_head.weight"]
+        
+        # Initialize weights
+        self.post_init()
+        
+        # Tie weights if configured
+        if config.tie_weights:
+            self.tie_weights()
+
+    def get_input_embeddings(self) -> nn.Module:
+        return self.wte
+
+    def set_input_embeddings(self, new_embeddings: nn.Module):
+        self.wte = new_embeddings
+        if getattr(self.config, "tie_weights", False):
+            self.tie_weights()
+
+    def get_output_embeddings(self) -> nn.Module:
+        return self.lm_head
+
+    def set_output_embeddings(self, new_embeddings: nn.Module):
+        self.lm_head = new_embeddings
+
+    def tie_weights(self):
+        # Use HF helper to tie or clone depending on config
+        if getattr(self.config, "tie_weights", False) or getattr(self.config, "tie_word_embeddings", False):
+            self._tie_or_clone_weights(self.lm_head, self.wte)
+    
+    def _init_weights(self, module: nn.Module):
+        """Initialize weights following GPT-2 style."""
+        if isinstance(module, nn.Linear):
+            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
+            if module.bias is not None:
+                torch.nn.init.zeros_(module.bias)
+        elif isinstance(module, nn.Embedding):
+            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
+        elif isinstance(module, nn.LayerNorm):
+            torch.nn.init.ones_(module.weight)
+            torch.nn.init.zeros_(module.bias)
+    
+    def forward(
+        self,
+        input_ids: torch.LongTensor,
+        attention_mask: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        labels: Optional[torch.LongTensor] = None,
+        return_dict: Optional[bool] = None,
+        **kwargs,
+    ) -> Union[Tuple, CausalLMOutputWithPast]:
+        """
+        Forward pass of the model.
+        
+        Args:
+            input_ids: Token IDs of shape (batch_size, seq_len).
+            attention_mask: Attention mask of shape (batch_size, seq_len).
+            position_ids: Position IDs of shape (batch_size, seq_len).
+            labels: Labels for language modeling loss.
+            return_dict: Whether to return a ModelOutput object.
+        
+        Returns:
+            CausalLMOutputWithPast containing loss (if labels provided) and logits.
+        """
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+        
+        batch_size, seq_len = input_ids.size()
+        device = input_ids.device
+        
+        # Create position IDs if not provided
+        if position_ids is None:
+            position_ids = torch.arange(seq_len, device=device).unsqueeze(0).expand(batch_size, -1)
+        
+        # Get embeddings
+        # DEBUG: Check indices
+        if input_ids.max() >= self.config.vocab_size:
+            raise ValueError(f"Input ID {input_ids.max()} exceeds vocab size {self.config.vocab_size}")
+        if input_ids.min() < 0:
+             raise ValueError(f"Input ID {input_ids.min()} is negative")
+
+        token_embeds = self.wte(input_ids)
+        position_embeds = self.wpe(position_ids)
+        hidden_states = self.drop(token_embeds + position_embeds)
+        
+        # Pass through transformer blocks
+        for block in self.h:
+            hidden_states = block(hidden_states, attention_mask=attention_mask)
+        
+        # Final layer norm
+        hidden_states = self.ln_f(hidden_states)
+        
+        # Get logits
+        logits = self.lm_head(hidden_states)
+        
+        # Compute loss if labels are provided
+        loss = None
+        if labels is not None:
+            # Shift logits and labels for next-token prediction
+            shift_logits = logits[..., :-1, :].contiguous()
+            shift_labels = labels[..., 1:].contiguous()
+            
+            # Flatten for cross-entropy
+            # Note: src/data.py uses -100 for ignored labels
+            loss_fct = nn.CrossEntropyLoss(ignore_index=-100)
+            loss = loss_fct(
+                shift_logits.view(-1, shift_logits.size(-1)),
+                shift_labels.view(-1),
+            )
+        
+        if not return_dict:
+            output = (logits,)
+            return ((loss,) + output) if loss is not None else output
+        
+        return CausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=None,
+            hidden_states=None,
+            attentions=None,
+        )
+    
+    @torch.no_grad()
+    def generate_move(
+        self,
+        input_ids: torch.LongTensor,
+        temperature: float = 1.0,
+        top_k: Optional[int] = None,
+        top_p: Optional[float] = None,
+    ) -> int:
+        """
+        Generate the next move given a sequence of moves.
+        
+        Args:
+            input_ids: Token IDs of shape (1, seq_len).
+            temperature: Sampling temperature (1.0 = no change).
+            top_k: If set, only sample from top k tokens.
+            top_p: If set, use nucleus sampling with this threshold.
+        
+        Returns:
+            The token ID of the predicted next move.
+        """
+        self.eval()
+        
+        # Get logits for the last position
+        outputs = self(input_ids)
+        logits = outputs.logits[:, -1, :] / temperature
+        
+        # Apply top-k filtering
+        if top_k is not None:
+            indices_to_remove = logits < torch.topk(logits, top_k)[0][..., -1, None]
+            logits[indices_to_remove] = float("-inf")
+        
+        # Apply top-p (nucleus) filtering
+        if top_p is not None:
+            sorted_logits, sorted_indices = torch.sort(logits, descending=True)
+            cumulative_probs = torch.cumsum(F.softmax(sorted_logits, dim=-1), dim=-1)
+            
+            # Remove tokens with cumulative probability above the threshold
+            sorted_indices_to_remove = cumulative_probs > top_p
+            sorted_indices_to_remove[..., 1:] = sorted_indices_to_remove[..., :-1].clone()
+            sorted_indices_to_remove[..., 0] = 0
+            
+            indices_to_remove = sorted_indices_to_remove.scatter(
+                dim=-1, index=sorted_indices, src=sorted_indices_to_remove
+            )
+            logits[indices_to_remove] = float("-inf")
+        
+        # Sample from the distribution
+        probs = F.softmax(logits, dim=-1)
+        next_token = torch.multinomial(probs, num_samples=1)
+        
+        return next_token.item()
+
+
+# Register the model with Auto classes for easy loading
+from transformers import AutoConfig, AutoModelForCausalLM
+
+AutoConfig.register("chess_transformer", ChessConfig)
+AutoModelForCausalLM.register(ChessConfig, ChessForCausalLM)

src/tokenizer.py

@@ -0,0 +1,278 @@
+"""
+Custom Chess Tokenizer for the Chess Challenge.
+
+This tokenizer treats each move as a single token using the extended UCI notation
+from the Lichess dataset (e.g., WPe2e4, BNg8f6).
+
+The dataset format uses:
+- W/B prefix for White/Black
+- Piece letter: P=Pawn, N=Knight, B=Bishop, R=Rook, Q=Queen, K=King
+- Source and destination squares (e.g., e2e4)
+- Special suffixes: (x)=capture, (+)=check, (+*)=checkmate, (o)/(O)=castling
+"""
+
+from __future__ import annotations
+
+import json
+import os
+from pathlib import Path
+from typing import Dict, List, Optional
+
+from transformers import PreTrainedTokenizer
+
+
+class ChessTokenizer(PreTrainedTokenizer):
+    """
+    A custom tokenizer for chess moves using extended UCI notation.
+    
+    This tokenizer maps each possible chess move to a unique token ID.
+    The vocabulary is built from the training dataset to ensure all moves
+    encountered during training have a corresponding token.
+    
+    Example:
+        >>> tokenizer = ChessTokenizer()
+        >>> tokenizer.encode("WPe2e4 BPe7e5")
+        [1, 42, 87, 2]  # [BOS, e2e4, e7e5, EOS]
+    """
+    
+    model_input_names = ["input_ids", "attention_mask"]
+    vocab_files_names = {"vocab_file": "vocab.json"}
+    
+    # Special tokens
+    PAD_TOKEN = "[PAD]"
+    BOS_TOKEN = "[BOS]"
+    EOS_TOKEN = "[EOS]"
+    UNK_TOKEN = "[UNK]"
+    
+    def __init__(
+        self,
+        vocab_file: Optional[str] = None,
+        vocab: Optional[Dict[str, int]] = None,
+        **kwargs,
+    ):
+        """
+        Initialize the chess tokenizer.
+        
+        Args:
+            vocab_file: Path to a JSON file containing the vocabulary mapping.
+            vocab: Dictionary mapping tokens to IDs (alternative to vocab_file).
+            **kwargs: Additional arguments passed to PreTrainedTokenizer.
+        """
+        # Initialize special tokens
+        self._pad_token = self.PAD_TOKEN
+        self._bos_token = self.BOS_TOKEN
+        self._eos_token = self.EOS_TOKEN
+        self._unk_token = self.UNK_TOKEN
+
+        # Remove any duplicate special-token entries passed through kwargs
+        # to avoid "multiple values for keyword" errors when loading from disk.
+        kwargs.pop("pad_token", None)
+        kwargs.pop("bos_token", None)
+        kwargs.pop("eos_token", None)
+        kwargs.pop("unk_token", None)
+        
+        # Load or create vocabulary
+        if vocab is not None:
+            self._vocab = vocab
+        elif vocab_file is not None and os.path.exists(vocab_file):
+            with open(vocab_file, "r", encoding="utf-8") as f:
+                self._vocab = json.load(f)
+        else:
+            # Create a minimal vocabulary with just special tokens
+            # The full vocabulary should be built from the dataset
+            self._vocab = self._create_default_vocab()
+        
+        # Create reverse mapping
+        self._ids_to_tokens = {v: k for k, v in self._vocab.items()}
+        
+        # Call parent init AFTER setting up vocab
+        super().__init__(
+            pad_token=self._pad_token,
+            bos_token=self._bos_token,
+            eos_token=self._eos_token,
+            unk_token=self._unk_token,
+            **kwargs,
+        )
+    
+    def _create_default_vocab(self) -> Dict[str, int]:
+        """
+        Create a minimal default vocabulary with just special tokens.
+        
+        For the full vocabulary, use `build_vocab_from_dataset()`.
+        This minimal vocab is just a placeholder - you should build from data.
+        """
+        special_tokens = [self.PAD_TOKEN, self.BOS_TOKEN, self.EOS_TOKEN, self.UNK_TOKEN]
+        vocab = {token: idx for idx, token in enumerate(special_tokens)}
+        return vocab
+    
+    @classmethod
+    def build_vocab_from_iterator(
+        cls,
+        iterator,
+        min_frequency: int = 1,
+    ) -> "ChessTokenizer":
+        """
+        Build a tokenizer vocabulary from an iterator of game strings.
+        
+        Args:
+            iterator: An iterator yielding game strings (space-separated moves).
+            min_frequency: Minimum frequency for a token to be included.
+        
+        Returns:
+            A ChessTokenizer with the built vocabulary.
+        """
+        from collections import Counter
+        
+        token_counts = Counter()
+        
+        for game in iterator:
+            moves = game.strip().split()
+            token_counts.update(moves)
+        
+        # Filter by frequency
+        tokens = [
+            token for token, count in token_counts.items()
+            if count >= min_frequency
+        ]
+        
+        # Sort for reproducibility
+        tokens = sorted(tokens)
+        
+        # Build vocabulary
+        special_tokens = [cls.PAD_TOKEN, cls.BOS_TOKEN, cls.EOS_TOKEN, cls.UNK_TOKEN]
+        vocab = {token: idx for idx, token in enumerate(special_tokens + tokens)}
+        
+        return cls(vocab=vocab)
+    
+    @classmethod
+    def build_vocab_from_dataset(
+        cls,
+        dataset_name: str = "dlouapre/lichess_2025-01_1M",
+        split: str = "train",
+        column: str = "text",
+        min_frequency: int = 500,
+        max_samples: Optional[int] = 100000,
+    ) -> "ChessTokenizer":
+        """
+        Build a tokenizer vocabulary from a Hugging Face dataset.
+        
+        Args:
+            dataset_name: Name of the dataset on Hugging Face Hub.
+            split: Dataset split to use.
+            column: Column containing the game strings.
+            min_frequency: Minimum frequency for a token to be included (default: 500).
+            max_samples: Maximum number of samples to process (default: 100k).
+        
+        Returns:
+            A ChessTokenizer with the built vocabulary.
+        """
+        from datasets import load_dataset
+        
+        dataset = load_dataset(dataset_name, split=split)
+        
+        if max_samples is not None:
+            dataset = dataset.select(range(min(max_samples, len(dataset))))
+        
+        def game_iterator():
+            for example in dataset:
+                yield example[column]
+        
+        return cls.build_vocab_from_iterator(game_iterator(), min_frequency=min_frequency)
+    
+    @property
+    def vocab_size(self) -> int:
+        """Return the size of the vocabulary."""
+        return len(self._vocab)
+    
+    def get_vocab(self) -> Dict[str, int]:
+        """Return the vocabulary as a dictionary."""
+        return dict(self._vocab)
+    
+    def _tokenize(self, text: str) -> List[str]:
+        """
+        Tokenize a string of moves into a list of tokens.
+        
+        Args:
+            text: A string of space-separated moves.
+        
+        Returns:
+            List of move tokens.
+        """
+        return text.strip().split()
+    
+    def _convert_token_to_id(self, token: str) -> int:
+        """Convert a token to its ID."""
+        return self._vocab.get(token, self._vocab.get(self.UNK_TOKEN, 0))
+    
+    def _convert_id_to_token(self, index: int) -> str:
+        """Convert an ID to its token."""
+        return self._ids_to_tokens.get(index, self.UNK_TOKEN)
+    
+    def convert_tokens_to_string(self, tokens: List[str]) -> str:
+        """Convert a list of tokens back to a string."""
+        # Filter out special tokens for cleaner output
+        special = {self.PAD_TOKEN, self.BOS_TOKEN, self.EOS_TOKEN, self.UNK_TOKEN}
+        return " ".join(t for t in tokens if t not in special)
+    
+    def save_vocabulary(
+        self,
+        save_directory: str,
+        filename_prefix: Optional[str] = None,
+    ) -> tuple:
+        """
+        Save the vocabulary to a JSON file.
+        
+        Args:
+            save_directory: Directory to save the vocabulary.
+            filename_prefix: Optional prefix for the filename.
+        
+        Returns:
+            Tuple containing the path to the saved vocabulary file.
+        """
+        if not os.path.isdir(save_directory):
+            os.makedirs(save_directory, exist_ok=True)
+        
+        vocab_file = os.path.join(
+            save_directory,
+            (filename_prefix + "-" if filename_prefix else "") + "vocab.json",
+        )
+        
+        with open(vocab_file, "w", encoding="utf-8") as f:
+            json.dump(self._vocab, f, ensure_ascii=False, indent=2)
+        
+        return (vocab_file,)
+
+
+def count_vocab_from_dataset(
+    dataset_name: str = "dlouapre/lichess_2025-01_1M",
+    split: str = "train",
+    column: str = "text",
+    max_samples: Optional[int] = 10000,
+) -> Dict[str, int]:
+    """
+    Count token frequencies in a dataset (useful for vocabulary analysis).
+    
+    Args:
+        dataset_name: Name of the dataset on Hugging Face Hub.
+        split: Dataset split to use.
+        column: Column containing the game strings.
+        max_samples: Maximum number of samples to process.
+    
+    Returns:
+        Dictionary mapping tokens to their frequencies.
+    """
+    from collections import Counter
+    from datasets import load_dataset
+    
+    dataset = load_dataset(dataset_name, split=split)
+    
+    if max_samples is not None:
+        dataset = dataset.select(range(min(max_samples, len(dataset))))
+    
+    token_counts = Counter()
+    
+    for example in dataset:
+        moves = example[column].strip().split()
+        token_counts.update(moves)
+    
+    return dict(token_counts)

src/train.py

@@ -0,0 +1,247 @@
+"""
+Training script for the Chess Challenge.
+
+This script provides a complete training pipeline using the Hugging Face Trainer.
+Students can modify this script to experiment with different training strategies.
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+from pathlib import Path
+
+import torch
+from transformers import (
+    Trainer,
+    TrainingArguments,
+    set_seed,
+)
+
+from src.data import ChessDataCollator, create_train_val_datasets
+from src.model import ChessConfig, ChessForCausalLM
+from src.tokenizer import ChessTokenizer
+from src.utils import count_parameters, print_parameter_budget
+
+
+def parse_args():
+    """Parse command line arguments."""
+    parser = argparse.ArgumentParser(
+        description="Train a chess-playing language model"
+    )
+    
+    # Model arguments
+    parser.add_argument(
+        "--vocab_size", type=int, default=1200,
+        help="Vocabulary size"
+    )
+    parser.add_argument(
+        "--n_embd", type=int, default=128,
+        help="Embedding dimension"
+    )
+    parser.add_argument(
+        "--n_layer", type=int, default=4,
+        help="Number of transformer layers"
+    )
+    parser.add_argument(
+        "--n_head", type=int, default=4,
+        help="Number of attention heads"
+    )
+    parser.add_argument(
+        "--n_ctx", type=int, default=256,
+        help="Maximum context length"
+    )
+    parser.add_argument(
+        "--n_inner", type=int, default=None,
+        help="Feed-forward inner dimension (default: 4 * n_embd)"
+    )
+    parser.add_argument(
+        "--dropout", type=float, default=0.1,
+        help="Dropout probability"
+    )
+    parser.add_argument(
+        "--no_tie_weights", action="store_true",
+        help="Disable weight tying between embedding and output layers"
+    )
+    
+    # Data arguments
+    parser.add_argument(
+        "--dataset_name", type=str, default="dlouapre/lichess_2025-01_1M",
+        help="Name of the dataset on Hugging Face Hub"
+    )
+    parser.add_argument(
+        "--max_train_samples", type=int, default=None,
+        help="Maximum number of training samples"
+    )
+    parser.add_argument(
+        "--val_samples", type=int, default=5000,
+        help="Number of validation samples"
+    )
+    
+    # Training arguments
+    parser.add_argument(
+        "--output_dir", type=str, default="./output",
+        help="Output directory for model and logs"
+    )
+    parser.add_argument(
+        "--num_train_epochs", type=int, default=3,
+        help="Number of training epochs"
+    )
+    parser.add_argument(
+        "--per_device_train_batch_size", type=int, default=32,
+        help="Training batch size per device"
+    )
+    parser.add_argument(
+        "--per_device_eval_batch_size", type=int, default=64,
+        help="Evaluation batch size per device"
+    )
+    parser.add_argument(
+        "--learning_rate", type=float, default=5e-4,
+        help="Learning rate"
+    )
+    parser.add_argument(
+        "--weight_decay", type=float, default=0.01,
+        help="Weight decay"
+    )
+    parser.add_argument(
+        "--warmup_ratio", type=float, default=0.1,
+        help="Warmup ratio"
+    )
+    parser.add_argument(
+        "--seed", type=int, default=42,
+        help="Random seed"
+    )
+    
+    # Logging arguments
+    parser.add_argument(
+        "--logging_steps", type=int, default=100,
+        help="Logging frequency"
+    )
+    parser.add_argument(
+        "--eval_steps", type=int, default=500,
+        help="Evaluation frequency"
+    )
+    parser.add_argument(
+        "--save_steps", type=int, default=1000,
+        help="Checkpoint saving frequency"
+    )
+    
+    return parser.parse_args()
+
+
+def main():
+    """Main training function."""
+    args = parse_args()
+    
+    # Set seed for reproducibility
+    set_seed(args.seed)
+    
+    print("=" * 60)
+    print("CHESS CHALLENGE - TRAINING")
+    print("=" * 60)
+    
+    # Build tokenizer from dataset
+    print("\nBuilding tokenizer from dataset...")
+    tokenizer = ChessTokenizer.build_vocab_from_dataset(
+        dataset_name=args.dataset_name,
+        min_frequency=500,  # Only keep moves that appear at least 500 times
+        max_samples=100000,  # Use 100k games to build vocabulary
+    )
+    print(f"   Vocabulary size: {tokenizer.vocab_size}")
+    
+    # Use the vocab size from tokenizer (override args if provided)
+    actual_vocab_size = tokenizer.vocab_size
+    
+    # Create model configuration
+    print("\nCreating model configuration...")
+    config = ChessConfig(
+        vocab_size=actual_vocab_size,
+        n_embd=args.n_embd,
+        n_layer=args.n_layer,
+        n_head=args.n_head,
+        n_ctx=args.n_ctx,
+        n_inner=args.n_inner,
+        dropout=args.dropout,
+        tie_weights=not args.no_tie_weights,
+        pad_token_id=tokenizer.pad_token_id,
+        bos_token_id=tokenizer.bos_token_id,
+        eos_token_id=tokenizer.eos_token_id,
+    )
+    
+    # Print parameter budget
+    print_parameter_budget(config)
+    
+    # Create model
+    print("\nCreating model...")
+    model = ChessForCausalLM(config)
+    n_params = count_parameters(model)
+    print(f"   Total parameters: {n_params:,}")
+    
+    if n_params > 1_000_000:
+        print("WARNING: Model exceeds 1M parameter limit!")
+    else:
+        print("✓  Model is within 1M parameter limit")
+    
+    # Load datasets
+    print("\nLoading datasets...")
+    train_dataset, val_dataset = create_train_val_datasets(
+        tokenizer=tokenizer,
+        dataset_name=args.dataset_name,
+        max_length=args.n_ctx,
+        train_samples=args.max_train_samples,
+        val_samples=args.val_samples,
+    )
+    print(f"   Training samples: {len(train_dataset):,}")
+    print(f"   Validation samples: {len(val_dataset):,}")
+    
+    # Create data collator
+    data_collator = ChessDataCollator(tokenizer, max_length=args.n_ctx)
+    
+    # Training arguments
+    training_args = TrainingArguments(
+        output_dir=args.output_dir,
+        num_train_epochs=args.num_train_epochs,
+        per_device_train_batch_size=args.per_device_train_batch_size,
+        per_device_eval_batch_size=args.per_device_eval_batch_size,
+        learning_rate=args.learning_rate,
+        weight_decay=args.weight_decay,
+        warmup_ratio=args.warmup_ratio,
+        logging_dir=os.path.join(args.output_dir, "logs"),
+        logging_steps=args.logging_steps,
+        eval_strategy="epoch",
+        save_strategy="epoch",
+        save_total_limit=3,
+        load_best_model_at_end=True,
+        metric_for_best_model="eval_loss",
+        greater_is_better=False,
+        seed=args.seed,
+        bf16=torch.cuda.is_available() and torch.cuda.is_bf16_supported(),
+        report_to=["none"],
+    )
+    
+    # Create trainer
+    trainer = Trainer(
+        model=model,
+        args=training_args,
+        train_dataset=train_dataset,
+        eval_dataset=val_dataset,
+        data_collator=data_collator,
+        tokenizer=tokenizer,
+    )
+    
+    # Train
+    print("\nStarting training...")
+    print(f"   Device: {training_args.device}")
+    trainer.train()
+    
+    # Save final model
+    print("\nSaving final model...")
+    trainer.save_model(os.path.join(args.output_dir, "final_model"))
+    tokenizer.save_pretrained(os.path.join(args.output_dir, "final_model"))
+    
+    print("\nTraining complete!")
+    print(f"   Model saved to: {args.output_dir}/final_model")
+
+
+if __name__ == "__main__":
+    main()

src/utils.py

@@ -0,0 +1,305 @@
+"""
+Utility functions for the Chess Challenge.
+
+This module provides helper functions for:
+- Parameter counting and budget analysis
+- Model registration with Hugging Face
+- Move validation with python-chess
+"""
+
+from __future__ import annotations
+
+from typing import Dict, Optional, TYPE_CHECKING
+
+import torch.nn as nn
+
+if TYPE_CHECKING:
+    from src.model import ChessConfig
+
+
+def count_parameters(model: nn.Module, trainable_only: bool = True) -> int:
+    """
+    Count the number of parameters in a model.
+    
+    Args:
+        model: The PyTorch model.
+        trainable_only: If True, only count trainable parameters.
+    
+    Returns:
+        Total number of parameters.
+    """
+    if trainable_only:
+        return sum(p.numel() for p in model.parameters() if p.requires_grad)
+    return sum(p.numel() for p in model.parameters())
+
+
+def count_parameters_by_component(model: nn.Module) -> Dict[str, int]:
+    """
+    Count parameters broken down by model component.
+    
+    Args:
+        model: The PyTorch model.
+    
+    Returns:
+        Dictionary mapping component names to parameter counts.
+    """
+    counts = {}
+    for name, module in model.named_modules():
+        if len(list(module.children())) == 0:  # Leaf module
+            param_count = sum(p.numel() for p in module.parameters(recurse=False))
+            if param_count > 0:
+                counts[name] = param_count
+    return counts
+
+
+def estimate_parameters(config: "ChessConfig") -> Dict[str, int]:
+    """
+    Estimate the parameter count for a given configuration.
+    
+    This is useful for planning your architecture before building the model.
+    
+    Args:
+        config: Model configuration.
+    
+    Returns:
+        Dictionary with estimated parameter counts by component.
+    """
+    V = config.vocab_size
+    d = config.n_embd
+    L = config.n_layer
+    n_ctx = config.n_ctx
+    n_inner = config.n_inner
+    
+    estimates = {
+        "token_embeddings": V * d,
+        "position_embeddings": n_ctx * d,
+        "attention_qkv_per_layer": 3 * d * d,
+        "attention_proj_per_layer": d * d,
+        "ffn_per_layer": 2 * d * n_inner,
+        "layernorm_per_layer": 4 * d,  # 2 LayerNorms, each with weight and bias
+        "final_layernorm": 2 * d,
+    }
+    
+    # Calculate totals
+    per_layer = (
+        estimates["attention_qkv_per_layer"] +
+        estimates["attention_proj_per_layer"] +
+        estimates["ffn_per_layer"] +
+        estimates["layernorm_per_layer"]
+    )
+    
+    estimates["total_transformer_layers"] = L * per_layer
+    
+    # LM head (tied with embeddings by default)
+    if config.tie_weights:
+        estimates["lm_head"] = 0
+        estimates["lm_head_note"] = "Tied with token embeddings"
+    else:
+        estimates["lm_head"] = V * d
+    
+    # Grand total
+    estimates["total"] = (
+        estimates["token_embeddings"] +
+        estimates["position_embeddings"] +
+        estimates["total_transformer_layers"] +
+        estimates["final_layernorm"] +
+        estimates["lm_head"]
+    )
+    
+    return estimates
+
+
+def print_parameter_budget(config: "ChessConfig", limit: int = 1_000_000) -> None:
+    """
+    Print a formatted parameter budget analysis.
+    
+    Args:
+        config: Model configuration.
+        limit: Parameter limit to compare against.
+    """
+    estimates = estimate_parameters(config)
+    
+    print("=" * 60)
+    print("PARAMETER BUDGET ANALYSIS")
+    print("=" * 60)
+    print(f"\nConfiguration:")
+    print(f"  vocab_size (V) = {config.vocab_size}")
+    print(f"  n_embd (d)     = {config.n_embd}")
+    print(f"  n_layer (L)    = {config.n_layer}")
+    print(f"  n_head         = {config.n_head}")
+    print(f"  n_ctx          = {config.n_ctx}")
+    print(f"  n_inner        = {config.n_inner}")
+    print(f"  tie_weights    = {config.tie_weights}")
+    
+    print(f"\nParameter Breakdown:")
+    print(f"  Token Embeddings:    {estimates['token_embeddings']:>10,}")
+    print(f"  Position Embeddings: {estimates['position_embeddings']:>10,}")
+    print(f"  Transformer Layers:  {estimates['total_transformer_layers']:>10,}")
+    print(f"  Final LayerNorm:     {estimates['final_layernorm']:>10,}")
+    
+    if config.tie_weights:
+        print(f"  LM Head:             {'(tied)':>10}")
+    else:
+        print(f"  LM Head:             {estimates['lm_head']:>10,}")
+    
+    print(f"  " + "-" * 30)
+    print(f"  TOTAL:               {estimates['total']:>10,}")
+    
+    print(f"\nBudget Status:")
+    print(f"  Limit:    {limit:>10,}")
+    print(f"  Used:     {estimates['total']:>10,}")
+    print(f"  Remaining:{limit - estimates['total']:>10,}")
+    
+    if estimates['total'] <= limit:
+        print(f"\n Within budget! ({estimates['total'] / limit * 100:.1f}% used)")
+    else:
+        print(f"\n OVER BUDGET by {estimates['total'] - limit:,} parameters!")
+    
+    print("=" * 60)
+
+
+def validate_move_with_chess(move: str, board_fen: Optional[str] = None) -> bool:
+    """
+    Validate a move using python-chess.
+    
+    This function converts the dataset's extended UCI format to standard UCI
+    and validates it against the current board state.
+    
+    Args:
+        move: Move in extended UCI format (e.g., "WPe2e4", "BNg8f6(x)").
+        board_fen: FEN string of the current board state (optional).
+    
+    Returns:
+        True if the move is legal, False otherwise.
+    """
+    try:
+        import chess
+    except ImportError:
+        raise ImportError("python-chess is required for move validation. "
+                         "Install it with: pip install python-chess")
+    
+    # Parse the extended UCI format
+    # Format: [W|B][Piece][from_sq][to_sq][suffix]
+    # Example: WPe2e4, BNg8f6(x), WKe1g1(o)
+    
+    if len(move) < 6:
+        return False
+    
+    # Extract components
+    color = move[0]  # W or B
+    piece = move[1]  # P, N, B, R, Q, K
+    from_sq = move[2:4]  # e.g., "e2"
+    to_sq = move[4:6]  # e.g., "e4"
+    
+    # Check for promotion
+    promotion = None
+    if "=" in move:
+        promo_idx = move.index("=")
+        promotion = move[promo_idx + 1].lower()
+    
+    # Create board
+    board = chess.Board(board_fen) if board_fen else chess.Board()
+    
+    # Build UCI move string
+    uci_move = from_sq + to_sq
+    if promotion:
+        uci_move += promotion
+    
+    try:
+        move_obj = chess.Move.from_uci(uci_move)
+        return move_obj in board.legal_moves
+    except (ValueError, chess.InvalidMoveError):
+        return False
+
+
+def convert_extended_uci_to_uci(move: str) -> str:
+    """
+    Convert extended UCI format to standard UCI format.
+    
+    Args:
+        move: Move in extended UCI format (e.g., "WPe2e4").
+    
+    Returns:
+        Move in standard UCI format (e.g., "e2e4").
+    """
+    if len(move) < 6:
+        return move
+    
+    # Extract squares
+    from_sq = move[2:4]
+    to_sq = move[4:6]
+    
+    # Check for promotion
+    promotion = ""
+    if "=" in move:
+        promo_idx = move.index("=")
+        promotion = move[promo_idx + 1].lower()
+    
+    return from_sq + to_sq + promotion
+
+
+def convert_uci_to_extended(
+    uci_move: str,
+    board_fen: str,
+) -> str:
+    """
+    Convert standard UCI format to extended UCI format.
+    
+    Args:
+        uci_move: Move in standard UCI format (e.g., "e2e4").
+        board_fen: FEN string of the current board state.
+    
+    Returns:
+        Move in extended UCI format (e.g., "WPe2e4").
+    """
+    try:
+        import chess
+    except ImportError:
+        raise ImportError("python-chess is required for move conversion.")
+    
+    board = chess.Board(board_fen)
+    move = chess.Move.from_uci(uci_move)
+    
+    # Get color
+    color = "W" if board.turn == chess.WHITE else "B"
+    
+    # Get piece
+    piece = board.piece_at(move.from_square)
+    piece_letter = piece.symbol().upper() if piece else "P"
+    
+    # Build extended UCI
+    from_sq = chess.square_name(move.from_square)
+    to_sq = chess.square_name(move.to_square)
+    
+    result = f"{color}{piece_letter}{from_sq}{to_sq}"
+    
+    # Add promotion
+    if move.promotion:
+        result += f"={chess.piece_symbol(move.promotion).upper()}"
+    
+    # Add suffix for captures
+    if board.is_capture(move):
+        result += "(x)"
+    
+    # Add suffix for check/checkmate
+    board.push(move)
+    if board.is_checkmate():
+        if "(x)" in result:
+            result = result.replace("(x)", "(x+*)")
+        else:
+            result += "(+*)"
+    elif board.is_check():
+        if "(x)" in result:
+            result = result.replace("(x)", "(x+)")
+        else:
+            result += "(+)"
+    board.pop()
+    
+    # Handle castling notation
+    if board.is_castling(move):
+        if move.to_square in [chess.G1, chess.G8]:  # Kingside
+            result = result.replace("(x)", "").replace("(+)", "") + "(o)"
+        else:  # Queenside
+            result = result.replace("(x)", "").replace("(+)", "") + "(O)"
+    
+    return result

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[BOS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[EOS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[BOS]",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[EOS]",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "tokenizer_class": "ChessTokenizer",
+  "unk_token": "[UNK]"
+}

vocab.json

@@ -0,0 +1,1684 @@
+{
+  "[PAD]": 0,
+  "[BOS]": 1,
+  "[EOS]": 2,
+  "[UNK]": 3,
+  "BBa5b6": 4,
+  "BBa6b7": 5,
+  "BBb4a5": 6,
+  "BBb4c3(x)": 7,
+  "BBb4c3(x+)": 8,
+  "BBb4c5": 9,
+  "BBb4d2(x)": 10,
+  "BBb4d2(x+)": 11,
+  "BBb4d6": 12,
+  "BBb4e7": 13,
+  "BBb6c7": 14,
+  "BBb6d4(x)": 15,
+  "BBb7a6": 16,
+  "BBb7c6": 17,
+  "BBb7c6(x)": 18,
+  "BBb7c8": 19,
+  "BBb7d5": 20,
+  "BBb7d5(x)": 21,
+  "BBb7e4(x)": 22,
+  "BBb7f3(x)": 23,
+  "BBb7g2(x)": 24,
+  "BBc5a7": 25,
+  "BBc5b4": 26,
+  "BBc5b4(+)": 27,
+  "BBc5b6": 28,
+  "BBc5d4": 29,
+  "BBc5d4(x)": 30,
+  "BBc5d6": 31,
+  "BBc5e3(x)": 32,
+  "BBc5e7": 33,
+  "BBc5f2(x+)": 34,
+  "BBc6b5": 35,
+  "BBc6d5": 36,
+  "BBc6d5(x)": 37,
+  "BBc6d7": 38,
+  "BBc6e4(x)": 39,
+  "BBc6f3(x)": 40,
+  "BBc8a6": 41,
+  "BBc8b7": 42,
+  "BBc8d7": 43,
+  "BBc8d7(x)": 44,
+  "BBc8e6": 45,
+  "BBc8e6(x)": 46,
+  "BBc8f5": 47,
+  "BBc8f5(x)": 48,
+  "BBc8g4": 49,
+  "BBc8g4(x)": 50,
+  "BBc8h3": 51,
+  "BBc8h3(x)": 52,
+  "BBd5e6": 53,
+  "BBd5f3(x)": 54,
+  "BBd6b4": 55,
+  "BBd6c5": 56,
+  "BBd6c5(x)": 57,
+  "BBd6c7": 58,
+  "BBd6e5": 59,
+  "BBd6e5(x)": 60,
+  "BBd6e7": 61,
+  "BBd6f4": 62,
+  "BBd6f4(x)": 63,
+  "BBd6g3(x)": 64,
+  "BBd6h2(x+)": 65,
+  "BBd7b5": 66,
+  "BBd7b5(x)": 67,
+  "BBd7c6": 68,
+  "BBd7c6(x)": 69,
+  "BBd7c8": 70,
+  "BBd7e6": 71,
+  "BBd7e8": 72,
+  "BBd7f5": 73,
+  "BBd7f5(x)": 74,
+  "BBd7g4": 75,
+  "BBe4f3(x)": 76,
+  "BBe4g6": 77,
+  "BBe5d6": 78,
+  "BBe5f6": 79,
+  "BBe5g7": 80,
+  "BBe6a2(x)": 81,
+  "BBe6b3(x)": 82,
+  "BBe6c4": 83,
+  "BBe6c4(x)": 84,
+  "BBe6d5": 85,
+  "BBe6d5(x)": 86,
+  "BBe6d7": 87,
+  "BBe6f5": 88,
+  "BBe6f5(x)": 89,
+  "BBe6f7": 90,
+  "BBe6g4": 91,
+  "BBe6h3(x)": 92,
+  "BBe7b4": 93,
+  "BBe7c5": 94,
+  "BBe7c5(x)": 95,
+  "BBe7d6": 96,
+  "BBe7d6(x)": 97,
+  "BBe7d8": 98,
+  "BBe7f6": 99,
+  "BBe7f6(x)": 100,
+  "BBe7f8": 101,
+  "BBe7g5": 102,
+  "BBe7g5(x)": 103,
+  "BBe7h4": 104,
+  "BBe7h4(x)": 105,
+  "BBf5c2(x)": 106,
+  "BBf5d3": 107,
+  "BBf5d3(x)": 108,
+  "BBf5d7": 109,
+  "BBf5e4": 110,
+  "BBf5e4(x)": 111,
+  "BBf5e6": 112,
+  "BBf5g4": 113,
+  "BBf5g6": 114,
+  "BBf6b2(x)": 115,
+  "BBf6c3(x)": 116,
+  "BBf6d4(x)": 117,
+  "BBf6e5": 118,
+  "BBf6e5(x)": 119,
+  "BBf6e7": 120,
+  "BBf6g5": 121,
+  "BBf6g7": 122,
+  "BBf8b4": 123,
+  "BBf8b4(+)": 124,
+  "BBf8c5": 125,
+  "BBf8c5(+)": 126,
+  "BBf8c5(x)": 127,
+  "BBf8d6": 128,
+  "BBf8d6(x)": 129,
+  "BBf8e7": 130,
+  "BBf8e7(x)": 131,
+  "BBf8g7": 132,
+  "BBf8h6": 133,
+  "BBg4d1(x)": 134,
+  "BBg4d7": 135,
+  "BBg4e2(x)": 136,
+  "BBg4e6": 137,
+  "BBg4f3(x)": 138,
+  "BBg4f5": 139,
+  "BBg4h5": 140,
+  "BBg5f6": 141,
+  "BBg6d3(x)": 142,
+  "BBg6e4(x)": 143,
+  "BBg6h7": 144,
+  "BBg7b2(x)": 145,
+  "BBg7c3(x)": 146,
+  "BBg7d4(x)": 147,
+  "BBg7e5": 148,
+  "BBg7e5(x)": 149,
+  "BBg7f6": 150,
+  "BBg7f6(x)": 151,
+  "BBg7f8": 152,
+  "BBg7h6": 153,
+  "BBg7h6(x)": 154,
+  "BBh3g2(x)": 155,
+  "BBh5f3(x)": 156,
+  "BBh5g6": 157,
+  "BBh6g7": 158,
+  "BKb6a5": 159,
+  "BKb6b5": 160,
+  "BKb6c5": 161,
+  "BKb6c6": 162,
+  "BKb6c7": 163,
+  "BKb7a6": 164,
+  "BKb7b6": 165,
+  "BKb7c6": 166,
+  "BKb7c7": 167,
+  "BKb8a7": 168,
+  "BKb8a8": 169,
+  "BKb8b7": 170,
+  "BKb8c7": 171,
+  "BKb8c8": 172,
+  "BKc5b4": 173,
+  "BKc5c4": 174,
+  "BKc5d4": 175,
+  "BKc5d6": 176,
+  "BKc6b5": 177,
+  "BKc6b6": 178,
+  "BKc6b7": 179,
+  "BKc6c5": 180,
+  "BKc6c7": 181,
+  "BKc6d5": 182,
+  "BKc6d6": 183,
+  "BKc6d7": 184,
+  "BKc7b6": 185,
+  "BKc7b7": 186,
+  "BKc7b8": 187,
+  "BKc7c6": 188,
+  "BKc7c8": 189,
+  "BKc7d6": 190,
+  "BKc7d7": 191,
+  "BKc7d8": 192,
+  "BKc8b7": 193,
+  "BKc8b8": 194,
+  "BKc8c7": 195,
+  "BKc8d7": 196,
+  "BKc8d8": 197,
+  "BKd4c3": 198,
+  "BKd5c4": 199,
+  "BKd5c5": 200,
+  "BKd5c6": 201,
+  "BKd5d4": 202,
+  "BKd5e4": 203,
+  "BKd5e6": 204,
+  "BKd6c5": 205,
+  "BKd6c6": 206,
+  "BKd6c7": 207,
+  "BKd6d5": 208,
+  "BKd6d7": 209,
+  "BKd6e5": 210,
+  "BKd6e6": 211,
+  "BKd6e7": 212,
+  "BKd7c6": 213,
+  "BKd7c7": 214,
+  "BKd7c8": 215,
+  "BKd7d6": 216,
+  "BKd7d8": 217,
+  "BKd7e6": 218,
+  "BKd7e7": 219,
+  "BKd7e8": 220,
+  "BKd8c7": 221,
+  "BKd8c8": 222,
+  "BKd8d7": 223,
+  "BKd8e7": 224,
+  "BKd8e8": 225,
+  "BKe4d3": 226,
+  "BKe5d4": 227,
+  "BKe5d5": 228,
+  "BKe5d6": 229,
+  "BKe5e4": 230,
+  "BKe5f4": 231,
+  "BKe5f5": 232,
+  "BKe5f6": 233,
+  "BKe6d5": 234,
+  "BKe6d6": 235,
+  "BKe6d7": 236,
+  "BKe6e5": 237,
+  "BKe6e7": 238,
+  "BKe6f5": 239,
+  "BKe6f6": 240,
+  "BKe6f7": 241,
+  "BKe7d6": 242,
+  "BKe7d7": 243,
+  "BKe7d8": 244,
+  "BKe7e6": 245,
+  "BKe7e8": 246,
+  "BKe7f6": 247,
+  "BKe7f7": 248,
+  "BKe7f8": 249,
+  "BKe8c8(O)": 250,
+  "BKe8d7": 251,
+  "BKe8d7(x)": 252,
+  "BKe8d8": 253,
+  "BKe8d8(x)": 254,
+  "BKe8e7": 255,
+  "BKe8e7(x)": 256,
+  "BKe8f7": 257,
+  "BKe8f7(x)": 258,
+  "BKe8f8": 259,
+  "BKe8g8(o)": 260,
+  "BKf4f3": 261,
+  "BKf5e4": 262,
+  "BKf5e5": 263,
+  "BKf5e6": 264,
+  "BKf5f4": 265,
+  "BKf5f6": 266,
+  "BKf5g4": 267,
+  "BKf5g5": 268,
+  "BKf5g6": 269,
+  "BKf6e5": 270,
+  "BKf6e6": 271,
+  "BKf6e7": 272,
+  "BKf6f5": 273,
+  "BKf6f7": 274,
+  "BKf6g5": 275,
+  "BKf6g6": 276,
+  "BKf6g7": 277,
+  "BKf7e6": 278,
+  "BKf7e7": 279,
+  "BKf7e8": 280,
+  "BKf7f6": 281,
+  "BKf7f8": 282,
+  "BKf7g6": 283,
+  "BKf7g7": 284,
+  "BKf7g8": 285,
+  "BKf8e7": 286,
+  "BKf8e8": 287,
+  "BKf8f7": 288,
+  "BKf8g7": 289,
+  "BKf8g8": 290,
+  "BKg5f4": 291,
+  "BKg5f5": 292,
+  "BKg5f6": 293,
+  "BKg5g4": 294,
+  "BKg5h4": 295,
+  "BKg6f5": 296,
+  "BKg6f6": 297,
+  "BKg6f7": 298,
+  "BKg6g5": 299,
+  "BKg6g7": 300,
+  "BKg6h5": 301,
+  "BKg6h6": 302,
+  "BKg6h7": 303,
+  "BKg7f6": 304,
+  "BKg7f7": 305,
+  "BKg7f8": 306,
+  "BKg7g6": 307,
+  "BKg7g8": 308,
+  "BKg7h6": 309,
+  "BKg7h7": 310,
+  "BKg7h8": 311,
+  "BKg8f7": 312,
+  "BKg8f7(x)": 313,
+  "BKg8f8": 314,
+  "BKg8f8(x)": 315,
+  "BKg8g7": 316,
+  "BKg8g7(x)": 317,
+  "BKg8h7": 318,
+  "BKg8h7(x)": 319,
+  "BKg8h8": 320,
+  "BKh5g4": 321,
+  "BKh5g6": 322,
+  "BKh5h4": 323,
+  "BKh6g5": 324,
+  "BKh6g6": 325,
+  "BKh6g7": 326,
+  "BKh6h5": 327,
+  "BKh6h7": 328,
+  "BKh7g6": 329,
+  "BKh7g7": 330,
+  "BKh7g8": 331,
+  "BKh7h6": 332,
+  "BKh7h8": 333,
+  "BKh8g7": 334,
+  "BKh8g8": 335,
+  "BKh8h7": 336,
+  "BNa5b3(x)": 337,
+  "BNa5c4": 338,
+  "BNa5c4(x)": 339,
+  "BNa5c6": 340,
+  "BNa6b4": 341,
+  "BNa6c5": 342,
+  "BNa6c7": 343,
+  "BNb4a6": 344,
+  "BNb4c2": 345,
+  "BNb4c2(x)": 346,
+  "BNb4c6": 347,
+  "BNb4d3": 348,
+  "BNb4d3(x)": 349,
+  "BNb4d5": 350,
+  "BNb6c4": 351,
+  "BNb6c4(x)": 352,
+  "BNb6d5": 353,
+  "BNb6d5(x)": 354,
+  "BNb6d7": 355,
+  "BNb8a6": 356,
+  "BNb8c6": 357,
+  "BNb8c6(x)": 358,
+  "BNb8d7": 359,
+  "BNb8d7(x)": 360,
+  "BNc2a1(x)": 361,
+  "BNc4b2(x)": 362,
+  "BNc4d6": 363,
+  "BNc4e5": 364,
+  "BNc5d3": 365,
+  "BNc5d3(x)": 366,
+  "BNc5d7": 367,
+  "BNc5e4": 368,
+  "BNc5e4(x)": 369,
+  "BNc5e6": 370,
+  "BNc6a5": 371,
+  "BNc6a7": 372,
+  "BNc6b4": 373,
+  "BNc6b4(x)": 374,
+  "BNc6b8": 375,
+  "BNc6d4": 376,
+  "BNc6d4(x)": 377,
+  "BNc6d8": 378,
+  "BNc6d8(x)": 379,
+  "BNc6e5": 380,
+  "BNc6e5(x)": 381,
+  "BNc6e7": 382,
+  "BNc6e7(x)": 383,
+  "BNd3b2(x)": 384,
+  "BNd4b3(x)": 385,
+  "BNd4c2(x)": 386,
+  "BNd4c6": 387,
+  "BNd4e2(+)": 388,
+  "BNd4e2(x+)": 389,
+  "BNd4e6": 390,
+  "BNd4f3(+)": 391,
+  "BNd4f3(x+)": 392,
+  "BNd4f5": 393,
+  "BNd5b4": 394,
+  "BNd5b6": 395,
+  "BNd5c3": 396,
+  "BNd5c3(x)": 397,
+  "BNd5e3": 398,
+  "BNd5e3(x)": 399,
+  "BNd5e7": 400,
+  "BNd5f4": 401,
+  "BNd5f4(x)": 402,
+  "BNd5f6": 403,
+  "BNd6e4": 404,
+  "BNd6f5": 405,
+  "BNd7b6": 406,
+  "BNd7b8": 407,
+  "BNd7c5": 408,
+  "BNd7c5(x)": 409,
+  "BNd7e5": 410,
+  "BNd7e5(x)": 411,
+  "BNd7f6": 412,
+  "BNd7f6(x)": 413,
+  "BNd7f8": 414,
+  "BNe3f1(x)": 415,
+  "BNe4c3": 416,
+  "BNe4c3(x)": 417,
+  "BNe4c5": 418,
+  "BNe4d2": 419,
+  "BNe4d2(x)": 420,
+  "BNe4d6": 421,
+  "BNe4f2(x)": 422,
+  "BNe4f6": 423,
+  "BNe4g3(x)": 424,
+  "BNe4g5": 425,
+  "BNe4g5(x)": 426,
+  "BNe5c4": 427,
+  "BNe5c4(x)": 428,
+  "BNe5c6": 429,
+  "BNe5d3": 430,
+  "BNe5d3(x)": 431,
+  "BNe5d7": 432,
+  "BNe5f3(+)": 433,
+  "BNe5f3(x+)": 434,
+  "BNe5g4": 435,
+  "BNe5g6": 436,
+  "BNe6d4": 437,
+  "BNe6f4": 438,
+  "BNe7c6": 439,
+  "BNe7c6(x)": 440,
+  "BNe7c8": 441,
+  "BNe7d5": 442,
+  "BNe7d5(x)": 443,
+  "BNe7f5": 444,
+  "BNe7f5(x)": 445,
+  "BNe7g6": 446,
+  "BNe8d6": 447,
+  "BNe8f6": 448,
+  "BNf4e2(+)": 449,
+  "BNf5d4": 450,
+  "BNf5d4(x)": 451,
+  "BNf5d6": 452,
+  "BNf5e3": 453,
+  "BNf5e3(x)": 454,
+  "BNf5e7": 455,
+  "BNf5h4": 456,
+  "BNf6d5": 457,
+  "BNf6d5(x)": 458,
+  "BNf6d7": 459,
+  "BNf6d7(x)": 460,
+  "BNf6e4": 461,
+  "BNf6e4(x)": 462,
+  "BNf6e8": 463,
+  "BNf6g4": 464,
+  "BNf6g4(x)": 465,
+  "BNf6g8": 466,
+  "BNf6h5": 467,
+  "BNf6h5(x)": 468,
+  "BNf6h7": 469,
+  "BNf8e6": 470,
+  "BNf8g6": 471,
+  "BNg4e3": 472,
+  "BNg4e3(x)": 473,
+  "BNg4e5": 474,
+  "BNg4e5(x)": 475,
+  "BNg4f2(x)": 476,
+  "BNg4f6": 477,
+  "BNg4h6": 478,
+  "BNg6e5": 479,
+  "BNg6e5(x)": 480,
+  "BNg6e7": 481,
+  "BNg6f4": 482,
+  "BNg6f4(x)": 483,
+  "BNg6h4": 484,
+  "BNg8e7": 485,
+  "BNg8f6": 486,
+  "BNg8f6(x)": 487,
+  "BNg8h6": 488,
+  "BNh5f4": 489,
+  "BNh5f4(x)": 490,
+  "BNh5f6": 491,
+  "BNh5g3(x)": 492,
+  "BNh6f5": 493,
+  "BNh6f7": 494,
+  "BNh6g4": 495,
+  "BNh7f6": 496,
+  "BNh7g5": 497,
+  "BPa2a1(Q)": 498,
+  "BPa3a2": 499,
+  "BPa4a3": 500,
+  "BPa4b3(x)": 501,
+  "BPa5a4": 502,
+  "BPa5b4(x)": 503,
+  "BPa6a5": 504,
+  "BPa6b5(x)": 505,
+  "BPa7a5": 506,
+  "BPa7a6": 507,
+  "BPa7b6(x)": 508,
+  "BPb2b1(Q)": 509,
+  "BPb3b2": 510,
+  "BPb4a3(x)": 511,
+  "BPb4b3": 512,
+  "BPb4c3(x)": 513,
+  "BPb5a4(x)": 514,
+  "BPb5b4": 515,
+  "BPb5c4(x)": 516,
+  "BPb6a5(x)": 517,
+  "BPb6b5": 518,
+  "BPb6c5(x)": 519,
+  "BPb7a6(x)": 520,
+  "BPb7b5": 521,
+  "BPb7b6": 522,
+  "BPb7c6(x)": 523,
+  "BPc2c1(Q)": 524,
+  "BPc3c2": 525,
+  "BPc4b3(x)": 526,
+  "BPc4c3": 527,
+  "BPc4d3(x)": 528,
+  "BPc5b4(x)": 529,
+  "BPc5c4": 530,
+  "BPc5d4(x)": 531,
+  "BPc6b5(x)": 532,
+  "BPc6c5": 533,
+  "BPc6d5(x)": 534,
+  "BPc7b6(x)": 535,
+  "BPc7c5": 536,
+  "BPc7c6": 537,
+  "BPc7d6(x)": 538,
+  "BPd3d2": 539,
+  "BPd4c3(x)": 540,
+  "BPd4d3": 541,
+  "BPd4e3(x)": 542,
+  "BPd5c4(x)": 543,
+  "BPd5d4": 544,
+  "BPd5e4(x)": 545,
+  "BPd6c5(x)": 546,
+  "BPd6d5": 547,
+  "BPd6e5(x)": 548,
+  "BPd7c6(x)": 549,
+  "BPd7d5": 550,
+  "BPd7d6": 551,
+  "BPe3e2": 552,
+  "BPe4d3(x)": 553,
+  "BPe4e3": 554,
+  "BPe4f3(x)": 555,
+  "BPe5d4(x)": 556,
+  "BPe5e4": 557,
+  "BPe5f4(x)": 558,
+  "BPe6d5(x)": 559,
+  "BPe6e5": 560,
+  "BPe6f5(x)": 561,
+  "BPe7d6(x)": 562,
+  "BPe7e5": 563,
+  "BPe7e6": 564,
+  "BPe7f6(x)": 565,
+  "BPf3f2": 566,
+  "BPf4e3(x)": 567,
+  "BPf4f3": 568,
+  "BPf4g3(x)": 569,
+  "BPf5e4(x)": 570,
+  "BPf5f4": 571,
+  "BPf5g4(x)": 572,
+  "BPf6e5(x)": 573,
+  "BPf6f5": 574,
+  "BPf6g5(x)": 575,
+  "BPf7e6(x)": 576,
+  "BPf7f5": 577,
+  "BPf7f6": 578,
+  "BPf7g6(x)": 579,
+  "BPg2g1(Q)": 580,
+  "BPg3g2": 581,
+  "BPg4f3(x)": 582,
+  "BPg4g3": 583,
+  "BPg4h3(x)": 584,
+  "BPg5f4(x)": 585,
+  "BPg5g4": 586,
+  "BPg5h4(x)": 587,
+  "BPg6f5(x)": 588,
+  "BPg6g5": 589,
+  "BPg6h5(x)": 590,
+  "BPg7f6(x)": 591,
+  "BPg7g5": 592,
+  "BPg7g6": 593,
+  "BPg7h6(x)": 594,
+  "BPh2h1(Q)": 595,
+  "BPh3h2": 596,
+  "BPh4g3(x)": 597,
+  "BPh4h3": 598,
+  "BPh5g4(x)": 599,
+  "BPh5h4": 600,
+  "BPh6g5(x)": 601,
+  "BPh6h5": 602,
+  "BPh7g6(x)": 603,
+  "BPh7h5": 604,
+  "BPh7h6": 605,
+  "BQa5b6": 606,
+  "BQa5c7": 607,
+  "BQa5d8": 608,
+  "BQb2a2(x)": 609,
+  "BQb4b2(x)": 610,
+  "BQb6a5": 611,
+  "BQb6b2(x)": 612,
+  "BQb6c6": 613,
+  "BQb6c7": 614,
+  "BQb6d4(x)": 615,
+  "BQb6d8": 616,
+  "BQc7a5": 617,
+  "BQc7b6": 618,
+  "BQc7b7": 619,
+  "BQc7c6": 620,
+  "BQc7d6": 621,
+  "BQc7d6(x)": 622,
+  "BQc7d7": 623,
+  "BQc7d8": 624,
+  "BQc7e5(x)": 625,
+  "BQc7e7": 626,
+  "BQd5a5": 627,
+  "BQd5d6": 628,
+  "BQd5d8": 629,
+  "BQd6c6": 630,
+  "BQd6c7": 631,
+  "BQd6d7": 632,
+  "BQd6e6": 633,
+  "BQd6e7": 634,
+  "BQd7c6": 635,
+  "BQd7c7": 636,
+  "BQd7d6": 637,
+  "BQd7e6": 638,
+  "BQd7e7": 639,
+  "BQd7f5": 640,
+  "BQd7g4": 641,
+  "BQd8a5": 642,
+  "BQd8a5(+)": 643,
+  "BQd8a8(x)": 644,
+  "BQd8b6": 645,
+  "BQd8b6(+)": 646,
+  "BQd8b8": 647,
+  "BQd8c7": 648,
+  "BQd8c8": 649,
+  "BQd8d1(x)": 650,
+  "BQd8d1(x+)": 651,
+  "BQd8d4": 652,
+  "BQd8d4(x)": 653,
+  "BQd8d5": 654,
+  "BQd8d5(x)": 655,
+  "BQd8d6": 656,
+  "BQd8d6(x)": 657,
+  "BQd8d7": 658,
+  "BQd8d7(x)": 659,
+  "BQd8e7": 660,
+  "BQd8e7(+)": 661,
+  "BQd8e7(x)": 662,
+  "BQd8e8": 663,
+  "BQd8f6": 664,
+  "BQd8f6(x)": 665,
+  "BQd8f8": 666,
+  "BQd8g5": 667,
+  "BQd8g5(x)": 668,
+  "BQd8h4": 669,
+  "BQd8h4(+)": 670,
+  "BQd8h4(x)": 671,
+  "BQe7c5": 672,
+  "BQe7c7": 673,
+  "BQe7d6": 674,
+  "BQe7d7": 675,
+  "BQe7d8": 676,
+  "BQe7e5": 677,
+  "BQe7e5(x)": 678,
+  "BQe7e6": 679,
+  "BQe7e6(x)": 680,
+  "BQe7f6": 681,
+  "BQe7f6(x)": 682,
+  "BQe7f7": 683,
+  "BQe7g5": 684,
+  "BQe7h4": 685,
+  "BQf6d8": 686,
+  "BQf6e5(x)": 687,
+  "BQf6e6": 688,
+  "BQf6e7": 689,
+  "BQf6f3(x)": 690,
+  "BQf6f5": 691,
+  "BQf6g5": 692,
+  "BQf6g6": 693,
+  "BQg5f6": 694,
+  "BQg5g6": 695,
+  "BQg6f6": 696,
+  "BRa2a1(+)": 697,
+  "BRa2b2": 698,
+  "BRa8a1(x)": 699,
+  "BRa8a2(x)": 700,
+  "BRa8a6": 701,
+  "BRa8a6(x)": 702,
+  "BRa8a7": 703,
+  "BRa8b8": 704,
+  "BRa8c8": 705,
+  "BRa8c8(x)": 706,
+  "BRa8d8": 707,
+  "BRa8d8(x)": 708,
+  "BRa8e8": 709,
+  "BRa8e8(x)": 710,
+  "BRa8f8": 711,
+  "BRa8f8(x)": 712,
+  "BRa8g8": 713,
+  "BRa8h8": 714,
+  "BRb2a2(x)": 715,
+  "BRb8a8": 716,
+  "BRb8b2": 717,
+  "BRb8b2(x)": 718,
+  "BRb8b6": 719,
+  "BRb8b7": 720,
+  "BRb8b7(x)": 721,
+  "BRb8c8": 722,
+  "BRb8d8": 723,
+  "BRb8e8": 724,
+  "BRb8f8": 725,
+  "BRc2a2(x)": 726,
+  "BRc2b2(x)": 727,
+  "BRc8a8": 728,
+  "BRc8b8": 729,
+  "BRc8c1(x)": 730,
+  "BRc8c1(x+)": 731,
+  "BRc8c2": 732,
+  "BRc8c2(x)": 733,
+  "BRc8c3": 734,
+  "BRc8c3(x)": 735,
+  "BRc8c4": 736,
+  "BRc8c4(x)": 737,
+  "BRc8c5": 738,
+  "BRc8c5(x)": 739,
+  "BRc8c6": 740,
+  "BRc8c6(x)": 741,
+  "BRc8c7": 742,
+  "BRc8c7(x)": 743,
+  "BRc8d8": 744,
+  "BRc8e8": 745,
+  "BRc8f8": 746,
+  "BRd7c7": 747,
+  "BRd7e7": 748,
+  "BRd8a8": 749,
+  "BRd8b8": 750,
+  "BRd8c8": 751,
+  "BRd8d1(+)": 752,
+  "BRd8d1(x)": 753,
+  "BRd8d1(x+)": 754,
+  "BRd8d2": 755,
+  "BRd8d2(x)": 756,
+  "BRd8d3": 757,
+  "BRd8d3(x)": 758,
+  "BRd8d4": 759,
+  "BRd8d4(x)": 760,
+  "BRd8d5": 761,
+  "BRd8d5(x)": 762,
+  "BRd8d6": 763,
+  "BRd8d6(x)": 764,
+  "BRd8d7": 765,
+  "BRd8d7(x)": 766,
+  "BRd8e8": 767,
+  "BRd8f8": 768,
+  "BRd8g8": 769,
+  "BRd8h8": 770,
+  "BRe7d7": 771,
+  "BRe8b8": 772,
+  "BRe8c8": 773,
+  "BRe8d8": 774,
+  "BRe8d8(x)": 775,
+  "BRe8e1(+)": 776,
+  "BRe8e1(x)": 777,
+  "BRe8e1(x+)": 778,
+  "BRe8e2": 779,
+  "BRe8e2(x)": 780,
+  "BRe8e3": 781,
+  "BRe8e3(x)": 782,
+  "BRe8e4": 783,
+  "BRe8e4(x)": 784,
+  "BRe8e5": 785,
+  "BRe8e5(x)": 786,
+  "BRe8e6": 787,
+  "BRe8e6(x)": 788,
+  "BRe8e7": 789,
+  "BRe8e7(x)": 790,
+  "BRe8f8": 791,
+  "BRe8g8": 792,
+  "BRf6g6": 793,
+  "BRf7e7": 794,
+  "BRf7f8": 795,
+  "BRf8a8": 796,
+  "BRf8a8(x)": 797,
+  "BRf8b8": 798,
+  "BRf8c8": 799,
+  "BRf8c8(x)": 800,
+  "BRf8d8": 801,
+  "BRf8d8(x)": 802,
+  "BRf8e8": 803,
+  "BRf8e8(+)": 804,
+  "BRf8e8(x)": 805,
+  "BRf8f1(x+)": 806,
+  "BRf8f2(x)": 807,
+  "BRf8f3(x)": 808,
+  "BRf8f4": 809,
+  "BRf8f4(x)": 810,
+  "BRf8f5": 811,
+  "BRf8f5(x)": 812,
+  "BRf8f6": 813,
+  "BRf8f6(x)": 814,
+  "BRf8f7": 815,
+  "BRf8f7(x)": 816,
+  "BRf8g8": 817,
+  "BRf8h8": 818,
+  "BRg8f8": 819,
+  "BRg8g6": 820,
+  "BRg8g7": 821,
+  "BRg8h8": 822,
+  "BRh8c8": 823,
+  "BRh8d8": 824,
+  "BRh8e8": 825,
+  "BRh8f8": 826,
+  "BRh8g8": 827,
+  "BRh8h6": 828,
+  "BRh8h7": 829,
+  "WBa3b2": 830,
+  "WBa4b3": 831,
+  "WBa4c2": 832,
+  "WBb2a3": 833,
+  "WBb2c1": 834,
+  "WBb2c3": 835,
+  "WBb2c3(x)": 836,
+  "WBb2d4": 837,
+  "WBb2d4(x)": 838,
+  "WBb2e5(x)": 839,
+  "WBb2f6(x)": 840,
+  "WBb2g7(x)": 841,
+  "WBb3a2": 842,
+  "WBb3c2": 843,
+  "WBb3d5": 844,
+  "WBb3d5(x)": 845,
+  "WBb3e6(x)": 846,
+  "WBb5a4": 847,
+  "WBb5c4": 848,
+  "WBb5c6(x)": 849,
+  "WBb5c6(x+)": 850,
+  "WBb5d3": 851,
+  "WBb5d7(x)": 852,
+  "WBb5d7(x+)": 853,
+  "WBb5e2": 854,
+  "WBc1a3": 855,
+  "WBc1b2": 856,
+  "WBc1d2": 857,
+  "WBc1d2(x)": 858,
+  "WBc1e3": 859,
+  "WBc1e3(x)": 860,
+  "WBc1f4": 861,
+  "WBc1f4(x)": 862,
+  "WBc1g5": 863,
+  "WBc1g5(x)": 864,
+  "WBc1h6": 865,
+  "WBc1h6(x)": 866,
+  "WBc2b3": 867,
+  "WBc2e4(x)": 868,
+  "WBc3d2": 869,
+  "WBc4a2": 870,
+  "WBc4b3": 871,
+  "WBc4b5": 872,
+  "WBc4b5(+)": 873,
+  "WBc4d3": 874,
+  "WBc4d5": 875,
+  "WBc4d5(x)": 876,
+  "WBc4e2": 877,
+  "WBc4e6(x)": 878,
+  "WBc4f7(x)": 879,
+  "WBc4f7(x+)": 880,
+  "WBd2b4": 881,
+  "WBd2b4(x)": 882,
+  "WBd2c1": 883,
+  "WBd2c3": 884,
+  "WBd2c3(x)": 885,
+  "WBd2e1": 886,
+  "WBd2e3": 887,
+  "WBd2f4": 888,
+  "WBd2f4(x)": 889,
+  "WBd2g5": 890,
+  "WBd3a6(x)": 891,
+  "WBd3b1": 892,
+  "WBd3b5": 893,
+  "WBd3b5(x)": 894,
+  "WBd3c2": 895,
+  "WBd3c4": 896,
+  "WBd3c4(x)": 897,
+  "WBd3e2": 898,
+  "WBd3e4": 899,
+  "WBd3e4(x)": 900,
+  "WBd3f1": 901,
+  "WBd3f5": 902,
+  "WBd3f5(x)": 903,
+  "WBd3g6(x)": 904,
+  "WBd3h7(x+)": 905,
+  "WBd4e3": 906,
+  "WBd4f6(x)": 907,
+  "WBd5b3": 908,
+  "WBe2b5": 909,
+  "WBe2c4": 910,
+  "WBe2c4(x)": 911,
+  "WBe2d1": 912,
+  "WBe2d3": 913,
+  "WBe2f1": 914,
+  "WBe2f3": 915,
+  "WBe2f3(x)": 916,
+  "WBe2g4": 917,
+  "WBe2g4(x)": 918,
+  "WBe2h5": 919,
+  "WBe2h5(x)": 920,
+  "WBe3a7(x)": 921,
+  "WBe3b6(x)": 922,
+  "WBe3c1": 923,
+  "WBe3c5": 924,
+  "WBe3c5(x)": 925,
+  "WBe3d2": 926,
+  "WBe3d4": 927,
+  "WBe3d4(x)": 928,
+  "WBe3f2": 929,
+  "WBe3f4": 930,
+  "WBe3f4(x)": 931,
+  "WBe3g5": 932,
+  "WBe3g5(x)": 933,
+  "WBe3h6": 934,
+  "WBe3h6(x)": 935,
+  "WBe4d3": 936,
+  "WBe4f3": 937,
+  "WBe5f6(x)": 938,
+  "WBe5g3": 939,
+  "WBf1b5": 940,
+  "WBf1b5(+)": 941,
+  "WBf1c4": 942,
+  "WBf1c4(x)": 943,
+  "WBf1d3": 944,
+  "WBf1d3(x)": 945,
+  "WBf1e2": 946,
+  "WBf1g2": 947,
+  "WBf1h3": 948,
+  "WBf3b7(x)": 949,
+  "WBf3c6(x)": 950,
+  "WBf3d5(x)": 951,
+  "WBf3e2": 952,
+  "WBf3e4": 953,
+  "WBf3e4(x)": 954,
+  "WBf3g2": 955,
+  "WBf3g4": 956,
+  "WBf4c7(x)": 957,
+  "WBf4d2": 958,
+  "WBf4d6": 959,
+  "WBf4d6(x)": 960,
+  "WBf4e3": 961,
+  "WBf4e5": 962,
+  "WBf4e5(x)": 963,
+  "WBf4g3": 964,
+  "WBf4g5": 965,
+  "WBf4h2": 966,
+  "WBf4h6": 967,
+  "WBg2b7(x)": 968,
+  "WBg2c6(x)": 969,
+  "WBg2d5(x)": 970,
+  "WBg2e4": 971,
+  "WBg2e4(x)": 972,
+  "WBg2f1": 973,
+  "WBg2f3": 974,
+  "WBg2f3(x)": 975,
+  "WBg2h3": 976,
+  "WBg3d6(x)": 977,
+  "WBg3e5": 978,
+  "WBg3e5(x)": 979,
+  "WBg3f2": 980,
+  "WBg3h2": 981,
+  "WBg3h4": 982,
+  "WBg4f3": 983,
+  "WBg5d2": 984,
+  "WBg5d8(x)": 985,
+  "WBg5e3": 986,
+  "WBg5e7(x)": 987,
+  "WBg5f4": 988,
+  "WBg5f6": 989,
+  "WBg5f6(x)": 990,
+  "WBg5h4": 991,
+  "WBg5h6": 992,
+  "WBh4e7(x)": 993,
+  "WBh4f6(x)": 994,
+  "WBh4g3": 995,
+  "WBh6f8(x)": 996,
+  "WBh6g5": 997,
+  "WBh6g7(x)": 998,
+  "WKb1a1": 999,
+  "WKb1a2": 1000,
+  "WKb1b2": 1001,
+  "WKb1c1": 1002,
+  "WKb1c2": 1003,
+  "WKb2a3": 1004,
+  "WKb2b3": 1005,
+  "WKb2c2": 1006,
+  "WKb2c3": 1007,
+  "WKb3a4": 1008,
+  "WKb3c2": 1009,
+  "WKb3c4": 1010,
+  "WKc1b1": 1011,
+  "WKc1b2": 1012,
+  "WKc1c2": 1013,
+  "WKc1d1": 1014,
+  "WKc1d2": 1015,
+  "WKc2b1": 1016,
+  "WKc2b2": 1017,
+  "WKc2b3": 1018,
+  "WKc2c3": 1019,
+  "WKc2d2": 1020,
+  "WKc2d3": 1021,
+  "WKc3b2": 1022,
+  "WKc3b3": 1023,
+  "WKc3b4": 1024,
+  "WKc3c4": 1025,
+  "WKc3d2": 1026,
+  "WKc3d3": 1027,
+  "WKc3d4": 1028,
+  "WKc4b5": 1029,
+  "WKc4c5": 1030,
+  "WKc4d5": 1031,
+  "WKd1c1": 1032,
+  "WKd1c2": 1033,
+  "WKd1d2": 1034,
+  "WKd1e1": 1035,
+  "WKd1e2": 1036,
+  "WKd2c1": 1037,
+  "WKd2c2": 1038,
+  "WKd2c3": 1039,
+  "WKd2d1": 1040,
+  "WKd2d3": 1041,
+  "WKd2e1": 1042,
+  "WKd2e2": 1043,
+  "WKd2e3": 1044,
+  "WKd3c2": 1045,
+  "WKd3c3": 1046,
+  "WKd3c4": 1047,
+  "WKd3d2": 1048,
+  "WKd3d4": 1049,
+  "WKd3e2": 1050,
+  "WKd3e3": 1051,
+  "WKd3e4": 1052,
+  "WKd4c3": 1053,
+  "WKd4c4": 1054,
+  "WKd4c5": 1055,
+  "WKd4d5": 1056,
+  "WKd4e3": 1057,
+  "WKd4e4": 1058,
+  "WKd4e5": 1059,
+  "WKd5c6": 1060,
+  "WKe1c1(O)": 1061,
+  "WKe1d1": 1062,
+  "WKe1d1(x)": 1063,
+  "WKe1d2": 1064,
+  "WKe1d2(x)": 1065,
+  "WKe1e2": 1066,
+  "WKe1e2(x)": 1067,
+  "WKe1f1": 1068,
+  "WKe1f2": 1069,
+  "WKe1f2(x)": 1070,
+  "WKe1g1(o)": 1071,
+  "WKe2d1": 1072,
+  "WKe2d2": 1073,
+  "WKe2d3": 1074,
+  "WKe2e1": 1075,
+  "WKe2e3": 1076,
+  "WKe2f1": 1077,
+  "WKe2f2": 1078,
+  "WKe2f3": 1079,
+  "WKe3d2": 1080,
+  "WKe3d3": 1081,
+  "WKe3d4": 1082,
+  "WKe3e2": 1083,
+  "WKe3e4": 1084,
+  "WKe3f2": 1085,
+  "WKe3f3": 1086,
+  "WKe3f4": 1087,
+  "WKe4d3": 1088,
+  "WKe4d4": 1089,
+  "WKe4d5": 1090,
+  "WKe4e3": 1091,
+  "WKe4e5": 1092,
+  "WKe4f3": 1093,
+  "WKe4f4": 1094,
+  "WKe4f5": 1095,
+  "WKe5d6": 1096,
+  "WKe5f6": 1097,
+  "WKf1e1": 1098,
+  "WKf1e2": 1099,
+  "WKf1f2": 1100,
+  "WKf1g1": 1101,
+  "WKf1g2": 1102,
+  "WKf2e1": 1103,
+  "WKf2e2": 1104,
+  "WKf2e3": 1105,
+  "WKf2f1": 1106,
+  "WKf2f3": 1107,
+  "WKf2g1": 1108,
+  "WKf2g2": 1109,
+  "WKf2g3": 1110,
+  "WKf3e2": 1111,
+  "WKf3e3": 1112,
+  "WKf3e4": 1113,
+  "WKf3f2": 1114,
+  "WKf3f4": 1115,
+  "WKf3g2": 1116,
+  "WKf3g3": 1117,
+  "WKf3g4": 1118,
+  "WKf4e3": 1119,
+  "WKf4e4": 1120,
+  "WKf4e5": 1121,
+  "WKf4f3": 1122,
+  "WKf4f5": 1123,
+  "WKf4g3": 1124,
+  "WKf4g4": 1125,
+  "WKf4g5": 1126,
+  "WKg1f1": 1127,
+  "WKg1f1(x)": 1128,
+  "WKg1f2": 1129,
+  "WKg1f2(x)": 1130,
+  "WKg1g2": 1131,
+  "WKg1g2(x)": 1132,
+  "WKg1h1": 1133,
+  "WKg1h2": 1134,
+  "WKg1h2(x)": 1135,
+  "WKg2f1": 1136,
+  "WKg2f2": 1137,
+  "WKg2f3": 1138,
+  "WKg2g1": 1139,
+  "WKg2g3": 1140,
+  "WKg2h1": 1141,
+  "WKg2h2": 1142,
+  "WKg2h3": 1143,
+  "WKg3f2": 1144,
+  "WKg3f3": 1145,
+  "WKg3f4": 1146,
+  "WKg3g2": 1147,
+  "WKg3g4": 1148,
+  "WKg3h2": 1149,
+  "WKg3h3": 1150,
+  "WKg3h4": 1151,
+  "WKg4f3": 1152,
+  "WKg4f4": 1153,
+  "WKg4f5": 1154,
+  "WKg4g3": 1155,
+  "WKg4g5": 1156,
+  "WKg4h3": 1157,
+  "WKg4h5": 1158,
+  "WKg5f6": 1159,
+  "WKh1g1": 1160,
+  "WKh1g2": 1161,
+  "WKh1h2": 1162,
+  "WKh2g1": 1163,
+  "WKh2g2": 1164,
+  "WKh2g3": 1165,
+  "WKh2h1": 1166,
+  "WKh2h3": 1167,
+  "WKh3g2": 1168,
+  "WKh3g3": 1169,
+  "WKh3g4": 1170,
+  "WKh3h2": 1171,
+  "WKh3h4": 1172,
+  "WKh4g3": 1173,
+  "WKh4g5": 1174,
+  "WKh4h5": 1175,
+  "WNa3b5": 1176,
+  "WNa3c2": 1177,
+  "WNa3c4": 1178,
+  "WNa4c3": 1179,
+  "WNa4c5": 1180,
+  "WNa4c5(x)": 1181,
+  "WNb1a3": 1182,
+  "WNb1c3": 1183,
+  "WNb1c3(x)": 1184,
+  "WNb1d2": 1185,
+  "WNb1d2(x)": 1186,
+  "WNb3c5": 1187,
+  "WNb3d2": 1188,
+  "WNb3d4": 1189,
+  "WNb5a3": 1190,
+  "WNb5c3": 1191,
+  "WNb5c7": 1192,
+  "WNb5d4": 1193,
+  "WNb5d6": 1194,
+  "WNb5d6(+)": 1195,
+  "WNb5d6(x)": 1196,
+  "WNc2e3": 1197,
+  "WNc3a2": 1198,
+  "WNc3a4": 1199,
+  "WNc3b1": 1200,
+  "WNc3b5": 1201,
+  "WNc3b5(x)": 1202,
+  "WNc3d1": 1203,
+  "WNc3d1(x)": 1204,
+  "WNc3d5": 1205,
+  "WNc3d5(x)": 1206,
+  "WNc3e2": 1207,
+  "WNc3e2(x)": 1208,
+  "WNc3e4": 1209,
+  "WNc3e4(x)": 1210,
+  "WNc4d2": 1211,
+  "WNc4d6": 1212,
+  "WNc4e3": 1213,
+  "WNc4e5": 1214,
+  "WNc4e5(x)": 1215,
+  "WNc5d3": 1216,
+  "WNc7a8(x)": 1217,
+  "WNd1e3": 1218,
+  "WNd2b1": 1219,
+  "WNd2b3": 1220,
+  "WNd2c4": 1221,
+  "WNd2c4(x)": 1222,
+  "WNd2e4": 1223,
+  "WNd2e4(x)": 1224,
+  "WNd2f1": 1225,
+  "WNd2f3": 1226,
+  "WNd2f3(x)": 1227,
+  "WNd3e5": 1228,
+  "WNd3f4": 1229,
+  "WNd4b3": 1230,
+  "WNd4b5": 1231,
+  "WNd4c6": 1232,
+  "WNd4c6(x)": 1233,
+  "WNd4e2": 1234,
+  "WNd4e6": 1235,
+  "WNd4e6(x)": 1236,
+  "WNd4f3": 1237,
+  "WNd4f5": 1238,
+  "WNd4f5(x)": 1239,
+  "WNd5c3": 1240,
+  "WNd5c7(x)": 1241,
+  "WNd5e3": 1242,
+  "WNd5e7(+)": 1243,
+  "WNd5e7(x)": 1244,
+  "WNd5e7(x+)": 1245,
+  "WNd5f4": 1246,
+  "WNd5f6(+)": 1247,
+  "WNd5f6(x+)": 1248,
+  "WNd6b7(x)": 1249,
+  "WNe1f3": 1250,
+  "WNe2c3": 1251,
+  "WNe2c3(x)": 1252,
+  "WNe2d4": 1253,
+  "WNe2d4(x)": 1254,
+  "WNe2f4": 1255,
+  "WNe2f4(x)": 1256,
+  "WNe2g3": 1257,
+  "WNe3c4": 1258,
+  "WNe3d5": 1259,
+  "WNe3f5": 1260,
+  "WNe3g4": 1261,
+  "WNe4c3": 1262,
+  "WNe4c5": 1263,
+  "WNe4c5(x)": 1264,
+  "WNe4d2": 1265,
+  "WNe4d6": 1266,
+  "WNe4d6(+)": 1267,
+  "WNe4d6(x)": 1268,
+  "WNe4f6(+)": 1269,
+  "WNe4f6(x+)": 1270,
+  "WNe4g3": 1271,
+  "WNe4g5": 1272,
+  "WNe5c4": 1273,
+  "WNe5c4(x)": 1274,
+  "WNe5c6": 1275,
+  "WNe5c6(x)": 1276,
+  "WNe5d3": 1277,
+  "WNe5d7": 1278,
+  "WNe5d7(x)": 1279,
+  "WNe5f3": 1280,
+  "WNe5f7(x)": 1281,
+  "WNe5g4": 1282,
+  "WNe5g4(x)": 1283,
+  "WNe5g6": 1284,
+  "WNe5g6(x)": 1285,
+  "WNe6f8(x)": 1286,
+  "WNf1e3": 1287,
+  "WNf1g3": 1288,
+  "WNf3d2": 1289,
+  "WNf3d2(x)": 1290,
+  "WNf3d4": 1291,
+  "WNf3d4(x)": 1292,
+  "WNf3e1": 1293,
+  "WNf3e5": 1294,
+  "WNf3e5(+)": 1295,
+  "WNf3e5(x)": 1296,
+  "WNf3g1": 1297,
+  "WNf3g5": 1298,
+  "WNf3g5(+)": 1299,
+  "WNf3g5(x)": 1300,
+  "WNf3h2": 1301,
+  "WNf3h4": 1302,
+  "WNf3h4(x)": 1303,
+  "WNf4d3": 1304,
+  "WNf4d5": 1305,
+  "WNf4d5(x)": 1306,
+  "WNf4e6(x)": 1307,
+  "WNf4h5": 1308,
+  "WNf5e3": 1309,
+  "WNf5e7(+)": 1310,
+  "WNf7h8(x)": 1311,
+  "WNg1e2": 1312,
+  "WNg1f3": 1313,
+  "WNg1f3(x)": 1314,
+  "WNg1h3": 1315,
+  "WNg3e2": 1316,
+  "WNg3e4": 1317,
+  "WNg3e4(x)": 1318,
+  "WNg3f5": 1319,
+  "WNg3f5(x)": 1320,
+  "WNg3h5": 1321,
+  "WNg4e3": 1322,
+  "WNg4e5": 1323,
+  "WNg5e4": 1324,
+  "WNg5e4(x)": 1325,
+  "WNg5e6": 1326,
+  "WNg5e6(x)": 1327,
+  "WNg5f3": 1328,
+  "WNg5f7(x)": 1329,
+  "WNg5h3": 1330,
+  "WNh2f3": 1331,
+  "WNh2g4": 1332,
+  "WNh3f2": 1333,
+  "WNh3f4": 1334,
+  "WNh3g5": 1335,
+  "WNh4f3": 1336,
+  "WNh4f5": 1337,
+  "WNh4f5(x)": 1338,
+  "WNh4g6(x)": 1339,
+  "WPa2a3": 1340,
+  "WPa2a4": 1341,
+  "WPa2b3(x)": 1342,
+  "WPa3a4": 1343,
+  "WPa3b4(x)": 1344,
+  "WPa4a5": 1345,
+  "WPa4b5(x)": 1346,
+  "WPa5a6": 1347,
+  "WPa5b6(x)": 1348,
+  "WPa6a7": 1349,
+  "WPa7a8(Q)": 1350,
+  "WPb2a3(x)": 1351,
+  "WPb2b3": 1352,
+  "WPb2b4": 1353,
+  "WPb2c3(x)": 1354,
+  "WPb3a4(x)": 1355,
+  "WPb3b4": 1356,
+  "WPb3c4(x)": 1357,
+  "WPb4a5(x)": 1358,
+  "WPb4b5": 1359,
+  "WPb4c5(x)": 1360,
+  "WPb5a6(x)": 1361,
+  "WPb5b6": 1362,
+  "WPb5c6(x)": 1363,
+  "WPb6b7": 1364,
+  "WPb7b8(Q)": 1365,
+  "WPc2b3(x)": 1366,
+  "WPc2c3": 1367,
+  "WPc2c4": 1368,
+  "WPc2d3(x)": 1369,
+  "WPc3b4(x)": 1370,
+  "WPc3c4": 1371,
+  "WPc3d4(x)": 1372,
+  "WPc4b5(x)": 1373,
+  "WPc4c5": 1374,
+  "WPc4d5(x)": 1375,
+  "WPc5b6(x)": 1376,
+  "WPc5c6": 1377,
+  "WPc5d6(x)": 1378,
+  "WPc6c7": 1379,
+  "WPc7c8(Q)": 1380,
+  "WPd2c3(x)": 1381,
+  "WPd2d3": 1382,
+  "WPd2d4": 1383,
+  "WPd3c4(x)": 1384,
+  "WPd3d4": 1385,
+  "WPd3e4(x)": 1386,
+  "WPd4c5(x)": 1387,
+  "WPd4d5": 1388,
+  "WPd4e5(x)": 1389,
+  "WPd5c6(x)": 1390,
+  "WPd5d6": 1391,
+  "WPd5e6(x)": 1392,
+  "WPd6d7": 1393,
+  "WPd7d8(Q)": 1394,
+  "WPe2e3": 1395,
+  "WPe2e4": 1396,
+  "WPe3d4(x)": 1397,
+  "WPe3e4": 1398,
+  "WPe3f4(x)": 1399,
+  "WPe4d5(x)": 1400,
+  "WPe4e5": 1401,
+  "WPe4f5(x)": 1402,
+  "WPe5d6(x)": 1403,
+  "WPe5e6": 1404,
+  "WPe5f6(x)": 1405,
+  "WPe5f6(xE)": 1406,
+  "WPe6e7": 1407,
+  "WPe6f7(x+)": 1408,
+  "WPf2e3(x)": 1409,
+  "WPf2f3": 1410,
+  "WPf2f4": 1411,
+  "WPf2g3(x)": 1412,
+  "WPf3e4(x)": 1413,
+  "WPf3f4": 1414,
+  "WPf3g4(x)": 1415,
+  "WPf4e5(x)": 1416,
+  "WPf4f5": 1417,
+  "WPf4g5(x)": 1418,
+  "WPf5e6(x)": 1419,
+  "WPf5f6": 1420,
+  "WPf5g6(x)": 1421,
+  "WPf6f7": 1422,
+  "WPg2f3(x)": 1423,
+  "WPg2g3": 1424,
+  "WPg2g4": 1425,
+  "WPg2h3(x)": 1426,
+  "WPg3f4(x)": 1427,
+  "WPg3g4": 1428,
+  "WPg3h4(x)": 1429,
+  "WPg4f5(x)": 1430,
+  "WPg4g5": 1431,
+  "WPg4h5(x)": 1432,
+  "WPg5f6(x)": 1433,
+  "WPg5g6": 1434,
+  "WPg5h6(x)": 1435,
+  "WPg6g7": 1436,
+  "WPg7g8(Q)": 1437,
+  "WPh2g3(x)": 1438,
+  "WPh2h3": 1439,
+  "WPh2h4": 1440,
+  "WPh3g4(x)": 1441,
+  "WPh3h4": 1442,
+  "WPh4g5(x)": 1443,
+  "WPh4h5": 1444,
+  "WPh5g6(x)": 1445,
+  "WPh5h6": 1446,
+  "WPh6h7": 1447,
+  "WPh7h8(Q)": 1448,
+  "WQa4b3": 1449,
+  "WQa4c2": 1450,
+  "WQb3b7(x)": 1451,
+  "WQb3c2": 1452,
+  "WQb3d1": 1453,
+  "WQc2b3": 1454,
+  "WQc2c3": 1455,
+  "WQc2d2": 1456,
+  "WQc2d3": 1457,
+  "WQc2e2": 1458,
+  "WQc2e4(x)": 1459,
+  "WQd1a1(x)": 1460,
+  "WQd1a4": 1461,
+  "WQd1a4(+)": 1462,
+  "WQd1b3": 1463,
+  "WQd1c1": 1464,
+  "WQd1c2": 1465,
+  "WQd1d2": 1466,
+  "WQd1d2(x)": 1467,
+  "WQd1d3": 1468,
+  "WQd1d3(x)": 1469,
+  "WQd1d4": 1470,
+  "WQd1d4(x)": 1471,
+  "WQd1d5": 1472,
+  "WQd1d5(x)": 1473,
+  "WQd1d6(x)": 1474,
+  "WQd1d8(x)": 1475,
+  "WQd1d8(x+)": 1476,
+  "WQd1e1": 1477,
+  "WQd1e2": 1478,
+  "WQd1e2(+)": 1479,
+  "WQd1e2(x)": 1480,
+  "WQd1f3": 1481,
+  "WQd1f3(x)": 1482,
+  "WQd1g4": 1483,
+  "WQd1g4(x)": 1484,
+  "WQd1h5": 1485,
+  "WQd1h5(+)": 1486,
+  "WQd1h5(x)": 1487,
+  "WQd2c2": 1488,
+  "WQd2c3": 1489,
+  "WQd2d3": 1490,
+  "WQd2e2": 1491,
+  "WQd2e3": 1492,
+  "WQd2e3(x)": 1493,
+  "WQd2f2": 1494,
+  "WQd2f4": 1495,
+  "WQd2f4(x)": 1496,
+  "WQd2g5": 1497,
+  "WQd2h6(x)": 1498,
+  "WQd3c2": 1499,
+  "WQd3d2": 1500,
+  "WQd3e2": 1501,
+  "WQd3e3": 1502,
+  "WQd3e4(x)": 1503,
+  "WQd3f3": 1504,
+  "WQd3g3": 1505,
+  "WQd4d1": 1506,
+  "WQd4d3": 1507,
+  "WQd4e3": 1508,
+  "WQe2c2": 1509,
+  "WQe2c4": 1510,
+  "WQe2c4(x)": 1511,
+  "WQe2d1": 1512,
+  "WQe2d2": 1513,
+  "WQe2d3": 1514,
+  "WQe2e3": 1515,
+  "WQe2e3(x)": 1516,
+  "WQe2e4": 1517,
+  "WQe2e4(x)": 1518,
+  "WQe2f2": 1519,
+  "WQe2f3": 1520,
+  "WQe2f3(x)": 1521,
+  "WQe2g4": 1522,
+  "WQe2h5": 1523,
+  "WQe3e2": 1524,
+  "WQe3f3": 1525,
+  "WQe3g3": 1526,
+  "WQf3b7(x)": 1527,
+  "WQf3d1": 1528,
+  "WQf3d3": 1529,
+  "WQf3d5(x)": 1530,
+  "WQf3e2": 1531,
+  "WQf3e3": 1532,
+  "WQf3e4(x)": 1533,
+  "WQf3f4": 1534,
+  "WQf3f6(x)": 1535,
+  "WQf3g3": 1536,
+  "WQf3g4": 1537,
+  "WQf3h3": 1538,
+  "WQf3h5": 1539,
+  "WQg3f3": 1540,
+  "WQg3g4": 1541,
+  "WQg3h4": 1542,
+  "WQg4f3": 1543,
+  "WQg4g3": 1544,
+  "WQh5f3": 1545,
+  "WRa1a2": 1546,
+  "WRa1a3": 1547,
+  "WRa1a6(x)": 1548,
+  "WRa1a7(x)": 1549,
+  "WRa1a8(x)": 1550,
+  "WRa1b1": 1551,
+  "WRa1c1": 1552,
+  "WRa1c1(x)": 1553,
+  "WRa1d1": 1554,
+  "WRa1d1(x)": 1555,
+  "WRa1e1": 1556,
+  "WRa1e1(x)": 1557,
+  "WRa1f1": 1558,
+  "WRa1f1(x)": 1559,
+  "WRa1g1": 1560,
+  "WRa1h1": 1561,
+  "WRb1a1": 1562,
+  "WRb1b2": 1563,
+  "WRb1b2(x)": 1564,
+  "WRb1b3": 1565,
+  "WRb1b7": 1566,
+  "WRb1b7(x)": 1567,
+  "WRb1c1": 1568,
+  "WRb1d1": 1569,
+  "WRb1e1": 1570,
+  "WRb1f1": 1571,
+  "WRb7a7(x)": 1572,
+  "WRc1a1": 1573,
+  "WRc1b1": 1574,
+  "WRc1c2": 1575,
+  "WRc1c2(x)": 1576,
+  "WRc1c3": 1577,
+  "WRc1c3(x)": 1578,
+  "WRc1c4(x)": 1579,
+  "WRc1c5": 1580,
+  "WRc1c5(x)": 1581,
+  "WRc1c6(x)": 1582,
+  "WRc1c7": 1583,
+  "WRc1c7(x)": 1584,
+  "WRc1c8(x)": 1585,
+  "WRc1d1": 1586,
+  "WRc1e1": 1587,
+  "WRc1f1": 1588,
+  "WRc7b7(x)": 1589,
+  "WRd1a1": 1590,
+  "WRd1b1": 1591,
+  "WRd1c1": 1592,
+  "WRd1d2": 1593,
+  "WRd1d2(x)": 1594,
+  "WRd1d3": 1595,
+  "WRd1d3(x)": 1596,
+  "WRd1d4": 1597,
+  "WRd1d4(x)": 1598,
+  "WRd1d5": 1599,
+  "WRd1d5(x)": 1600,
+  "WRd1d6": 1601,
+  "WRd1d6(x)": 1602,
+  "WRd1d7": 1603,
+  "WRd1d7(+)": 1604,
+  "WRd1d7(x)": 1605,
+  "WRd1d8(+)": 1606,
+  "WRd1d8(x)": 1607,
+  "WRd1d8(x+)": 1608,
+  "WRd1e1": 1609,
+  "WRd1e1(x)": 1610,
+  "WRd1f1": 1611,
+  "WRd1g1": 1612,
+  "WRd1h1": 1613,
+  "WRd2e2": 1614,
+  "WRd7b7(x)": 1615,
+  "WRe1a1": 1616,
+  "WRe1b1": 1617,
+  "WRe1c1": 1618,
+  "WRe1d1": 1619,
+  "WRe1d1(x)": 1620,
+  "WRe1e2": 1621,
+  "WRe1e2(x)": 1622,
+  "WRe1e3": 1623,
+  "WRe1e3(x)": 1624,
+  "WRe1e4": 1625,
+  "WRe1e4(x)": 1626,
+  "WRe1e5": 1627,
+  "WRe1e5(x)": 1628,
+  "WRe1e6": 1629,
+  "WRe1e6(x)": 1630,
+  "WRe1e7": 1631,
+  "WRe1e7(x)": 1632,
+  "WRe1e8(+)": 1633,
+  "WRe1e8(x)": 1634,
+  "WRe1e8(x+)": 1635,
+  "WRe1f1": 1636,
+  "WRe1g1": 1637,
+  "WRe1h1": 1638,
+  "WRe2d2": 1639,
+  "WRe2e3": 1640,
+  "WRe3f3": 1641,
+  "WRe3g3": 1642,
+  "WRf1a1": 1643,
+  "WRf1a1(x)": 1644,
+  "WRf1b1": 1645,
+  "WRf1c1": 1646,
+  "WRf1c1(x)": 1647,
+  "WRf1d1": 1648,
+  "WRf1d1(x)": 1649,
+  "WRf1e1": 1650,
+  "WRf1e1(+)": 1651,
+  "WRf1e1(x)": 1652,
+  "WRf1f2": 1653,
+  "WRf1f2(x)": 1654,
+  "WRf1f3": 1655,
+  "WRf1f3(x)": 1656,
+  "WRf1f4": 1657,
+  "WRf1f4(x)": 1658,
+  "WRf1f5(x)": 1659,
+  "WRf1f6(x)": 1660,
+  "WRf1f7": 1661,
+  "WRf1f7(x)": 1662,
+  "WRf1f8(x+)": 1663,
+  "WRf1g1": 1664,
+  "WRf1h1": 1665,
+  "WRf2e2": 1666,
+  "WRf3g3": 1667,
+  "WRf3h3": 1668,
+  "WRg1e1": 1669,
+  "WRg1f1": 1670,
+  "WRg1g2": 1671,
+  "WRg1g3": 1672,
+  "WRg1h1": 1673,
+  "WRh1c1": 1674,
+  "WRh1d1": 1675,
+  "WRh1e1": 1676,
+  "WRh1f1": 1677,
+  "WRh1g1": 1678,
+  "WRh1h2": 1679,
+  "WRh1h3": 1680,
+  "WRh1h5(x)": 1681
+}