ippity/chessLM-0.01-llama-3.1-8b

Chess AI Model (1300-1700 ELO)

Model Description

This finetune aims to simulate a human chess player with an ELO range between 1300 and 1700 on Lichess. The ultimate goal is to create a model that feels like you're playing against a human, as existing bots on platforms like Chess.com and Lichess often feel inorganic and aren't conducive to learning how to improve your ELO.

The model is based on Llama 3.1 8b and trained on 20k+ chess games. It navigates the opening phase well and plays moves that feel natural and organic. However, there are areas that need improvement:

1. Lack of positional awareness.
2. Performance deteriorates after about 20 moves.
3. Future training plans involve higher ELO ranges and narrower skill levels to improve gameplay quality.

Prompt Format

The model is finetuned using an Alpaca-style prompt. Below is an example of how the prompt format looks:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Given the moves so far in a chess game and the legal moves in the current position, predict the subsequent moves until the end of the game.

### Input:
Moves so far: e4 d5 exd5
Legal Moves: Qd8+ Qc8+ Qb8+ Qxf7...

### Response:

Game Loop Usage Example

You can play against the model using the following Python code, which includes a chess game loop. The code interacts with the model to generate moves based on the current state of the board:

import chess
import transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Define the Alpaca prompt format
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

# Enable faster inference using FastLanguageModel
FastLanguageModel.for_inference(model)  # Enable native 2x faster inference

def generate_model_move(board, move_history):
    # Get the legal moves in the current position
    legal_moves = [board.san(move) for move in board.legal_moves]

    # Prepare the input for the model
    instruction = "Given the moves so far in a chess game, predict the subsequent moves until the end of the game."
    input_text = f"Moves so far: {' '.join(move_history)}\nLegal moves: {' '.join(legal_moves)}"

    inputs = tokenizer(
    [
        alpaca_prompt.format(instruction, input_text, "")
    ], return_tensors = "pt").to("cuda")

    # Generate the model's response
    outputs = model.generate(
        **inputs,
        max_new_tokens=16,
        use_cache=True
    )

    # Decode the output and extract the moves
    decoded_output = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

    # Extract the response after the '### Response:' section
    response_marker = "### Response:"
    if response_marker in decoded_output:
        response = decoded_output.split(response_marker)[-1].strip()
    else:
        response = decoded_output.strip()

    # The model's output is the remaining moves; we need the first move
    predicted_moves = response.split()
    if not predicted_moves:
        print("Model did not provide any moves.")
        return None

    print(predicted_moves)

    # Validate the model's move
    for move_san in predicted_moves:
        try:
            move = board.parse_san(move_san)
            if move in board.legal_moves:
                return move_san
        except ValueError:
            continue  # Invalid SAN notation, try next move

    print("Model did not provide a valid legal move.")
    return None

def main():
    # Initialize the chess board
    board = chess.Board()
    move_history = []

    print("Welcome to Chess AI Game!")
    print("You are playing as White. Enter your moves in SAN notation (e.g., e4, Nf3).")
    print(board)

    while not board.is_game_over():
        # User's turn
        while True:
            user_move_san = input("Your move: ")
            try:
                user_move = board.parse_san(user_move_san)
                if user_move in board.legal_moves:
                    board.push(user_move)
                    move_history.append(user_move_san)
                    break
                else:
                    print("Illegal move. Please try again.")
            except ValueError:
                print("Invalid move notation. Please try again.")

        if board.is_game_over():
            break

        # Model's turn
        print("AI is thinking...")
        model_move_san = generate_model_move(board, move_history)
        if model_move_san is None:
            print("AI failed to make a move. You win!")
            break

        model_move = board.parse_san(model_move_san)
        board.push(model_move)
        move_history.append(model_move_san)
        print(f"AI move: {model_move_san}")
        print(board)

    # Game over
    result = board.result()
    print(f"Game over. Result: {result}")

if __name__ == "__main__":
    main()

Future Improvements

I plan to continue improving the model in several areas:

1. I aim to enhance the model’s understanding of mid- and late-game positions.
2. The model struggles with accurate predictions past 20 moves, so further training on more extensive game data is necessary.
3. In the future, I plan to train the model on higher ELO games, particularly in narrower ELO brackets, to fine-tune its skill. Additionally, I am working on software to make it easier to play against this model without needing to set everything up manually. Example games and further documentation will be added in future updates.