README.md

---
license: apache-2.0
base_model: unsloth/Qwen2.5-Coder-7B-Instruct
tags:
  - godot
  - godot4
  - gdscript
  - game-development
  - lora
  - qlora
  - unsloth
  - code
  - saltshakerstudio
datasets:
  - glaiveai/godot_4_docs
language:
  - en
library_name: peft
pipeline_tag: text-generation
---

# Qwen2.5-Coder-7B-Instruct — Godot 4 / GDScript LoRA

A LoRA adapter for [unsloth/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct), fine-tuned on Godot 4 documentation Q&A to improve knowledge of modern Godot 4 GDScript syntax and APIs.

**Motivation:** Most open GDScript fine-tunes (e.g. godot-dodo, 2023) were trained on Godot 3 code and produce outdated syntax. Even current code models frequently mix Godot 3 and Godot 4 patterns. This adapter is a first attempt at nudging a modern 7B coder model toward Godot 4 conventions, trained locally on a single consumer GPU.

## Training details

| | |
|---|---|
| Base model | unsloth/Qwen2.5-Coder-7B-Instruct |
| Method | QLoRA (4-bit), via Unsloth Fine-tuning Studio |
| Dataset | [glaiveai/godot_4_docs](https://huggingface.co/datasets/glaiveai/godot_4_docs) (~3,490 Q&A pairs generated from Godot 4 documentation) |
| Epochs | 2 |
| Learning rate | 2e-4, linear schedule |
| LoRA rank / alpha / dropout | 16 / 16 / 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Batch size | 2 (gradient accumulation 4, effective 8) |
| Optimizer | AdamW 8-bit, weight decay 0.001 |
| Max sequence length | 4096 |
| Hardware | Single RTX 4080 (16 GB), Windows 11 |
| Training time | ~45 minutes |

Eval loss fell from ~0.83 to ~0.775 and plateaued. Runs at 1, 2, and 3 epochs showed 2 epochs to be the sweet spot for this dataset: 1 epoch left eval loss still declining, 3 epochs overfit (eval loss climbed after epoch 2 and chat-test quality regressed).

## What it improves

Compared to the base model in side-by-side chat tests, the adapter more consistently produces some Godot 4 conventions, for example the `@export` annotation:

```gdscript
@export var speed: int = 300   # Godot 4 ✅
# instead of:
export var speed = 300         # Godot 3 ❌
```

Responses also tend to be more concise and code-focused, reflecting the Q&A style of the training data.

## Known limitations — read before using

**This adapter does not fully solve the Godot 3 → 4 problem.** In testing, both the base model and this fine-tune still frequently produce Godot 3 patterns, especially:

- **Signal connections** — often writes the old form `button.connect("pressed", self, "_on_pressed")` instead of the Godot 4 form `button.pressed.connect(_on_pressed)`
- **Character movement** — may use `Sprite2D` or `KinematicBody2D` instead of `CharacterBody2D`, and `move_and_slide(velocity)` instead of `move_and_slide()`

The training dataset is derived from Godot 4 documentation and appears to be thin on these common game-programming patterns, so the model had little opportunity to learn them. Treat generated code as a draft and verify against the [current Godot documentation](https://docs.godotengine.org/en/stable/).

Other caveats:

- Trained and tested in English only
- Not evaluated on Godot C#, shaders (GDShader), or Godot 3 back-compatibility questions
- No safety or alignment tuning beyond what the base model provides

## Usage

Load the adapter on top of the base model with PEFT:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen2.5-Coder-7B-Instruct",
    load_in_4bit=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-Coder-7B-Instruct")
model = PeftModel.from_pretrained(base, "YOUR_USERNAME/YOUR_REPO_NAME")
```

Or merge and export to GGUF for local runners (llama.cpp, Ollama, LM Studio) — Unsloth Fine-tuning Studio provides an Export to GGUF option.

## Intended use

Hobbyist / experimental. A starting point for anyone interested in local Godot 4 coding assistants — including eventual use alongside a Godot MCP server, where better Godot 4 knowledge should translate into more correct tool calls. Contributions of better Godot 4 training data (especially signals, movement, and node-setup examples) would likely help more than additional training epochs on the current dataset.

## Acknowledgements

- Base model: Qwen team / Unsloth quantization
- Dataset: Glaive AI's godot_4_docs
- Training: Unsloth Fine-tuning Studio