SaltShakerStudio/Qwen2.5-GodotCoder

Qwen2.5-Coder-7B-Instruct — Godot 4 / GDScript LoRA

A LoRA adapter for unsloth/Qwen2.5-Coder-7B-Instruct, fine-tuned on Godot 4 documentation Q&A to improve knowledge of modern Godot 4 GDScript syntax and APIs.

Motivation: Most open GDScript fine-tunes (e.g. godot-dodo, 2023) were trained on Godot 3 code and produce outdated syntax. Even current code models frequently mix Godot 3 and Godot 4 patterns. This adapter is a first attempt at nudging a modern 7B coder model toward Godot 4 conventions, trained locally on a single consumer GPU.

Training details

Base model	unsloth/Qwen2.5-Coder-7B-Instruct
Method	QLoRA (4-bit), via Unsloth Fine-tuning Studio
Dataset	glaiveai/godot_4_docs (~3,490 Q&A pairs generated from Godot 4 documentation)
Epochs	2
Learning rate	2e-4, linear schedule
LoRA rank / alpha / dropout	16 / 16 / 0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Batch size	2 (gradient accumulation 4, effective 8)
Optimizer	AdamW 8-bit, weight decay 0.001
Max sequence length	4096
Hardware	Single RTX 4080 (16 GB), Windows 11
Training time	~45 minutes

Eval loss fell from ~0.83 to ~0.775 and plateaued. Runs at 1, 2, and 3 epochs showed 2 epochs to be the sweet spot for this dataset: 1 epoch left eval loss still declining, 3 epochs overfit (eval loss climbed after epoch 2 and chat-test quality regressed).

What it improves

Compared to the base model in side-by-side chat tests, the adapter more consistently produces some Godot 4 conventions, for example the @export annotation:

@export var speed: int = 300   # Godot 4 ✅
# instead of:
export var speed = 300         # Godot 3 ❌

Responses also tend to be more concise and code-focused, reflecting the Q&A style of the training data.

Known limitations — read before using

This adapter does not fully solve the Godot 3 → 4 problem. In testing, both the base model and this fine-tune still frequently produce Godot 3 patterns, especially:

• Signal connections — often writes the old form button.connect("pressed", self, "_on_pressed") instead of the Godot 4 form button.pressed.connect(_on_pressed)
• Character movement — may use Sprite2D or KinematicBody2D instead of CharacterBody2D, and move_and_slide(velocity) instead of move_and_slide()

The training dataset is derived from Godot 4 documentation and appears to be thin on these common game-programming patterns, so the model had little opportunity to learn them. Treat generated code as a draft and verify against the current Godot documentation.

Other caveats:

• Trained and tested in English only
• Not evaluated on Godot C#, shaders (GDShader), or Godot 3 back-compatibility questions
• No safety or alignment tuning beyond what the base model provides

Usage

Load the adapter on top of the base model with PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen2.5-Coder-7B-Instruct",
    load_in_4bit=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-Coder-7B-Instruct")
model = PeftModel.from_pretrained(base, "YOUR_USERNAME/YOUR_REPO_NAME")

Or merge and export to GGUF for local runners (llama.cpp, Ollama, LM Studio) — Unsloth Fine-tuning Studio provides an Export to GGUF option.

Intended use

Hobbyist / experimental. A starting point for anyone interested in local Godot 4 coding assistants — including eventual use alongside a Godot MCP server, where better Godot 4 knowledge should translate into more correct tool calls. Contributions of better Godot 4 training data (especially signals, movement, and node-setup examples) would likely help more than additional training epochs on the current dataset.

Acknowledgements

• Base model: Qwen team / Unsloth quantization
• Dataset: Glaive AI's godot_4_docs
• Training: Unsloth Fine-tuning Studio