| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-Coder-7B-Instruct |
| datasets: |
| - Pinkstack/Roblox_Luau_CoT_conversational_sharegpt_lqv1 |
| language: |
| - en |
| tags: |
| - luau |
| - roblox |
| - code |
| - gguf |
| - qwen2.5 |
| --- |
| |
|  |
|
|
| qode1 is a specialised code generation model fine-tuned from `Qwen/Qwen2.5-Coder-7B-Instruct`. It is specifically optimised to generate valid, high-quality **Luau (Roblox Lua)** code using Chain-of-Thought reasoning. |
|
|
| ## Model Description |
|
|
| - **Developed by:** qodelabs |
| - **Shared by:** qodelabs |
| - **Model type:** Coder / Fine-tuned Large Language Model |
| - **Language(s) (NLP):** English |
| - **License:** Apache 2.0 |
| - **Finetuned from model:** Qwen/Qwen2.5-Coder-7B-Instruct |
|
|
| # Benchmarks |
|  |
| qode-luau-95 is an automated test suite benchmark containing 95 coding challenges designed for conversational instruct models. It is designed specifically to evaluate how accurately an AI model writes Luau code. |
| Rather than testing general knowledge, qode-luau-95 executes generated code inside a test runner to measure how well the model handles actual Luau scripting requirements. |
|
|
| qode-luau-95 is broken up into 4 unseen sections: |
| 1. Pure Luau Logic & Types (25 questions) |
| 2. Spatial Math & Vectors (20 questions) |
| 3. Data Structures & Systems (25 questions) |
| 4. Defensive Logic & Data (25 questions) |
|
|
| Below are the scores for each section (all models are their respective instruct variants): |
| | Section | qode1:7b | qwen2.5-coder:7b | codegemma:7b | llama3:8b | Average | |
| | :--- | :---: | :---: | :---: | :---: | :---: | |
| | 1 — Pure Luau Logic & Types | **10/25 (40.0%)** | 9/25 (36.0%) | 9/25 (36.0%) | 9/25 (36.0%) | 37.0% | |
| | 2 — Spatial Math & Vectors | **8/20 (40.0%)** | **8/20 (40.0%)** | 5/20 (25.0%) | 6/20 (30.0%) | 33.8% | |
| | 3 — Data Structures & Systems | 5/25 (20.0%) | 5/25 (20.0%) | **7/25 (28.0%)** | 3/25 (12.0%) | 20.0% | |
| | 4 — Defensive Logic & Data | **11/25 (44.0%)** | 9/25 (36.0%) | 9/25 (36.0%) | 4/25 (16.0%) | 33.0% | |
|
|
|
|
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| qode1 is intended for developers building games, scripts, and systems within the Roblox ecosystem using Luau. It can assist with: |
| - Writing object-oriented Luau modules and classes. |
| - Implementing vector math, CFrame operations, and spatial transformations. |
| - Building game logic, data structures, and state management systems. |
|
|
| ### Out-of-Scope Use |
|
|
| - General-purpose non-coding tasks (e.g., creative writing, general Q&A). |
| - Generating code for languages outside of Luau (though base capabilities for Python, C++, etc., may partially remain, the model is specialized for Luau). |
|
|
| ## Bias, Risks, and Limitations |
|
|
| - **Syntax Bleed:** The model may occasionally output C-style operators (e.g., ternary `? :` or logical `&&`) due to base model pre-training. Using an explicit system prompt is recommended to strictly enforce Luau syntax rules. |
| - **Nil Returns in Constructors:** When generating complex OOP structures, always verify that constructors explicitly return `self` or object instances. |
